Separate three different activities
An AI heuristic review can flag possible usability problems. An automated browser check can verify that a button or flow functions. A usability study observes actual people attempting tasks. All three can help, but they answer different questions. Never report generated personas or AI role-play as participant findings.
Step 1: write a focused research question
Choose one journey and the decision the study will support. For a fictional booking application, ask whether a first-time user can identify an available time and understand the confirmation. Define observable completion before testing. Identify the relevant audience and accessibility needs.
Step 2: draft tasks that do not reveal the answer
Ask AI to remove leading language. “You need an appointment next week; show me what you would do” is more useful than “Click the calendar and select the green slot.” Prepare neutral follow-ups such as “What did you expect to happen?” and “What are you looking for now?”
Review this usability study plan for leading questions, missing accessibility considerations and unclear success criteria. Rewrite tasks without naming interface controls. For each task list the behavior we will observe and what would count as completion, partial completion or failure. Do not create participant responses or predicted findings.
Step 3: recruit and run real sessions
Select participants who match the target use case. A small formative study can reveal friction, but does not estimate population-wide prevalence. Obtain appropriate consent for recording and transcription. Use approved storage and de-identify material before sharing with an AI tool.
Keep a session log: participant ID, task, outcome, observation, timestamp and moderator intervention. If you helped someone complete a task, mark that completion as assisted. Preserve negative cases and unexpected workarounds.
Step 4: synthesize without changing the evidence
Ask AI to group observations and attach source IDs to every finding. Check each citation against the session record. Distinguish observation (“participant returned to the previous screen”) from interpretation (“confirmation may be unclear”) and recommendation (“test a clearer confirmation”).
| Finding field | What to record |
|---|---|
| Evidence | Participant IDs and exact observed behavior |
| Impact | Whether it blocks, delays or confuses task completion |
| Scope | Which task and segment were studied |
| Confidence | Strength and limits of the evidence |
| Next step | Owner, proposed change and follow-up question |
Step 5: review and retest
Prioritize issues based on task impact and repeated evidence, with care for severe issues observed only once. Ask design and engineering to review feasibility. Retest a changed journey instead of assuming that implementing a recommendation solved the problem.
Practice assignment
Spend 45 minutes preparing a study and another session reviewing permitted notes. Deliver a neutral task guide and an evidence-linked findings table. You pass when every finding can be traced to an actual observation, assistance is recorded and sample limitations are visible. Research time and recruitment are separate from this preparation exercise.