Getting useful feedback on a robot prototype

“It's cute” is pleasant feedback, but it does not tell a robotics team what to build next. Neither does a list of features someone might enjoy someday. A useful prototype session helps answer a specific uncertainty: can a person recognize the ready state, start the intended action, understand the response, or recover when the device stops? Choose the uncertainty before inviting people to see the robot.

Physical prototypes have a particular complication. A polished shell can make an incomplete device look finished, while exposed wiring can make a promising interaction look fragile. Participants need a clear account of what is being tested. Explain which functions work, which are simulated, and when the facilitator may need to intervene. That context protects the usefulness of the feedback without requiring a long technical lecture.

The following example is a planning exercise for an adult desktop-robot prototype. The device runs a timer, signals completion with a moving part, and has a quiet control. Its shape and behavior are fictional. The test is intended to uncover interaction problems before a wider study, not to estimate market demand or establish a statistical performance claim.

Turn the team's argument into a question

Suppose a team disagrees about whether a completion gesture is clear enough. One person wants a larger motion; another wants a sound. Write the research question as “How do participants notice and interpret the end of a timer while doing a desk task?” That keeps the study open to possibilities the team has not proposed, such as a held position or a clearer status display.

Define what an observation can support. If a participant notices the gesture and says the timer has ended, that is evidence about recognition in the test conditions. It is not evidence that they would use the product every day. If they say they would buy it, record the comment as a stated preference rather than a sale. Keep those categories separate in the final notes.

The GOV.UK guide to moderated usability testing recommends realistic tasks that do not reveal the answer and a discussion guide for consistent sessions. Those principles translate well to a physical prototype, while the team must separately address the hardware's operating conditions and any safety constraints.

Recruit for the context you need to understand

Choose participants who resemble the intended adult users in the way that matters for the question. For a desk notification, their work environment and interruption habits may matter more than how enthusiastic they are about robots. Include people who use different desk layouts or rely on different ways of receiving information. Record those differences so the observations have context.

Avoid letting the team's close collaborators stand in for unfamiliar customers. They may know where the hidden control is or understand the motion from earlier conversations. Colleagues can help rehearse the session, but label that work as a pilot. The pilot is a chance to discover a confusing instruction, a broken prop, or a prototype state that is hard to reset.

Tell participants what the session involves before they arrive, including the expected duration, the recording choice, and any accessibility arrangements. Obtain appropriate consent for research and recording. Keep participation separate from permission to use someone's face or words in a public launch video. Those are different requests with different consequences for the participant.

Prepare a script with room to listen

Start with a short explanation of the prototype and remind the participant that the design is being evaluated. Then ask about a recent work session in which they used a timer or missed a reminder. A concrete example gives context for the later task. Avoid asking them to imagine an ideal robot before they have tried the behavior under study.

GOV.UK's in-depth interview guidance encourages open, neutral questions and discussion of actual experiences. For this session, a useful opening is “Tell me about the last time you used a timer while working.” Follow the participant's answer rather than steering them toward the problem the team hopes to solve.

The task instruction could be: “You want to spend a short period on this sketch. Use the device to let you know when that period ends.” Do not mention the dial or the completion flag if discovering those controls is part of the test. Have the participant perform a simple realistic task while the timer runs, so the signal is observed in context instead of under a spotlight.

Record behavior and interpretation separately

Use a note sheet with four columns: time or event, observed action, participant's words, and possible explanation. “Reached behind the device” belongs in the observation column. “Thought the switch was at the back” belongs there only if the participant said it. “Control placement may be confusing” is an interpretation to consider later, not a fact to merge into the record.

Write down any help the facilitator gives. If the participant completes the task after being told which button to press, preserve that distinction. A successful demonstration with assistance can still teach something, but it answers a different question from an unassisted attempt. Record prototype faults too, so a mechanical failure is not mistaken for a participant's misunderstanding.

For the timer example, an observation might read: “At the end, participant glanced at the raised arm and continued drawing. After twenty seconds, asked whether the timer was still running.” That note suggests a follow-up question. It does not yet prove whether the problem was the movement, the display, the instructions, or a preference to finish the current sketch stroke.

Ask about the moment while it is still clear

After a task, return to specific events. “What did you think had happened when the arm moved?” is more useful than “Was the notification clear?” The first allows an unexpected interpretation. The second invites a quick judgment using the team's own framing. If the participant proposes a new feature, ask which difficulty it would address before adding it to a roadmap.

Include a cancellation task. Ask the person to make the device quiet before a pretend call. Observe how they attempt it and what they believe remains active afterward. This can expose an important mismatch: a quiet control might silence sound while the person expects it to stop all motion. The label and behavior may need to change together.

End by asking which part would need improvement before the person would want to try the prototype again in their own setting. That grounds preference in the experience they just had. Leave room for a negative answer. A participant should not have to perform enthusiasm to be polite to the person who built the object.

Convert the session into a decision

After each session, identify observations that relate directly to the original question. Group similar issues, but keep counterexamples. If one person understood a signal and another did not, the difference may point to viewing angle, expectations, or timing. The useful output is a set of plausible explanations and a next experiment, not a forced consensus that the design is good or bad.

Write each proposed change beside the evidence it is intended to address. For example: “Hold the completion pose until acknowledgment; test whether users can determine the timer state after missing the initial movement.” This connects the design change to a question that can be tested in the next round. Assign someone to make the change and someone to review whether it answers that question.

Keep the original notes securely, limit access to the research team, and follow the retention terms given to participants. Share a concise finding with colleagues who need the design decision, without circulating unnecessary personal detail. The next session should begin with a revised question or prototype. Repeating the same demonstration without changing what the team can learn is rarely the best use of another participant's time.