Formative vs summative usability testing for medical devices
Gideon Awolesi ·
Formative testing is how you find problems while you can still change the design. Summative testing is how you show, at the end, that people can use the finished device safely. A lot of medical device teams treat them as two versions of the same test, and that's usually when the summative study turns up a surprise nobody has time to fix. This guide explains the difference, using IEC 62366-1 and the FDA's human factors guidance, which was reissued on 3 August 2026.
01The definitions
IEC 62366-1 defines formative evaluation as a "user interface evaluation conducted with the intent to explore user interface design strengths, weaknesses, and unanticipated use errors." Summative evaluation is a "user interface evaluation conducted at the end of the user interface development with the intent to obtain objective evidence that the user interface can be used safely."
The FDA uses the term human factors validation testing for the end-of-development test. Its guidance notes that this "is sometimes referred to as summative testing", but that some definitions of summative testing leave out essential parts of validation testing. So if you're submitting to the FDA, design your final study to the FDA's requirements, whatever you call it.
02Side by side
| Formative | Summative / validation | |
|---|---|---|
| Purpose | Find design strengths, weaknesses, and unanticipated use errors | Show with objective evidence that the device can be used safely |
| When | Throughout design, as often as you need | At the end of user interface development |
| What you test | Mock-ups, early prototypes, or more advanced prototypes | The final design of the user interface, including final labelling |
| Participants | Small groups are fine, and your own staff can take part | Representative users. The FDA expects at least 15 from each distinct user population, and not your employees unless all users are employees |
| Tasks | Whatever you're unsure about | All critical tasks |
| Output | Design changes | Evidence for your usability engineering file and submission |
03How many participants you need
For validation testing, the FDA's guidance says at least 15 participants, and if you have more than one distinct user population, "at least 15 participants from each user population." Populations are distinct when their characteristics would likely affect how they use the device, or when they do different tasks. A nurse and a patient using the same device at home are usually two populations, so that's at least 30 participants.
IEC 62366-1 doesn't set a number. For formative work there's no fixed minimum, because the goal is to learn and improve the design.
04Formative methods that work
The FDA's guidance describes several formative methods, and in practice you combine them:
- Task analysis, to break down what users do and find the critical tasks
- Heuristic evaluation and expert review, before you put anything in front of users
- Contextual inquiry and interviews, to understand the environment the device is used in
- Cognitive walkthroughs, where users talk through how they'd complete a task
- Simulated-use testing with a prototype, which is where we usually find the most useful problems
05When validation finds a new use error
This is the situation formative testing exists to prevent. The FDA's guidance is clear that serious use errors found in validation aren't acceptable in a premarket submission, unless you can show that further risk reduction isn't possible or practical and the benefits outweigh the risks. Saying you'll fix it in a later version isn't acceptable. The guidance also says such findings might indicate that not enough formative evaluation was done.
In practice, that means redesigning and testing again, at the most expensive point in the project. A few rounds of formative testing on prototypes cost far less than one failed validation study.
06What the FDA now expects in a submission
The FDA's guidance on the content of human factors information in marketing submissions became final on 29 May 2026, replacing the 2022 draft. It sorts submissions into three categories. A change that doesn't affect the user interface, users, uses, environment, training, or labelling needs only a short justification (category 1). A change with no new or affected critical tasks needs a rationale (category 2). Where critical tasks are involved and the risk controls aren't already proven, you need a full human factors report, including validation testing (category 3). Submissions received before 1 August 2026 generally aren't expected to follow it.
07How we plan it
We plan the formative rounds alongside the design, so each round tests the riskiest flows while they're still prototypes. When a device already on the market gets a new feature, we separate the new tasks from the ones that were already validated, so the testing focuses on where the new risk is.
If your summative study is coming up and you're not sure your usability engineering file will hold up, our Usability Review checks it clause by clause against IEC 62366-1 and the FDA's guidance.