Learning how to build an adaptive assessment is mostly learning to build an item bank, because the branching logic is straightforward and the scoring is arithmetic. The item bank is where the work lives, and it is where most attempts collapse. Teams design elaborate routing rules over thirty questions, discover that thirty is nowhere near enough, and quietly revert to a fixed quiz. This walks through the sequence that works, whether you implement it in Adaptive Assessment or build it yourself.
Before You Start
- The item bank is 80% of the work. Budget accordingly, because branching logic is comparatively trivial.
- You need roughly 5 to 8 items per difficulty level per objective. Below that, learners see repeats.
- Difficulty must be measured from learner data, not estimated by the author. Author estimates are consistently wrong.
- Branch on demonstrated understanding of an objective, not on a running total score.
- Report the objective a learner failed, not just the percentage, because the percentage is not actionable.
🖥️ Sign In to Access Your Dashboard
Step One: Define Objectives Before Writing Items
An adaptive assessment routes learners based on what they have demonstrated about a specific objective. So the objectives come first, and they need to be narrow enough to be individually testable.
“Understands data protection” is not testable. “Can identify which categories of personal data require explicit consent” is. If an objective cannot be assessed by a single well-written question, split it.
Six to ten objectives is a workable range for a module. More than twelve and the item bank requirement becomes impractical for most teams.
Step Two: Build the Item Bank
This is the part that gets underestimated. For each objective you need items at three difficulty levels, and enough at each level that a learner retrying does not see the same question.
| Objectives | Difficulty levels | Items per level | Total items needed |
| 6 | 3 | 5 | 90 |
| 8 | 3 | 6 | 144 |
| 10 | 3 | 8 | 240 |
Ninety items for a six-objective module surprises people. It is the honest number, and generating a first draft from your existing course material is considerably faster than authoring from scratch, though every item still needs review.
Three rules for writing items:
One objective per item: An item testing two objectives cannot route, because a wrong answer does not tell you which objective failed.
Distractors must be plausible: A wrong answer nobody selects carries no information. Good distractors encode common misconceptions, which makes the wrong answer itself diagnostic.
Difficulty comes from cognitive demand, not obscurity: A hard item requires applying a principle to a novel situation. It does not require remembering a footnote.
Step Three: Calibrate Difficulty From Data
Author-estimated difficulty is unreliable. Experts consistently misjudge what novices find hard, which is a well-documented effect known as the curse of knowledge.
Run the full bank flat, non-adaptively, on your first cohort. Record the proportion answering each item correctly. That proportion is your difficulty measure.
- Above 85% correct: easy
- 50 to 85% correct: medium
- Below 50% correct: hard
- Below 20% correct: inspect the item, because it is more likely broken than hard
Recalibrate annually, since items drift as content and audience change. Formal approaches such as Item Response Theory go further and are worth reading if your assessment carries certification weight.
📄 Generate a Free PDF Sample Course in Your Cloned Voice
Step Four: Write the Branching Rules
Keep them simple. Complex routing is harder to debug and rarely outperforms a straightforward rule.
The rule that works for most corporate assessment:
- Start every objective at medium difficulty.
- Correct answer, move up one level. Incorrect, move down one.
- Two consecutive correct at the same level marks the objective as demonstrated. Move to the next objective.
- Two consecutive incorrect at the easiest level marks the objective as not demonstrated. Move on and flag it.
- Cap at five items per objective regardless, so the assessment terminates.
Branch per objective, not on a running total. A learner strong in four areas and weak in two should get harder questions in the four and easier ones in the two, which a total score cannot express.
Step Five: Score and Report
The output should be an objective-level profile rather than a percentage.
| Objective | Status | Highest level demonstrated |
| Consent categories | Demonstrated | Hard |
| Retention periods | Demonstrated | Medium |
| Breach notification | Not demonstrated | Easy failed |
| Transfer rules | Demonstrated | Medium |
This tells an instructor exactly where to intervene. A score of 74% does not. It also feeds directly into remediation, since you now know which module to send the learner back to rather than making them repeat everything. Where the same objective fails across many learners, the content is the problem rather than the cohort, which connects to how you diagnose drop-off and confusion signals in existing material.
Solving the Item Bank Problem
To be clear about scope: adaptive assessment does not decide whether someone is certified. Pass thresholds, awarding rules, and any regulatory requirement stay with you and your accrediting body.
What it affects is the item bank problem. Generating a first-draft bank from your existing course content turns a multi-week authoring project into a review exercise, and calibration data accumulates automatically as cohorts run.
For a provider building assessment across an 8-objective module
| Metric | Before | After |
| Time to author a 144-item bank | 5 weeks | 4 days including review |
| Difficulty calibration | Author estimate | Cohort data |
| Reporting granularity | Percentage | Per objective |
| Learners repeating full module on failure | All | Failed objectives only |
Building Your First Adaptive Assessment
- Write 6 to 10 narrow, individually testable objectives.
- Generate a first-draft item bank at three difficulty levels, then review every item.
- Run the bank flat on your first cohort to calibrate difficulty from actual results.
- Implement the simple branching rule and cap items per objective.
- Report at objective level and route remediation to the specific failed objective.
- Recalibrate annually and inspect any item below 20% correct.
Frequently Asked Questions
Roughly 5 to 8 items per difficulty level per objective, across three levels. For a six-objective module that is about 90 items, and for ten objectives closer to 240. Below this, learners retrying will see repeated questions and the routing degrades.
From cohort data, not author estimates, since experts consistently misjudge what novices find hard. Run the bank flat on a first cohort and use the proportion answering correctly. Above 85% is easy, 50 to 85% is medium, and below 50% is hard.
Start at medium, move up one level on a correct answer and down on an incorrect one. Two consecutive correct at a level marks the objective demonstrated. Cap at five items per objective so the assessment terminates. Branch per objective rather than on total score.
A profile. An objective-level breakdown tells an instructor exactly where to intervene and lets remediation target the specific failed objective. A percentage tells you a learner scored 74% without indicating which of eight objectives they missed.
It can generate a first draft from your existing course material, which converts a multi-week authoring project into a review exercise. Every item still needs human review, particularly the distractors, since plausible wrong answers are what make an item diagnostic.
If your last attempt at adaptive assessment reverted to a fixed quiz, the item bank was almost certainly too small. Count what you would need before designing any routing.
