PLAYBOOK
AI grading is only as good as its alignment to your marking scheme. Here's how to set that up so scores match the rubric, not a generic model's guess.
CrazyGoldFish
IN SHORT
AI grades reliably when it scores against your actual marking scheme, not a generic notion of a good answer. That means digitising the scheme into a per-question rubric, mapping it to the evaluation layer, calibrating on a sample against teacher marks, routing the question types prone to disagreement to a teacher, and keeping review and override on. Done this way, the AI becomes a rubric-faithful second reader rather than an opaque scorer.
// WHAT.YOU'LL.DO
CHECKBreak the scheme down to what earns each mark on each question, including partial credit.
WHYHow granular to go. The more explicit the rubric, the more faithfully it can be enforced — vague schemes produce vague scoring, by AI or human.
CHECKAttach the rubric and the model answer to each question so the AI scores against them, not against a generic standard.
WHYWhere model answers come from (yours, or co-authored). The model answer is the anchor the feedback references.
CHECKGrade a real sample with both AI and teachers, compare, and tune the rubric where they diverge.
WHYYour acceptable agreement band. Calibration is where a marking scheme gets tightened and the scoring gets trustworthy.
CHECKIdentify the question formats where AI and teacher most often disagree and send those to a teacher rather than auto-scoring.
WHYYour thresholds. Most disagreement concentrates in a few question types; routing them is cheap and raises trust in the rest.
CHECKSurface every score for teacher review with edit and override, logged to an audit trail.
WHYYour approval states. Overrides feed back as calibration signals, so alignment improves over time.
// CHECKLIST
KEEP READING
// COMMON.QUESTIONS
Why does marking-scheme alignment matter?
Because a score is only meaningful against the scheme it's supposed to reflect. Aligning the AI to your rubric and model answers makes the scoring faithful to your intent, not a generic model's guess.
How do we know the AI follows the rubric?
Calibration. You grade a sample with both AI and teachers, compare, and tune the rubric where they diverge — and the feedback explicitly references the model answer, which is the check against drift.
What about partial credit?
Partial credit is part of the rubric. When the scheme specifies what earns half a mark, the AI can apply and explain it — and those explanations are part of the per-question feedback.
Can teachers still override?
Always. Review and override stay on, every change is logged, and overrides feed back as calibration signals that improve alignment over time.