Why EdTech Platforms Are Throwing Away Their Most Valuable Assessment Data

EdTech platforms generate their most valuable data at the moment of student evaluation — then discard it. When a student answers an open-ended question, the answer contains their reasoning, their misconceptions, and the specific gap that needs closing. Most platforms record only the score. The reasoning disappears. This is the EdTech data problem.

A student writes three sentences on a history test. The AI scores it 6/10. The score goes into the gradebook. The three sentences — what the student understood, where their logic broke, what they almost-got — disappear.

This happens approximately 350 million times per year in India alone. Every evaluation event generates a signal. Most EdTech platforms record the score — the assessment data they keep is the number, not what's inside the answer.

That's not an oversight. It's a structural choice baked into how EdTech was built. And it's a choice the industry is only now beginning to question.

---

What EdTech Actually Captures from Assessments

Modern EdTech platforms are sophisticated about tracking everything that happens around learning. Login frequency. Time-on-task. Completion rates. Drop-off points in video. Pages visited, modules completed, sessions per week.

What they're structurally blind to is what happens inside learning — specifically, what a student writes, draws, or speaks when they attempt an open-ended answer.

The 1EdTech analytics workstream, which sets interoperability standards for how platforms exchange data, defines assessment data in terms of outcomes and completion events. The xAPI and Caliper specifications are built for "the student completed this" and "the student scored X." They have no native representation for "the student wrote this, reasoned this way, and was wrong about this specific thing."

This isn't a technical limitation of those standards. It's a reflection of what EdTech platforms have historically decided to store. EdTech assessment data, as the industry defines it, means scores, completion status, and session metadata. It does not mean the content of the answer.

The content of the answer — the student's own words, their working, their reasoning — is typically not stored anywhere after the evaluation is complete. The score replaces it. The answer disappears.

---

The Score Is a Compression Artifact

A score compresses 200 words of student reasoning into a single integer.

That compression is not neutral. It's irreversible. Once you've scored the answer and moved on, the signal inside it is gone. You cannot reconstruct a student's misconception from a 6/10. You cannot derive an error type from a 72%. You cannot generate targeted remediation from a grade.

Scores were designed for administrative systems — gradebooks, transcripts, reports, rank lists. The decision to store a score and discard the answer was made by bureaucratic infrastructure in the 1950s, not by any educational theory about what drives learning improvement. EdTech inherited this decision without questioning it.

The result is visible in the data. The Annual Status of Education Report (ASER) 2024 found that more than 50% of Grade 5 students in rural India cannot read a Grade 2 text, despite years of annual assessment. The scores exist. They are recorded, reported, and filed. The signal that could have diagnosed exactly where the reading failure lives — the specific phonemic gap, the word-class confusion, the comprehension breakdown — has been discarded at every evaluation point along the way.

The score told us there was a problem. It couldn't tell us what the problem was. And so the problem persisted.

---

What Student Assessment Data Actually Contains

A handwritten open-ended answer contains, at minimum, four distinct signals.

What the student knows. What they think they know but are wrong about. How they reason from premise to conclusion. And what would close the specific gap in their understanding.

These four signals collapse into a single number when you score the answer. What looks like a 6/10 might be a student who understood the core concept but misapplied the framing, which suggests one remediation path. Or it might be a student with a foundational misconception that will compound across every subsequent topic, which suggests a completely different one. The score cannot tell you which.

At CrazyGoldFish, we've seen what becomes visible when you stop discarding the answer. In a deployment with Educate Girls, evaluating 2,297 Hindi handwritten student answers in field conditions, 39% of AI-generated feedback named the exact error and provided the next step to close it. 58% included forward-looking guidance. Mean feedback length was 216 characters — approximately 30 Hindi words — specific enough to act on, not generic. Internal reliability across 11 tests reached α=0.808 (Educate Girls internal validation, Jan-Feb 2026), meeting the accepted threshold for reliable measurement even at mixed script and varied handwriting quality.

In a separate government school pilot in Andhra Pradesh, structured evaluation of open-ended answers achieved 93% inter-rater reliability (ICC), published in the IndiaAI Impact Summit 2026 Compendium, MeitY. This wasn't a controlled lab condition — it was a parallel grading study on field-produced scripts in a public education system.

The signal is in the answer. It has always been there. The question is whether the infrastructure exists to capture it.

---

What Changes When You Treat Evaluation as Infrastructure

Three things become possible when you stop discarding the answer.

First, error-type analytics across a cohort. If 60% of a class missed the same concept in the same way, a teacher can know this — not by reviewing 60 scripts, but because the evaluation infrastructure extracted and structured that signal across every answer simultaneously. This is a different kind of teaching decision than "class average is 62%."

Second, longitudinal signal. A score tells you where a student landed on a particular day. Ten structured answer evaluations over a semester tell a story — how reasoning developed, which gaps closed, which misconceptions proved persistent. A score sequence shows a line. A structured signal sequence shows a learning arc.

Third, content generation can close the loop. When you know the specific error in a specific student's answer, you can generate remedial content targeted at that gap — not a generic worksheet on fractions, but a worksheet that addresses the particular misunderstanding visible in what the student actually wrote. This is what evaluation infrastructure makes possible.

"Infrastructure" here is not a marketing word. It means the signal persists and compounds. A score is a transaction — it records an outcome and closes. A structured answer-signal is an asset — it accumulates, enables new decisions, and becomes more valuable as it compounds with subsequent evaluations.

The evaluate-to-personalise-to-generate loop only works if the evaluation layer structures and retains the signal. If the score replaces the answer, the loop breaks at the first step.

---

Why This Hasn't Been Solved Until Recently

The problem persisted for a structural reason: open-ended answers are hard to evaluate reliably at scale.

Multiple-choice answers have been auto-graded since the 1920s. The mechanics are trivial. Open-ended answers — handwritten or typed, multilingual, varied in structure, requiring rubric-based judgment — require something closer to human assessment capability. That capability at production scale, at accuracy levels suitable for instructional decision-making, in languages beyond English, is a 2024-2025 development.

LLMs capable of evaluating open-ended answers in Hindi, Tamil, and mixed-script handwriting at 90%+ accuracy on structured rubrics did not exist at meaningful cost in 2020. They do now.

The barrier was not philosophical. EdTech platforms did not choose to discard the signal because they didn't care about it. They discarded it because capturing formative assessment data at scale — reliably, in multilingual handwritten contexts — was not possible. That barrier has dropped. What remains is whether platforms will build the infrastructure to capture what they can now reliably extract — or continue discarding it by default because the gradebook was built for scores.

---

Every platform evaluating open-ended answers at scale is making a choice right now.

The answer can be an event — scored, recorded, and forgotten. Or it can be a signal — captured, structured, and compounded into something that actually tells you whether learning happened and what to do about it.

That choice determines whether the data you're already generating becomes an asset or a disposal operation.

For more on how evaluation infrastructure works in practice, see AI evaluation of open-ended Hindi answers and the evaluation infrastructure overview.

---

Frequently Asked Questions

What is assessment data in EdTech? Assessment data in EdTech typically refers to scores, completion rates, and time-on-task metrics collected after a student submits work. In open-ended evaluation, the answer itself — the student's reasoning, working, and misconceptions — is a richer signal that most platforms don't capture.

What are learning signals in education? Learning signals are structured data extracted from a student's response that go beyond a score: error type, misconception classification, rubric alignment, and gap-to-close. They require evaluation infrastructure that processes the content of the answer, not just the outcome.

Why do EdTech platforms not use open-ended answer data? Three reasons: (1) open-ended answers were historically hard to evaluate reliably at scale, (2) existing gradebook infrastructure was designed for scores, not structured signals, and (3) LLM-based evaluation capable of extracting reliable signals at scale is a recent development (2024-2025).

What is evaluation infrastructure in education? Evaluation infrastructure is the layer that converts student assessments into structured learning signals — rubric alignment, error classification, feedback generation — that persist, compound, and drive downstream personalisation and content generation. Distinct from a "grading tool," which produces a score and stops.

How does AI evaluation infrastructure work? AI evaluation infrastructure processes student submissions (handwritten, typed, audio) through OCR and LLM layers to produce structured output: score, rubric alignment, error type, feedback, and confidence band. Human-in-the-loop review handles edge cases. The signal is stored, not discarded, enabling longitudinal learner profiles.