A checker you can't verify is just an opinion. So we test Pedagrade on quizzes and lessons where we already know the answer, and publish what it catches, what it misses, and how often it raises a false alarm. Weak spots included.
Every test uses material where the right answer is known before Pedagrade sees it. We run the same service you use.
326 questions from a public, MIT-licensed academic question bank, kept only where we verified the answer key, plus 15 course questions we wrote, each checked 3 times. Most quizzes get one planted mistake: a wrong key, a joke option, a giveaway, or a missing key. Some stay clean on purpose.
Across 24 course topics, from feedback skills and phishing to medication safety and food safety. Each question is written to demand one level. 120 use a misleading verb, like "Evaluate which of these is part of..." for a recall question.
96 lessons with known objectives and one planted problem each: a quiz that tests an objective too easily, questions from an unrelated topic, both, or none. Plus 7 hand-built lessons from real courses, each checked 3 times.
"Caught" is how often it found a problem we planted. "False alarms" is how often it flagged something that was fine.
| Check | Tested on | Caught | False alarms | |
|---|---|---|---|---|
| Wrong answer keyThe marked answer isn't the right one | 23 planted, 371 checks | 94% to 100% | 0 | Strong |
| Missing answer keyOne question has no key when the others do | 19 planted | 100% | 0 | Strong |
| Practice with no feedbackLearners see right or wrong, but not why | 3 planted | 100% | 0 | Strong |
| Answer giveawayThe right option stands out, like being far longer | 19 planted | 88% to 100% | 12 of 326 clean questions | Good, some noise |
| Wrong options nobody would pickA joke or obviously wrong distractor | 20 planted | 94% to 100% | 33 of 326 clean questions | Good, noisy |
| Two right answersMore than one option can be defended | 3 checks (small test so far) | 1 of 3 | 0 | Weak, see below |
| Bloom's levelThe level a question actually demands | 432 questions | 97% to 99% exact 98% to 99% within one level | n/a | Strong |
| Misleading verbsA higher-sounding verb on a simpler question | 120 questions | 100% | n/a | Strong |
| Lesson alignmentTaught, practiced, and tested at the objective's level | 103 lessons | 89 of 96 fully right 7 of 7 real lessons, every check | Rare, counted in the 7 misses | Strong |
We'd rather you hear these from us.
Pedagrade is built to stay quiet when it isn't sure. That's why it has zero false accusations. The cost: on our small test of questions with two defensible answers, it spoke up 1 time in 3. We chose that trade on purpose.
On questions with nothing planted, it suggested a stronger wrong option about 1 time in 10. Some of those are fair. These show as suggestions, not errors, and bringing the rate down is on our list.
7 of 96 test lessons had a problem it missed, mostly a quiz question that asks learners to recall what the objective asks them to compare or judge.
On three test courses with real alignment gaps, the review listed every gap but the overall score still came out between 84 and 92. Until we fix that, read the list of gaps, not just the number.
Reviewing the same course twice usually lands within about 2.5 points. The findings themselves are more stable than the score.
We'd rather miss a problem than accuse something that's fine.
Every finding points to the text it's about, so you can check it.
We publish these numbers with every release, including the ones that get worse.
If it gets your course wrong, tell us. Real mistakes become new test cases.
Found a mistake? Email support@pedagrade.com with what it said and what was right.