Scores on a scale
When it rates something on a scale, how close does it land?
Off by one level is a different mistake from off by four. We report exact hits, within-one hits and the average distance in levels.
Only one score track in this version.
Accuracy, score exact level on a five-level scale
Jev 1.14
TypeSafe
Jev 1.13
TypeSafe
Clef
Cloudflare
Clef-Flash
Cloudflare
pplx-decider v1.2
Perplexity
pplx-decider-flash v1.2
Perplexity
OpenAI Decisions
OpenAI
Microsoft-Decision-1
Microsoft
Kev-4B
Jared Palmer
Share of article quality ratings where the model's top level matches the WikiProject assessment exactly.
Score within one level five-level article quality scale
Jev 1.14
TypeSafe
Jev 1.13
TypeSafe
Clef
Cloudflare
Clef-Flash
Cloudflare
pplx-decider v1.2
Perplexity
pplx-decider-flash v1.2
Perplexity
OpenAI Decisions
OpenAI
Microsoft-Decision-1
Microsoft
Kev-4B
Jared Palmer
Share of article quality ratings within one level of the WikiProject assessment.
Score error in levels mean absolute error, in levels
Jev 1.14
TypeSafe
Jev 1.13
TypeSafe
Clef
Cloudflare
Clef-Flash
Cloudflare
pplx-decider v1.2
Perplexity
pplx-decider-flash v1.2
Perplexity
OpenAI Decisions
OpenAI
Microsoft-Decision-1
Microsoft
Kev-4B
Jared Palmer
Mean absolute distance, in levels, between the model's top level and the assessed level.
Test items and precision
- Items
- 500 article quality ratings per test set on five levels
- Precision
- About ±3 points within one level; about ±5 exact.