None of these
What does it do when none of the listed options is right?
No decision API we found has a built-in way to abstain, so we add a "None of these" option the way a caller would.
Without that option, we check whether the model's confidence drops when nothing fits.
Picks "None of these" when nothing fits the right option was removed
Jev 1.14
TypeSafe
Jev 1.13
TypeSafe
Clef
Cloudflare
Clef-Flash
Cloudflare
pplx-decider v1.2
Perplexity
pplx-decider-flash v1.2
Perplexity
OpenAI Decisions
OpenAI
Microsoft-Decision-1
Microsoft
Kev-4B
Jared Palmer
Share of decisions with the right option removed and a "None of these" option added where the model picks "None of these".
Picks "None of these" wrongly a listed option was right
Jev 1.14
TypeSafe
Jev 1.13
TypeSafe
Clef
Cloudflare
Clef-Flash
Cloudflare
pplx-decider v1.2
Perplexity
pplx-decider-flash v1.2
Perplexity
OpenAI Decisions
OpenAI
Microsoft-Decision-1
Microsoft
Kev-4B
Jared Palmer
Share of decisions with every option present where the model still picks "None of these".
Test items and precision
- Items
- 400 decisions with the right option removed and 400 controls per test set
- Precision
- About ±5 points on picking "None of these", about ±3 on picking it wrongly.