decidingbench

9 decision models, every provider

Each model is tested on every provider that serves it, because price, speed and limits differ from host to host.

Every model on every provider

ModelInputWeightsProviderList priceStatus
Jev 1.14TypeSafeText and imagesClosedTypeSafe APIserverless$0.042 per million input tokens, no output chargeNot run yet
OpenRouterserverless$0.042 per million input tokens (to confirm)Not tested yet: needs an account
Jev 1.13TypeSafeText onlyClosedTypeSafe APIserverless$0.042 per million input tokens, no output chargeNot run yet
OpenRouterserverless$0.042 per million input tokens (to confirm)Not tested yet: needs an account
ClefCloudflareText and imagesOpen, Apache 2.0Cloudflare Workers AIserverless$0.24 per million input tokens, no output chargeNot run yet
Clef-FlashCloudflareText and imagesOpen, Apache 2.0Cloudflare Workers AIserverless$0.038 per million input tokens, no output chargeNot run yet
pplx-decider v1.2PerplexityText and imagesClosedPerplexity APIserverless$0.02 per million input tokens, no output chargeNot tested yet: needs an account
pplx-decider-flash v1.2PerplexityText and imagesClosedPerplexity APIserverless$0.01 per million input tokens, no output chargeNot tested yet: needs an account
OpenAI DecisionsOpenAIText and imagesClosedOpenAI APIserverless$0.10 per million input tokens, no output chargeNot tested yet: needs an account
Microsoft-Decision-1MicrosoftText onlyClosedMicrosoft Foundry (Azure)serverlessnot published on Microsoft's pageNot tested yet: needs an account
OpenRouterserverless$0.042 per million input tokens (to confirm)Not tested yet: needs an account
Kev-4BJared PalmerText onlyOpen, Apache 2.0OpenRouterserverless$0.042 per million input tokens, no output chargeNot tested yet: needs an account
Our Mac Mini (run it yourself)self-runno per-decision priceOptional, after hosted results

Not tested yet

  • Kev-9B: no hosted API; self-host only.
  • Von and Laya: no hosted API; small enough for the later run-it-yourself track.
  • H2O-Lightning-4B: no hosted API; candidate for the run-it-yourself track.
  • CLM-8B: no hosted API with a published model ID or price.
  • Chon-1.0-12B, basal-omni-1.0-ultra, Quyet-1.0-Large: no hosted API; would need a paid GPU host.
  • OpenJev: no hosted API; non-commercial licence to check.

As of 20 Oct 2026, decidingbench has not published results. The planned lineup is Jev 1.14, Jev 1.13, Clef, Clef-Flash, pplx-decider v1.2, pplx-decider-flash v1.2, OpenAI Decisions, Microsoft-Decision-1 and Kev-4B. Published figures for these models so far come from vendors (Cloudflare's leaderboard) or from boards that use written or academic items (JevBench, the Artificial Analysis Decision Index). As of this date we found no independent test against real recorded outcomes.