Files
agent-runtime-framework/benchmarks/artificial-analysis/2026-10-13.csv
Paul O'Reilly 418d634afa benchmarks: add Artificial Analysis snapshot and placeholder M3.1-Flash estimate
- artificial-analysis/2026-10-13.csv: Anthropic, Qwen, MiniMax-M3 rows with
  cost per task and derived cost per intelligence point
- minimax-m3-1-flash-unverified.md: operator-supplied KingBench figures, unverified
- estimates.csv: TODO placeholder for M3.1 Flash (0.9x M3 cost, index scaled by
  KingBench ratio) -- replace with measured/published values
2026-10-13 14:33:53 +13:00

36 lines
2.1 KiB
CSV

model,creator,context,intelligence_index,cost_per_task_usd,cost_per_intelligence_point_usd,median_tokens_per_s
Claude Opus 5.5 (max with fallback),Anthropic,1M,58,5.98,0.1031,96
Claude Sonnet 5.5 (max with fallback),Anthropic,1M,56,5.46,0.0975,135
Claude Opus 5.5 (xhigh with fallback),Anthropic,1M,56,3.46,0.0618,81
Claude Opus 5.5 (high with fallback),Anthropic,1M,54,1.82,0.0337,82
Claude Fable 5.1 (max with fallback),Anthropic,1M,53,7.63,0.1440,67
Claude Fable 5.1 (xhigh with fallback),Anthropic,1M,53,5.98,0.1128,62
Claude Sonnet 5.5 (xhigh with fallback),Anthropic,1M,52,2.01,0.0387,103
Claude Opus 5.5 (medium with fallback),Anthropic,1M,51,1.34,0.0263,79
Claude Fable 5.1 (high with fallback),Anthropic,1M,51,3.91,0.0767,52
Claude Fable 5.1 (medium with fallback),Anthropic,1M,49,2.98,0.0608,54
Claude Fable 5.1 (low with fallback),Anthropic,1M,47,2.37,0.0504,56
Claude Sonnet 5.5 (high with fallback),Anthropic,1M,47,0.88,0.0187,96
Claude Haiku 5.5 (max),Anthropic,1M,43,0.21,0.0049,236
Claude Opus 5.5 (low with fallback),Anthropic,1M,42,0.55,0.0131,80
Claude Haiku 5.5 (xhigh),Anthropic,1M,41,0.12,0.0029,187
Claude Sonnet 5.5 (medium with fallback),Anthropic,1M,41,0.48,0.0117,95
Claude Haiku 5.5 (high),Anthropic,1M,38,0.08,0.0021,175
Claude Sonnet 5.5 (low with fallback),Anthropic,1M,36,0.35,0.0097,100
Claude Haiku 5.5 (medium),Anthropic,1M,34,0.05,0.0015,171
Claude Haiku 5.5 (low),Anthropic,1M,29,0.02,0.0007,164
MiniMax-M3,MiniMax,1M,29,0.51,0.0176,107
Qwen3.8 Max (0902),Alibaba,984k,45,5.41,0.1202,36
Qwen3.8 2.4T A95B,Alibaba,262k,40,2.16,0.0540,37
Qwen3.8-Flash-Next,Alibaba,256k,40,0.37,0.0092,51
Qwen3.8 27B (xhigh),Alibaba,256k,34,1.01,0.0297,46
Qwen3.8 27B (medium),Alibaba,256k,28,1.13,0.0404,50
Qwen3.8 27B (low),Alibaba,256k,26,1.05,0.0404,53
Qwen3.7 Plus,Alibaba,1M,25,0.22,0.0088,55
Qwen3.8 27B (non-reasoning),Alibaba,256k,20,2.49,0.1245,52
Qwen3.5 397B A17B,Alibaba,262k,18,0.47,0.0261,88
Qwen3.6 35B A3B,Alibaba,262k,18,0.48,0.0267,129
Qwen3.5 122B A10B,Alibaba,262k,16,0.32,0.0200,129
Qwen3.5 9B,Alibaba,262k,11,0.21,0.0191,58
Qwen3 Coder Next,Alibaba,256k,9,0.78,0.0867,73