LLM leaderboard

Composite formula

Scheme Agent/system 70% · Reasoning 20% · Chat 10%. AA · DeepSWE 20% each; TB · BrowseComp · HLE 15% each.

Agent Arena Agent Arena is a rank-linear percentile, 100×(N−rank)/(N−1), so rank 1 is exactly 100% and the board ends near 0%. Against the scored set that compresses z to roughly ±1.35 by construction — much tighter than free metric scales. It carries weight but does not count toward the ≥4 direct-source floor.

Coverage & missing A dash means the model has no score on that active source. The primary Index uses z = 0 for that cell; the sensitivity band shows weight renormalization and appears only when coverage is incomplete. Imputed cells do not count toward the inclusion floor.

Sources Tool-use board audit 2026-08-30: BFCL v4 (BenchLM) was ≤7d fresh with 13 rows but 0 models in the Hermes frontier seed after name cleaning — fails ≥5 frontier hits. tau-bench (BenchLM) published an empty leaderboard (display-only / outdated tasks). Neither is a primary source. SWE-Atlas-QnA weight is 0 (not Hermes task mix). Terminal-Bench is pinned to release 4.0 (BenchLM mirror of tbench.ai; best harness/effort per model). Scores are not comparable to the prior TB 2.x column; cross-version fallback into one z-pool is forbidden.

CIs Only GPQA currently shows a ±95% Wald interval (n≈198 multiple-choice items). LMSYS is Elo, not a binomial accuracy; Agent Arena is a rank transform; BrowseComp, Terminal-Bench 4.0, DeepSWE, and HLE are publisher point estimates without a stable public n for the same binomial model. Extending CIs needs per-source n or SEs.

MAD ε Scale floor agent_lmsys ε=1, browsecomp ε=1, deepswe ε=1, gpqa_diamond ε=1, hle ε=1, lmsys ε=10, terminalbench ε=1. Z clamped to ±3.0.

Picks Picks use the same top-15 board as the public leaderboard (models with ≥4 active directs, ranked by Hermes Index). The Hermes Index itself is capability-only and never uses price. Price enters only Cheap and Default. List $/1M means input+output USD per 1M tokens. Task $ means DeepSWE average $/task (real multi-step agent spend on that board). Frontier is the best Hermes Index (#1); the runner-up is the closest peer among ranks 2–5 within 2.0 Index points (higher Index then board rank on ties; no price), otherwise #2 by Index rank. Cheap is the lowest DeepSWE $/task inside that top-15 (list $/1M is only a tie-break or fallback when task $ is missing); its runner-up is the highest Index within 2× that task $ (else the next-cheapest by the same key). Default is the highest Hermes Index among models whose task $ is within 2× Cheap's task $ (list $ defines the band only when Cheap lacks task $), excluding Frontier #1 — the best daily driver still near the cheap frontier, not max Index÷$. The Default runner-up is the next-highest Index in that same band (if the band is a singleton, the next-cheapest model). Default may equal Cheap when Cheap also leads the band on Index; the rule never elevates a dominated model just to keep three category names distinct. Aggregate ≠ task proof.