Session Log -- 2026-10-13

Summary

Moved the framework's MiniMax content to MiniMax-M3.1-Flash-Preview (new context, Flash models/registry, bare minimax/minimax-m2 slugs removed; d7decc1), then added a benchmarks/ directory with an Artificial Analysis snapshot, unverified M3.1 Flash figures, and a flagged placeholder estimate (418d634). Both pushed to main.

Decisions

Decision: new context harnesses/contexts/minimax-m31-flash/v1 (env pin MiniMax-M3.1-Flash-Preview, no init.sh) replaces minimax-m3-max/v1 in scaffolding-repo, integration-direct, test-writing-repo, best-practices-minimax-repo; the old context stays for rollback -- Rationale: Flash ignores MAX_THINKING_TOKENS; effort comes from --effort via default_flags.reasoning_effort.
Decision: deleted model-registry/minimax.yaml (endpoint minimax never existed) and repointed the six task-templates/minimax-*.yaml and the three *-review-loop@1 model: minimax-m2 nodes to minimax-m31-flash-max -- Rationale: MiniMax silently maps unknown model names to M3.
Decision: benchmark data lives in benchmarks/ beside the registry, not inside registry entries -- Rationale: RegistryEntry (pydantic default extra=ignore) silently drops unknown fields, and the loader never reads this dir.
Decision: cost factor recorded is "cost per intelligence point" = AA cost per task / AA Intelligence Index (derived; the site has no such column) -- Rationale: operator request.
Decision: M3.1 Flash placeholder = 0.9x M3 cost per task ($0.46) and M3's index scaled by the KingBench-3 ratio (29 x 2.12 = 61.5), flagged ESTIMATE-TODO -- Rationale: operator stop-gap; 61.5 exceeds Opus 5.5 max (58), so implausible. Planner FUTURE item 629829d3 (agent-runtimes) tracks replacing it.

Gotchas Discovered

[minimax] Symptom: unknown model names (minimax, minimax-m31-flash-max) return HTTP 200 with model: MiniMax-M3 -- Fix: send the provider id; translation now done by agent-runtimes ef822703 (deployed 0.0.5.1817). Check the echoed model.
[artificial-analysis] Symptom: WebFetch of artificialanalysis.ai/models returns only headline numbers (tables are JS-rendered) -- Fix: open the page in Chrome (claude-in-chrome), wait ~4 s, read document.querySelector('table') rows via javascript_tool (269 rows on /leaderboards/models; columns: model, context, creator, index, cost per task, tokens/s, latency).
[artificial-analysis] MiniMax-M3.1-Flash-Preview is not on the leaderboard (no row; /models/minimax-m3-1-* 404 as of 2026-10-13); only MiniMax-M3 (index 29, $0.51/task) exists for MiniMax.
[benchmarks] Cost vs intelligence correlates strongly only within one model family across effort levels (Claude: r 0.93-0.99, cost x4-6 per +10 index points); across vendors it is weak (r 0.53) and Qwen3.8 27B is inverted (cost does not fall with lower effort), so cross-vendor cost estimates for M3.1 Flash are unreliable.
[git] The framework checkout was on the user's feature branch feat/hatchmate-per-project-fork-key; moving my uncommitted work to main needed git stash push -- <paths> (untracked new files stay in place), checkout main, ff-pull, stash pop.

Open Questions

Which Qwen3.8 size do airouter-qwen38* models actually call? (AA has 27B, 2.4T A95B, Max, Flash-Next.)
Registry model ids claude-sonnet-5 / claude-opus-5 / claude-haiku-4-5 predate the 5.5 releases AA lists -- verify before mapping AA rows to them.
Source for the KingBench 3 figures; scoring rule to map AA index / cost per intelligence point onto the registry's 0-10 dimensions.

Key Context

Files: benchmarks/artificial-analysis/{2026-10-13.csv,README.md} (34 rows: Anthropic, Qwen, MiniMax-M3), benchmarks/minimax-m3-1-flash-unverified.md, benchmarks/estimates.csv.
Live: CRS checkout for this repo is at d7decc1 on the controlplane (verified); 418d634 adds only benchmarks (nothing reads it).
Registry scores/costs for minimax-m31-flash-{max,medium} are still placeholders (copies of the M3 entries; costs just under them).

Process Notes

Before rebasing onto a feature-branch checkout, check git branch --show-current; land work on main explicitly.
Record derived metrics with their derivation (README) and mark stop-gap numbers with a TODO in the data file name/status column AND a durable planner item.

4.3 KiB Raw Blame History