N12 — the Bantu numeracy board
Can a model operate a Bantu language's number system — primitives, composition from units to a trillion, reading numbers back, agreement with nouns, measures and arithmetic? Each cell is a language's N12 index (0–100); the composite is the mean over the suite's languages and is ranked only when every language is scored.
Ranking
By family — suite mean of each family (its weight in the index)
By component — suite mean of each component
By language — hardest first: mean over the ranked models
Where models break — share right by size of number, every component that has a size
Drop = the largest fall from units-to-hundreds to any larger size.
Profiles — the diagnosis in each language (number of languages)
Key under review
When very different models fail a cell identically, the answer key is suspected before the models: a frame word, register or construction the native data does not yet cover. These cells still count — nothing is dropped silently — and each is checked against native data; any form found is added with its source and the stored answers are re-scored without asking the models again. Corrected so far this way: Kiswahili worked sums (ni sawa na), Zulu money (ayi-), dates with a cardinal day.
Anchor trend — the comparable number as the suite grows
Suites are nested: S10 ⊂ S15 ⊂ S20. The anchor is a model's composite over the S10 languages, reported on every larger suite — solid lines. The suite composite (dashed) falls as harder languages join; the anchor is what stays comparable. Each session draws fresh items, so the anchor moves by about its interval between sessions.
What the items look like
One item per component from a published sample seed — never used for a scored session. Answers are never shown: every spelling a native speaker uses is accepted, and showing one would coach the next model.