The public record of our own errors
Every figure we have got wrong.
We ask communities and governments to publish numbers that can be checked. This is the same standard applied to us. Each entry names the wrong figure, the one that replaced it, why it happened, and what changed so it cannot happen the same way again.
Errors caught before anyone saw them are listed too. Leaving those out would make the page an advertisement rather than a record.
9
corrections
4
were public
5
caught first
8
now guarded in code
- RestatedWas public
A count of "verified" community programs, where no row has a reviewer
- What we said
- "981 verified programs", and six other magnitudes saying the same thing — 1,021 · 1,081 · 1,165 · 1,568 · 1,741 · 495 — across 53 source files. Live on the Wall of Proof, the funder pages, the state pages, four outbound message generators and the letter a constituent sends their MP.
- What is true
- 1,770 community alternatives DOCUMENTED, excluding the 379 rows a machine invented. The word "verified" is withdrawn everywhere and not replaced by another credential. /proof now carries a second tile reading 0, "checked by a named person", at the same size as the count beside it.
- Why it happened
- verification_status is a column a machine writes about its own output, and it was read as though a person had signed it. reviewed_by is NULL on all 2,149 rows, including all 14 marked human_verified and all 44 marked community_verified. The magnitudes were separately stale, each frozen at whatever the table held on the day that file was written. This is the fourth appearance of one shape: a trust flag set by the same process it is supposed to check.
- What settled it
- Live query against alma_interventions, 2026-08-30: 2,149 rows total, 1,770 excluding ai_generated, 0 rows with reviewed_by set. CLAUDE.md has recorded the prohibition since March.
- What stops it recurring
- src/__tests__/lib/alma-verified-count-claim.test.ts fails on the PREDICATE — the word "verified" beside an ALMA noun — not on any number. A regex on the literal 981 would have passed /proof, which computes the count live from the database.
- QuarantinedWas public
The fabricated articles were quarantined in the table and not in the search index
- What we said
- 68 of the 76 fabricated articles had vector chunks in civic_intelligence_chunks, which the public unauthenticated assistant ranks by similarity and hands to the model as sourced coverage. The direct read of alma_media_articles had been filtered since 2026-07-31; the retrieval path over the same text never was.
- What is true
- Both retrieval paths now re-apply the quarantine and report what they withheld. The keyword fallback — reached exactly when the embedding call fails — was unfiltered too, and is filtered now.
- Why it happened
- The quarantine is a property of the RECORD, and a COPY of the record's sentences was made somewhere the property did not travel with it. A filtered view over the articles table, which is the fix already written for the row-level leak, would not have caught this either: the chunk table holds the text, not a join.
- What settled it
- Measured 2026-08-30: 76 rows quarantined in alma_media_articles, 68 of them present as source_id in civic_intelligence_chunks.
- What stops it recurring
- withholdFabricatedChunks() in src/lib/media/articles.ts, which fails CLOSED — a failed lookup withholds every media chunk rather than passing them through.
- QuarantinedWas public
76 fabricated news articles were reachable in production
- What we said
- 76 rows in alma_media_articles with example.com URLs, attributed to The Guardian Australia, NITV News and ABC News, carrying invented statistics about Indigenous programs including "Indigenous youth diversion program shows 40% reduction in reoffending".
- What is true
- All 76 are quarantined and reach no surface. 508 genuine articles remain. Nothing was deleted: the rows stay inspectable as evidence of what happened.
- Why it happened
- The quarantine had been written weeks earlier and never deployed. It sat on a feature branch with seven other commits while production served the table unfiltered. The code was correct; the gap was between the code and the deploy, and nothing was watching that gap.
- What settled it
- Row-level read of alma_media_articles before wiring it to a surface, 2026-07-31.
- What stops it recurring
- A database view (alma_media_articles_publishable) plus a grep guard test that fails the build if any file queries the base table without the quarantine. A quarantine you have to remember is a habit, not a quarantine.
- RestatedCaught internally
A 55-to-1 detention comparison that used the wrong denominator
- What we said
- "$1.1bn a year on detention against $20m a year for justice reinvestment", a 55-to-1 ratio, drafted as a national frame.
- What is true
- Detention gets $2.20 for every $1 spent on community-based supervision. $1,141,155,000 against $519,560,000, national, 2024-25.
- Why it happened
- Three errors at once: different years (2024-25 actual against a 2026-27 commitment), different bases (audited expenditure against a budget line), and above all the wrong denominator. Australia does not spend $20m on community alternatives; it spends $519.6m. The comparison swapped "all community-led youth justice" for "one Commonwealth program" and understated the real figure twenty-six fold.
- What settled it
- Report on Government Services 2026, table 17A.10, financial year 2024-25, national, government real recurrent expenditure. Same table, same basis, same year, which is the only reason the two may be divided.
- What stops it recurring
- Recorded in the project memory as a figure-canon entry, with the note that the two service lines do not sum to the total and must never be shown as a two-part pie.
- Note correctedCaught internally
Our own note said most intervention coordinates were fabricated
- What we said
- "Most alma_interventions coordinates are FABRICATED", carried in CLAUDE.md and used to plan a re-geocoding project of ~900 rows.
- What is true
- 733 of 929 coordinates sat on unique points and look like real geocodes. The damage was concentrated in 12 shared points, not general. And 1,151 of 1,741 programs should carry no pin at all, because they name a state or the nation rather than a place.
- Why it happened
- The note was written from a real problem and generalised beyond it. A note that overstates the rot steers work as badly as one that hides it: it made an 84%-sound layer feel unusable, and framed the fix as geocoding when most rows are correctly pinless.
- What settled it
- Point-distribution analysis of alma_interventions, 2026-07-31.
- What stops it recurring
- src/lib/geo/geography-scope.ts classifies what precision a geography can honestly support, so "should this have a pin" is answered by code rather than by a note.
- RestatedWas public
52 map pins were in the wrong place, including every Palm Island program
- What we said
- Every Palm Island Community Company program was pinned in Brisbane CBD, roughly 1,000km from Palm Island, while its own geography read "QLD | Palm Island | Very Remote". Bawinanga's Maningrida services were 838km out. Ukrainian, Cambodian and Laotian programs were pinned in Queensland.
- What is true
- 52 coordinates cleared. A further 142 are marked as locality fallbacks and no longer drawn as precise pins. The old values are kept, with the reason each was rejected, so any verdict of ours can be reversed.
- Why it happened
- An enrichment pass assigned fallback coordinates in bulk. They were never obviously wrong on screen because a dot in Brisbane looks like a dot anywhere else.
- What settled it
- src/lib/geo/coordinate-verdict.ts, which asks only whether a coordinate is contradicted by its own geography, and accuses rather than guesses.
- What stops it recurring
- coordinate_precision on alma_interventions records what each pin is worth. Nothing has earned "geocoded" yet, because earning it means checking against a source.
- WithdrawnCaught internally
The claim that 57% of Queensland early intervention money goes to detention
- What we said
- "57% of early intervention is detention", from a $560M denominator.
- What is true
- Withdrawn, not replaced. The denominator mixed multi-year capital with annual program funding, so no ratio could be drawn from it.
- Why it happened
- justice_funding.amount_dollars holds the ANNUAL line, while public announcements quote the MULTI-YEAR total. Adding one year to four looks like arithmetic rather than a category error, which is why it survived review.
- What settled it
- Queensland service delivery statements against the March 2026 briefing table.
- What stops it recurring
- compareOnSameBasis() in src/lib/funding/measure-period.ts parses the period out of project_description and REFUSES a comparison across bases. Guard test: src/__tests__/lib/measure-period-guard.test.ts.
- RestatedCaught internally
The replacement ratio was wrong in the same way as the claim it replaced
- What we said
- "$25 of detention capital for every $1 of community grants", comparing $318M of multi-year capital against $12.5M of annual grants.
- What is true
- About six to one. Four years against four years: $318M of detention capital against $50M of Kickstarter grants. Annualised it is $87.4M against $12.5M, about seven to one, and the two landing in the same place is the test that the figure is sound.
- Why it happened
- The same annual-versus-multi-year mixing, made while correcting the first instance of it. Knowing about an error is not the same as having a guard against it.
- What settled it
- Question on Notice 640 of 2026 (Mr S King to Minister Gerber, 3 June 2026): "Through the $50 million Kickstarter program...". The row's own project_description had said so all along.
- What stops it recurring
- Same measure-period guard. The lesson recorded alongside it: read project_description before trusting amount_dollars.
- RestatedCaught internally
Kickstarter was recorded at $115M
- What we said
- ~$115M, in the March 2026 briefing table.
- What is true
- $50M over four years, which is $12.5M a year. The stored annual line was correct; the briefing figure was not.
- Why it happened
- Three figures were in play and nobody reconciled them against the source description.
- What settled it
- Question on Notice 640 of 2026, Minister Gerber, 3 June 2026.
Found something else that is wrong?
Records about a community belong to that community, and a figure you can disprove is worth more to us than one nobody checks.