This methodology document exists so any reader can walk every claim on /intelligence/civic back to a row in our database, a Hansard speech, a published diary entry, or an oversight body’s own recommendation. If a claim cannot be traced, it should not be on the page.
What we measure
Three chapters.
Access. Who delivers youth justice programs on the ground, and who gets meetings and contracts from the ministers who set youth justice policy. The question is not "how many meetings" but "which kinds of organisations get the meetings, and which kinds get the work."
Promises. What governments have publicly committed to in charters, election platforms, and parliamentary statements, and what state of completion those commitments are in. The question is not "did they say it" but "did the words travel to action."
Oversight. What independent reviewers have recommended, and the disposition of those recommendations: accepted, deferred, rejected, silent. The question is not "what does the evidence say" but "what happens to the evidence after the report lands on the desk."
The Tier 1 universe
Australia has roughly six hundred organisations doing some kind of youth-justice-adjacent work. Some are running on-Country diversion. Some are publishing white papers. Some are charging consulting fees. Tier 1 separates one from the others.
Tier 1: Primary frontline youth justice service. An Aboriginal community-controlled organisation, community-based not-for-profit, or small social-enterprise hybrid whose primary delivered service is one of: pre-court diversion, bail support, on-Country mentoring, post-release reintegration, family-led conferencing, legal first-response for under-eighteens. The organisation must employ frontline workers, hold relationships with young people, and operate in or adjacent to a community affected by youth criminalisation.
Tier 2: Adjacent service. Peak bodies, statewide legal services, advocacy organisations, research centres that work alongside Tier 1 but do not themselves carry the daily case load.
Tier 3: System actors. Consultancies, government departments, large generalist NFPs without a primary YJ stream, funders.
A v1 ship visualises Tier 1 only. Tier 2 and Tier 3 are documented and present in the underlying data but do not anchor the public ratio. The Tier 1 list is hand-confirmed; the Tier 2 and Tier 3 lists are agent-proposed only.
How we built the Tier 1 list
For v1, Tier 1 covers the Northern Territory and Queensland.
- Universe. All organisations linked by
alma_interventions.operating_organization_idwhereverification_status != 'ai_generated', filtered tostate = 'NT'orstate = 'QLD'. Three hundred and thirty-eight candidates on 2026-05-15. - Agent proposal. An LLM reviewed each candidate against the Tier 1 definition above, using the organisation’s name, ABN, registry record, indigenous-org flag, ALMA intervention categories, and any free-text description on file. Output: proposed tier, proposed sector category, confidence score, evidence snippet.
- Human sweep. Every candidate was reviewed and confirmed or overridden in a single admin session by the JusticeHub team. Confirmed Tier 1 entries are visible on the page; unconfirmed entries do not contribute to claims.
The access ratio: what it is and what it is not
The headline ratio compares money not meetings. Specifically:
access_ratio_qld = sum(YJ-relevant consultancy spend in QLD) / sum(YJ-relevant grants to Tier 1 frontline orgs in QLD)A ratio of N reads as: "for every dollar of QLD government youth justice funding that reached a confirmed primary frontline organisation, the government spent N dollars on consulting and advisory firms doing youth-justice-related work."
Why funding and not meetings
We originally designed this page around a meeting-frequency ratio (consultancy meetings vs frontline meetings in the ministerial diary register). The empirical work on 2026-05-15 showed this is not supportable. The QLD diary register contains 1,728 entries; only two are with consulting firms (both Ernst & Young, neither with the youth justice minister). The 160 entries that match a youth-justice keyword filter contain zero consultancy meetings.
The likely explanation: consultancies engaged on youth justice work are procured through departmental procurement channels rather than through ministerial meetings. Their advice reaches the minister via the Director-General, which appears in the diary register as a departmental meeting. The diary register is therefore an honest record of who-met-the-minister but a poor proxy for who-shapes-the-policy.
Procurement spend is the honest proxy. It is also harder to spin away. Every dollar lives in a published contract.
How we built the funding ratio
- Universe. All
justice_fundingrows wherestate = 'QLD'AND the recipient name matches one of: consulting, advisory, Deloitte, KPMG, PwC, Ernst & Young, McKinsey, Boston Consulting, Accenture, or other named Big-4 / Big-3 firm. 433 rows on 2026-05-15. - YJ-relevance pass. An LLM reviewed each row's recipient + program name + project description and classified it as
direct_yj_service,yj_research_or_review,yj_advisory_consultancy,yj_infrastructure_or_capital,broader_justice_includes_yj, ornot_yj_related. - Human confirmation. Every classified row was reviewed in the admin sweep workflow. Only rows with
confirmed_at IS NOT NULLcount toward the ratio. - Numerator. Sum of
amount_dollarsfor confirmed YJ-relevant consultancy rows in QLD. - Denominator. Sum of
amount_dollarsfor alljustice_fundingrows wherealma_organization_idis a confirmed Tier 1 frontline org in QLD.
What the meeting data still tells us
The diary data is not discarded. The Access chapter carries a secondary claim: the youth justice minister meets internal departmental staff approximately twenty times for every one meeting with a Tier 1 frontline organisation. This is supportable on the data we have. It complements but does not replace the funding ratio.
The promises tracker
Promises come from three sources.
civic_charter_commitments: public charter pledges and election commitments.civic_hansard: ministerial statements made in Parliament naming a specific deliverable, filtered for YJ relevance.funding_outcome_commitments: funding commitments with named outcomes.
Each promise carries a status: made, in_progress, delivered, quietly_dropped, or unknown. Status is set by human review against publicly available evidence: budget papers, departmental reports, freedom-of-information disclosures. Any promise with unknown status is flagged on the page as "no public evidence of action."
The oversight ledger
Oversight recommendations come from oversight_recommendations. The page surfaces:
- Recommendations from named bodies: Queensland Sentencing Advisory Council, NT Children’s Commissioner, the Royal Commission into the Protection and Detention of Children in the Northern Territory.
- The government’s published disposition:
accepted,accepted_in_principle,deferred,rejected,no_response. - Time elapsed since the recommendation was published.
The page does not editorialise. The dispositions are the government’s own words.
Sources by record
Every snapshot statistic on the page has a Cite button. Clicking it copies a citation in the form:
"{stat}. Source: JusticeHub Civic Intelligence, computed {date}. Methodology: /intelligence/civic/methodology#{claim_id}"
The underlying claim row carries:
- The SQL or computational process that produced the value.
- The list of source record IDs used in the computation.
- The list of source document URLs (Hansard URLs, oversight report PDFs, charter pages).
- The timestamp the claim was last computed.
Anyone can request the underlying record set: civic@justicehub.com.au.
What we retired and why
Two claims were retired between the design brief and ship day. Both retirements are documented here so the methodological evolution is visible, not hidden.
Retired claim 1: "678 organisations deliver programs, ministers met 192 organisations, overlap = 10." This conflated the full uncurated YJ org universe (Tier 1 plus Tier 2 plus Tier 3) with the meeting register. Many of the "192 organisations met" are government departments or consulting firms, not service deliverers. The overlap number was therefore misleading. Replaced by the tiered ratio framing.
Retired claim 2: meeting-frequency ratio between consultancies and frontline orgs. The original design intended a ratio of consultancy meetings to frontline meetings in the ministerial diary register. Empirical work on 2026-05-15 showed the diary register contains only two consultancy meetings across 1,728 entries (both Ernst & Young, neither with the youth justice minister). The framing was not supportable on diary data. Replaced by the funding ratio (procurement spend). The dept-to-frontline meeting asymmetry survives as a secondary claim.
Both retirements were the same kind of decision: the data did not match the design intent, so the design moved to match the data. The page reports what the data can defend.
What v1 does not claim
This page does not claim to be a complete map of Australian youth justice. v1 covers NT and QLD only. NSW, Victoria, WA, SA, Tasmania, and the ACT are marked as "coming v2". The data exists in our ingest pipeline but has not been through the curation and tagging process that Tier 1 demands.
This page also does not claim Tier 1 status for any organisation that does not appear on the published list. An organisation can be brilliant frontline-relevant work and not be on the v1 list, either because the curation has not reached it yet, or because the available registry data was insufficient to confirm primary-frontline status.
Hostile-reader preempt
The strongest attacks on a page like this:
- "Your Tier 1 list is arbitrary." It is hand-curated against a written definition by named reviewers. The list is public; the definition is on this page. Disagreement is welcome and goes to civic@justicehub.com.au with proposed additions or removals.
- "Consulting spend is different in kind. It's procurement, not capture." True. But the funding ratio is not a claim of capture. It is a claim about where dollars flow when the policy is being shaped. Readers are free to draw their own inferences.
- "The overlap number you retired was the whole point. What changed?" The overlap was built on an uncurated org universe. The funding ratio is tighter and more defensible. Both retirements are documented above so the methodological shift is visible.
- "The data is incomplete." Yes. v1 ships NT and QLD only. We name the gaps because pretending they don't exist is what gets pages like this discredited.
Versioning
This methodology is versioned. The v1 document is frozen on ship day; subsequent revisions land at /intelligence/civic/methodology?v=N. Every snapshot claim carries the version it was computed under.
Last updated 2026-08-26. Computed claims under v1.