A deterministic gate that catches hallucinated numbers before they reach an audience. Built for briefs where an LLM drafts financial or research content and a specific, stated requirement is: don't let it make numbers up. What follows is real output from a real program.
⚠ Demo only — the research data below is fictionalEvery numeric claim in a draft is checked against a source-of-truth fact bank after generation — a mechanism, not a prompting technique.
Unsourced: no fact matches the claim's topic at all. Contradicts source: a fact matches the topic, but the number is wrong — the more dangerous, more plausible-sounding kind of hallucination.
Fact-check gate — data/draft-good.md PASS 1/ Sector revenue rose 12% in Q2 — one of the strongest prints this year. ↳ matches F1: "Q2 sector revenue growth" = 12% PASS 2/ Valuations aren't stretched: forward P/E sits at 18.4, in line with the 5-year average. ↳ matches F2: "Forward P/E ratio for the sector" = 18.4 PASS 3/ The central bank held its benchmark rate at 5.25%, as expected. ↳ matches F3: "Central bank benchmark rate held at" = 5.25% PASS 4/ Unemployment held steady at 4.1%, signalling a resilient labour market. ↳ matches F4: "Unemployment rate reported at" = 4.1% PASS 5/ The 10-year treasury yield closed at 4.32%, up slightly on the week. ↳ matches F5: "10-year treasury yield closed at" = 4.32% 5 passed, 0 blocked, out of 5 numeric claim(s). ✅ All claims cleared. Written to output/approved/…
Same draft, two lines changed: a P/E figure inflated from 18.4 to a fabricated 34.7, and an invented "insiders expect a rate cut" prediction with a number that appears nowhere in the source.
Fact-check gate — data/draft-bad.md PASS 1/ Sector revenue rose 12% in Q2 — one of the strongest prints this year. ↳ matches F1: "Q2 sector revenue growth" = 12% BLOCK 2/ Valuations are stretched: forward P/E has spiked to 34.7, the highest since 2000. ↳ topic matches F2 but no source value matches the claimed number(s) [34.7, 2000] F2 actually says: "Forward P/E ratio for the sector" = 18.4 PASS 3/ The central bank held its benchmark rate at 5.25%, as expected. ↳ matches F3: "Central bank benchmark rate held at" = 5.25% PASS 4/ Unemployment held steady at 4.1%, signalling a resilient labour market. ↳ matches F4: "Unemployment rate reported at" = 4.1% BLOCK 5/ Insiders now expect a surprise rate cut to 3.75% within weeks. ↳ topic matches F3, F4 but no source value matches the claimed number(s) [3.75%] F3 actually says: "Central bank benchmark rate held at" = 5.25% F4 actually says: "Unemployment rate reported at" = 4.1% 3 passed, 2 blocked, out of 5 numeric claim(s). ⛔ Blocked — will not publish as-is. Report written to output/blocked/…
An unsourced claim is easy to catch — it's simply not on-topic. A claim that contradicts the source is harder and more dangerous: it's on-topic, plausible, and wrong. Both fabricated lines above are this second kind — a real metric (P/E, a rate) with an invented value. That's the shape hallucinated financial content actually takes, and it's what this gate is built to catch.