Enterprise AI can’t do your arithmetic — LG Electronics measured it. Here’s the fix.
Published: 3 August 2026 · Updated: 3 August 2026
There is a number from a 2025 research paper that belongs on the wall of every team connecting AI to a company database: 4%. That is the accuracy a modern, enterprise-optimized language model reached on arithmetic-reasoning questions built from LG Electronics’ own internal sales data — the same model that scored 93% on simple aggregation. The demo works; the production deployment does not; and LG measured the exact distance between the two. This article walks through what they found, what it looks like on the ground in our region, and the structural fix. Sources are linked throughout.
What LG actually tested — and why it is your test too
The paper — Fact-Consistency Evaluation of Text-to-SQL Generation for Business Intelligence (arXiv:2505.00060) — is not a toy benchmark. The researchers built 219 business questions in five complexity levels, straight from a real BigQuery environment, each with a gold-standard SQL query and a verified correct answer. Then they graded not whether the SQL merely ran, but whether the answer was factually right.
The failures were not random. They clustered exactly where a CFO cares: misapplied arithmetic logic (4% accuracy on the hardest tier), incomplete filtering, incorrect grouping (31%). The model did not crash. It produced confident, well-formatted, wrong numbers. The authors’ conclusion is blunt: enterprise use needs fact-consistency validation layers — a way to know the answer is right before it reaches the person who acts on it.
The same movie, on the ground in our region
A payroll platform in the Gulf connected an AI assistant to its HR database and asked the simplest question in the book: how many employees do we have? The AI said 100 — confident, instant, well-phrased. The real answer was 80. Twenty of those rows were people who had been terminated, flagged as such in a status column, but nothing in the database told the AI that “terminated” means “does not count.” That rule lived in application code and in one developer’s head. The SQL was syntactically perfect. The answer was wrong by 25% — and nobody would have known until a headcount number walked into a board meeting.
Same story, different company: an employee asked an AI for her working-days balance. The number did not exist in the database at all — it was computed in the frontend and never stored. The AI, helpful as ever, produced a number anyway. Fluent, friendly, fabricated. These are not edge cases; this is what “connect the model to the database” produces on real company data, on the first Tuesday.
The fix is structural, and it has two parts
First, business rules must become explicit, machine-readable facts. “Headcount excludes terminated and deleted statuses.” “Working days = hire date to today, minus weekends and holidays.” Once those are written down as certified definitions — reviewed and signed off by the business owner, not guessed by the model — the 100-vs-80 class of error does not get smaller; it disappears. “How many employees?” stops being a generation problem and becomes a routing problem: run the query that was already proven, and show the filter it applied.
Second, every answer needs a receipt. Not “trust me” — a line that says what was asked, what data was touched, what definition was used, and what left the environment (nothing). When the answer carries its own evidence, a wrong rule gets spotted by the person who knows the truth in seconds, instead of surfacing in a quarterly review. That is what the LG researchers call a fact-consistency validation layer, translated from academic English to operational English: certified answers, with receipts, or an honest “I don’t know.”
Why this matters more in our region than anywhere else
The Gulf is not a market where confident wrong numbers are a rounding error. Saudi Arabia’s PDPL — fully enforceable since September 2024, with 48 SDAIA violation decisions in 2025 — carries fines up to SAR 5 million. DIFC’s Regulation 10, enacted September 2023 and amended in July 2025, already applies data-protection obligations to autonomous and semi-autonomous systems, and is being revised again through the June 2026 public consultation. Qatar’s central bank can audit AI systems in finance directly.
And the money at stake is generational: PwC estimates AI will contribute $320 billion to the Middle East economy by 2030, with Saudi Arabia alone at $135.2 billion, and the GCC AI market is projected to grow from $6.2 billion to $23 billion by 2034. Every riyal of that flows through questions exactly like “how many employees?” — thousands of times a day, across thousands of companies. The companies that win that decade will not be the ones with the flashiest demo. They will be the ones whose AI can prove its answers.
The doctrine is converging
We are far from the only ones who see it. Earlier this year, IBM published a governance-by-construction architecture for generalist agents (arXiv:2605.20874) — policy enforcement at planning time, human approval for high-risk actions, output control — which lands on remarkably similar doctrine. A companion paper on pre-action governance reasoning (arXiv:2604.25684) reports 95% compliance accuracy with zero false escalations. When the biggest enterprise labs alive converge on the same homework, the question stops being “is this real?” and becomes “who executes it best for my data, my language, and my regulator?”
Frequently asked questions
Is the 4% figure about one model or about AI in general?
It is one well-run study of one enterprise-optimized model on one real dataset — but the pattern (strong on simple lookups, collapsing on arithmetic and grouped reasoning) is consistent with what enterprises report in production. Read it as a warning about improvisation, not about one vendor.
Why not just use a better model?
Because the failure is structural, not intellectual. “Terminated doesn’t count” is not in any model’s weights — it is a rule about YOUR business that lives in application code and people’s heads. Until rules like it are written down explicitly, a smarter model just produces more confident wrong answers.
What is a certified answer, practically?
A business question the company has answered once, correctly, on purpose: the metric, its filters, its definition — declared as data and signed off by the business owner. When someone asks “how many employees?”, the system runs that proven query and shows the filter it used, instead of improvising a new one each time.
Where does DEBO fit?
DEBO is the governed layer that runs exactly this pattern: certified definitions with visible filters, computed statistics with their caveats, an audit line per answer, and an honest decline when the certified ground runs out. The model never sees a row of your data; it never improvises a money answer.
See a certified answer on your own data
A 30-minute demo on your use case: watch a money question answered from a certified definition, filter shown, audit line attached — no improvisation, no rows leaving.
Book a demo