We red-teamed our own AI layer 63 times. Here’s the scoreboard.
Published: 3 August 2026 · Updated: 3 August 2026
Every AI vendor’s security page says the same words: enterprise-grade, SOC2-ready, zero-trust. They all mean “trust us.” We decided to do something less comfortable: attack our own layer, over and over, in the languages and industries our customers actually operate in — and publish the scoreboard. What broke, what held, and what it taught us. You will not find our internals here (that is the point of a boundary). You will find the evidence of how hard we tried to break it.
The setup: 63 attacks, no gentleness
We built four waves of adversarial cases against DEBO’s own guard chain. Invented bilingual companies (HR, banking, health, logistics, manufacturing). Attacks in Arabic and English. Five database shapes (PostgreSQL, MySQL, SQL Server, and the non-SQL question). The goal was not a demo — it was to find the holes before anyone else does. The fake model in the harness was the attacker: it tried to smuggle names out, widen tenant scope, launder PII through aliases, and turn rendering bugs into data leaks.
The categories: tenant-isolation evasion (can one company’s user read another’s rows?), egress (can a value reach the model?), disclosure (can a caller see a field their role forbids?), question scrubbing (can a name typed in a question leak?), and structural evasion (can SQL be smuggled past the guards?). Sixty-three cases in total, each logged pass or bypass.
What broke — and what we did about it
Things broke. That is the point of a red team, and we are publishing these because a vendor that only shows wins is marketing, not engineering. Among the real findings: a 16-digit card number was invisible to the pattern layer (one digit above its ceiling — the single most regulated number in fintech, sailing through). A JSON escaping artifact let anchored patterns miss values adjacent to escaped newlines. Column aliases could hide a PII value from the tagged-value matcher. Arabic and Turkish typing variants defeated exact-name matching. Locale-specific phone formats (9-digit Gulf mobiles, spaced Turkish numbers) passed under the digit floor.
Every one of those became a fix with a regression test attached. The full suite today stands at 800+ tests, with each attack that ever succeeded now pinned as a case that must stay fixed. When we say the boundary is tested, we mean specifically: these 63 attacks, plus the waves before them, and every fix proven by a test that runs on every change.
What held
With a certified catalog on the governed path: zero breaches across all 63 cases. Tenant isolation held against OR-widening and CTE laundering. Zero values reached the model on the governed path — verified in both Arabic and English turns. Non-SQL traffic (a MongoDB-style pipeline) failed closed: rejected as unverifiable, never waved through. Role enforcement held at the field level: HR sees names, a branch manager sees their branch, everyone else sees counts.
The honest summary sentence we use internally: everything that broke, broke at the two places we already knew were weakest — the legacy fallback path and the name-shaped edges of pattern matching — and both are now closed or guarded with an explicit, documented control.
Why we publish this
Because the AI industry has a trust-words problem, and the only cure is receipts. If you are a CISO evaluating any AI vendor — including us — ask for the red-team log. Not the compliance PDF, not the badge: the cases, the outcomes, and the fixes. A vendor that has one will show you. A vendor that doesn’t will say “enterprise-grade” again.
This is also why independent benchmarks matter more than vendor claims — the same culture that made us publish the LG Electronics text-to-SQL study and the agentic-AI threat model in our own articles. The category only grows up if the receipts are public.
Frequently asked questions
Will you share the full attack cases?
The scoreboards and findings summaries, yes — that is this article. The full case files and internals, no: a boundary’s construction is exactly what a red team exists to protect. What we can always show in a technical session is the test suite running and the scoreboard reproducing.
Did anything remain unfixed?
Everything that produced a live bypass is fixed and test-pinned. A small set of documented residuals remains — for example, locale-specific identifier formats that need per-tenant pattern packs, which exist precisely because no global regex can be both safe and sane everywhere. Each residual has a named control and a named owner.
Is 63 cases enough?
It is a strong start, not a finish. Red-teaming is a habit, not an event: new waves are added for new sectors and new database shapes, and the CI suite runs every case on every change. The number that matters is zero — live bypasses on the governed path.
Bring your own attack
A 30-minute technical session with your security team: the scoreboard, the fix register, and the suite running live — then we take your hardest question and run it through the boundary.
Book a demo