Article URL: https://github.com/sqlsure/sqlsure Comments URL: https://news.ycombinator.com/item?id=48875342 Points: 10 # Comments: 0

A query can be perfectly valid, run without error, and return a number that's silently wrong — revenue double-counted by a join, an average summed, a patient identifier exposed. Databases don't catch this. Linters don't catch this. LLMs reviewing their own SQL don't catch this. Proof, not promises: we ran sqlsure over the gold answers of the two benchmarks every text-to-SQL model is graded on. 2,568 expert-written queries, 45 flags, zero false alarms — including a BIRD dev gold answer that is provably wrong by 8× from the exact bug class sqlsure targets, and a schema defect now filed upstream. sqlsure judges SQL against facts your team already declared — dbt unique tests become grain, relationships tests become join cardinality, one-line meta tags mark what's safe to sum. No new language to learn, no model to maintain by hand. Rules are dictionary lookups, not LLM calls: same input, same verdict, every time, offline. Every rejection carries a machine-actionable fix, so AI agents self-repair: draft → check → fix → check → execute. In our benchmark, applying the fix verbatim produced a passing query 10/10 times. 3. Library — embed check() inside any text-to-SQL product or agent framework. A drop-in SemanticGate wraps Vanna/WrenAI-style generators; a semantic eval metric scores NL2SQL output where execution-accuracy is blind. When sqlsure can't verify something, it says "can't verify" — never "looks fine." Honest uncertainty is a feature. dbt (works today): manifest.json or schema.yml — the tests teams already wrote become enforceable semantics, zero config The live database itself (works today): no semantic layer at all? sqlsure.introspect builds the rulebook from the catalog — SQLite PRAGMAs or information_schema PK/FK (postgres/mysql). Introspecting BIRD's own database files recovered 2 foreign keys missing from the benchmark's published schema (bird-bench/mini_dev#37)