Article URL: https://github.com/yashmahajan10/llm-differential-privacy-gateway Comments URL: https://news.ycombinator.com/item?id=49113543 Points: 8 # Comments: 0

Give an AI agent query access to sensitive data, with a mathematical guarantee that no individual's record can leak — even if the agent is wrong, manipulated, or adversarial. A recorded Claude Desktop session (replies trimmed; the chart cards are the session's own). An AI agent breaks 20 patients down by diagnosis, and the ±12 noise swamps every bin. Reminded that it cannot turn the noise off, it drains a three-answer budget until the gate returns a refusal instead of a quieter answer. On the 32,561-row census, a too-narrow slice is rejected at the trust boundary, while a full education breakdown comes back clean at scale. The refusal and the rejection are the live gateway's real enforcement, reproduced by python scripts/render_demo_gif.py. Try it from Use it from Claude Desktop. Five minutes to evaluate: watch the GIF, then skim the attack gallery. Deep review: DESIGN.md. You ask questions about a sensitive dataset in plain English. An LLM compiles each question into a small, constrained query; a differential-privacy engine executes it under a tracked privacy budget and returns a deliberately noisy answer with a stated confidence interval. The name fits. Like its audio namesake, the gateway keeps every signal below a set threshold under the noise floor: any one individual's contribution is drowned out, while population-scale signals pass through nearly untouched. No individual record can be reconstructed from the answers, no matter what is asked or how cleverly it is phrased. The interesting part isn't that an LLM can write queries. It's that the privacy guarantee does not depend on the LLM being trustworthy. The model is a convenience that proposes a query; it enforces nothing. Every privacy property is enforced downstream, by components that would behave the same way if a human typed the query by hand. This is the trust-boundary discipline you'd apply to any untrusted input in a production system, applied here to an AI agent. Anyone can claim privacy. This repository ships the exploits that would break the claim, runs them against its own engine, and pins the outcomes in CI. The fastest way to understand what the gateway guarantees is to watch it defeat three classic attacks that break naive "query a database" systems. A differencing attack isolates one person by asking two aggregate questions that differ by exactly that person. Both queries are "just aggregates." Neither names a single row. Yet together they expose an individual. The gallery (attacks/differencing.py) shows this attack succeeding with privacy disabled: the target's private value is recovered exactly. (The salary sketch above is illustrative; on the real UCI Adult data, "Alice" is the unique holder of her group's maximum capital gain.) It then shows the same attack defeated once DP is on: the calibrated noise on each answer makes the subtraction useless, and the budget accountant charges for the information released across both queries rather than treating them as independent.