yagobski/agentleak

Audit every execution channel, score it deterministically, gate it in CI.
Quickstart  · 
Documentation  · 
Benchmark  · 
Scoring  · 
Website
Your agent's final answer can look perfectly clean while sensitive data leaks
through its tool calls, shared memory, inter-agent messages, and logs, the
channels that output-only audits never inspect. AgentLeak tests for exactly
that, locally, before you ship.
text
Risk Index: 0.440 / 1.0 High risk (privacy 56/100)
Final output: clean ✓
Shared memory: L4 health identifier + diagnosis leaked ✗
Inter-agent: L4 diagnosis leaked ✗
Logs: L2 email leaked ✗
Key insight: the final answer appears safe, but sensitive data leaked through
internal channels. (The SIN, medication and address the agent received stayed
contained, which is why RI is 0.440 and not 1.0.)
A trace goes in, a privacy report comes out. Leakage is scored with
AgentRisk, a severity-weighted, density-normalized Risk
Index grounded in GDPR Article 9 and Québec Law 25. No cloud, no LLM dependency,
no data ever leaves your machine.
## What is in this repository
AgentLeak-authored software and scenarios are released under MIT. Bundled
third-party-derived research fixtures retain their upstream licenses and
attribution: PrivacyLens-derived fixtures are CC-BY-4.0 and AgentDojo-derived
fixtures are MIT. The wheel includes the applicable license texts and NOTICE.
| | Where |
| SDK, CLI, engine, AgentLeak-authored scenarios, Action, local UI | this repository, MIT |
| PrivacyLens-derived fixtures | bundled research fixtures, CC-BY-4.0 |
| AgentDojo-derived fixtures | bundled research fixtures, MIT |
| Hosted workspace | agentleak.org/app |
| Published benchmark | agentleak.org/benchmark |
## Install
bash
pip install agentleak # core
pip install "agentleak[gui]" # + local web UI
pip install "agentleak[mcp]" # + MCP tools for coding agents
pip install "agentleak[presidio]" # + Presidio
pip install "agentleak[full]" # everything above
From source:
bash
git clone https://github.com/yagobski/agentleak.git
cd agentleak
pip install -e ".[dev]"
## Platform (web UI)
bash
pip install 'agentleak[gui]'
agentleak serve # opens http://127.0.0.1:8000/app/
A full local platform built with React, Tailwind and shadcn/ui (black
theme), fully self-contained (no CDN, self-hosted fonts), with a left-sidebar
navigation:
Projects — each is an agent under test. Run a real agent against any
scenario, or use the built-in scripted agent offline. Connect your own agent
via the SDK (the Connect tab generates a copy-paste snippet).
Red Team — 62 executable plugin IDs (24 native, plus 38 transposed for
Promptfoo compatibility so an existing taxonomy keeps working —
not a scoreboard)
privacy/security transpositions, mapped to 46
observable attack classes across 6 families, combined with 10 delivery
strategies (direct, jailbreak framing, markup, encodings, Unicode, and
multi-turn Crescendo). Run a zero-cost scripted baseline or attack the real
configured agent; every probe persists with ASR / ELR / CLR evidence.
Runs — every analysis is stored locally (SQLite); view, compare
(weight-robust dominance), export (JSON / MD / HTML), delete.