skillberry-ai/cap-evolve

watch capability evolve
**cap-evolve improves an AI agent's prompts, tools, and skills by learning from failed
evaluation traces.**
You bring the agent and the eval you already have. cap-evolve runs the loop — evaluate →
diagnose the failures → propose an edit → keep it only if it beats a held-out split by a
significant margin → commit — and reports one honest number. It optimizes what your agent
*reads*, not its weights.
tag: GitHub's README sanitizer only honors for
its own attachment hosts, so an external src renders as an empty gap. The thumbnail
opens GitHub's own player. The site embeds the real — Pages serves it as
video/mp4, which raw.githubusercontent does not. -->
▶︎ Watch
the 85-second demo · plays inline on the site
Quickstart ·
Documentation ·
Examples ·
Results ·
Contributing
Why cap-evolve
- **Optimize more than prompts.** System prompts, executable **tool code**, MCP tool
surfaces, and whole **skill packages** — pick one or several and optimize them jointly.
- **Learn from real agent failures.** Every iteration reads full trajectories and per-task
causal feedback (which task ids a prior edit *broke* and *fixed*), so edits are large and
don't regress the wins.
- **Keep evaluation honest.** Acceptance is a val-only significance gate (Δ > k·SE); the
test split is sealed and scored exactly once. Both live in the core, not in editable docs.
- **Inspect every change.** Each candidate is a git commit; the dashboard shows costs,
timing, diffs, lineage, and a tasks × iterations pass/fail heatmap.
Try it in two minutes — no API key required
toy_calc is a deterministic stand-in agent that only answers correctly when its system
prompt contains a [CALC] marker. The mock optimizer adds it, so the score provably
rises — **no model is called**.
Expected — the seed prompt scores 0.0 on val; the optimized prompt is gate-accepted and
scores 1.0 on the sealed test split:
Open the printed dashboard.html in any browser. Full walkthrough:
Getting started.
The CLI
Start with no arguments — cap-evolve prints a branded home screen with the golden path and
every command grouped by what it's for.
doctor is the one to run before spending anything. Every failing row names the fix:
See what actually changed
Every candidate is a snapshot, so you can read the edit that moved the number — unified below
120 columns, side-by-side above:
Watch a run, live
The live view. The masthead answers is this the run I meant to launch? — resolved
spec, algorithm and mode, split sizes, gate bar. Then the cumulative-best stair, the lineage
with the paired-gate reason behind every accept ( ✓ ), reject ( ✗ ) and
indecisive ( ~ ) step, a per-task heatmap that marks
not evaluated distinctly from failed , and spend split into
runner / optimizer / intake.
The dashboard
Every run gets the same tabs whatever algorithm produced it — Overview, Candidates, Gate,
Tasks, Cost, Logs, Diffs, Trajectories, Memory, Files — and an algorithm that has extra signal
gets an extra tab rather than a different dashboard. GEPA's minibatch-vs-full-val gates and
Pareto selection, SkillOpt's epochs and edit-budget schedule, and
agent-optimize's free-form rounds are all read from events the engine already emitted.
Logs: every line of events.jsonl , phase-tagged and filterable — including the
optimizer's own stderr and each budget warning. Model- and subprocess-authored text is
sanitized and rendered as text nodes only, so a log line can never drive the page.
The dashboard and the terminal are the same projection (events.jsonl → reduce_run), so they
cannot disagree about what happened. run also writes a self-contained dashboard.html that
needs no server.
Choose your path
| Path | Use it when | Start |
|---|---|---|
| **Claude Code plugin** | You use Claude Code and want slash commands + honesty hooks | claude --plugin-dir ./plugins/cap-evolve then follow RUN.md |