J
jessejamesrich/edd
File-based evaluation method for building non-deterministic AI systems
更新 2026-11-18
在 GitHub 查看 ↗直链 · 无中转页
- ★ 数取值
- 0 ★ · jessejamesrich/edd · 取自 GitHub 搜索 API · (2026-11-19 20:00:48 核对)
- 来源明细
- 从该仓库的 GitHub 页面直接读取 · 条目类型:开源项目
- 收录与活跃
- 收录于 2026-11-18 · 最近见到 2026-11-20 · 共见到 3 次 · 被 1 个来源列出
- 评分构成
- 综合分 103 · 1 个来源 · repo
常见问题(答案都来自我们的记录)
它适合谁 / 用来做什么?
我们把它归在这些标签下: 开源项目。标签来自条目自身,不是我们强加的分类。
谁维护它、还在更新吗?
仓库
jessejamesrich/edd;我们收录于 2026-11-18;最近见到 2026-11-20。这里没有"官方认证"或"质量评级"——我们只记录来源与时间。README 原文(英文,未翻译) · 在 GitHub 读完整 README →
# Evaluation-Driven Development (EDD)
以下摘录自该仓库的 README(片段)
A lightweight methodology for building non-deterministic systems — AI
generation, retrieval, agents, classifiers — where "quality" is a spectrum, not
a boolean. Based on the scientific method: observe, annotate, hypothesize,
experiment, measure.
EDD is context as files. Every investigation lives as a small YAML document
in your repo, versioned alongside the code. The primary reader isn't a
dashboard — it's the next agent (or human) to pick up the work. They read
the story and know exactly where things stand without digging through hundreds
of lines of chat history or scrollback.
> Pairs with ghostbus.
> ghostbus is the live wire — a tmux-backed bus for real-time coordination
> between multiple local coding agents. EDD is the durable record — the
> file-based memory that outlives any single session. Bus for now, files for
> memory. Neither depends on the other; they just compose well.
## In production
edd runs a client data project at OMTM.ai: 99 stories and about 2,000 logged
runs between September and November 2026, covering ingestion from ten source
systems, a read-only access audit, dataset anonymization and a training proof
of concept. Numbers the project reports to the client are backed by executable
checks that recompute them from raw data.
## Why
Traditional tests say pass/fail. That's the right tool for deterministic code:
a function returns the wrong bucket → fix → test → done.
But when you're tuning a generator, a retrieval pipeline, or an agent, the
output is non-deterministic and quality is a gradient. "Did it pass?" is the
wrong question. The right questions are how good, why, and what changed since
last time? A green checkmark can't hold a hypothesis, a run history, or the
reason you tried the thing you tried.
EDD gives that state a home: a mutable story that names the problem, holds
the current theory, and accumulates an append-only list of runs. It's
deliberately low-ceremony — plain YAML, plain directories, greppable, no
service to stand up.
## Concepts
### Story
A mutable working document that names the problem, holds the current theory,
and lists runs. The story is the entry point to ongoing work.
Active stories live in .edd/active/ /story.yaml
Parked stories live in .edd/parked/ /story.yaml
Resolved stories live in .edd/archived/ /story.yaml
A story has:
name — human-readable title
status — one of active, resolved, rejected, superseded, parked
problem — what's wrong, in 1–3 sentences
theory — current hypothesis about why, and what fix to try
refs — loose association to branches, commits, docs
fixture — how to reproduce / test (a command, a file reference, or a
description)
runs — append-only list of results, newest or oldest first but
consistently one or the other
Common optional fields:
priority — p0 immediate, p1 what makes the product wrong or broken,
p2 real and worth doing with nothing visibly broken, p3 lowest.
p3 is not a parking space. Say parked once, with status, or the day
comes when a story is parked by one signal and open by the other
depends_on — story that should resolve before this one starts
blocked_by — the immediate blocker. A blocked story's status stays
active or parked; blocking is a fact about the work, not a phase of it
generation, retrieval, agents, classifiers — where "quality" is a spectrum, not
a boolean. Based on the scientific method: observe, annotate, hypothesize,
experiment, measure.
EDD is context as files. Every investigation lives as a small YAML document
in your repo, versioned alongside the code. The primary reader isn't a
dashboard — it's the next agent (or human) to pick up the work. They read
the story and know exactly where things stand without digging through hundreds
of lines of chat history or scrollback.
> Pairs with ghostbus.
> ghostbus is the live wire — a tmux-backed bus for real-time coordination
> between multiple local coding agents. EDD is the durable record — the
> file-based memory that outlives any single session. Bus for now, files for
> memory. Neither depends on the other; they just compose well.
## In production
edd runs a client data project at OMTM.ai: 99 stories and about 2,000 logged
runs between September and November 2026, covering ingestion from ten source
systems, a read-only access audit, dataset anonymization and a training proof
of concept. Numbers the project reports to the client are backed by executable
checks that recompute them from raw data.
## Why
Traditional tests say pass/fail. That's the right tool for deterministic code:
a function returns the wrong bucket → fix → test → done.
But when you're tuning a generator, a retrieval pipeline, or an agent, the
output is non-deterministic and quality is a gradient. "Did it pass?" is the
wrong question. The right questions are how good, why, and what changed since
last time? A green checkmark can't hold a hypothesis, a run history, or the
reason you tried the thing you tried.
EDD gives that state a home: a mutable story that names the problem, holds
the current theory, and accumulates an append-only list of runs. It's
deliberately low-ceremony — plain YAML, plain directories, greppable, no
service to stand up.
## Concepts
### Story
A mutable working document that names the problem, holds the current theory,
and lists runs. The story is the entry point to ongoing work.
Active stories live in .edd/active/ /story.yaml
Parked stories live in .edd/parked/ /story.yaml
Resolved stories live in .edd/archived/ /story.yaml
A story has:
name — human-readable title
status — one of active, resolved, rejected, superseded, parked
problem — what's wrong, in 1–3 sentences
theory — current hypothesis about why, and what fix to try
refs — loose association to branches, commits, docs
fixture — how to reproduce / test (a command, a file reference, or a
description)
runs — append-only list of results, newest or oldest first but
consistently one or the other
Common optional fields:
priority — p0 immediate, p1 what makes the product wrong or broken,
p2 real and worth doing with nothing visibly broken, p3 lowest.
p3 is not a parking space. Say parked once, with status, or the day
comes when a story is parked by one signal and open by the other
depends_on — story that should resolve before this one starts
blocked_by — the immediate blocker. A blocked story's status stays
active or parked; blocking is a fact about the work, not a phase of it
收录徽章(可选)
想把收录标记放进 README 就复制这行,可选,不影响收录。
[](https://anclum.com/zh/s/jessejamesrich-edd/)
- 来源
- github · https://github.com/jessejamesrich/edd
- 按领域浏览
- 开源项目
- 收录于
- 今天
更多条目
所属领域与标签
每个标签都能点进去看同类的全部条目。数字是当前库里的真实条数。