U
Ustay-coder/verdict
Write the rubric once.
更新 2026-11-19
在 GitHub 查看 ↗直链 · 无中转页
我们没有可直接复制的安装命令,也没有公开的远端端点,所以不出配置。上面那条命令仅供参考,不是我们能担保的可复制命令——编一条出来比不给更糟。 各客户端的接法 →
- ★ 数取值
- 0 ★ · Ustay-coder/verdict · 取自 GitHub 搜索 API · (2026-11-19 20:00:48 核对)
- 来源明细
- 收录自 punkpeye/awesome-mcp-servers 榜单(我们从中收集的条目之一) · 条目类型:MCP 服务
- 收录与活跃
- 收录于 2026-11-18 · 最近见到 2026-11-20 · 共见到 3 次 · 被 3 个来源列出
- 评分构成
- 综合分 300 · 3 个来源 · mcp
常见问题(答案都来自我们的记录)
它适合谁 / 用来做什么?
我们把它归在这些标签下: MCP 服务。标签来自条目自身,不是我们强加的分类。
谁维护它、还在更新吗?
仓库
Ustay-coder/verdict;我们收录于 2026-11-18;最近见到 2026-11-20。这里没有"官方认证"或"质量评级"——我们只记录来源与时间。README 原文(英文,未翻译) · 在 GitHub 读完整 README →
# Verdict — one rubric, every decision model
以下摘录自该仓库的 README(片段)
Write the rubric once. Run it on Jev, Clef, OpenAI's Decisions API, Gemini's and any other decision model, and see where they disagree on your own cases.
Decision models answer typed questions (yes or no, one of these, a level on a scale) with probabilities instead of prose, in a fraction of a second. Every vendor publishes benchmarks; none of them is about your cases, and each vendor wants its questions in its own shape. Verdict turns a rubric — criteria, scales, levels — into the questions each model takes, sends the same questions to all of them, and puts the answers side by side.
bash
npm install -g @verdict-ai/cli
verdict run -r examples/banking77/intent.yaml -d examples/banking77/cases.jsonl --models jev,clef,clef-flash,openrouter,openai,gemini
text
Transfer intent × cases.jsonl: 80 case(s), 6 model(s), the same questions to each
model answered by agreed with labels failed asked a human input tokens calls time
jev typesafe/jev-1.14.0 66/80 (83%) 0 0 44,560 80 2.1 s
clef cloudflare/clef 70/80 (88%) 0 0 31,259 80 6.9 s
clef-flash cloudflare/clef-flash 73/80 (91%) 0 0 31,259 80 4.5 s
openrouter openrouter/typesafe/jev-1.13-20260917 67/80 (84%) 0 0 44,560 80 3.9 s
openai openai/gpt-6-luna 69/80 (86%) 0 0 25,254 80 7.1 s
gemini gemini/gemini-decision-1 68/80 (85%) 0 0 25,364 80 7.1 s
the models gave one answer on 70 of 80 question(s) they all answered; they differ on 10:
A real run (2026-11-12) of eighty customer messages from a public dataset, on six models: examples/banking77 has the rubric, the cases, the whole output and how to read it. It is not a ranking: eighty cases separate the models by a handful of answers. The ten cases where they differ are the ones worth reading, and on your cases they will be different ones.
## Try it
Node 22 or newer. No key is needed for the first run:
bash
npm install -g @verdict-ai/cli
git clone https://github.com/Ustay-coder/verdict && cd verdict/examples/banking77
verdict run -r intent.yaml -d cases.jsonl --models mock # a stand-in model: shows the flow, not real answers
With a key for any one service, name its model instead (TYPESAFE_API_KEY for jev; CLOUDFLARE_ACCOUNT_ID and CLOUDFLARE_API_TOKEN for clef; OPENROUTER_API_KEY; OPENAI_API_KEY). A model that cannot answer keeps its row and says why.
Then your own cases:
bash
verdict rubric init rubric.yaml # a starter rubric to edit
verdict run -r rubric.yaml -d cases.jsonl --models jev,clef # cases.jsonl: one {"id", "inputs", "expected"} per line
verdict diff --base run_… --head run_… # any two runs, case by case
The rubric file is not edited: --models re-points its decision judges, one run per model, and every run is kept. Getting started walks through it; Decision models explains the table.
## Models and inputs
| Name it | Model | Service | Checked against the live service | Pictures and PDFs |
| jev | Jev | TypeSafe | Yes | No, text only |
| clef | Clef | Cloudflare Workers AI | Yes | Pictures, up to 4 |
| clef-flash | Clef-flash | Cloudflare Workers AI | Yes | Pictures, up to 4 |
| openrouter: | For example typesafe/jev-1.13 | OpenRouter | Yes | No |
| openai: | OpenAI's Decisions API, for example gpt-6-luna | OpenAI | Yes | Pictures |
Decision models answer typed questions (yes or no, one of these, a level on a scale) with probabilities instead of prose, in a fraction of a second. Every vendor publishes benchmarks; none of them is about your cases, and each vendor wants its questions in its own shape. Verdict turns a rubric — criteria, scales, levels — into the questions each model takes, sends the same questions to all of them, and puts the answers side by side.
bash
npm install -g @verdict-ai/cli
verdict run -r examples/banking77/intent.yaml -d examples/banking77/cases.jsonl --models jev,clef,clef-flash,openrouter,openai,gemini
text
Transfer intent × cases.jsonl: 80 case(s), 6 model(s), the same questions to each
model answered by agreed with labels failed asked a human input tokens calls time
jev typesafe/jev-1.14.0 66/80 (83%) 0 0 44,560 80 2.1 s
clef cloudflare/clef 70/80 (88%) 0 0 31,259 80 6.9 s
clef-flash cloudflare/clef-flash 73/80 (91%) 0 0 31,259 80 4.5 s
openrouter openrouter/typesafe/jev-1.13-20260917 67/80 (84%) 0 0 44,560 80 3.9 s
openai openai/gpt-6-luna 69/80 (86%) 0 0 25,254 80 7.1 s
gemini gemini/gemini-decision-1 68/80 (85%) 0 0 25,364 80 7.1 s
the models gave one answer on 70 of 80 question(s) they all answered; they differ on 10:
A real run (2026-11-12) of eighty customer messages from a public dataset, on six models: examples/banking77 has the rubric, the cases, the whole output and how to read it. It is not a ranking: eighty cases separate the models by a handful of answers. The ten cases where they differ are the ones worth reading, and on your cases they will be different ones.
## Try it
Node 22 or newer. No key is needed for the first run:
bash
npm install -g @verdict-ai/cli
git clone https://github.com/Ustay-coder/verdict && cd verdict/examples/banking77
verdict run -r intent.yaml -d cases.jsonl --models mock # a stand-in model: shows the flow, not real answers
With a key for any one service, name its model instead (TYPESAFE_API_KEY for jev; CLOUDFLARE_ACCOUNT_ID and CLOUDFLARE_API_TOKEN for clef; OPENROUTER_API_KEY; OPENAI_API_KEY). A model that cannot answer keeps its row and says why.
Then your own cases:
bash
verdict rubric init rubric.yaml # a starter rubric to edit
verdict run -r rubric.yaml -d cases.jsonl --models jev,clef # cases.jsonl: one {"id", "inputs", "expected"} per line
verdict diff --base run_… --head run_… # any two runs, case by case
The rubric file is not edited: --models re-points its decision judges, one run per model, and every run is kept. Getting started walks through it; Decision models explains the table.
## Models and inputs
| Name it | Model | Service | Checked against the live service | Pictures and PDFs |
| jev | Jev | TypeSafe | Yes | No, text only |
| clef | Clef | Cloudflare Workers AI | Yes | Pictures, up to 4 |
| clef-flash | Clef-flash | Cloudflare Workers AI | Yes | Pictures, up to 4 |
| openrouter: | For example typesafe/jev-1.13 | OpenRouter | Yes | No |
| openai: | OpenAI's Decisions API, for example gpt-6-luna | OpenAI | Yes | Pictures |
收录徽章(可选)
想把收录标记放进 README 就复制这行,可选,不影响收录。
[](https://anclum.com/zh/s/ustay-coder-verdict/)
- 来源
- awesome · https://github.com/Ustay-coder/verdict
- 按领域浏览
- MCP 服务
- 收录来源
- 3 个独立来源
- 收录于
- 今天
更多 MCP 服务器
所属领域与标签
每个标签都能点进去看同类的全部条目。数字是当前库里的真实条数。