hidai25/eval-view

Record what your agent does today. Get told when it silently changes.
---
Your agent returns 200 and looks fine. But a model update, a provider change, or a one-line prompt edit just made it skip a clarification, call the wrong tool, or quietly drop output quality. Your tests still pass. Your users notice before you do.
**EvalView snapshots your agent's behavior — the tools it calls, in what order, with what output — and tells you the moment that behavior changes.** Like Jest snapshots, but for tool-calling, multi-turn agents.
↑ 30-second live demo — no API key needed
Quick Start
> **OpenAI adapter migration:** OpenAI shut down the Assistants API on **August 26, 2026**.
> The latest published EvalView release, **0.8.1**, still uses that API; the Responses API
> migration is currently **unreleased source**. If you use openai-assistants, follow the
> migration guide before running your tests. An assistant_id
> alone cannot preserve your agent's configuration. Other adapters are unaffected.
That's the whole loop. check returns one of:
It diffs the **whole trajectory** — tool names, parameters, and order — not just the final string. The deterministic tool + sequence diff runs offline, with no API key. Add an LLM judge only when you want output-quality scoring.
Executing your agent can still incur backend API charges: --no-judge skips the
judge, not those calls. Embedding-based semantic comparison is opt-in.
No agent yet? See it work in 30 seconds:
Why snapshot testing (and not assertions)?
Most eval tools ask you to *write down what "good" looks like* — assertions, metrics, rubrics. That's a lot of upfront work, and you can only catch the failures you thought to assert.
EvalView inverts it: **it records what your agent actually does now, and flags any drift from that.** You catch regressions you never anticipated, with zero assertions written. When the new behavior is correct, evalview snapshot accepts it as the new baseline — same as updating a snapshot in Jest.
| | EvalView | Assertion-based eval tools |
|---|---|---|
| Setup | Record current behavior | Write assertions/metrics first |
| Catches | Any drift from baseline | Only what you asserted |
| Non-determinism | Multi-variant baselines (up to 5 valid paths) | You handle it |
| Unit of comparison | Full tool-call trajectory | Usually final output |
This makes EvalView a **merge-time regression gate**, which is a different job from observability (Langfuse, LangSmith) or metric scoring (promptfoo, DeepEval, Braintrust). Many teams run one of those for visibility **and** EvalView as the gate. Honest comparisons →
EvalView tests itself in public, every day
Every day at 09:00 UTC, on pull requests, and on pushes to main,
Core Dogfood exercises the non-live test suite,
type checks, local mock-agent snapshot / check, evalview demo, end-to-end
flows, and an evalview monitor smoke test. It uses no paid API credentials and
makes no paid inference calls. GitHub runner usage is separate.
Live Provider Checks test the real evaluator and
chat assistant only when a maintainer explicitly opts into paid API use on main.