richer-richard/cochlea

**A headless audio engine for agents.** Write a score as data, render it
offline to deterministic PCM, then *listen through numbers* — loudness,
onsets, pitch, key, spectrograms — and assert what you heard. Compose →
render → probe → verify, with no human ear (and no audio device) in the
loop.
*What the agent sees: the mel spectrogram of examples/scores/first_light.ron
— the score used in the example below — after render and probe. No PCM in
sight.*
Or entirely from the command line, score as RON:
cochlea probe works on **any** WAV, plus FLAC (decoded bit-exact), mp3,
and ogg — still without ffmpeg, and with no score required. That's the
front door: point it at audio you didn't render, and you get the same JSON
report and spectrogram an agent uses to review its own work.
How an agent listens
compose → render → probe (JSON) → spectrogram (one vision call) → verify
1. **compose** a score as data (RON, or the Rust builder above).
2. **render** it to deterministic PCM — cochlea render score.ron --out mix.wav.
3. **probe** the mix into a compact JSON report (loudness, onsets, pitch,
key, silence, clipping) — cochlea probe mix.wav --json report.json.
No image, no audio: the agent reads numbers.
4. **look**, when numbers aren't enough — cochlea spectro mix.wav --out
spec.png renders one small PNG the agent reviews in a single vision
call instead of reasoning about raw samples.
5. **verify** — cochlea render score.ron --verify runs the score's
embedded assertions and exits nonzero on failure. An agent can retry on
its own, without a human confirming "yes, that sounds right."
When something in the middle of a long file needs a closer listen, every
read tool takes --from/--to. Probe just bars 17–19, or draw a
spectrogram of just the drop. The cut is frame-exact, report times are
relative to it, and source.start_ms records where it came from. That
turns the whole stack into a zoom lens instead of a whole-file-only
report.
The economics are the point here, not an afterthought. The first_light
render above is 7 seconds of 48 kHz 32-bit-float PCM, which is 2.7 MB. A
3-minute piece at the same settings runs about 66 MB. You would not hand
that to an agent as text, and reading it sample by sample is worse.
Its probe report is a few KB of JSON instead. Here is schema v5, trimmed
to the interesting fields. Note pitch.melody: the piece's bass line and
melody read back as *note events*, which is the read-back half of the
compose loop.
And the spectrogram is one small image. Here's the title_cue demo — a
pad whose cutoff_hz automation sweeps 250 Hz → 5000 Hz across bars 1–3:
*The dark band at the top of the frame narrows as the sweep runs, letting
more high-frequency energy through over time. An agent reads that
straight off the image. The demo's Monotone(track: "pad", param:
"cutoff_hz", ...) assertion checks the same thing numerically.*
To get a whole piece in one image no matter how long it is, --sheet
tiles the spectrogram into a contact sheet instead of one long strip.
Two bars per tile here, via --bars-per-tile 2:
Reading audio without a context window
probe --digest skips JSON and prints a deterministic text summary
instead: one line per feature dimension, then a windowed timeline capped
at about 40 rows. Here's real output for the drum_groove demo — 20.8
seconds and four tracks, with the rhythm, stereo, and structure
dimensions all in one screenful:
Tempo and rhythm are reported as *separate axes*, because they change
independently. A drum solo can hold a rock-steady pulse while its pattern
turns unrecognizable, and that difference is exactly what an agent needs
to see.
Here the tempo reads 110.3 BPM, matching the authored 110, and
stability 1.00 says the speed never moves across the piece. The alts
list surfaces the genuine half-tempo reading at 54.9 BPM — which is
actually the stronger raw peak, with the octave prior breaking the tie
toward the beat. Metrical ambiguity like that is data an agent can weigh,
rather than a coin flip hidden inside the detector.
The rhythm line then says how the *hits* relate to that pulse: 98% of
onsets sit on the beat-subdivision grid and 56% land on off-beat
subdivisions, so clear=true. That's an eighth-note hat groove, with its
syncopation reported as a number.