iyerbrah/QuoteRunner

An AI broker agent: one sentence of chat becomes a verified commercial insurance application
(ACORD 125-style), which is submitted to three carrier portals and comes back as quotes compared side
by side. It asks only for what's missing and never guesses.
Live demo · How it works ·
Eval results · 60-second demo video (link coming)
## Results
Measured on 50 simulated small-business owners talking to the same code the site runs
(every conversation is public). The two latest full runs use
identical code, so the ranges show run-to-run variation, not cherry-picking.
| | Runs 3 and 4 |
| Field accuracy | 99.3–99.8% |
| Invented values (a field filled that the owner never stated) | 0–1 in 50 conversations |
| Conversations finished | 90–98% |
| Questions asked | 6.8–7.5, instead of a 25-field form |
| Cost per conversation | $0.0026–0.0029 (gpt-5-nano), plus $0.0008 for the simulated customer |
The finished applications were then sent to the three carrier portals
(quotes runs 2 and 3, identical results):
| | Quotes runs 2 and 3 |
| Values received by carriers that were correct | 100% of 1,285 |
| Invented values that reached a carrier | 0 across 252 portal submissions |
| Carrier decisions matching their rules applied to the true facts | 100% |
| Businesses quoted | 45 of 45, after one follow-up for facts the owner didn't know (3 before it) |
| Speed and cost | portal mapping learned once for $0.004, then 6 ms per business |
In insurance, a made-up answer can void a policy, so invented values is the number that matters
most: every persona has 1–2 facts they genuinely don't know, and any guess shows up in the score.
## How it works
One customer message, end to end:
mermaid
flowchart TD
B["Browser chat + live application"] -->|POST /api/chat| X["LLM call 1: extract forced tool call"]
X --> V["Code: Pydantic validation and the no-guess rules"]
V --> N["Code: next_question() priority order, industry follow-ups"]
N -->|a question is left| W["LLM call 2: word the question"]
N -->|nothing left| S["Code: summary for the customer to approve"]
V --> R["Code: coverage and risk-flag rules"]
W --> B
S --> B
X & W -.-> L[("llm.py: cost per call, session and daily caps, SQLite call log")]
Then, once the customer approves the application, the submission agent takes it to every carrier at once:
mermaid
flowchart TD
A["Verified application"] --> G["GET the carrier's quote form (HTTP)"]
G --> F["Code: read the form's fields, labels, options and hints"]
F --> M["LLM, 3 samples in parallel: map form fields to application fields (names only, never values)"]
M --> K["Code: keep only what most samples agree on, check transforms and options, cache it"]
K --> P["Code: fill every value from the application, units from the form's own words"]
P -->|a required fact is unknown| Q["Needs info: ask the customer, resubmit"]
P --> S["POST the form, read the result"]
S --> C["Quote / decline / not offered, cheapest first"]
The carriers (Granite Mutual, Harborline Specialty, Lakeside Commercial) are fictional portals built
into the app, because real carriers only give portal access to licensed, appointed brokers. They
differ the way real ones do: different field names, revenue in dollars on one and in thousands on