sifter-ai/sifter

**Your documents are a dark database.**
Open-source document intelligence engine — schema-driven extraction, NL query, MCP server, Python and TypeScript SDKs. Self-hostable under MIT.
---
Why not RAG?
RAG is built for retrieval — *find me chunks similar to this query*. It breaks on homogeneous collections like invoices, contracts, or receipts where every document looks alike and the question is an aggregation, not a search.
Sifter's approach: extract structured fields once (*client, date, total*), store them as typed records, query with real filters and aggregations. The answer is exact and reproducible — because it's a database query, not a similarity search.
---
Quickstart
Open http://localhost:3000 — create a sift, upload documents, query results.
---
Python SDK
TypeScript SDK
---
MCP server (Claude Desktop / Cursor / AI agents)
Then ask:
> *"What's the total unpaid across all invoices from last quarter?"*
> *"Show me all contracts expiring in the next 90 days."*
> *"Which candidates have Python and more than 5 years experience?"*
Sifter answers with structured data — exact counts, sums, filtered rows. Not a text blob.
Want a remote MCP URL without running a local server? → Sifter Cloud
---
Dashboard
Sifter includes a built-in dashboard — no Metabase, no Grafana, no SQL required.
Describe what you want to see in plain language:
Produces KPI tiles, breakdowns, and time-series — updated automatically on every extraction.
---
What's included
- **Schema-driven extraction** — describe what to extract in natural language; schema is inferred automatically and exported as Pydantic / TypeScript types
- **NL query** — ask questions in plain language; Sifter generates inspectable MongoDB aggregation pipelines
- **MCP server** — stdio transport, read + write tools, zero custom integration code
- **REST API + SDKs** — full OpenAPI spec, typed clients for Python and TypeScript
- **Webhooks** — HMAC-signed HTTP callbacks on every extraction event
- **Spec-driven dashboards** — short NL spec → auto-generated board (KPI, breakdown, table, time series)
- **CLI** — sifter extract, sifter records, sifter sifts for terminal workflows and CI
- **Self-hostable** — Docker Compose, bring your own MongoDB and LLM API key