ayushagrawal288/memex

A production-grade persistent memory service for AI agents. Agents forget everything between sessions by default — memex fixes that. It stores, retrieves, and ranks conversation memory using semantic search with recency decay, so agents surface what's relevant *and* recent, not just what's semantically closest.
---
Architecture
**Write path:** content → fastembed ONNX inference (local, ~12 ms CPU, BAAI/bge-small-en-v1.5) → INSERT with 384-dim vector → return memory ID.
**Read path:** query → embed → pgvector cosine search (top_k × 3 candidates) → re-rank with recency decay in Python → return top_k results with scores.
---
Design decisions
1. Recency decay on top of semantic search
Pure vector similarity returns the most semantically similar memories, not the most useful ones. A fact from 90 days ago that's a 0.95 similarity match is often less useful than a 0.80 match from yesterday.
Score formula:
Where λ = ln(2) / half_life_days (default: 30 days, so a 30-day-old memory has 50% recency weight).
α is configurable per request (default 0.7). Task-focused agents use higher α (semantic dominates). Conversational agents use lower α (recency matters more).
2. Fetch 3× candidates, re-rank in Python
The pgvector query returns top_k × 3 candidates sorted by pure similarity. Python re-ranks with the decay formula and slices to top_k. This prevents recency decay from starving high-similarity older memories — they're still in the candidate pool.
At 10× scale (>1M memories per agent): push the scoring into a Postgres function using pg_proc to eliminate the Python re-ranking round-trip.
3. asyncpg + explicit pool sizing over SQLAlchemy async
SQLAlchemy adds ORM overhead on every query. The hot retrieval path — embed, query, re-rank — needs to be tight. asyncpg gives direct control over pool min/max (same instinct as tuning HikariCP in Java). pgvector queries require raw SQL for the operator anyway.
Pool defaults: min=5, max=20. Right-size for a single-instance deployment. Override via DB_MAX_POOL_SIZE env var.
4. Rate limiting in Postgres, not Redis
Sliding window counter via upsert. One fewer dependency. Correct under concurrent requests (transactional upsert). At 10× scale with distributed deployments: replace with Redis INCR + EXPIRE — atomic operations, no lock contention.
5. ivfflat index, not HNSW
ivfflat has lower build cost and lower memory footprint — the right tradeoff at small-to-medium scale ( Run on MacBook M-series, Docker Desktop (4 CPUs), 4 uvicorn workers, 16 threads/worker.
> Embeddings: local ONNX (BAAI/bge-small-en-v1.5) — zero external API calls, zero cost.
**Why the ceiling is ~120 RPS:**
Every write and every search requires one ONNX inference (~10–15 ms on CPU). With 4 Docker CPUs: 4 cores / 12 ms ≈ 333 embeddings/s theoretical max. After Python overhead, DB queries, and asyncio scheduling: ~120 RPS actual.
**Path to higher throughput:**
| Approach | Expected gain | Complexity |
|---|---|---|
| Embedding cache (Redis, key = SHA256 of text) | 2–3× (40–60% hit rate on repeated agent queries) | Low |
| Horizontal scaling (N replicas behind a load balancer) | N× linear | Medium |
| GPU inference (swap ONNX runtime → CUDA) | 10–50× | Medium |
| Voyage-3 API (offload to Anthropic's inference fleet) | Scales to thousands of RPS, limited by API quota | Low code change |
---
Project structure
---
Observability
docker compose up starts Prometheus and Grafana alongside the API:
| Service | URL | Credentials |
|---|---|---|
| REST API docs | http://localhost:8000/docs | — |
| MCP server | http://localhost:8001/mcp/ | — |