pranit-p/memory-reuse

An execution cache layer for AI agents that cuts LLM and tool call costs by avoiding redundant computation. Drop it into any Python agent or LangGraph workflow with a single decorator.
- **Framework-agnostic** — LangGraph, LiteLLM, or any plain Python function.
- **Zero required dependencies** — the core runs on the standard library alone.
- **Safe by default** — per-user / per-session scoping prevents cross-user cache leaks.
- **Typed** — ships with py.typed, fully type-hinted.
---
How it works
When your agent calls an LLM or a tool, memory-reuse checks the cache first.
On a hit it returns the stored result instantly — no tokens spent, no API call
made. On a miss it runs the real call and stores the result for next time:
The lookup always tries the fastest, cheapest path first. An **exact** hash
match is attempted before anything else; only on an exact miss — and only if the
semantic cache is enabled — is the query embedded and matched by similarity. So
identical repeats never pay for an embedding.
The three cache types
memory-reuse supports three kinds of lookup. Use one, or combine them.
| Type | How it works | Best for |
|---|---|---|
| **Exact cache** | Hashes the input and looks for an identical match. | Repeated identical LLM prompts. |
| **Tool cache** | Hashes a tool name + its arguments, with a TTL, and looks for an identical match. | API calls, DB queries, search — anything with expiry. |
| **Semantic cache** | Embeds the input and finds the closest previous request by cosine similarity. | Same intent phrased differently ("reworded" questions). |
> **Exact/tool are on by default; semantic is opt-in.** Enabling the
> semantic cache is what lets reworded-but-equivalent
> requests reuse a stored answer. See the
> Architecture guide
> for the full component and data-flow diagrams.
> **Node-level and graph-level caching are here.** Beyond the three cache types
> above, memory-reuse can skip whole LangGraph nodes or replay an entire agent
> run from cache. See Graph-level execution cache.
---
Install
Works with both **pip** and **uv** — pick whichever you use.
**pip**
**uv**
Optional extras
| Extra | What it adds | pip | uv |
|---|---|---|---|
| redis | Redis backend support | pip install memory-reuse[redis] | uv add memory-reuse[redis] |
| litellm | LiteLLM cached wrappers | pip install memory-reuse[litellm] | uv add memory-reuse[litellm] |
| semantic | Semantic cache with **API** embeddings (OpenAI / LiteLLM) — no torch | pip install memory-reuse[semantic] | uv add memory-reuse[semantic] |
| semantic-local | Semantic cache with **local** embeddings (sentence-transformers, pulls in torch) | pip install memory-reuse[semantic-local] | uv add memory-reuse[semantic-local] |
| strands | Strands Agents cached_tool integration | pip install memory-reuse[strands] | uv add memory-reuse[strands] |
| crewai | CrewAI cached_tool integration | pip install memory-reuse[crewai] | uv add memory-reuse[crewai] |
| agentcore | AWS AgentCore shared backend | pip install memory-reuse[agentcore] | uv add memory-reuse[agentcore] |
| prometheus | Prometheus analytics exporter | pip install memory-reuse[prometheus] | uv add memory-reuse[prometheus] |
| opentelemetry | OpenTelemetry analytics exporter | pip install memory-reuse[opentelemetry] | uv add memory-reuse[opentelemetry] |
| langchain | Faithful LangChain message (de)serialization for wrap_graph | pip install memory-reuse[langchain] | uv add memory-reuse[langchain] |
| all | Everything above | pip install memory-reuse[all] | uv add memory-reuse[all] |
> **Note:** uv is a fast Python package manager. If you don't have it yet:
> pip install uv or see docs.astral.sh/uv
---
Quick start
---
Usage patterns
1 — Basic exact cache (LLM responses)
2 — LangGraph node caching
The decorator reads user_id from the state dict automatically, or from
cache.set_context(user_id=...).
3 — LangGraph tool caching
4 — LiteLLM (works with OpenAI, Claude, Bedrock, Groq, Ollama, and 100+ more)
5 — Strands / CrewAI tool caching
Strands and CrewAI each get a cached_tool decorator that mirrors the LangGraph
one — same signature cached_tool(cache, *, scope="global", ttl=300,
semantic=False, exact_only=False), same keying, scope resolution, TTL expiry,
and exact-vs-semantic routing. They delegate to the shared caching machinery, so
behaviour is identical across frameworks.
**Strands** — requires pip install "memory-reuse[strands]":
**CrewAI** — requires pip install "memory-reuse[crewai]":