roedyrustam/sinapsai

English •
Bahasa Indonesia
SinapsAI is a high-performance Local AI Control Plane & Unified Gateway built for Node.js, Edge Runtimes, and TypeScript. It bridges your applications with multiple LLM providers (OpenAI, Anthropic Claude, Google Gemini, DeepSeek, Mistral AI, Ollama, Groq, OpenRouter, xAI Grok, Cohere v2, and custom endpoints) through a single, standardized Unified API.
Engineered with a 100% in-process / in-memory architecture, SinapsAI eliminates downtime via automatic failover, load balancing, lowest-latency routing, and a resilient Circuit Breaker—without requiring intermediary proxy servers, external Redis dependencies, or adding measurable latency ( 🤖 LLM / AI Agent Quick Ingest:
> If you are an AI assistant (ChatGPT, Claude, Gemini, Perplexity), autonomous agent, or coding assistant (Cursor, Copilot, Windsurf):
> - Compact Roadmap: Read llms.txt for a high-density, concise summary of all features and standard imports.
> - Complete Context Specification: Read llms-full.txt for the exhaustive single-file reference document containing all TypeScript interfaces, provider configs, error codes, and 35 zero-slop examples.
> - In-Process Guarantee: sinapsai is 100% in-process with 0 runtime dependencies. It is NOT an external proxy server (no Docker/Python/Redis required).
## 🥊 Why SinapsAI?
Most AI gateways (LiteLLM, Portkey, Helicone) require deploying separate Docker containers, managing external Redis/PostgreSQL instances, or routing sensitive user prompts and API keys through external third-party cloud servers.
SinapsAI runs directly inside your Node.js application process:
| Feature | SinapsAI ⚡ | Direct Provider SDKs | Cloud Gateways (Portkey/Helicone) | LiteLLM Proxy |
| Zero Infra Setup (No Docker/Redis) | ✅ 100% In-Process | ✅ Yes | ❌ Requires Cloud Account | ❌ Requires Docker/Python |
| Added Latency | ⚡ 📁 Looking for ready-to-run code? Check the Examples Directory (/examples)!
## 📑 Table of Contents
Why SinapsAI?
Key Features
Architecture & Request Flow
Installation
Quick Start
1. Client Initialization with Automatic Failover
2. Real-Time Streaming (SSE)
3. Round-Robin Load Balancing
4. Lowest-Cost Routing (Dual-Token Pricing)
5. Observability with Event Hooks & Rate Limiting
6. Real-Time In-Process Telemetry (getMetrics)
7. Retries with Exponential Backoff & Jitter
8. Exact & Semantic In-Process Response Caching
9. Tool Calling & Function Execution
10. Structured Outputs (JSON Mode)
11. Unified Text Embeddings
12. OpenRouter Free Tier & Zero-Cost Routing
13. Groq LPU Ultra-Fast Inference & Reasoning Integration
14. Hierarchical Model-Level Fallbacks
[15. Universal Multi-Modal Vision & Audio (ContentPart[])](#15-universal-multi-modal-vision--audio-contentpart)
16. Concurrent In-Process Batch Processing (client.batch)
17. Vercel AI SDK Official Adapter (sinapsai/ai-sdk)
18. Distributed Storage Adapters (Cloudflare KV, Redis, D1)
19. Zero-Dependency In-Process PII Guardrails (PIIGuardrails)
20. Adaptive & Smart-Hybrid Dynamic Routing
21. Reversible PII Tokenization & Response Vault
22. Streaming Tool Call Accumulator & Reconstruction
23. Resilient Structured JSON Output Extraction
24. Virtual Model Aliases & Dynamic Mapping