Cognipeer/console

Self-hosted, open-source AI gateway and control plane: OpenAI-compatible LLM routing, RAG, an MCP hub, guardrails and agent tracing for multi-tenant teams.
Docs · Quick start · Discussions · Changelog · Contributing · Contact
Adoption: in active use at 15 companies that Cognipeer supports as enterprise customers. Open-source users are not included in this count.
Community edition is available under AGPL-3.0. Commercial licensing, hosted deployments, and support agreements are available separately through Cognipeer.
## Features
Gateway
LLM Gateway — OpenAI-compatible chat completions, embeddings, batch, and moderations across OpenAI, Anthropic, AWS Bedrock, Google Vertex AI, vLLM, Ollama, and more
Model Hub & Dynamic Routing — Fallback chains and per-model quota/rate limiting
Realtime (Enterprise) — WebSocket voice agents with barge-in and tool-call events
Agents & Automation
MCP Hub — Curate, host, and discover Model Context Protocol servers (OpenAPI, remote, stdio) with per-tool controls
Sandbox (Enterprise) — Isolated runtime environments for code, agents and previews with snapshot/fork/resume and port-forwarded previews
Browser Automation — Managed browser profiles, live sessions, and per-browser MCP endpoints
A2A Protocol — Publish Console-managed agents as Agent-to-Agent servers with agent cards
Prompt Management — Versioned templates with environment-based deployment (dev/staging/prod); the Prompt Optimizer is Enterprise
Knowledge & Retrieval
RAG / Knowledge Engine — Document ingestion, chunking, embedding, and retrieval
Web Crawler & Web Search — Scheduled or ad-hoc crawling plus a built-in search provider
Vector Store Management — Multi-provider vector operations (Pinecone, Chroma, Qdrant, and more) with built-in SQLite vector support
OCR & Rerankers — Layout/table-aware document extraction and Cohere-compatible reranking
Semantic Memory — Scoped memory stores with vector-based recall
Observability & Evaluation
Agent Tracing — Batch and streaming ingest, OTLP/HTTP JSON, and an OpenTelemetry exporter
Evaluations — Datasets, AI-assisted labeling, and accuracy scoring
Inference Monitoring & Alerts — Real-time monitoring for self-hosted inference servers with rule-based alerting
Safety & Compliance
Authorization Engine — Roles, group grants, project permissions and API token scopes are decided by one policy engine (Cedar) on every request, with custom allow/deny policies, an access check, decision ids on every refusal and an audit trail (docs)
Guardrails — PII detection, content moderation, prompt shields, and custom LLM-based evaluators
AI Red-Teaming — OWASP LLM Top 10 probes, campaigns, and attack runs
Audit Logging — Tenant-scoped audit trails
Infrastructure & Cost
GPU Fleet (Enterprise) — Pooled GPU inference with MIG slicing, Hugging Face catalog deploys, and capacity planning
Cost & Optimization (Enterprise) — Budgets and automated optimization prescriptions on top of spend and usage tracking
Multi-Tenant Architecture — Complete data isolation per tenant, projects and groups; SSO (OIDC) and LDAP sign-in are Enterprise
File Management — Multi-provider file storage with automatic Markdown conversion