divyanshu-iitian/SearchForge

Open-source web search API and MCP server for LLMs, agents, and RAG.
**One local gateway. Intent-aware search. Clean Markdown. REST, MCP, CLI, and TypeScript.**
Website · Quick start · Free tools · MCP · API · Design
---
**SearchForge is a free, open-source web search API and MCP server for LLMs, AI agents, and retrieval-augmented generation (RAG) pipelines.** It provides a predictable retrieval layer without forcing every project to integrate a paid search vendor. SearchForge routes each query to the right source, isolates provider failures, deduplicates URLs, fuses rankings, and turns public pages into LLM-ready Markdown.
It does **not** generate answers, hide citations, scrape public SearXNG instances, or send telemetry.
What you get
| Capability | Default source | Cost / credentials |
| ----------- | ----------------------------------- | ------------------------------ |
| auto | Intent-routed source mix | No key by default |
| web | Wikipedia; optional private SearXNG | No key / self-hosted |
| code | GitHub repository search | No key; token optional |
| academic | Crossref works and DOI metadata | No key |
| community | Hacker News via Algolia | No key, community service |
| read_url | Jina Reader | No key, currently rate-limited |
SearchForge starts with all no-key adapters enabled. auto is the default and routes code, research, and current/community intent to relevant sources while retaining a web fallback. A GitHub token only raises the public API quota, and Brave remains an optional keyed backend. Broad, independent web metasearch is provided by the included SearXNG stack.
Quick start
Try it without cloning
The first run downloads and builds the package from GitHub. Searches use intent-aware auto routing unless you select a category.
Zero-key local CLI
Full web search with private SearXNG
This starts SearchForge on port 3000 and a private, JSON-enabled SearXNG on port 8080. Before exposing the stack, change the SearXNG secret, set SEARCHFORGE_API_KEY, and terminate TLS at a trusted proxy.
Free tools
Search by capability
The default auto category detects code, academic, and current/community signals and queries the matching source families alongside the web fallback. Explicit categories prevent irrelevant providers from being queried. An explicit providers list overrides category routing, which is useful for evaluations.
Read a URL as Markdown
read_url accepts public HTTP(S) URLs only. Credentials, localhost, private IP literals, and non-web protocols are rejected. Responses are size-bounded, timed out, and cached.
Diagnose the whole retrieval path
Doctor performs real, bounded probes and reports each provider's access tier, capability, latency, and error. A failed source produces degraded, not a misleading all-or-nothing status.
MCP
SearchForge exposes three stdio tools:
- web_search — routed, citation-ready structured search
- read_url — clean Markdown from a public URL
- search_status — live capability and latency report
The search and status tools return MCP structured content as well as readable text.
Run the MCP server straight from GitHub without a clone:
SearchForge is also published in the
official MCP Registry
as io.github.divyanshu-iitian/searchforge. To run the registry-backed OCI image