Role1776/mcp-retrieval

An MCP server that gives an LLM three web tools: search, image search, and page scraping — no API keys required.
Tools ·
Quick start ·
Configuration ·
Retrieval engine ·
Architecture ·
Contributing
tag
inline when the src points at an uploaded asset URL
(https://github.com/user-attachments/assets/...). -->
---
What it is
mcp-retrieval is a Model Context Protocol server written in Go. It exposes web retrieval capabilities to any MCP-compatible client (Claude Desktop, IDE agents, custom LLM apps) as three read-only tools. Under the hood it uses the retrieval-go library to search the web and fetch pages, returning results as clean Markdown ready to hand to a model.
The library needs **no API keys**: web search goes through DuckDuckGo Lite, image search through Bing Images, and page fetching runs the HTML through a readability extractor before converting it to Markdown. To stay reliable against bot protection it impersonates real browsers at the TLS level and can rotate both browser fingerprints and proxies — see Retrieval engine.
Both transports the MCP SDK supports are available and expose the identical tool set:
- **stdio** — the client launches the binary and talks over stdin/stdout (the default, ideal for desktop clients).
- **http** — a long-running streamable HTTP server (useful for remote/shared deployments).
---
Tools
| Tool | Description |
| :--- | :--- |
| **web_search** | Runs one or more queries in parallel and returns per-query deduplicated, reranked snippets with links. |
| **web_search_images** | Runs one or more image queries in parallel and returns per-query deduplicated image results. |
| **web_scrape** | Downloads one or more pages in parallel and returns the main article text as Markdown. |
All three are annotated as **read-only**. Each tool returns a structured JSON payload that matches its output schema; the SDK mirrors the same JSON into the text content block for clients that do not read structuredContent.
web_search
| Parameter | Type | Default | Notes |
| :--- | :--- | :--- | :--- |
| queries | []string | — | **Required.** Executed in parallel. |
| max_results | int | 5 | Snippets per query, capped at max_results config (20). |
| timeout_ms | int64 | 5000 | Whole-call timeout; clamped to [min, max] from config. |
| date | string | — | Freshness filter: d (day), w (week), m (month), y (year). |
web_search_images
| Parameter | Type | Default | Notes |
| :--- | :--- | :--- | :--- |
| queries | []string | — | **Required.** Executed in parallel. |
| max_images | int | 5 | Images per query, capped at max_images config (10). |
| timeout_ms | int64 | 5000 | Whole-call timeout; clamped to [min, max] from config. |
| date | string | — | Freshness filter: d / w / m / y. |
web_scrape
| Parameter | Type | Default | Notes |
| :--- | :--- | :--- | :--- |
| urls | []string | — | **Required.** Downloaded in parallel. |
| robots_txt | bool | false | Respect the page's robots.txt. |
| timeout_ms | int64 | 5000 | Whole-call timeout; clamped to [min, max] from config. |
| remove_links | bool | false | Strip Markdown links from the text. |
| max_chars | int | 20000 | Truncate page text to N characters, capped at max_document_chars config (20000). |
> Both queries/urls lists are capped at max_queries (10) items per call. Queries must be ≤ 512 characters; URLs ≤ 2048 characters and http/https only.
Results and counts
Every call fans out across the input list and returns one entry per query/URL, each with its own status — success, failed, or timeout — so a partial failure still returns the items that did work.
count is the number of items actually returned, and it can be **lower than the requested max_results / max_images**: duplicates within a single query's results are removed before the limit is applied, and the upstream may simply have fewer items to give. A smaller count is a normal outcome, not an error.
Deduplication is **per query, not across queries**. Each entry is deduplicated on its own, so a link found by two of the queries in the same call appears in both entries — dedupe the union yourself if you need it.