Fetch Guard
以下摘录自该仓库的 README(片段) · 在 GitHub 读完整 README →
An MCP server and CLI tool that fetches URLs and returns clean, LLM-ready markdown. A purpose-built extraction pipeline sanitizes HTML, pulls structured metadata, detects prompt injection attempts, and handles the edge cases that break naive fetchers: bot blocks, paywalls, login walls, non-HTML content types, and pages that require JavaScript to render.
The core problem is straightforward: LLMs need web content, but raw HTML is noisy and potentially hostile. Fetched pages can contain hidden text, invisible Unicode, off-screen elements, and outright prompt injection attempts embedded in the content itself. This pipeline strips all of that before the content reaches the model.
Three layers handle the injection defense specifically:
1. **Pre-extraction sanitization** removes hidden elements (display:none, visibility:hidden, opacity:0, font-size:0, transform:scale(0), clip:rect(0,0,0,0), zero-height overflow containers, and elements with matching foreground and background colors), elements hidden via CSS class/ID rules in tags, off-screen positioned content, aria-hidden elements, and tags, and 26 categories of non-printing Unicode characters including bidi isolates and Unicode Tags. This happens before content extraction, so trafilatura never sees the attack vectors.
2. **Pattern scanning** runs a four-phase scan against the extracted text and metadata fields. Phase one applies 50 compiled regex patterns covering system prompt overrides, ignore-previous instructions, role injection, fake conversation tags, and hidden instruction markers, in English, Spanish, French, German, Japanese, Simplified Chinese, and Portuguese. Phase two normalizes the text via NFKC and confusable-character mapping, then rescans to catch homoglyph bypasses (Cyrillic or mathematical Unicode characters substituted for Latin, etc.). Phase three finds base64, hex-encoded, and URL percent-encoded blocks, decodes them, and scans against high-severity patterns. Phase four decodes the full document with ROT13 and scans against high-severity patterns. Metadata fields (title, description, og:title, etc.) are scanned independently with matches namespaced to their source field.
3. **Session-salted output wrapping** generates a random 8-character hex salt per invocation and wraps the body in tags. Since the salt is unpredictable, injected content cannot spoof the wrapper boundaries.
One Tool
This is a single-tool MCP server. It exposes one tool — fetch — that runs a full extraction pipeline behind a consistent interface. No tool selection, no routing, no multi-step workflows. One URL in, one structured result out, configurable via parameters.
Quick Start
Prerequisites
- Python 3.10+
- pip
Install
For JavaScript rendering (optional):
Configure Your MCP Client
Add the following to your MCP client config. Works with Claude Code, Claude Desktop, Cursor, or any MCP-compatible client.
**Via uvx (recommended):**
**Via pip install:**
**From source:**
**Via Docker:**
> **Note:** The Docker image does not include Playwright. JavaScript rendering (js: true) is not available when running via Docker. Use the uvx or pip install if you need JS rendering.
Verify
Ask your AI assistant to fetch any URL. If it returns structured content with a status header, metadata, and risk assessment, you're connected.
CLI
| Flag | Default | Description |
|---|---|---|
| --timeout N | 180 | Request timeout in seconds |
| --max-words N | none | Word cap on extracted body content. Also disables the automatic size guard |
| --js | off | Use Playwright for JS-rendered pages |
| --strict | off | Exit code 2 on high-risk injection |
| --links MODE | domains | domains for unique external domains, full for all URLs with anchor text |
| --header KEY:VALUE | none | Custom HTTP header (repeatable) |