jae-jae/fetcher-mcp

Deutsch |
Español |
français |
日本語 |
한국어 |
Português |
Русский
Fetcher MCP
MCP server for fetch web page content using Playwright headless browser.
> 🌟 **Recommended**: OllaMan - Powerful Ollama AI Model Manager.
Advantages
- **JavaScript Support**: Unlike traditional web scrapers, Fetcher MCP uses Playwright to execute JavaScript, making it capable of handling dynamic web content and modern web applications.
- **Intelligent Content Extraction**: Built-in Readability algorithm automatically extracts the main content from web pages, removing ads, navigation, and other non-essential elements.
- **Flexible Output Format**: Supports both HTML and Markdown output formats, making it easy to integrate with various downstream applications.
- **Parallel Processing**: The fetch_urls tool enables concurrent fetching of multiple URLs, significantly improving efficiency for batch operations.
- **Resource Optimization**: Automatically blocks unnecessary resources (images, stylesheets, fonts, media) to reduce bandwidth usage and improve performance.
- **Robust Error Handling**: Comprehensive error handling and logging ensure reliable operation even when dealing with problematic web pages.
- **Configurable Parameters**: Fine-grained control over timeouts, content extraction, and output formatting to suit different use cases.
Quick Start
Run directly with npx:
First time setup - install the required browser by running the following command in your terminal:
HTTP and SSE Transport
Use the --transport=http parameter to start both Streamable HTTP endpoint and SSE endpoint services simultaneously:
After startup, the server provides the following endpoints:
- /mcp - Streamable HTTP endpoint (modern MCP protocol)
- /sse - SSE endpoint (legacy MCP protocol)
Clients can choose which method to connect based on their needs.
Debug Mode
Run with the --debug option to show the browser window for debugging:
Configuration MCP
Configure this MCP server in Claude Desktop:
On MacOS: ~/Library/Application Support/Claude/claude_desktop_config.json
On Windows: %APPDATA%/Claude/claude_desktop_config.json
Docker Deployment
Running with Docker
Deploying with Docker Compose
Create a docker-compose.yml file:
Then run:
Features
- fetch_url - Retrieve web page content from a specified URL
- Uses Playwright headless browser to parse JavaScript
- Supports intelligent extraction of main content and conversion to Markdown
- Supports the following parameters:
- url: The URL of the web page to fetch (required parameter)
- timeout: Page loading timeout in milliseconds, default is 30000 (30 seconds)
- waitUntil: Specifies when navigation is considered complete, options: 'load', 'domcontentloaded', 'networkidle', 'commit', default is 'load'
- extractContent: Whether to intelligently extract the main content, default is true
- maxLength: Maximum length of returned content (in characters), default is no limit
- returnHtml: Whether to return HTML content instead of Markdown, default is false
- waitForNavigation: Whether to wait for additional navigation after initial page load (useful for sites with anti-bot verification), default is false
- navigationTimeout: Maximum time to wait for additional navigation in milliseconds, default is 10000 (10 seconds)