ScrapeGraphAI/scrapegraph-mcp

A production-ready Model Context Protocol (MCP) server that provides seamless integration with the ScrapeGraph AI API. This server enables language models to leverage advanced AI-powered web scraping capabilities with enterprise-grade reliability.
## Table of Contents
Key Features
Quick Start
Available Tools
Setup Instructions
Remote Server Usage
Local Usage
Google ADK Integration
Example Use Cases
Error Handling
Common Issues
Development
Contributing
Documentation
Technology Stack
License
## API v2
This MCP server targets ScrapeGraph API v2 (https://v2-api.scrapegraphai.com/api), aligned 1:1 with
scrapegraph-py PR #84. Auth uses the
SGAI-APIKEY header. Environment variables mirror the Python SDK:
SGAI_API_URL — override the base URL (default https://v2-api.scrapegraphai.com/api)
SGAI_TIMEOUT — request timeout in seconds (default 120)
SGAI_API_KEY — API key (can also be passed via MCP scrapegraphApiKey or X-API-Key header)
> Legacy aliases (still honored): SCRAPEGRAPH_API_BASE_URL for SGAI_API_URL, SGAI_TIMEOUT_S for SGAI_TIMEOUT.
## Key Features
Scrape & extract: scrape (POST /scrape, multi-format), extract (POST /extract, URL + prompt)
Search: search (POST /search; num_results clamped 3–20)
Crawl: Async multi-page crawl with crawl_start / crawl_get_status / crawl_stop / crawl_resume
Schema: schema (POST /schema) — generate or augment a JSON Schema from a prompt
Monitors: Scheduled jobs via monitor_create, monitor_list, monitor_get, pause/resume/delete, monitor_activity (paginated tick history)
Account: credits, history
Easy integration: Claude Desktop, Cursor, Smithery, HTTP transport
Developer docs: .agent/ folder
## Migration: v2 → v3
v3 renames every MCP tool that diverged from the v2 API docs. Hard rename, no aliases.
| v2 (old) | v3 (new) |
| smartscraper | extract |
| searchscraper | search |
| smartcrawler_initiate | crawl_start |
| smartcrawler_fetch_results | crawl_get_status |
| sgai_history | history |
| generate_schema | schema |
| markdownify | removed — use scrape with output_format="markdown" |
## Quick Start
### 1. Get Your API Key
Sign up and get your API key from the ScrapeGraph Dashboard
### 2. Install with Smithery (Recommended)
bash
npx -y @smithery/cli install @ScrapeGraphAI/scrapegraph-mcp --client claude
### 3. Start Using
Ask Claude or Cursor:
"Convert https://scrapegraphai.com to markdown"
"Extract all product prices from this e-commerce page"
"Research the latest AI developments and summarize findings"
That's it! The server is now available to your AI assistant.
## Available Tools
| Tool | Role |
| scrape | POST /scrape (output_format: markdown, html, screenshot, branding, links, images, summary) |
| extract | POST /extract (requires website_url + user_prompt; optional output_schema) |
| search | POST /search (num_results 1–20; supports country_search, time_range, output_schema) |
| crawl_start | POST /crawl — extraction_mode markdown / html / links / images / summary / branding / screenshot |
| crawl_get_status | GET /crawl/:id (poll until status: completed) |
| crawl_stop, crawl_resume | POST /crawl/:id/stop \| resume |
| schema | POST /schema (generate or augment a JSON Schema from a prompt) |
| credits | GET /credits |