U
unclecode/crawl4ai
Official Crawl4AI Cloud: a hosted web scraping, extraction, and search API for AI agents. Free API key, 1,000 credits a month, no card. Fully open source: this repo is the Apache-2.0 crawler that powers it, and you can self-host it.
View on GitHub ↗Link checked 2026-11-12 · HTTP 200 · opens the vendor's own site
Install & config
- Copy the command above
- Paste it as one entry under
mcpServersinclaude_desktop_config.json— local servers take command + args (example: clients) - Restart your client
Install command from the repository
pip install -U crawl4ai
Parsed from the README we fetched on 2026-10-26; we have not executed it.
This is the install method the repository states, not an MCP client config, so no JSON is generated.
Key names differ by client: Clients & CLI
- Star count
- 87,314 ★ · unclecode/crawl4ai · read from the GitHub search API · (checked 2026-11-13 20:00:50)
- Source
- read directly from its GitHub page · type: MCP server
- Harvest history
- first seen 2026-10-26 · last seen 2026-11-09 · seen 16× · listed by 1 sources
- Install
pip install -U crawl4ai
FAQ (answers come from our records)
How do I install it?
The repository provides this install command:
pip install -U crawl4ai. It comes from the repository itself; we cannot guarantee it runs as-is in your client.Who is it for?
We have no tags recorded for it yet.
Who maintains it?
repository
unclecode/crawl4ai；first catalogued 2026-10-26；last seen 2026-11-09。There is no official certification or quality rating here — we record sources and dates only.# 🚀🤖 Crawl4AI: the open-source web crawler for LLMs and AI agents
以下摘录自该仓库的 README（片段） · 在 GitHub 读完整 README →
Latest: v0.9.5 (14 Oct 2026) · all releases →
Crawl4AI turns any website into clean, LLM-ready Markdown for RAG, AI agents and data pipelines. Run the open-source web crawler and scraper yourself, free forever, or use it hosted with one key: /scrape a page to Markdown, /extract it to JSON, /search the web, /serp for Google's results page as data, /answer a question with sources; MCP for your agent. Free: 1,000 credits every month, 10 a minute, 2 at a time. Paid: 120 a minute, 10 at a time, from $10 a month.
## Two ways to use Crawl4AI
### 🐍 Run it yourself: open source, forever
bash
pip install -U crawl4ai
crawl4ai-setup # installs the browser, once
python
import asyncio
from crawl4ai import AsyncWebCrawler
async def main():
async with AsyncWebCrawler() as crawler:
result = await crawler.arun(url="https://news.ycombinator.com")
print(result.markdown)
asyncio.run(main())
Docker server, CLI and every option: Installation · docs.crawl4ai.com
### ☁️ Or use the cloud: no browsers, no proxies
1.
Verify your email and 1,000 free credits are yours every month. No card.
2. Get any page as Markdown:
bash
curl -s https://api.crawl4ai.com/scrape \
-H "Authorization: Bearer $CRAWL4AIKEY" \
-H "Content-Type: application/json" \
-d '{"url": "https://news.ycombinator.com"}' | jq -r .markdown
The same key works for /extract, /search, /serp, /answer and many URLs at once (/scrape/batch, /scrape/jobs). Pay as you go: live prices.
3. Give it to your AI agent. Claude Code shown; Codex, Cursor and OpenCode →
bash
claude mcp add --transport http crawl4ai https://api.crawl4ai.com/mcp \
--header "Authorization: Bearer $CRAWL4AIKEY"
### Which one?
| | 🐍 Library | 🐳 Your own server | ☁️ Crawl4AI Cloud |
| Runs the browsers | you, in your Python process | you, in Docker on your machine | we do |
| JS-heavy pages and bot walls | your settings, your proxies | your settings, your proxies | handled for you, automatically |
| Web search | – | – | /search, /serp and /answer |
| Price | free, forever | free (your hosting) | 1,000 free credits a month; plans from $10 |
🤓 My Personal Story
I grew up on an Amstrad, thanks to my dad, and never stopped building. In grad school I specialized in NLP and built crawlers for research. That’s where I learned how much extraction matters.
In 2023, I needed web-to-Markdown. The “open source” option wanted an account, API token, and $16, and still under-delivered. I went turbo anger mode, built Crawl4AI in days, and it went viral. Now it’s the most-starred crawler on GitHub.
I made it open source for availability, anyone can use it without a gate. Now I’m building the platform for affordability, anyone can run serious crawls without breaking the bank. If that resonates, join in, send feedback, or just crawl something amazing.
That platform is live now: Crawl4AI Cloud.
Why developers pick Crawl4AI
LLM-ready output: smart Markdown with headings, tables, code and citation hints
Fast in practice: async browser pool, caching, minimal hops
Full control: sessions, proxies, cookies, user scripts, hooks
Adaptive intelligence: learns site patterns, explores only what matters
Deploy anywhere: no keys needed, CLI and Docker, or the hosted cloud
Crawl4AI turns any website into clean, LLM-ready Markdown for RAG, AI agents and data pipelines. Run the open-source web crawler and scraper yourself, free forever, or use it hosted with one key: /scrape a page to Markdown, /extract it to JSON, /search the web, /serp for Google's results page as data, /answer a question with sources; MCP for your agent. Free: 1,000 credits every month, 10 a minute, 2 at a time. Paid: 120 a minute, 10 at a time, from $10 a month.
## Two ways to use Crawl4AI
### 🐍 Run it yourself: open source, forever
bash
pip install -U crawl4ai
crawl4ai-setup # installs the browser, once
python
import asyncio
from crawl4ai import AsyncWebCrawler
async def main():
async with AsyncWebCrawler() as crawler:
result = await crawler.arun(url="https://news.ycombinator.com")
print(result.markdown)
asyncio.run(main())
Docker server, CLI and every option: Installation · docs.crawl4ai.com
### ☁️ Or use the cloud: no browsers, no proxies
1.
Verify your email and 1,000 free credits are yours every month. No card.
2. Get any page as Markdown:
bash
curl -s https://api.crawl4ai.com/scrape \
-H "Authorization: Bearer $CRAWL4AIKEY" \
-H "Content-Type: application/json" \
-d '{"url": "https://news.ycombinator.com"}' | jq -r .markdown
The same key works for /extract, /search, /serp, /answer and many URLs at once (/scrape/batch, /scrape/jobs). Pay as you go: live prices.
3. Give it to your AI agent. Claude Code shown; Codex, Cursor and OpenCode →
bash
claude mcp add --transport http crawl4ai https://api.crawl4ai.com/mcp \
--header "Authorization: Bearer $CRAWL4AIKEY"
### Which one?
| | 🐍 Library | 🐳 Your own server | ☁️ Crawl4AI Cloud |
| Runs the browsers | you, in your Python process | you, in Docker on your machine | we do |
| JS-heavy pages and bot walls | your settings, your proxies | your settings, your proxies | handled for you, automatically |
| Web search | – | – | /search, /serp and /answer |
| Price | free, forever | free (your hosting) | 1,000 free credits a month; plans from $10 |
🤓 My Personal Story
I grew up on an Amstrad, thanks to my dad, and never stopped building. In grad school I specialized in NLP and built crawlers for research. That’s where I learned how much extraction matters.
In 2023, I needed web-to-Markdown. The “open source” option wanted an account, API token, and $16, and still under-delivered. I went turbo anger mode, built Crawl4AI in days, and it went viral. Now it’s the most-starred crawler on GitHub.
I made it open source for availability, anyone can use it without a gate. Now I’m building the platform for affordability, anyone can run serious crawls without breaking the bank. If that resonates, join in, send feedback, or just crawl something amazing.
That platform is live now: Crawl4AI Cloud.
Why developers pick Crawl4AI
LLM-ready output: smart Markdown with headings, tables, code and citation hints
Fast in practice: async browser pool, caching, minimal hops
Full control: sessions, proxies, cookies, user scripts, hooks
Adaptive intelligence: learns site patterns, explores only what matters
Deploy anywhere: no keys needed, CLI and Docker, or the hosted cloud
Listed badge (optional)
Copy this line into the README if you want the listing mark. Optional, and it does not affect being listed.
[](https://anclum.com/en/s/unclecode-crawl4ai/)
- Source
- github · https://github.com/unclecode/crawl4ai
- Traction
- 87,314 ★
- added
- 5d ago