lfnovo/content-core

Extract, process, and summarize content from URLs, files, and text through a unified async Python API, CLI, or MCP server.
Supported Formats
| Category | Formats |
|----------|---------|
| Web | URLs, HTML pages, YouTube videos, Reddit posts |
| Documents | PDF, DOCX, PPTX, XLSX, ODT, ODS, ODP, EPUB, HTML, Markdown, plain text |
| Media | MP3, WAV, M4A, FLAC, OGG (audio); MP4, AVI, MOV, MKV (video) |
Quick Start
Or with zero install:
CLI Usage
Content Core provides a unified content-core command with subcommands for extraction, summarization, and MCP server.
Extract
Summarize
MCP Server
Configuration
Config is stored in ~/.content-core/config.toml. Priority: command flags > env vars > config file > defaults.
Zero-Install with uvx
All commands work without installation using uvx:
Python API
Extraction
Summarization
Configuration
MCP Integration
Content Core includes a Model Context Protocol (MCP) server for use with Claude Desktop and other MCP-compatible applications.
Add to your claude_desktop_config.json:
The MCP server exposes two tools: extract_content and summarize_content. Both return plain text.
For detailed setup, see the MCP documentation.
Agent Skill (Claude Code & Codex)
Content Core ships an Agent Skill that teaches AI agents how to use it for extracting content from external sources. This repository is also a plugin marketplace, so the skill installs natively in both harnesses.
**Claude Code** — add the marketplace and install the plugin:
**Codex** — the repository carries a Codex plugin manifest (.codex-plugin/plugin.json) and marketplace catalog (.agents/plugins/marketplace.json) pointing at the same skill.
**Manual fallback** — copy the skill file directly into your project:
Once installed, the agent can use content-core to extract content from URLs, documents, and media files — either via CLI (uvx content-core) or MCP if configured.
AI Providers
Content Core uses Esperanto to support multiple LLM and STT providers. Switch providers by changing the config — no code changes needed:
Supported providers include OpenAI, Anthropic, Google, Groq, DeepSeek, Ollama, and more. See the Esperanto documentation for the full list.
Configuration
Content Core uses ContentCoreConfig powered by pydantic-settings. Settings are resolved in priority order: constructor args > env vars (CCORE_*) > config file (~/.content-core/config.toml) > defaults.
Environment Variables
| Variable | Description | Default |
|----------|-------------|---------|
| CCORE_URL_ENGINE | URL extraction engine (auto, simple, firecrawl, jina, crawl4ai) | auto |
| CCORE_DOCUMENT_ENGINE | Document extraction engine (auto, simple, docling) — docling raises ConfigurationError if the extra is not installed; auto falls back silently | auto |
| CCORE_AUDIO_CONCURRENCY | Concurrent audio transcriptions (1-10) | 3 |
| CCORE_AUDIO_SEGMENT_MINUTES | Length of the segments long audio is split into; 0 sends the file whole | 10 |
| CRAWL4AI_API_URL | Crawl4AI Docker API URL (omit for local browser mode) | - |
| CRAWL4AI_API_TOKEN | Bearer token for the Crawl4AI Docker API (required by Crawl4AI >= 0.9.0) | - |