call518/MCP-Airflow-API

> Revolutionary Open Source Tool for Managing Apache Airflow with Natural Language
---
Architecture & Internal (DeepWiki)
---
📋 Overview
Have you ever wondered how amazing it would be if you could manage your Apache Airflow workflows using natural language instead of complex REST API calls or web interface manipulations? **MCP-Airflow-API** is the revolutionary open-source project that makes this goal a reality.
---
🎯 What is MCP-Airflow-API?
MCP-Airflow-API is an MCP server that leverages the **Model Context Protocol (MCP)** to transform Apache Airflow REST API operations into natural language tools. This project hides the complexity of API structures and enables intuitive management of Airflow clusters through natural language commands.
🆕 Multi-Version API Support (NEW!)
**Now supports both Airflow API v1 (2.x) and v2 (3.0+)** with dynamic version selection via environment variable:
- **API v1**: Full compatibility with Airflow 2.x clusters (54 tools) - Documentation
- **API v2**: Enhanced features for Airflow 3.0+ including asset management for data-aware scheduling (56 registered tools) - Documentation
**Key Architecture**: Single MCP server with shared common tools (54) plus v2-exclusive asset tools (2) - dynamically loads appropriate toolset based on AIRFLOW_API_VERSION environment variable!
**Traditional approach (example):**
**MCP-Airflow-API approach (natural language):**
> "Show me the currently running DAGs"
---
🚀 Quickstart
> **📝 Need a test Airflow cluster?** Use our companion project **Airflow-Docker-Compose** with support for both **Airflow 2.x** and **Airflow 3.x** environments!
Flow Diagram of Quickstart/Tutorial
🎯 Recommended: Docker Compose (Complete Demo Environment)
**For quick evaluation and testing:**
Getting Started with OpenWebUI (Docker Option)
> 📌 **Note**: Web-UI configuration instructions are based on OpenWebUI **v0.6.22**. Menu locations and settings may differ in newer versions.
1. Access http://localhost:3002/
2. Log in with admin account
3. Go to "Settings" → "Tools" from the top menu
4. Add Tool URL: http://localhost:8002/airflow-api
5. Configure your LLM provider (Ollama, OpenAI, etc.)
---
📦 MCP Server Installation Methods
Method 1: Direct Installation from PyPI
Method 2: Claude-Desktop MCP Client Integration
**Local Access (stdio mode)**
**Remote Access (streamable-http mode without authentication)**
**Remote Access (streamable-http mode with Bearer token authentication - Recommended)**
**Multiple Airflow Clusters with Different Versions**
> **💡 Pro Tip**: Use the test clusters from Airflow-Docker-Compose for the above configuration - they run on ports 38080 (2.x) and 48080 (3.x) respectively!
Method 3: Development Installation
---
🌟 Key Features
1. **Natural Language Queries**
No need to learn complex API syntax. Just ask as you would naturally speak:
- "What DAGs are currently running?"
- "Show me the failed tasks"
- "Find DAGs containing ETL"
2. **Comprehensive Monitoring Capabilities**
Real-time cluster status monitoring:
- Cluster health monitoring
- DAG status and performance analysis
- Task execution log tracking
- XCom data management
3. **Dynamic API Version Support**
Single MCP server adapts to your Airflow version:
- **API v1**: 54 shared tools for Airflow 2.x compatibility
- **API v2**: 54 shared tools + 2 asset management tools for Airflow 3.0+
- **Environment Variable Control**: Switch versions instantly with AIRFLOW_API_VERSION
- **Zero Configuration Changes**: Same tool names, enhanced capabilities
- **Efficient Architecture**: Shared common codebase eliminates duplication
4. **Comprehensive Tool Coverage**
Covers almost all Airflow API functionality:
- DAG management (trigger, pause, resume)
- Task instance monitoring
- Pool and variable management
- Connection configuration
- Configuration queries
- Event log analysis
4. **Large Environment Optimization**
Efficiently handles large environments with 1000+ DAGs:
- Smart pagination support
- Advanced filtering options
- Batch processing capabilities
---
🛠️ Technical Advantages
- **Leveraging Model Context Protocol (MCP)**
MCP is an open standard for secure connections between AI applications and data sources, providing:
- Standardized interface
- Secure data access
- Scalable architecture
- **Support for Two Transport Modes**
- stdio mode: Direct MCP client integration for local environments
- streamable-http mode: HTTP-based deployment for Docker and remote access
**Environment Variable Control:**