marcomq/mq-bridge

crossing streams · library
One Rust engine to move data reliably between brokers, databases, files and HTTP — embedded in your Rust, Python or Node.js service.
🖥️ mq-bridge-app ·
📊 ETL benchmarks ·
🏗️ Architecture ·
⚙️ Config ·
📚 Middleware & endpoint reference
## What is mq-bridge?
mq-bridge is an asynchronous Rust library that moves messages and data between systems: Kafka, NATS, RabbitMQ, MQTT, MongoDB, PostgreSQL/MySQL/SQLite, Postgres CDC, ClickHouse, Redis Streams, HTTP, WebSocket, gRPC, ZeroMQ, AWS SQS/SNS, IBM MQ, cloud object storage, files, and in-memory channels. Every endpoint sits behind the same batch-shaped receivebatch / sendbatch traits, so any source can feed any sink.
A route connects one input to one output. In between, it can transform, filter, fan out, retry, rate-limit, deduplicate, or answer a request, all by configuration. Where configuration isn't enough, you add a handler in Rust, Python or Node.js. Batching, ack/nack, commit ordering and broker I/O stay in the Tokio-based core, so your code works with plain CanonicalMessages and doesn't handle Kafka offsets or AMQP nacks.
It is a library you embed in your own service, not a daemon or platform you operate. If you'd rather not write code, mq-bridge-app runs the same engine and config as a CLI, server, desktop app, and MCP server.
## Use cases
| Problem | Guide with a runnable example |
| A RAG or semantic-search index drifts from its Postgres source | Keep a vector index in sync with Postgres: CDC → embeddings → Qdrant |
| Search results show what MongoDB no longer has | Keep a search index in sync with MongoDB: change stream → Meilisearch |
| Free-text messages need routing by what they say | Classify and route messages with an LLM |
| Events need a database record and per-key history to act on | Enrich events and keep per-key history in Postgres |
| A service must update its database and publish an event atomically | Transactional outbox with Postgres and Kafka |
| Tests for Kafka, NATS or RabbitMQ consumers need a running broker | Test message handlers without a broker |
## Quick Start
Install the library for your language:
| Language | Package | Install |
| Rust | mq-bridge | cargo add mq-bridge --features kafka,nats,yaml |
| Python | mq-bridge (PyPI) | pip install mq-bridge |
| Node.js | mq-bridge (npm) | npm install mq-bridge |
| C / C++ | mqbridge.h | cargo build --release -p mq-bridge-c (build from source) |
Describe a route — here Kafka → NATS with retries — in routes.yaml:
yaml
kafkatonats:
input:
kafka: { url: "localhost:9092", topic: "orders", groupid: "bridge" }
output:
middlewares:
retry: { maxattempts: 5 }
nats: { url: "nats://localhost:4222", subject: "orders.processed" }
Run it:
python
from mqbridge import Route
Route.fromfile("routes.yaml", "kafkatonats").run() # blocks; use start() to keep going
js
import { Route } from "mq-bridge";
Route.fromFile("routes.yaml", "kafkatonats").start();
rust
mqbridge::deployfile("routes.yaml").await?; // every route in the file, in the background
Attach a handler (withhandler / withHandler) when you need business logic in between. No code at all? mq-bridge-app runs the same engine from the command line or a desktop UI: