NVIDIA/TensorRT-Model-Connect

Deploy supported Hugging Face models for end-to-end TensorRT inference in just two commands.
Documentation   |   Quick Start   |   Model Support   |   API Reference
> Public Preview — TensorRT-Model-Connect is evolving rapidly and is
> intended for evaluation and feedback. APIs, scope, and direction may change.
> See Project status.
## 📰 News and Updates
2026-08-23 · Blog — AI-Native by Design: What We Learned Building
TensorRT-Model-Connect
2026-08-18 · Announcement — TensorRT-Model-Connect enters public
preview
## 💻 Example Code
bash
python -m tensorrtmodelconnect build Qwen/Qwen3-0.6B \
--max-sequence-length 16384 \
--output qwen3-0.6b.bundle
trtmc run ./qwen3-0.6b.bundle \
--runtime-root /opt/trtmc/lib \
--prompt "What is the capital of France? Answer in one word." \
--use-chat-template true \
--enable-thinking false
# Generated text: Paris
The same bundle works from
C++:
cpp
#include
#include
#include
#include
auto task = trtmc::loadtask("./qwen3-0.6b.bundle", "/opt/trtmc/lib");
auto text = dynamiccast (task.get());
if (text == nullptr) throw std::runtimeerror("unexpected task");
std::cout generate("What is the capital of France? Answer in one word.").text
## 🌟 Get Started & Stay Tuned
Ready to try TensorRT-Model-Connect? Follow the
Quick Start
to build and run your first supported model.
We're glad you're here. Star the repository
to keep TensorRT-Model-Connect on your radar as we share new model integrations,
releases, examples, and community updates. Ideas and feedback are always
welcome through the issue chooser.
## 🔎 What is TensorRT-Model-Connect?
TensorRT Model Connect is an extensive collection of AI Model reference implementations in C++, on top of NVIDIA TensorRT. Model Connect is powered by an agentic workflow that continuously adds support for upcoming models, drastically reducing integration effort on user side and time until new models become compatible.
## 🧭 Choose the right abstraction layer
Use TensorRT-Model-Connect to explore models quickly and evaluate broad
model coverage.
For production LLM/VLM deployment on NVIDIA edge platforms where performance
is the priority, start directly with
TensorRT Edge-LLM.
## 💡 Why TensorRT-Model-Connect?
Start from a supported Hugging Face or local checkpoint and build TensorRT
engines without an intermediate ONNX export step.
Hand a versioned .bundle artifact from the Python-first build environment
to native C++ task APIs such as text generation, transcription, image and
video generation, segmentation, embedding, and forecasting.
Use model-family-owned builders, runtime pipelines, helper kernels, and
validation contracts as concrete blueprints for modification and
customization.
Keep native TensorRT and optional TensorRT-RTX execution behind the same
task-oriented application boundary.
Read the Architecture Overview
for the architecture boundary, intended users, and comparison with other
TensorRT integration paths.
TensorRT-Model-Connect is a reference implementation. Users are responsible
for trusting the checkpoints, bundles, native libraries, and local environment