omni-mcp/isaac-sim-mcp

The MCP Server and its extension leverage the Model Context Protocol (MCP) framework to enable natural language control of NVIDIA Isaac Sim, transforming conversational AI inputs into precise simulation manipulation. This expansion bridges the MCP ecosystem with embodied intelligence applications.
Features
- Natural language control of Isaac Sim
- Dynamic robot positioning and movement
- Custom lighting and scene creation
- Advanced robot simulations with obstacle navigation
- Interactive code preview before execution
Requirements
- NVIDIA Isaac Sim 4.2.0 or higher
- Python 3.9+
- Cursor AI editor for MCP integration
**Mandatory** Pre-requisite
- Install uv/uvx: https://github.com/astral-sh/uv
- Install mcp[cli] to base env: [uv pip install "mcp[cli]"](https://pypi.org/project/mcp/)
Installation
Install and Enable Extension
Isaac Sim extension folder should point to your project folder:
- Extension location: ~/Documents/isaac-sim-mcp
- Extension ID: isaac.sim.mcp_extension
Verify the extension starts successfully. The output should look like:
The extension should be listening at **localhost:8766** by default.
Install MCP Server
1. Go to terminal and run, make sure mcp server could start sucessfully at terminal with base venv.
2. Start Cursor and open the folder ~/Documents/isaac-sim-mcp
3. Go to Cursor preferences, choose MCP and add a global MCP server:
Development Mode
To develop the MCP Server, start the MCP inspector:
You can visit the debug page through http://localhost:5173
Example Prompts for Simulation
Notice: Switch to Agent mode in top left of Chat dialog before you type prompt and choose sonnet 3.7 for better coding.
Robot Party
Factory Setup
Vibe Coding from scratch
Gen3D with beaver3d model support
USD search
MCP Tools
The Isaac Sim MCP Extension provides several specialized tools that can be accessed through natural language in Cursor AI. These tools enable you to control and manipulate NVIDIA Isaac Sim with simple commands:
Connection and Scene Management
- **get_scene_info** - Pings the Isaac Sim Extension Server to verify connection status and retrieve basic scene information. Always use this first to ensure the connection is active.
Physics and Environment Creation
- **create_physics_scene** - Creates a physics scene with configurable parameters:
- objects: List of objects to create (each with type and position)
- floor: Whether to create a ground plane (default: true)
- gravity: Vector defining gravity direction and magnitude (default: [0, -0.981, 0])
- scene_name: Name for the scene (default: "physics_scene")
Robot Creation and Control
- **create_robot** - Creates a robot in the scene at a specified position:
- robot_type: Type of robot to create (options: "franka", "jetbot", "carter", "g1", "go1")
- position: [x, y, z] position coordinates
Omniverse Kit and Scripting
- **omni_kit_command** - Executes an Omni Kit command: