joinly-ai/joinly

**joinly.ai** is a connector middleware designed to enable AI agents to join and actively participate in video calls. Through its MCP server, joinly.ai provides essential meeting tools and resources that can equip any AI agent with the skills to perform tasks and interact with you in real time during your meetings.
> Want to dive right in? Jump to the Quickstart!
> Want to know more? Visit our website!
> [!IMPORTANT]
> Don't want the hustle of setting everything up? Try our cloud first! ☁️🚀
:sparkles: Features
- **Live Interaction**: Lets your agents execute tasks and respond in real-time by voice or chat within your meetings
- **Conversational flow**: Built-in logic that ensures natural conversations by handling interruptions and multi-speaker interactions
- **Cross-platform**: Join Google Meet, Zoom, and Microsoft Teams (or any available over the browser)
- **Bring-your-own-LLM**: Works with all LLM providers (also locally with Ollama)
- **Choose-your-preferred-TTS/STT**: Modular design supports multiple services - Whisper/Deepgram for STT and Kokoro/ElevenLabs/Deepgram for TTS (and more to come...)
- **100% open-source, self-hosted and privacy-first** :rocket:
:video_camera: Demos
GitHub
> In this demo video, joinly answers the question 'What is Joinly?' by accessing the latest news from the web. It then creates an issue in a GitHub demo repository.
Notion
> In this demo video, we connect joinly to our notion via MCP and let it edit the content of a page content live in the meeting.
Any ideas what we should build next? Write us! :rocket:
:zap: Quickstart
Run joinly via Docker with a basic conversational agent client.
> [!IMPORTANT]
> **Prerequisites**: Docker installation
Create a new folder joinly or clone this repository (not mandatory for the following steps). In this directory, create a new .env file with a valid API key for the LLM provider you want to use, e.g. OpenAI:
> [!TIP]
> You can find the OpenAI API key here
> [!NOTE]
> See .env.example for complete configuration options including Anthropic (Claude) and Ollama setups. Replace the placeholder values with your actual API keys and adjust the model name as needed. Delete the placeholder values of the providers you don't use.
Pull the Docker image (~2.3GB since it packages browser and models):
Launch your meeting in Zoom, Google Meet or Teams and let joinly join the meeting using the meeting link as . Then, run the following command from the folder where you created the .env file:
> :red_circle: Having trouble getting started? Let's figure it out together on our discord!
:technologist: Run a custom client
In Quickstart, we ran the Docker Container directly as a client using --client. But we can also run it as a server and connect to it from outside the container, which allows us to connect other MCP servers. Here, we run a custom client using the joinly-client package and connect it to the joinly MCP server.
> [!IMPORTANT]
> **Prerequisites**: do the Quickstart (except the last command), install uv, and open two terminals
Start the joinly server in the first terminal (note, we are not using --client here and publish port 8000 on localhost only):
> [!WARNING]
> The joinly MCP server has no authentication and accepts client-supplied configuration, so it is meant to run locally with a single trusted client. Bind it to localhost (as above) and do not expose the port to a network.
While the server is running, start the example client implementation in the second terminal window to connect to it and join a meeting:
Add MCP servers to the client
Add the tools of any MCP server to the agent by providing a JSON configuration. The configuration file can contain multiple entries under "mcpServers" which will all be available as tools in the meeting (see fastmcp client docs for config syntax):