tan-yong-sheng/ai-vision-mcp

A powerful Model Context Protocol (MCP) server that provides AI-powered image and video analysis using Google Gemini and Vertex AI models.
Features
- **Dual Provider Support**: Choose between Google Gemini API and Vertex AI
- **Multimodal Analysis**: Support for both image and video content analysis
- **Flexible File Handling**: Upload via multiple methods (URLs, local files, base64)
- **Storage Integration**: Built-in Google Cloud Storage support
- **Comprehensive Validation**: Zod-based data validation throughout
- **Error Handling**: Robust error handling with retry logic and circuit breakers
- **TypeScript**: Full TypeScript support with strict type checking
Quick Start
Pre-requisites
You could choose either to use google provider or vertex_ai provider. For simplicity, google provider is recommended.
Below are the environment variables you need to set based on your selected provider. (Note: It’s recommended to set the timeout configuration to more than 5 minutes for your MCP client).
(i) **Using Google AI Studio Provider**
Get your Google AI Studio's api key here
(ii) **Using Vertex AI Provider**
Refer to the guideline here on how to set this up.
Installation
Below are the installation guide for this MCP on different MCP clients, such as Claude Desktop, Claude Code, Cursor, Cline, etc.
Claude Desktop
Add to your Claude Desktop configuration:
(i) Using Google AI Studio Provider
(ii) Using Vertex AI Provider
Claude Code
(i) Using Google AI Studio Provider
(ii) Using Vertex AI Provider
Note: Increase the MCP startup timeout to 1 minutes and MCP tool execution timeout to about 5 minutes by updating ~\.claude\settings.json as follows:
Cursor
Go to: Settings -> Cursor Settings -> MCP -> Add new global MCP server
Pasting the following configuration into your Cursor ~/.cursor/mcp.json file is the recommended approach. You may also install in a specific project by creating .cursor/mcp.json in your project folder. See Cursor MCP docs for more info.
(i) Using Google AI Studio Provider
(ii) Using Vertex AI Provider
Cline
Cline uses a JSON configuration file to manage MCP servers. To integrate the provided MCP server configuration:
1. Open Cline and click on the MCP Servers icon in the top navigation bar.
2. Select the Installed tab, then click Advanced MCP Settings.
3. In the cline_mcp_settings.json file, add the following configuration:
(i) Using Google AI Studio Provider
(ii) Using Vertex AI Provider
Other MCP clients
The server uses stdio transport and follows the standard MCP protocol. It can be integrated with any MCP-compatible client by running:
MCP Tools
The server provides four main MCP tools:
1) analyze_image
Analyzes an image using AI and returns a detailed description.
**Parameters:**
- imageSource (string): URL, base64 data, or file path to the image
- prompt (string): Question or instruction for the AI
- mode (string, optional): Analysis mode - one of:
- general (default) - General image analysis
- palette - Extract design tokens (colors, spacing, typography)
- hierarchy - Analyze visual hierarchy and eye flow
- components - Catalog UI components and design system maturity
- options (object, optional): Analysis options including temperature and max tokens
**Examples:**
1. **General image analysis:**
2. **Extract design tokens:**
3. **Analyze visual hierarchy:**
4. **Component inventory:**
2) compare_images
Compares multiple images using AI and returns a detailed comparison analysis.
**Parameters:**
- imageSources (array): Array of image sources (URLs, base64 data, or file paths) - minimum 2, maximum 4 images
- prompt (string): Question or instruction for comparing the images
- options (object, optional): Analysis options including temperature and max tokens
**Examples:**
1. **Compare images from URLs:**