itsalissonsilva/ModelSafetyMCP

Malicious or unsafe ML artifacts are a real supply-chain problem. There have already been real cases of malicious or suspicious models being discovered in the wild, including models hosted on public platforms and larger batches of unsafe AI/ML artifacts. See RL identifies malware ML model hosted on Hugging Face and Over 100 Malicious AI/ML Models Found on Hugging Face.
One of the biggest reasons this matters is deserialization risk. Many model formats, especially pickle-based and framework-specific formats, can execute code or invoke unsafe logic while being loaded or reconstructed. That means a model file can become an execution vector, not just a passive blob of weights.
model-safety is an MCP server for inspecting machine learning model artifacts before you load, ship, or trust them.
It is designed for practical triage:
- scan a local model file
- scan a downloadable model URL
- triage a whole directory of artifacts
- combine heuristic checks with dedicated scanners
- return normalized findings and concrete next actions
What It Can Do
The server currently exposes these tools:
- available_scanners
Shows which scanner backends are installed and ready.
- artifact_safety_report
Runs the broadest scan available on one model artifact and returns per-scanner results, normalized findings, highest severity, and recommended actions.
- modelscan_scan
Runs ModelScan directly against a local file or URL.
- picklescan_scan
Runs PickleScan directly against a local file or URL.
- deep_model_inspect
Runs lightweight structural checks for risky extensions, embedded pickle members, and suspicious packaging patterns.
- scan_directory
Runs artifact_safety_report across every file in a directory and aggregates the risky files.
Quick Start
1. Add the MCP server to Cursor or Claude Code using the local Python runtime in this repo.
2. Start with artifact_safety_report on a local file or direct URL.
3. Review normalized_findings, highest_severity, and recommended_actions.
4. Use scan_directory when you need to triage a whole model drop.
Minimal Cursor config:
Current Detection Strengths
This MCP is strongest when scanning:
- PyTorch checkpoints such as .pt and .pth
- pickle-like artifacts such as .pkl, .pickle, and .joblib
- Keras and TensorFlow HDF5 models such as .h5
- model bundles that are actually ZIP or TAR containers
It currently combines:
- modelscan
Best general-purpose backend, especially for model-specific unsafe patterns like Keras Lambda.
- picklescan
Best supporting backend for pickle-oriented artifacts.
- deep_model_inspect
Fast heuristic fallback that catches risky packaging even when specialized scanners are quiet.
Supported Inputs
You can scan either:
- a local filesystem path
- a direct http or https artifact URL
For Hugging Face specifically, use the direct file URL:
- Good: https://huggingface.co/ /resolve/main/model.h5
- Not ideal: https://huggingface.co/ /blob/main/model.h5
blob URLs usually return an HTML page, while resolve URLs return the real artifact bytes.
Runtime
This repo includes a local Python 3.12 runtime in python312. That is the supported runtime because it works with:
- mcp
- modelscan
- picklescan
- h5py
If you ever need to rebuild that runtime manually: