官方说明开源项目5 天前
onlymuneeb38-glitch/ios-agent-skills-evaluator
iOS Agent Skills 2026: Test 11 Tasks, 260+ Scenarios, 850+ Assertions on 3 Models
在 GitHub 查看 ↗直链 · 无中转页
- ★ 数取值
- 3 ★ · onlymuneeb38-glitch/ios-agent-skills-evaluator · 取自 GitHub 搜索 API · （2026-10-16 20:00:20 核对）
- 来源明细
- 从该仓库的 GitHub 页面直接读取 · 条目类型：开源项目
- 收录与活跃
- 收录于 2026-10-12 · 最近见到 2026-10-16 · 共见到 7 次 · 被 1 个来源列出
- 评分构成
- 综合分 103 · 3 ★ · 1 个来源 · repo
常见问题（答案都来自我们的记录）
怎么安装？
我们【没有】从该仓库取到安装命令（我们的 install 覆盖率只有约 4%）。请看它的 README 或 releases 页。
它适合谁 / 用来做什么？
我们把它归在这些标签下: 开源项目。标签来自条目自身，不是我们强加的分类。
谁维护它、还在更新吗？
仓库
onlymuneeb38-glitch/ios-agent-skills-evaluator；我们收录于 2026-10-12；最近见到 2026-10-16。这里没有"官方认证"或"质量评级"——我们只记录来源与时间。第三方质量标记
iOS Agent Skills Pro: Intelligent Test Automation Framework for Mobile AI Evalua
以下摘录自该仓库的 README（片段） · 在 GitHub 读完整 README →
🚀 The Ultimate Testbed for iOS AI Agent Capabilities
Welcome to **iOS Agent Skills Pro** — a comprehensive, open-source evaluation framework designed to stress-test the cognitive and operational skills of AI agents operating on iOS devices. This repository is not just another testing toolkit; it is a **neural gymnasium** where your AI agents train, compete, and evolve through 11 meticulously crafted tasks, 260+ real-world navigation scenarios, and 850+ granular assertions across three cutting-edge models.
Whether you are a machine learning researcher, a QA engineer, or a developer building autonomous iOS assistants, this framework provides the structured chaos needed to validate agent performance in production-like environments.
---
🧠 Repository Vision: Beyond Benchmarks
Traditional agent testing is like teaching a parrot to recite phrases. **iOS Agent Skills Pro** is different — it is the **Swiss Army knife of agent evaluation**. We simulate ambiguous user intents, broken UI states, and multi-step reasoning chains. Our 850+ assertions act as **microscopic neural checkpoints**, ensuring your agent doesn't just *appear* smart but *actually* navigates iOS interfaces with human-like adaptability.
**Key differentiators from existing repos:**
- Not a static benchmark but a **living test suite** that grows with iOS updates
- Supports **Claude API and OpenAI API** for model-agnostic evaluation
- Generates **radar charts of agent weaknesses** (e.g., fails on dynamic lists, succeeds on static buttons)
- Includes **responsive UI validation** — tests how agents react to device rotation, split-screen, and dark mode
---
📥 Quick Start: Download and Setup
Prerequisites
- iOS Simulator (Xcode 15+)
- Python 3.10+
- OpenAI API key or Anthropic API key
Installation in 30 Seconds
1. Download the repository from the link above
2. Run pip install -r requirements.txt
3. Configure your API keys in config.yaml
---
🔮 Architecture Overview (Mermaid Diagram)
*Figure 1: The agent evaluation pipeline — from user intent to assertion verification.*
---
⚙️ Example Profile Configuration
Create a profile.yaml file to customize your agent's behavioral parameters:
**Why this matters:** The max_steps parameter prevents infinite loops during tricky tasks like "find the WiFi password in Settings > General > About." The forbidden_actions field mimics real-world safety guardrails.
---
💻 Example Console Invocation
Run a full evaluation suite with a single command:
Expected output:
*This real-time feedback loop is the **heartbeat** of the framework — you know exactly where your agent stumbles.*
---
📱 Operating System Compatibility
| iOS Version | Status | Notes |
|-------------|--------|-------|
| iOS 18.0+ | ✅ Full Support | All 260+ scenarios verified |
| iOS 17.4 - 17.9 | ✅ Supported | ClipKit and Live Activities may vary |
| iOS 16.6 - 17.3 | ⚠️ Partial | Some gesture-based assertions disabled |
| iOS 15.x | ❌ Deprecated | Minimum requirement is iOS 16.6 |
**Emoji Legend:** ✅ = Certified | ⚠️ = Limited | ❌ = Not Supported
The framework auto-detects the OS version and adjusts assertion thresholds accordingly — a **feature born from frustration** with brittle test suites that break on every iOS update.
---
🌟 Comprehensive Feature List
Core Capabilities
- **11 Task Categories** — from basic tapping to complex multi-step workflows involving Apple Pay, Shortcuts, and Siri integration
Welcome to **iOS Agent Skills Pro** — a comprehensive, open-source evaluation framework designed to stress-test the cognitive and operational skills of AI agents operating on iOS devices. This repository is not just another testing toolkit; it is a **neural gymnasium** where your AI agents train, compete, and evolve through 11 meticulously crafted tasks, 260+ real-world navigation scenarios, and 850+ granular assertions across three cutting-edge models.
Whether you are a machine learning researcher, a QA engineer, or a developer building autonomous iOS assistants, this framework provides the structured chaos needed to validate agent performance in production-like environments.
---
🧠 Repository Vision: Beyond Benchmarks
Traditional agent testing is like teaching a parrot to recite phrases. **iOS Agent Skills Pro** is different — it is the **Swiss Army knife of agent evaluation**. We simulate ambiguous user intents, broken UI states, and multi-step reasoning chains. Our 850+ assertions act as **microscopic neural checkpoints**, ensuring your agent doesn't just *appear* smart but *actually* navigates iOS interfaces with human-like adaptability.
**Key differentiators from existing repos:**
- Not a static benchmark but a **living test suite** that grows with iOS updates
- Supports **Claude API and OpenAI API** for model-agnostic evaluation
- Generates **radar charts of agent weaknesses** (e.g., fails on dynamic lists, succeeds on static buttons)
- Includes **responsive UI validation** — tests how agents react to device rotation, split-screen, and dark mode
---
📥 Quick Start: Download and Setup
Prerequisites
- iOS Simulator (Xcode 15+)
- Python 3.10+
- OpenAI API key or Anthropic API key
Installation in 30 Seconds
1. Download the repository from the link above
2. Run pip install -r requirements.txt
3. Configure your API keys in config.yaml
---
🔮 Architecture Overview (Mermaid Diagram)
*Figure 1: The agent evaluation pipeline — from user intent to assertion verification.*
---
⚙️ Example Profile Configuration
Create a profile.yaml file to customize your agent's behavioral parameters:
**Why this matters:** The max_steps parameter prevents infinite loops during tricky tasks like "find the WiFi password in Settings > General > About." The forbidden_actions field mimics real-world safety guardrails.
---
💻 Example Console Invocation
Run a full evaluation suite with a single command:
Expected output:
*This real-time feedback loop is the **heartbeat** of the framework — you know exactly where your agent stumbles.*
---
📱 Operating System Compatibility
| iOS Version | Status | Notes |
|-------------|--------|-------|
| iOS 18.0+ | ✅ Full Support | All 260+ scenarios verified |
| iOS 17.4 - 17.9 | ✅ Supported | ClipKit and Live Activities may vary |
| iOS 16.6 - 17.3 | ⚠️ Partial | Some gesture-based assertions disabled |
| iOS 15.x | ❌ Deprecated | Minimum requirement is iOS 16.6 |
**Emoji Legend:** ✅ = Certified | ⚠️ = Limited | ❌ = Not Supported
The framework auto-detects the OS version and adjusts assertion thresholds accordingly — a **feature born from frustration** with brittle test suites that break on every iOS update.
---
🌟 Comprehensive Feature List
Core Capabilities
- **11 Task Categories** — from basic tapping to complex multi-step workflows involving Apple Pay, Shortcuts, and Siri integration
- 来源
- github · https://github.com/onlymuneeb38-glitch/ios-agent-skills-evaluator
- 按领域浏览
- 开源项目
- Traction
- 3 ★
- 收录于
- 昨天
更多开源项目
所属领域与标签
每个标签都能点进去看同类的全部条目。数字是当前库里的真实条数。