Google today open-sourced an AI-driven Android automation system called ARTEMIS on GitHub, which can turn everyday English instructions directly into reliable automation operations on Android devices. According to the officially published AndroidWorld benchmark data, ARTEMIS achieved a completion rate of over 99% across more than 20 real-world apps and over 100 complex multi-step tasks.
What is ARTEMIS? More than just a testing framework.
According to the official documentation on GitHub, ARTEMIS is an AI agent with autonomous capabilities, fundamentally different in positioning from traditional testing frameworks. It can handle complex cross-application workflows, such as “setting a driving route on Google Maps and calculating the total duration, then opening YouTube to play a Coldplay song,” which falls under everyday tasks that require execution across different apps.
ARTEMIS is open-sourced under the Apache-2.0 license, supports macOS, Linux, and Windows platforms, and requires an Android device with USB debugging enabled or an emulator, along with a Python 3.12+ environment. It also integrates support for multiple large multimodal models, including Gemini, Claude, GPT-4o, and Qwen-VL.
ARTEMIS – GitHub
Two execution modes: Flash and Pro
ARTEMIS offers two distinctly different execution modes to meet different automation needs:
Flash mode (`–profile flash`)This is a fast reactive loop, with each step taking about 3–5 seconds. The model directly observes the current screen, thinks, and acts, without any additional graphical orchestration. It is suitable for routine, deterministic UI tasks, such as setting options, filling out forms, and other application scenarios that can be completed within 25–30 steps. It does not support pre-execution safety checks, checkpoint verification, final reports, or ADB shell.
Pro Mode (`–profile pro`)This is a multi-agent graphical architecture, with each step taking approximately 15–40 seconds. The Planner creates a dynamic Markdown task plan, marking milestones and verification checkpoints; the Operator executes tasks using a full toolset. Every action passes through a pre-execution Safety Net check, employing a dual verification mechanism that prioritizes XML with pixel-based fallback. It supports long-horizon workflows of 100+ steps, extended continuous monitoring, and optional written reports. Pro mode can also handle execution incidents caused by UI delays or pop-ups, recovering directly within the Operator’s context without needing an additional repair agent.
Core Architecture: Three-Layer Progressive Positioning Engine
The core technology of ARTEMIS is a three-stage progressive positioning engine:
Layer 1: On-device OCR + Accessibility Level(approximately 150 ms, 0 token consumption), drives over 85% of standard UI operations via accessibility hierarchy indexing, unaffected by resolution drift or UI redesign.
Layer 2: Spatial Vision ModelWhen the first layer cannot locate elements, fall back to the vision model to handle custom Canvas, Compose, or Flutter self-drawn UIs.
Layer 3: Sandboxed CV ProbeUsed for subtle pixel state detection, to determine details that are difficult to judge directly with the naked eye, such as button enabled states, progress bar progress, etc.
Additionally, the Pre-Touch Pixel Gate and Speculative Chaining mechanisms can intercept unexpected pop-up dialog windows a few milliseconds before an action is dispatched, and perform predictive sequential clicks on transient UI elements (such as auto-fading video control bars), eliminating silent misclicks caused by inference latency race conditions.
MCP Protocol Integration: Directly Control Your Phone Within the IDE
ARTEMIS includes a native MCP (Model Context Protocol) server, enabling AI coding assistants to directly drive real test devices. Supported IDEs include Antigravity, Claude Code, Codex, Cursor, Windsurf, VS Code, Cline, and OpenClaw.
Developers only need to enter natural language instructions in the IDE chat window, for example:
Build the latest changes into an APK, install it on the connected device, open the login screen with a test account, check for any abnormal pop-ups after logging in, and send back a screenshot of the final page.
ARTEMIS will automatically execute this entire process, automatically capturing Logcat crash stacks and key frame screenshots along the way. The actual integration can be done with a single command: `uv run artemis mcp –install all` will automatically mount the MCP configuration and test mindset rules into all supported IDEs.
Automated Testing vs. Daily Assistant
ARTEMIS has a dual positioning. For test engineers, it is a zero-maintenance test automation engine: traditional Appium or Espresso frameworks rely on fragile XPath/ID selectors, so test code must be modified whenever the UI is redesigned; ARTEMIS’s dynamic priority-based locating engine, on the other hand, remains resilient to UI changes, system updates, and resolution variations.
It can also be used as an everyday autonomous assistant. Officially demonstrated uses include cross-app information extraction and operation, continuous exploration testing sessions lasting over 10 hours, and integration with CI/CD pipelines for automated regression testing. The Python SDK provides a standard `ArtemisClient` interface that can be embedded into the pytest testing framework and supports strongly typed Pydantic structured output.
Roadmap: iOS and Android Studio native plugins
According to the plan on GitHub, ARTEMIS’s development roadmap includes native plugin integration for Android Studio, iOS platform expansion, on-device lightweight VLM (Vision-Language Model), and bidirectional voice interaction. This indicates that Google is positioning this technology as a cross-platform, universal mobile AI automation infrastructure.
Conclusion
The open-sourcing of ARTEMIS effectively puts AI-driven mobile automation capabilities—previously affordable only to large test teams—into the hands of any developer. Together, the 99%+ benchmark completion rate and MCP protocol integration turn “writing tests in natural language” from a concept into a directly usable tool. For end users, this also signals that AI agent capabilities on Android devices are about to enter a new phase: no longer just passively answering questions, but actively operating the phone for you.
Source: KOCPC Chinese