Mobilegym

MobileGym is a purpose-built sandbox for researchers who need deterministic, parallelizable mobile agent evaluation. Its core insight — represent entire app states as structured JSON — lets you snapshot, fork, and reset in milliseconds, a feat impossible with real devices or emulators. The programmatic judges (zero false accept/reject over 416 tasks) give RL practitioners something they rarely get: a clean reward signal. The 28 simulated apps cover Chinese daily-use ecosystems (WeChat, Alipay, 12306) that benchmarks typically avoid because they're unreadable, unresettable, and irreversible on real hardware. Where it bites: this is a research tool, not a QA platform. Simulated apps are React/TypeScript reimplementations — they mimic Android behavior but aren't actual APKs. If your goal is testing a production app's UI, you need real devices or emulators running the real OS. MobileGym also assumes technical familiarity with Python, RL training loops, and agent frameworks. Non-technical users will find the CLI and live demo limited for exploration. Compared to alternatives like Android Studio Emulator or BrowserStack, MobileGym sacrifices authenticity for controllability. The emulator gives you a real OS but no app-level state injection or parallel rollouts; BrowserStack offers many real devices but no simulation sandbox. MobileGym occupies a specific niche: online RL research on GUI agents where reproducibility and batch training matter more than OS fidelity. In practice, we'd reach for this when training models like Qwen3-VL-4B with GRPO — the paper reports +40.7 pt real-device gain with 95.1% retention. For production CI/CD testing, pass. For multi-agent benchmarks requiring dozens of environments in parallel, it's currently the best option available.