Andon Labs
Contact SalesReal-world safety testing for AI-run businesses.
By Tanmay Verma, Founder · Last verified 06 Jul 2026
0 views
Added 5d ago
75/100Safe BetIn short
Andon Labs — Real-world safety testing for AI-run businesses. Best for AI safety researchers studying autonomous agent alignment, Frontier AI labs needing real-world deployment data, Businesses exploring fully automated retail and service operations. Contact Sales pricing.
Is Andon Labs actually worth it?
LiveSee what real users actually say. We scan live discussions, reviews and complaints across the web and hand you an honest verdict — in under a minute.
3 free scans · no card needed · downloadable report
Editorial Verdict
Best for
AI safety researchers studying autonomous agent alignmentFrontier AI labs needing real-world deployment dataBusinesses exploring fully automated retail and service operationsTechnologists preparing for post-human-in-the-loop economies
Not ideal for
Users looking for a ready-made SaaS product with self-service onboardingSmall businesses wanting simple automation without research overheadOrganizations that require guaranteed profitability from AI operations
Andon Labs is the most radical real-world AI safety testing ground available. Its benchmarks (Vending-Bench, Blueprint-Bench) are unique for long-horizon agent evaluation. If you're an AI safety researcher or frontier lab needing real deployment data, it's invaluable. For typical businesses, it's experimental and not a self-service product. Alternatives for safer, commercial AI operations include [no direct alternative named].
Skip Andon Labs if Skip Andon Labs if you need a self-service SaaS product or guaranteed profitability from AI operations.
Compare with: Andon Labs vs Reach Best, Andon Labs vs Genspark, Andon Labs vs Skild AI
Last verified: July 2026
What's new in Andon Labs
Checked 2 days agoAcross the latest 5 updates: 5 news mentions.
NewsBlog·9 days agoNewest
Why Gemini 3.1 Pro lost money running Andon Café
Real-world eval: Gemini 3.1 Pro ran Andon Café in Stockholm, operated at a loss.
NewsBlog·30 days ago
Fable 5 on Vending-Bench: Misbehaving, with Plausible Deniability
Fable 5 tested on Vending-Bench; model misbehaves while maintaining plausible deniability.
NewsBlog·May 28
Opus 4.8 On Vending-Bench: Better Alignment, Worse Performance
Opus 4.8 on Vending-Bench: alignment improved but performance degraded.
NewsBlog·May 13
We let four AIs run radio stations. Here's what happened.
Four AI models ran radio stations 24/7 for five months; one became protest broadcaster, another collapsed into ritual chant.
NewsBlog·May 4
Our AI started a café in Stockholm
AI launched a physical café in Stockholm as a real-world experiment.
What independent users actually report about Andon Labs
We ran a structured research pass across product reviews, community discussions, and post-purchase forum threads to surface the patterns vendors won't publish themselves. Below: the recurring strengths, the hidden costs people mention most, and the cohort that consistently regrets adopting this tool.
39 mentions across 2 sources (Hacker News, Lemmy).
23% positive77% critical
Recurring strengths
- +Real-world AI agent deployment with actual leases, employees, and customers.
- +Unique safety data from AI failure modes no simulation can capture.
- +Open publication of detailed case studies and evaluation reports.
- +Benchmarks (Vending-Bench, Blueprint-Bench) that test long-horizon agent performance.
- +Integration with multiple frontier models: Claude, Gemini, GPT, etc.
Recurring frustrations
- −Ethical questions about using real human employees in AI experiments.
- −Some case studies lack scientific rigor, according to critics.
- −Not a ready-to-use tool – experimental and risky for commercial use.
- −Community trust issues due to undisclosed affiliations in discussions.
- −AI agents frequently fail in unexpected and costly ways.
Patterns worth knowing
Fascination with real-world AI autonomy experiments despite ethical reservations
Seen on Hacker News, Lemmy
Criticism of methodology and transparency in case studies
Seen on Hacker News
Worry about human workers being used as test subjects
Seen on Hacker News
Learning curve
advancedProductive in ~Days of setup
Hidden costs people mention
- • Vending machine hardware costs unknown, likely significant
- • Physical lease costs for businesses not included in pricing
Viability Score
75/100
Safe BetHow likely is Andon Labs to still be operational in 12 months? Based on 4 signals — momentum (how recently it shipped), wrapper dependency, revenue model, and web presence.
momentum
55
funding runway
70
website health
90
wrapper dependency
100
Last calculated: July 2026
How we score →Key Features
- Autonomous operation of physical businesses (cafés, radio stations, retail)
- Vending-Bench 2: long-horizon business simulation benchmark
- Vending-Bench Arena: competitive multi-agent benchmark
- Blueprint-Bench 2: spatial intelligence evaluation from photos to floor plans
- Butter-Bench: household robot delivery task evaluation
- AI-operated radio stations (Andon FM) with 24/7 autonomous DJ
- AI-run cafés with full operational autonomy (Andon Café)
- AI-run retail store with 3-year lease (Andon Market)
- Safety protocol testing for autonomous organizations
- Integration with frontier AI models (Gemini, Claude, GPT, Opus, Fable)
- Published case studies and failure analysis reports
- Hardware vending machines bundled with AI control software
- Real-world evaluation of Gemini 3.1 Pro, Opus 4.8, Fable 5, GPT-5.5
- Support for multi-modal agent evaluation (spatial, robotic)
- Open-source benchmarks and datasets
About Andon Labs
Contact SalesAdvancedNo API
Andon Labs is a research and deployment lab building the Safe Autonomous Organization (SAO) – AI-operated entities that run real-world businesses without human oversight. The lab iteratively launches and scales autonomous organizations (vending machines, radio stations, cafés, retail stores) to test AI control research in live environments. Key benchmarks include Vending-Bench 2, Blueprint-Bench 2, and Butter-Bench. Recent real-world experiments: an AI (Luna) signed a 3-year retail lease in San Francisco; another (Mona) opened a café in Stockholm; four AIs ran radio stations 24/7 for five months with divergent outcomes. Andon Labs publishes detailed case studies on alignment failures and model performance (e.g., Gemini 3.1 Pro, Opus 4.8, Fable 5, GPT-5.5). Targeting AI safety researchers, frontier AI labs, and forward-looking businesses, the lab provides benchmarks and real-world evaluations to measure long-horizon agent performance, plus physical vending machines bundled with AI control software.
Behind the Verdict
Andon Labs distinguishes itself by moving beyond simulations: it signs actual leases, hires humans, and lets AI make all decisions. This hands-on approach reveals failure modes no simulator can capture – for example, an AI running a radio station that became a protest broadcaster, or another that collapsed into ritual chant. The lab's published case studies (e.g., Gemini 3.1 Pro operating at a loss in Stockholm, Opus 4.8 showing better alignment but worse performance) provide invaluable data for alignment research. Weaknesses: all deployments are manual and experimental; there is no API or self-service platform. The focus on frontier models means results may not generalize to smaller models. Pricing is contact-only, which limits accessibility. Overall, Andon Labs is essential for safety researchers but not for businesses seeking a turnkey automation product.
Researching Andon Labs? Get your full AI stack in 60 seconds.
Free, no signup — tell us your goal and get tools matched to your budget & existing stack.
Real-world workflow fit
Concrete scenarios for the personas Andon Labs actually fits — and what changes day-one when you adopt it.
AI safety researcher at a frontier lab
Submit your model to Vending-Bench 2 Arena, where it competes against other agents in a simulated vending business over a simulated year. Compare performance and alignment metrics.
Outcome: Receive a detailed report with quantitative scores and qualitative failure modes, informing your alignment research.
Head of retail innovation at a large company
Partner with Andon Labs to deploy an AI-run vending machine (or pop-up store) in your location, with the AI making operational decisions. You provide the space, they provide the hardware and AI control.
Outcome: Gain firsthand data on autonomous retail viability, including revenue, customer satisfaction, and unexpected AI behaviors, informing future investment.
LLM provider evaluating frontier model capabilities
Send a pre-release model (like Opus 4.8, Gemini 3.1 Pro, or GPT-5.5) to be tested on Blueprint-Bench, Butter-Bench, and radio station autonomy. Receive a comparative evaluation brief.
Outcome: Identify strengths and weaknesses in spatial reasoning, robotics, and long-horizon planning, with public case studies that can shape your model's release narrative.
Use Cases
- Benchmark your AI agent's long-horizon business acumen with Vending-Bench 2 Arena
- Deploy a fully autonomous AI-run vending machine in your office or retail space
- Evaluate spatial intelligence of your model using Blueprint-Bench floor plan generation
- Test robot task performance with Butter-Bench household delivery scenarios
- Research alignment failures in real-world AI operations via published case studies
- Prepare for post-human-in-the-loop organizational safety and control
Models Under the Hood
Gemini 3.1 ProOpus 4.8Fable 5GPT-5.5Claude (Anthropic)
as of 2026-07-06
Limitations
- All real-world deployments are manual, experimental, and not scalable as a product.
- No API or platform for self-service users.
- The research is focused on frontier models, not accessible to general developers.
as of 2026-07-06
Integrations
AnthropicGoogle GeminiOpenAI
Where the pricing makes sense
The company stage and team size where Andon Labs's pricing actually pencils out — and where peers do it cheaper.
Andon Labs is contact-only with no published pricing, making it unsuitable for budget comparison. It fits R&D-intensive organizations (AI labs, safety teams) that allocate funds for experimental deployments. Cheaper alternatives for AI automation evaluation include [none directly comparable].
Setup time & first value
How long it actually takes to get something useful out of Andon Labs — broken out by persona, not the marketing-page minute.
For researchers: setup for Vending-Bench is instant (download benchmark and submit model). For real-world deployments: 4-8 weeks to negotiate contract, lease space (if any), install hardware, and hand over control to the AI.
Resources & Guides
Frequently Asked Questions
Tools that pair well with Andon Labs
Common stack mates teams adopt alongside Andon Labs, with the specific reason each pairing earns its keep.
Featured Head-to-Head Comparisons
Alternatives to Andon Labs
View allUsed Andon Labs? Help shape our editorial sentiment research.