Visualwebarena vs Presto Voice

What users praise

• • Execution-based evaluation with visual metrics is a true innovation.
• • Set-of-Marks annotation makes it easy to identify interactable elements.
• • Covers 910 tasks across classifieds, shopping, and Reddit platforms.
• • Free and open-source with publicly available code and leaderboard.

What frustrates them

• • Setup is plagued by Docker and Elasticsearch/MySQL issues.
• • Annotation errors in benchmark tasks require manual fixes.
• • Configuration for reproducing model results is poorly documented.
• • Only 910 tasks may be limited for thorough evaluation.

What users praise

• • Vendor claims up to 95% non-intervention on drive-thru orders, saving labor costs.
• • Claims an upselling engine with 88% offer acceptance and 6% revenue lift.
• • Runs a spectrum of Voice AI models to adapt to different menus and speech patterns.
• • Has 15+ years of restaurant-industry experience and national rollout references.

What frustrates them

• • No independent user reviews or community validation of the vendor's claims.
• • Brand name 'Presto' is confused with unrelated products like canners and transit cards.
• • App Store reviews for a similarly named app cite glitches, autoload failures, and connection errors.
• • Tourists report being locked out of the related Presto transit app without a Canadian address.