Potato
Free open-source annotation for text, images, audio, video, and AI agents.
Potato delivers near-commercial annotation power at zero cost, with agent evaluation and quality-control features that beat many paid tools. Pick it if you can self-host; skip it if you need managed SaaS, but the feature depth is hard to beat for the price.
Verified 3d ago · liveness 66/100 · cite: rightaichoice.com/tools/potato
- Academic researchers needing free, flexible annotation for papers
- NLP teams building SFT/DPO datasets from agent traces
- Computer vision teams labeling images with bounding boxes/polygons
- AI agent evaluators who need to calibrate judges and rank models
- Teams without DevOps capacity to self-host and maintain servers
- Organizations needing managed cloud SaaS with support
- Enterprise deployments requiring role-based access control (RBAC)
We scan live Reddit threads, YouTube comments, X posts, G2 reviews and other communities — and hand you an honest verdict in under a minute.
- Honest verdict, not marketing
- Real pros & cons from real users
- Attributed quotes with receipts
3 free scans · no card needed
Skip Potato if you need a fully managed SaaS with zero DevOps, enterprise-grade RBAC/audit trails, or real-time simultaneous collaboration on the same session.
Self-hosting requires your own server, storage, and maintenance — there's no managed cloud option, so you'll spend time on deployment and scaling.
Potato is free, self-hosted, and open-source, making it ideal for academics, startups, and teams with DevOps skills. For managed SaaS alternatives, Labelbox and Prodigy offer cloud convenience but cost thousands per year. Potato's $0 price tag undercuts them, but you trade off convenience for control.
In short
Potato — Free open-source annotation for text, images, audio, video, and AI agents. Best for Academic researchers needing free, flexible annotation for papers, NLP teams building SFT/DPO datasets from agent traces, Computer vision teams labeling images with bounding boxes/polygons. Free to use.
What's new in Potato
Checked yesterdayAcross the latest 7 updates: 5 feature updates, 1 launch and 1 changelog entry.
Agreement Cannot Catch Rubber-Stamping
Potato discusses how rubber-stamping pre-labels inflates agreement metrics; timing is the only signal left.
Partial Is a Real Outcome: Annotating Robot Episodes
Potato's timeline-shaped interface for robot episode annotations handles partial successes with min/max downsampling and hindsight relabelling.
Shipping Segmentation That Runs in a Browser Tab
Potato runs click-to-segment and open-vocabulary text prompting client-side, no GPU, with hand-written tokenizer and verified encoder contract.
Raw IoU Is Not Agreement
Potato highlights that raw IoU between annotators is ~0.95 even for non-lookers; proposes chance-corrected alternatives.
Potato 2.8: Annotate Anything, Then Measure It
Potato 2.8 adds computer vision, gigapixel zoom, 3D point clouds, depth maps, robot episodes, video eval, VLM grounding, and chance-corrected agreement.
Reading the Writing Process: Keystroke Logging for Free-Text Annotation
Potato logs pauses, revisions, and pastes in free-text annotation without recording raw keystrokes, flagging pasted responses.
Potato 2.7.1: The Transcript Already Exists
Potato 2.7.1 reads 21 transcript/subtitle formats, loads sidecar files, and includes a converter for ASR output to annotation-ready data.
What people actually say about Potato — is it worth it?
We ran a structured research pass across product reviews, community discussions, and post-purchase forum threads to surface the patterns vendors won't publish themselves. Below: the recurring strengths, the hidden costs people mention most, and the cohort that consistently regrets adopting this tool.
95 mentions across 7 sources (Hacker News, YouTube, Product Hunt, App Store, Stack Overflow, GitHub, Lemmy) · researched Aug 30, 2026.
- +Free and open-source, eliminating licensing costs for academic labs.
- +Supports 61 annotation types covering text, audio, images, and more.
- +Built-in AI hints via OpenAI, Claude, and Gemini accelerate labeling.
- +Active learning with five query strategies reduces annotation effort.
- +QDA Mode and living codebook suit qualitative research workflows.
- −Zero real user reviews means reliability and UX are unproven.
- −Self-hosting demands DevOps skills; no community guides exist.
- −Name collision with food and VPN apps makes support search chaotic.
- −No evidence of enterprise-grade scalability or uptime at scale.
- −Intermediate skill level may intimidate non-technical annotators.
- • No official hidden costs, but self-hosting requires server infrastructure, maintenance, and DevOps time.
- • Potential costs for AI API usage if using OpenAI, Claude, or Gemini hints at scale.
Viability Score
How well maintained and how widely used is Potato? Built from what the vendor actually publishes (docs, changelog, tutorials, integrations, pricing), whether the site is live, and how much real users discuss it. How we calculate this
Last calculated: September 2026
How we score →Key Features
- 61 annotation types including classification, span, bounding box, polygon, and more
- Agent evaluation on clickable interaction graphs for multi-agent teams
- LLM hints from OpenAI, Claude, and Gemini for label suggestions
- Active learning with uncertainty sampling, diversity-based, BADGE, BALD, hybrid ensemble
- QDA Mode with living codebook, in-vivo codes, memos, cases
- Audio waveform annotation
- Image annotation with bounding boxes and polygons
- Video playback controls for annotation
- IRT error bars for label confidence
- Multiplayer norming rooms with live agreement meter
- Counterfactual boundary probes
- Peer-prediction scoring
- Think-Aloud Mode for local voice rationales
- Keystroke logging for free-text answers
- Reads 21 transcript formats and converts ASR folders
About Potato
Potato is a free, open-source annotation platform developed by the University of Michigan, designed for labeling text, audio, images, video, and qualitative data. It supports a wide range of annotation types, from classification and span labeling to bounding boxes, polygons, and codebook-driven coding. What sets Potato apart is its focus on AI agent evaluation: you can import traces from any framework, visualize multi-agent interactions on a clickable graph, and label them for training or quality control. Setup is entirely YAML-based, so researchers can launch a complete annotation interface with zero code in minutes. The tool includes 61 annotation types, covering everything from radio buttons and Likert scales to best-worst scaling, pairwise comparison, and qualitative coding with a living codebook. For multimedia, Potato offers audio waveform annotation, image bounding boxes and polygons, and video playback controls. AI integration is built in, with LLM hints from OpenAI, Claude, and Gemini for label suggestions, keyword highlighting, and option highlighting. Active learning uses five query strategies—uncertainty sampling, diversity-based selection, BADGE, BALD, and hybrid ensemble—plus an LLM cold start to choose initial instances when no labels exist yet. Recent updates have strengthened quality control and transcript handling. Potato 2.7.1 reads 21 transcript formats, loads sidecar files, and converts ASR folders into annotation-ready files. Newer features include keystroke logging for free-text answers (recording pauses, revisions, and pastes as auditable flags), IRT error bars for label confidence, multiplayer norming rooms with a live agreement meter, counterfactual boundary probes, peer-prediction scoring, and Think-Aloud Mode that records reasoning locally without an LLM. Potato competes with commercial tools like Labelbox and Prodigy but stays free and self-hosted. It is best suited for academic labs, NLP teams, and agent evaluators who need
Behind the Verdict
Potato is the tool we'd reach for when budget is zero and we need serious annotation muscle. The breadth is staggering—61 annotation types, multimedia support, and agent evaluation on interactive graphs—all free. It's not just a toy; the active learning strategies and IRT error bars show it's built for real research. Where it bites is in deployment. You have to self-host, manage a server, and handle the YAML config yourself. That's fine for a lab with a grad student who can run pip install, but it's a non-starter for teams that want a managed cloud solution. If you're evaluating with Labelbox or Prodigy, you're paying for simplicity and support that Potato doesn't offer. The agent annotation angle is genuinely different. Being able to import traces from any framework and label multi-agent interactions on a graph is something we haven't seen elsewhere at this price. It's a strong fit if you're building SFT or DPO datasets from agent runs. Quality control features like keystroke logging and peer-prediction scoring are ahead of many commercial tools. If you need to audit annotator behavior or catch sloppy work, this is a major plus. But don't expect real-time collaboration on the same session. Multiplayer norming rooms help, but for teams needing simultaneous editing, look elsewhere. Also, RBAC is missing, so large enterprise deployments might struggle with access control. In practice, we'd use Potato for research projects, paper data sets, and internal agent evals. It's not a turnkey SaaS, but the trade-off is worth it for the feature depth and the price.
Researching Potato? Get your full AI stack in 60 seconds.
Free, no signup — tell us your goal and get tools matched to your budget & existing stack.
Real-world workflow fit
Concrete scenarios for the personas Potato actually fits — and what changes day-one when you adopt it.
You need to annotate a corpus of text for a study on sentiment.
Outcome: Install Potato via pip, configure YAML for your labels, launch the server, and have RA annotate in minutes — all free.
You're creating SFT/DPO data from agent traces.
Outcome: Import traces from LangChain, use the interactive graph to label actions, and export to JSON for training.
You need to code interview transcripts with a codebook.
Outcome: Use QDA Mode to build a living codebook, add in-vivo codes and memos, and analyze with full-text search.
Use Cases
- Annotate text for sentiment analysis, NER, and classification
- Evaluate multi-agent AI systems by labeling traces on interaction graphs
- Calibrate an LLM judge against human labels
- Edit agent trajectories into SFT and DPO training data
- Conduct qualitative coding with a living codebook and memos
- Set up active learning pipelines to prioritize uncertain instances
Models Under the Hood
as of 2026-08-28
Limitations
- Potato is a free, open-source annotation tool that requires self-hosting, typically started via a CLI command and accessed through a web browser.
- While configuration is YAML-based and code-free, advanced features may require understanding of configuration and deployment.
- The platform is developed at the University of Michigan and is primarily aimed at researchers and annotation teams.
- Users are responsible for their own server deployment, scaling, and data privacy when self-hosting.
as of 2026-08-24
Verification history
We have re-verified Potato 6 times since . Each pass re-reads the vendor's own pages and re-checks every listed field against that evidence; passes where nothing had changed are marked as such.
- — re-verified summary, description, our verdict, our analysis, pricing model, pricing tiers, features, integrations, who it suits, who should skip it
- — re-verified summary, description, our verdict, our analysis, pricing model, pricing tiers, features, integrations, who it suits, who should skip it
- — re-verified summary, description, our verdict, our analysis, pricing model, pricing tiers, features, integrations, who it suits, who should skip it
- — re-verified summary, description, our verdict, our analysis, pricing model, pricing tiers, features, integrations, who it suits, who should skip it
- — re-verified summary, description, our verdict, our analysis, pricing model, pricing tiers, features, integrations, who it suits, who should skip it
- — re-verified summary, description, our verdict, our analysis, pricing model, pricing tiers, features, integrations, who it suits, who should skip it
Free to cite with attribution — this page re-verifies continuously.
12-month cost
Project the real annual outlay, including the implied monthly cost when only an annual tier is published.
Vendor list price only. Add-on usage, seat overages, and contract minimums are surfaced under Hidden costs & gotchas.
Plans compared
For each published Potato tier: who it actually fits, and what it adds vs. the previous tier. Cross-reference the cost calculator above for projected annual outlay.
Open Source (Self-Hosted)
$0/mo
Ideal for
Academic researchers, independent developers, and small teams needing a free, customizable annotation platform with full control over data and infrastructure.
What this tier adds
Starting tier: free and open-source, includes all features like 61 annotation types, agent evaluation, and AI hints, but requires self-hosting.
Where the pricing makes sense
The company stage and team size where Potato's pricing actually pencils out — and where peers do it cheaper.
Potato is free, self-hosted, and open-source, making it ideal for academics, startups, and teams with DevOps skills. For managed SaaS alternatives, Labelbox and Prodigy offer cloud convenience but cost thousands per year. Potato's $0 price tag undercuts them, but you trade off convenience for control.
Setup time & first value
How long it actually takes to get something useful out of Potato — broken out by persona, not the marketing-page minute.
Install via pip and run your first task in under 10 minutes. For a simple YAML config, you can be annotating within the hour. Advanced features like agent evaluation may take a few hours to configure.
Switching to or from Potato
How to bring data in from common predecessors and how to get it back out — written for the switcher, not the buyer.
- →From Labelbox: Export your labels and import them into Potato's JSON format; recreate your annotation interface in YAML.
- →From Prodigy: Migrate your recipes and data by converting your configuration to Potato's YAML.
- →From manual Excel/CSV: Import your data files and define your annotation scheme in YAML.
- ↗To Labelbox: Export Potato annotations as JSON and import them into Labelbox using their API.
- ↗To Prodigy: Convert Potato's JSON to Prodigy's format using a custom script.
- ↗To a custom pipeline: Potato's JSON export can be read by Python scripts for any downstream use.
Integrations
Resources & Guides
Tutorials & Learning
Official links
Tools that pair well with Potato
Common stack mates teams adopt alongside Potato, with the specific reason each pairing earns its keep.
Featured Head-to-Head Comparisons
Potato vs Screenplayiq
Choose ScreenplayIQ if you're a screenwriter or producer needing data-driven box office predictions and structural feedback on feature scripts. Choose Potato if you need a free, flexible annotation platform for multi-modal data (text, audio, images, video, agent traces) and are comfortable self-hosting.
Potato vs Praktika
Praktika and Potato serve completely different purposes: Praktika is a mobile language learning app focused on conversation practice with AI tutors, while Potato is an open-source annotation tool for data labeling and AI agent evaluation. Choose Praktika if you want to improve your speaking fluency through real-time feedback; choose Potato if you need to annotate text, audio, images, video, or evaluate AI agents for research. They are not direct competitors.
Alternatives to Potato
View all
Deepfabric
Open-source Python framework for generating grounded synthetic datasets from real tool execution traces.
Frequently Asked Questions
Categories
Used Potato? Help shape our editorial sentiment research.