Potato

Potato is a free, open-source annotation platform developed at the University of Michigan. It supports text, audio, images, video, and AI agent traces, enabling researchers and teams to quickly set up annotation tasks without coding—everything is configured in YAML. The tool offers over 30 annotation types, including classification, span labeling, bounding boxes, polygons, Likert scales, pairwise comparisons, and qualitative coding (QDA Mode) with living codebooks and memos. AI assistance integrates with OpenAI, Claude, Gemini, and local LLMs via Ollama for intelligent hints, label suggestions, and keyword highlighting. Active learning leverages five query strategies: uncertainty, diversity, BADGE, BALD, and hybrid ensemble. For agent evaluation, Potato can import traces from LangChain, CrewAI, AutoGen, and OpenTelemetry, visualize agent actions on an interactive graph, and allow annotators to edit trajectories into SFT and DPO training data. It also supports LLM-as-judge calibration against human labels, model arena comparisons, and CI gating. Potato is licensed under GPL-3.0-or-later and received Best Demo at HCOMP 2024 and was featured at ACL 2026. Compared to commercial alternatives like Labelbox or Prodigy, Potato offers a free, self-hostable solution with unique agent evaluation capabilities, but requires technical ability to deploy.