Guidance

When to pick this: Choose Guidance when you need guaranteed output format (e.g., JSON, code, enumerated choices) from an LLM, especially in production pipelines where reliability matters. It's excellent for data extraction, tool-use agents, and multi-step reasoning where intermediate results must follow a schema. When to pass: Skip Guidance if you only need free-form chat completion with no structure constraints—simple prompting or OpenAI's chat API will be simpler. Also avoid if you cannot install Python packages or are using a backend without full Guidance support (some older local models may not fully respect constraints). Comparison to closest alternative: vs. LMQL or SGLang, Guidance is more mature (21.5k stars) and focuses on a simple Pythonic API with inline constraint syntax. Its @guidance decorator for custom functions is unique. However, LMQL offers more sophisticated query optimization. Real-world usage caveats: Constrained generation may increase token usage for some grammars; always test with your model. Offline debugging with Mock is a huge time-saver. The tool is under active development—check for breaking changes when upgrading. Overall, Guidance is a serious tool for serious LLM engineering.

Guidance is an efficient programming paradigm for controlling and steering large language models (LLMs). It allows users to constrain generation using regex and context-free grammars (CFGs), interleave control logic (conditionals, loops, tool use) with generation seamlessly, and capture structured output. This approach reduces latency and cost compared to conventional prompting or fine-tuning. The tool provides a Pythonic interface that integrates with multiple backends including Transformers, llama.cpp, and OpenAI. Key features include constrained generation via regex or select(), offline grammar debugging with a Mock model, and the ability to create custom guidance functions using the @guidance decorator. Guidance is ideal for developers and researchers who need reliable, structured output from LLMs for applications like data extraction, form filling, code generation, and chatbot development. Unlike simple prompting frameworks, Guidance enforces output structure at the grammar level, ensuring correctness even with weaker models. It is an open-source library with an MIT license, available on GitHub and PyPI.