Lmql

What LMQL actually is: a query language and runtime for LLM prompting, created by the SRI Lab at ETH Zurich and contributors. It is not an orchestration framework with a plugin marketplace; it is a language with its own semantics. Strengths. First, constraints are real. A where clause such as len(RESPONSE) < 10 or STOPS_AT(ANSWER, ".") is translated into subtoken masks and enforced during generation, so you are restricting the decoder, not asking politely in the prompt. Second, typed variables: [NUM: int] gives you an int, and regex constraints give you a concrete output format like DD/MM/YYYY. Third, scripted prompting — the program body is Python, so loops (for i in range(4)) and branching (if "test" not in RESPONSE) control prompt construction, which is how the packing list example generates four list items from a fixed THING set. Fourth, nested queries, the feature the site leads with: modularized local instructions and reusable prompt components, brought to prompting as procedural programming. Fifth, portability — the site states LMQL makes your code portable across backends and switchable with one line: llama.cpp, OpenAI, Azure, Replicate and Hugging Face Transformers. Weaker spots. The constraint semantics and the language both have a learning curve; the documentation itself is a prerequisite read, not optional. Constraint enforcement has runtime cost, so complex constraints can affect generation speed, and performance varies by backend. LMQL is also a smaller, research-originated project — it offers integration pages for LangChain and LlamaIndex, but the wider enterprise tooling layer is thinner than in a managed platform. The web Playground IDE is for experimentation; self-hosted models via Transformers or llama.cpp require a local install. If you only need single-turn prompting without output control, LMQL is more machinery than the job requires. Where it fits: teams that treat prompts as code and want verifiable generations, researchers experimenting with constrained decoding and decoders like sample, beam and best_k, and anyone who needs to move the same query between a local llama.cpp model and a hosted API. Not for teams wanting a managed service, and not for non-programmers.