Tokf
Open-source Rust CLI that filters command output before it hits your LLM
tokf is a genuinely useful free tool for heavy CLI+LLM users. It tackles a real problem—token bloat from noisy command output—with a clean, privacy-first design. The 63 built-in filters cover the most common commands, and the RAC test confirms dramatic token savings. v0.2.52 now rejects nondeterministic filters at publish time, and the ongoing changelog shows active maintenance. It's not for everyone: you need terminal comfort and willingness to tweak TOML filters. If you live in Claude Code or
Verified 2d ago · liveness 76/100 · cite: rightaichoice.com/tools/tokf
- Developers using Claude Code or Copilot for daily coding
- Teams wanting to reduce LLM token costs from CLI output
- Users of task runners (make, just, mise) who integrate LLM assistants
- Privacy-conscious developers needing offline AI tooling
- Non-developers or those not using CLI-based AI assistants
- Users who do not work with LLMs or AI coding tools
- Teams needing a GUI or web interface
We scan live Reddit threads, YouTube comments, X posts, G2 reviews and other communities — and hand you an honest verdict in under a minute.
- Honest verdict, not marketing
- Real pros & cons from real users
- Attributed quotes with receipts
3 free scans · no card needed
Skip tokf if you don't use CLI-based AI coding assistants or aren't comfortable working in a terminal.
No hidden costs—tokf is fully free and open source with no paid tiers, but you may need to invest time learning TOML syntax to write custom filters.
tokf is free and open source (MIT), which makes it a no-brainer for individual developers and small teams. Compared to token-optimization services that charge per token or per seat, tokf saves you money on every LLM API call by reducing input tokens. There's no cheaper option—it's $0.
In short
Tokf — Open-source Rust CLI that filters command output before it hits your LLM. Best for Developers using Claude Code or Copilot for daily coding, Teams wanting to reduce LLM token costs from CLI output, Users of task runners (make, just, mise) who integrate LLM assistants. Free to use.
What's new in Tokf
Checked 8 days agoAcross the latest 5 updates: 5 changelog entries.
v0.2.52 — reject nondeterministic filters at publish time
Rejects nondeterministic filters at publish time; makes run overrides honest about what was executed; propagates --no-mask-exit-code into hook-generated rewrites.
v0.2.51 — rarity-weighted richness metric
Adds rarity-weighted richness metric to tokf verify; annotates filtered output with history entry ID; asserts byte-stable filter output; recalibrates token estimator.
v0.2.50 — unwraps local environment wrappers
Unwraps local environment wrappers such as nix develop -c.
v0.2.49 — fixes multibyte UTF-8 compound rewrites
Fixes multibyte UTF-8 compound rewrites.
v0.2.48 — preserves newline in nested-list segments
Preserves newline after operator in nested-list segments.
What people actually say about Tokf — is it worth it?
We ran a structured research pass across product reviews, community discussions, and post-purchase forum threads to surface the patterns vendors won't publish themselves. Below: the recurring strengths, the hidden costs people mention most, and the cohort that consistently regrets adopting this tool.
41 mentions across 4 sources (Hacker News, YouTube, GitHub, Lemmy) · researched Aug 25, 2026.
- +Achieves dramatic token reduction, up to 98% on verbose outputs.
- +Runs fully local with no telemetry, respecting privacy.
- +Automatic git hook integration simplifies setup for git workflows.
- +Transparently wraps make, just, and mise task runners.
- +Includes 63 built-in command patterns for popular tools.
- −Early-stage project with few stars and open issues.
- −Requires learning TOML filter syntax and Luau for advanced use.
- −Filter maintenance could become tedious as outputs evolve.
- −Potential to filter out critical warning signs if misconfigured.
- −Lack of extensive community validation and user reviews.
- • Time investment to write and maintain custom filters.
- • Potential need to compile from source for some platforms.
Viability Score
How well maintained and how widely used is Tokf? Built from what the vendor actually publishes (docs, changelog, tutorials, integrations, pricing), whether the site is live, and how much real users discuss it. How we calculate this
Last calculated: September 2026
How we score →Key Features
- TOML-based filter configuration
- Automatic git hook integration
- Transparent wrapper for make, just, mise
- 63 built-in command patterns
- Luau scripting escape hatch
- JSONPath (RFC 9535) extraction
- Optional cloud sync (opt-in)
- Full offline / air-gapped operation
- tokf doctor subcommand
- tokf discover for missed savings
- Reduction statistics & compression indicator
- Telemetry status subcommand
- OpenTelemetry OTLP metrics exporter
- Codex PreToolUse integration
- Multibyte UTF-8 compound rewrite support
About Tokf
tokf is an open-source, Rust-based command-line tool that intercepts the output of terminal commands and runs it through configurable TOML filters before the data reaches your AI coding assistant. It's built for developers who use Claude Code, Copilot, Cursor, Aider, Gemini CLI, or Codex and want to cut token waste from verbose compiler logs and repetitive test output. For a 1.7 KB cargo test output, tokf can reduce it to 35 bytes—a 98% cut. Setup is one-time: tokf automatically installs git hooks so every commit, push, and pull is filtered, and it transparently wraps make, just, and mise targets. Filter files in `.tokf/filters/` are version-controlled and reviewable. For logic TOML can't express, tokf includes a sandboxed Luau scripting escape hatch. The tool ships with 63 built-in command patterns covering git, cargo, docker, npm, pnpm, go, GitHub CLI, kubectl, Next.js, TypeScript, pytest, and more. AI agents can even author filters for you via `tokf skill install`, including a PreToolUse hook for Claude Code real-time filtering. Privacy is core: everything runs 100% locally by default—no telemetry, no network requests, no account. Optional cloud sync is inert until you run `tokf auth login`. Recent releases add Codex PreToolUse integration, a `tokf doctor` subcommand to detect problematic filters, `tokf discover` to find missed token savings, and a generic fallback command. tokf differs from prompt compression libraries by working at the source, keeping the original command intact while eliminating noise—a more efficient first-line defense against token waste.
Behind the Verdict
tokf earns its place in the toolkit of any developer who has watched half their LLM context window vanish into a wall of cargo warnings. The premise is simple—intercept command output, filter it, hand the model only the signal—and the execution is refreshingly honest. The one-time setup of git hooks and task runner wrapping means you don't have to remember to prefix every command; tokf quietly works in the background. The 98% reduction on a cargo test run isn't just marketing: the math is transparent on the homepage. Where tokf really shines is in its privacy posture. Everything is local by default, with no telemetry and no network requests. The optional cloud sync is inert until you explicitly log in, which is a rarity in AI tooling. For anyone who works in air-gapped environments or simply distrusts cloud dependencies, that's a decisive advantage. But this tool isn't for everyone. It demands terminal comfort and a willingness to tweak TOML filters. If you're not using CLI-based AI assistants like Claude Code or Copilot, you won't get much value. And if your command output is already minimal—simple echo, ls, or basic git status—tokf is overkill. The learning curve for authoring custom filters, especially with the Luau escape hatch, is real. Compared to prompt compression libraries that shrink your entire context window, tokf attacks the problem at the source. That's a smarter approach: it doesn't compress the signal, it removes the noise. The built-in library of 63 patterns means you get value immediately, and the active changelog—recently adding Codex PreToolUse support and a doctor subcommand—shows the project is evolving. In practice, we'd reach for tokf when our main LLM assistant is a CLI tool and our daily workflow involves build systems, tests, or container
Researching Tokf? Get your full AI stack in 60 seconds.
Free, no signup — tell us your goal and get tools matched to your budget & existing stack.
Real-world workflow fit
Concrete scenarios for the personas Tokf actually fits — and what changes day-one when you adopt it.
Running cargo build in a large Rust project, output floods Claude Code context.
Outcome: tokf compresses the output from 1.7 KB to 35 bytes, letting Claude focus on actual errors and reducing token usage.
Team runs heavy test suites with verbose output, racking up LLM token bills.
Outcome: tokf automatically filters git hooks and task runner output, cutting token consumption by up to 98% across the team.
You want to use AI assistants but don't want terminal output sent to external services.
Outcome: tokf processes everything locally with no telemetry or phone-home, so your data never leaves your machine.
Use Cases
- Cut token usage by compressing cargo build output before sending to Claude Code
- Automatically strip noise from git diff output when using Copilot inline chat
- Set up git hooks to filter output from every commit and push command
- Wrap make targets so build logs are condensed before reaching an LLM
- Use tokf doctor to identify filters that confuse AI agents
- Share custom filter sets across a team via optional cloud sync
- Use tokf discover to find unfiltered commands in past AI sessions and apply filters retroactively
Limitations
- tokf is a CLI tool that filters command output for LLMs; it requires comfort with terminals and command-line workflows.
- Cloud sync is optional and fully inert until explicit login.
- Advanced filtering may require knowledge of TOML and, for complex logic, Luau scripting.
- The tool is in early versioning (v0.2.x) and while stable, features may evolve.
as of 2026-08-25
Verification history
We have re-verified Tokf 8 times since . Each pass re-reads the vendor's own pages and re-checks every listed field against that evidence; passes where nothing had changed are marked as such.
- — re-verified summary, description, our verdict, our analysis, pricing model, pricing tiers, features, integrations, who it suits, who should skip it
- — re-verified summary, description, our verdict, our analysis, pricing model, pricing tiers, features, integrations, who it suits, who should skip it
- — re-checked, vendor evidence unchanged
- — re-checked, vendor evidence unchanged
- — re-verified summary, description, our verdict, our analysis, pricing model, pricing tiers, features, integrations, who it suits, who should skip it
- — re-verified summary, description, our verdict, our analysis, pricing model, pricing tiers, features, integrations, who it suits, who should skip it
Showing the 6 most recent of 8 verification passes.
Free to cite with attribution — this page re-verifies continuously.
12-month cost
Project the real annual outlay, including the implied monthly cost when only an annual tier is published.
Vendor list price only. Add-on usage, seat overages, and contract minimums are surfaced under Hidden costs & gotchas.
Plans compared
For each published Tokf tier: who it actually fits, and what it adds vs. the previous tier. Cross-reference the cost calculator above for projected annual outlay.
Open Source
$0/mo
Ideal for
Any developer or team using CLI-based AI assistants who wants to cut token usage at no cost.
What this tier adds
Free entry point with all features—no paid tiers, no premium filters, no feature gating.
Where the pricing makes sense
The company stage and team size where Tokf's pricing actually pencils out — and where peers do it cheaper.
tokf is free and open source (MIT), which makes it a no-brainer for individual developers and small teams. Compared to token-optimization services that charge per token or per seat, tokf saves you money on every LLM API call by reducing input tokens. There's no cheaper option—it's $0.
Setup time & first value
How long it actually takes to get something useful out of Tokf — broken out by persona, not the marketing-page minute.
For a single developer using Claude Code with cargo, expect under 5 minutes to install via Homebrew or Cargo, run `tokf setup` for the interactive wizard, and have your first git hooks active. For teams, add time for reviewing and versioning the .tokf/filters directory (30-60 minutes).
Switching to or from Tokf
How to bring data in from common predecessors and how to get it back out — written for the switcher, not the buyer.
- →From manual grep/sed pipelines: Replace ad-hoc output cleaning with tokf's declarative TOML filters—no more fragile shell scripts.
- →From no filtering: Install tokf and enable git hooks to automatically filter all commit/push output with zero changes to your workflow.
- ↗To a different CLI filter: Export your .tokf/filters as reference, but note the TOML format is specific to tokf.
- ↗To manual filtering: If you stop using tokf, you'd need to re-implement output scrubbing in your own scripts or rely on the AI tool's context management.
Integrations
Resources & Guides
- Documentationtokf.net
Docs · Tokf
Full product docs from tokf.net
- Quickstarttokf.net
Getting Started · Tokf
Get up and running fast from tokf.net
- Documentationtokf.net
Writing Filters · Tokf
Full product docs from tokf.net
- Documentationtokf.net
Lua Escape Hatch · Tokf
Full product docs from tokf.net
- Documentationtokf.net
Token Tracking · Tokf
Full product docs from tokf.net
Tutorials & Learning
Official links
Tools that pair well with Tokf
Common stack mates teams adopt alongside Tokf, with the specific reason each pairing earns its keep.
Featured Head-to-Head Comparisons
Tokf vs Spider Cloud
Choose Spider Cloud if you need real-time, structured web data for AI agents or RAG pipelines, especially with its new browser AI commands and 1000+ scrapers. Choose Tokf if you're a developer using Claude Code or Copilot and want to slash token costs from CLI output—it's free, offline, and can reduce payload by 98%. They solve completely different problems, so your decision depends on whether you need to gather web data or compress command output.
Tokf vs Voyage Ai
Voyage AI and tokf address completely different needs: Voyage AI improves retrieval accuracy in RAG pipelines with fine-tuned embeddings and rerankers, while tokf reduces token costs by compressing CLI output before it reaches an LLM assistant. Choose Voyage AI if you're building enterprise RAG on specialized domains; choose tokf if you're a developer wanting to cut token waste from command output in your AI coding workflow.
Tokf vs Temporal Ai
Temporal AI is the right choice for teams building durable, failure-resistant AI agents and orchestrating complex workflows with retries and rollbacks. Tokf is a niche but powerful cost-saving tool for developers who feed terminal output into LLMs. Choose Temporal for orchestration, Tokf for token efficiency.
Alternatives to Tokf
View allFrequently Asked Questions
Best-of guides
Used Tokf? Help shape our editorial sentiment research.