Features
Architecture-specific comparison (Ampere to Hopper)
Performance bottleneck identification
GitHub pull request review for kernel code
Research articles and notes on GPU execution
Fast & lossless inference for 1-bit LLMs (BitNet b1.58)
Optimized CPU kernels for ARM & x86 architectures
Official GPU inference kernel (released 05/2025)
Parallel kernel implementations with configurable tiling
Embedding quantization for 1.15x–2.1x additional speedup
1.37x–6.17x CPU speedup vs baseline
55%–82% CPU energy reduction
Run 100B BitNet b1.58 on single CPU (5-7 tok/s)
Lookup Table kernels built on T-MAC methodologies
Support for Hugging Face 1-bit models
Conda environment setup script (setup_env.py)
Inference server (run_inference_server.py)
Lossless inference—no accuracy degradation
Falcon3 family and Llama3-8B-1.58 model support
Supports models up to 100B parameters
