quant.cpp — LLM на CPU без GPU и зависимостей

★ 6.7 · vibe-coding

score is a Claude Code skill for working with quant.cpp, an open-source C library that runs large language models on consumer hardware with zero external dependencies. It achieves 6.4× KV-cache compression at under 3% perplexity cost, enabling 128K-token context on a 16 GB Mac — measured at 9.5 GB RSS for Llama 3.2 3B. After `pip install quantcpp`, the CLI offers `pull`, `run`, `serve`, and `client` subcommands; the server exposes an OpenAI-compatible `POST /v1/chat/completions` endpoint with SSE streaming, and the Python API requires just three lines. Benchmarks show full-document inference scoring 7/7 versus 0/7 for chunk-RAG when the wrong section is retrieved, eliminating hallucinations caused by mis-retrieved chunks. The skill targets developers who need local, GPU-free LLM inference and want to replace RAG pipelines with whole-document understanding.