Конвертация и квантизация моделей в GGUF

★ 7.7 · ai-tooling

gguf is a Claude Code skill that guides converting and quantizing language models into the GGUF format for efficient llama.cpp inference on CPUs, Apple Silicon, and consumer hardware without mandatory GPU requirements. It covers the full pipeline: converting HuggingFace models with convert_hf_to_gguf.py, quantizing to types from Q2_K through Q8_0 using K-quant methods, generating an importance matrix via llama-imatrix for better low-bit quality, and running inference through the CLI or llama-cpp-python bindings. Q4_K_M is the recommended default, balancing file size and output quality; Q8_0 preserves near-original accuracy. The resulting GGUF files work directly with LM Studio, Ollama, koboldcpp, and text-generation-webui, making this skill ideal for developers deploying local LLMs on laptops, desktops, or M1/M2/M3 Macs.