bitsandbytes — квантизация LLM до 4 и 8 бит

★ 7.7 · ai-tooling

bitsandbytes is a Claude Code skill that quantizes large language models to 8-bit or 4-bit precision, reducing GPU memory usage by 50–75% with less than 1% accuracy loss. It supports INT8, NF4, and FP4 formats, QLoRA fine-tuning on consumer GPUs, and 8-bit Adam/AdamW optimizers that cut optimizer memory by up to 75%. The skill integrates with HuggingFace Transformers via BitsAndBytesConfig and relies on bitsandbytes, accelerate, and torch — a single config object is enough to load, for example, Llama-2-7B in 3.5 GB instead of 14 GB. Ideal for engineers who need to run or fine-tune billion-parameter models on hardware with limited VRAM.