TRL — дообучение LLM с RLHF и DPO

★ 7.7 · ai-tooling

trl-fine-tuning is a Claude Code skill that provides a complete post-training pipeline for large language models using HuggingFace's TRL library. It covers four training methods: SFT for instruction tuning, DPO for preference alignment without a separate reward model, PPO and GRPO for reward-based policy optimization, and standalone reward model training. The full RLHF workflow follows a three-stage sequence — SFT → reward model → PPO — producing a model aligned with human feedback. Required dependencies are trl, transformers, datasets, peft, accelerate, and torch; code examples use Qwen2.5-0.5B as the base model. Aimed at ML engineers and researchers who need to align a language model with human preferences or fine-tune it for instruction following.