SimPO — выравнивание LLM без эталонной модели

★ 7.7 · ai-tooling

simpo is a Claude Code skill that implements Simple Preference Optimization — a reference-free LLM alignment method that outperforms DPO by 6.4 points on AlpacaEval 2.0 without requiring a separate reference model. The skill covers the full training pipeline: ready-made configs for Mistral 7B, Llama 3 8B, and DeepSeek Math, Accelerate launches with DeepSpeed ZeRO-3, tuning of beta and gamma_beta_ratio hyperparameters, and optional SFT regularization to prevent capability forgetting. Dependencies include torch, transformers, trl, accelerate, and Flash Attention 2; an NVIDIA A100 40GB or better is recommended. It targets ML engineers who need preference alignment training from chosen/rejected pairs with less complexity and compute than DPO or PPO, on a single-node setup. The underlying method was presented at NeurIPS 2024.

• #preference-optimization
• #llm-alignment
• #post-training
• #dpo-alternative
• #efficient-training