verl — обучение LLM методами RL

★ 7.7 · ai-tooling

verl is a Claude Code skill that guides engineers through reinforcement learning post-training of large language models using the verl (Volcano Engine RL) library from ByteDance's Seed team. It covers PPO, GRPO, DAPO, RLOO, REINFORCE++, and SPIN algorithms, with training backends FSDP, FSDP2, and Megatron-LM, and rollout engines vLLM and SGLang. The underlying HybridFlow framework (EuroSys 2025) separates a single-process Ray controller from multi-process workers — ActorRolloutRef, Critic, and RewardManager — enabling flexible backend swapping without rewriting training logic. Supported models range from 0.5B to 671B parameters (Qwen-3, Llama-3.1, DeepSeek, Gemma-2), with additional support for multi-turn tool-call rollouts, LoRA RL, and vision-language model training. The skill suits teams building production RLHF pipelines who need scalable, configurable infrastructure rather than a fixed training stack.

• #reinforcement-learning
• #rlhf
• #grpo
• #ppo
• #distributed-training
• #bytedance
• #post-training