DeepSpeed — распределённое обучение больших моделей

★ 7.7 · engineering

deepspeed is a Claude Code skill that provides expert guidance on distributed training with DeepSpeed, covering ZeRO optimization stages, pipeline parallelism, mixed precision (FP16/BF16/FP8), 1-bit Adam, and sparse attention for scaling large models across multiple GPUs. It also addresses DeepNVMe — a high-performance I/O layer built on NVMe SSDs, Linux Async I/O, and NVIDIA GPUDirect Storage — with blocking, non-blocking, and parallel tensor write patterns using aio_handle and gds_handle abstractions. Dependencies include deepspeed, torch, transformers, and accelerate. The skill is generated from official DeepSpeed documentation and targets ML engineers and researchers optimizing memory usage, training throughput, or debugging DeepSpeed configurations for large-scale model training.