Очередь ML-экспериментов на удалённых GPU
★ 7.1 · orchestration
experiment-queue is a Claude Code skill that manages an SSH job queue for multi-seed and multi-config ML experiments on remote GPU servers, with full state tracking, OOM-aware retry, stale screen cleanup, and wave-transition race prevention. It runs its own detached server-side scheduler polling every 60 seconds — wrapping it in an additional cron or loop would duplicate the scheduler and break wave-transition logic. Input can be a YAML manifest, a Cartesian grid spec, or a plain-language description; the skill builds the job list and distributes work across specified GPUs while enforcing preconditions such as teacher checkpoints existing before student distillation starts. Each job moves through a state machine: pending → running → completed / failed_oom (retried up to N times after a configurable delay) / stuck. The right fit is batches of 10 or more jobs, sweeps like 21 seeds × 12 cells, or teacher-then-student chains where /run-experiment falls short.
- #experiment-orchestration
- #job-queue
- #multi-seed-sweep
- #gpu-scheduling
- #ml-batch-processing