RWKV — работа с гибридной RNN-трансформер моделью

★ 7.7 · ai-tooling

rwkv is a Claude Code skill that covers working with the RWKV (Receptance Weighted Key Value) architecture — an RNN and Transformer hybrid delivering O(n) linear inference complexity, infinite context length, and no KV cache. The skill walks through installation via pip with CUDA support, loading models in both GPT mode (parallel processing) and RNN mode (sequential processing), streaming token-by-token generation, and processing sequences of one million tokens or more with constant O(1) memory usage. Fine-tuning workflows using PyTorch Lightning and DeepSpeed are also included. RWKV is a Linux Foundation AI project used in production at Windows, Office, and NVIDIA NeMo; the latest RWKV-7 release (March 2025) scales up to 14B parameters. It targets developers building memory-constrained deployments, streaming applications, or long-document pipelines where standard Transformer attention becomes impractical.