Async GRPO with LoRA across HF Jobs: a bucket, a proxy, and no NCCL

Hugging Face just figured out how to run DeepSeek-style RL (GRPO) using LoRA across standard cloud jobs without needing expensive, specialized hardware interconnects. It swaps out heavy NCCL dependencies for a simple bucket and a proxy.

This lowers the barrier for teams to train reasoning models. You no longer need a massive, specialized GPU cluster with perfect networking to experiment with the latest RL techniques.

It shifts the advantage from well-funded labs with InfiniBand clusters to agile startups using standard cloud compute. This opens up the reasoning model race.

• →Agile labs can iterate on reasoning capabilities like DeepSeek-R1 using much cheaper, fragmented compute.
• →Vertical AI startups can fine-tune specialized reasoning models on niche datasets without a massive infra team.
• →Hugging Face solidifies its position as the default factory for the model training lifecycle, not just a model repo.

• →Performance overhead: Using a bucket and proxy instead of NCCL is inherently slower, so there is a ceiling on how fast this can scale.
• →Dependency lock-in: Teams might become overly reliant on the Hugging Face Jobs ecosystem to manage these complex async workflows.