Hugging Face has published a technical deep dive into implementing asynchronous Gradient Reversal Policy Optimization (GRPO) using LoRA across its job infrastructure. This approach demonstrates a method for distributed machine learning that bypasses the need for NCCL, leveraging a bucket and proxy system for efficiency.
Source: Hugging Face