Low-Rank Adaptation

FLoCoRA: Federated Learning Compression With Low-Rank Adaptation

Low-Rank Adaptation (LoRA) methods have gained popularity in efficient parameter fine-tuning of models containing hundreds of billions of parameters. In this work, instead, we demonstrate the application of LoRA methods to train small-vision models …