LoRA / QLoRA

LoRA (Low-Rank Adaptation) — parameter-efficient fine-tuning technique that injects trainable low-rank matrices into frozen model weights, drastically reducing GPU memory requirements.

QLoRA — quantised version of LoRA; loads the base model in 4-bit precision, enabling fine-tuning of large models on consumer hardware.

Related

• Fine-Tuning
• Pre-training
• Llama
• Gemma
• vLLM