Beyond Static Ranks: The Power of Dynamic Quantization in LLM Fine-Tuning

QDyLoRA enables efficient, multi-rank LLM fine-tuning in one session by combining quantization with dynamic adaptation, beating standard QLoRA