Nvidia researchers unlock 4-bit LLM training that matches 8-bit performance

Nvidia has introduced NVFP4, a new 4-bit quantization technique for training large language models. NVFP4 addresses the challenges of maintaining accuracy in low-precision formats by employing a sophisticated multi-level scaling approach to handle outlier values. It uses a mixed-precision strategy, quantizing most layers while keeping crucial ones in higher precision, and adjusts gradient calculations to reduce low-precision biases. NVFP4-trained models achieve performance comparable to FP8 models while using half the memory and fewer computations. Testing showed NVFP4 models closely matched FP8 in training loss and task accuracy across various domains. Compared to MXFP4, NVFP4 converges to a better loss score and requires less data to achieve the same level of performance. This efficiency allows for faster inference, higher throughput, and quicker return on investment for AI factories. The technology enables the training of customized, high-performance AI models by a wider range of organizations. NVFP4 proves that precision can be optimized without sacrificing quality, paving the way for more efficient AI design. This opens opportunities for future research into lower precisions and optimized architectures, particularly for agentic systems.