Nvidia’s Llama-3.1-Minitron 4B is a small language model that punches above its weight

Nvidia researchers used model pruning and distillation to create a small language model (SLM) at a fraction of the base cost.