VentureBeat
Follow
AI21’s Jamba Reasoning 3B Redefines What “Small” Means in LLMs — 250K Context on a Laptop
AI21 Labs introduces Jamba Reasoning 3B, a "tiny" open-source model designed for enterprise use on devices like laptops and phones. This model can handle extended reasoning, code generation, and ground-truth-based responses, handling over 250,000 tokens. AI21 sees small models as crucial for enterprises, reducing data center load by shifting inference to devices, addressing expensive data center costs. Jamba Reasoning 3B combines Mamba and Transformers, enabling a large context window and faster inference speeds, tested at 35 tokens per second on a MacBook Pro. The hybrid architecture also minimizes memory requirements, improving computing efficiency. The model excels at tasks like function calling and policy-grounded generation, making it suitable for simpler requests. Jamba Reasoning 3B’s performance surpasses other small models in benchmarks such as IFBench and Humanity's Last Exam. Compared to other models, such as Qwen 4B and Llama 3.2B-3B, Jamba Reasoning 3B offers superior steerability and enhanced privacy for enterprises, as inference stays local. Enterprises are increasingly adopting small models, with competitors like Meta, Google, and FICO also releasing their own focused models. AI21's co-CEO believes that optimizing for customer experience through on-device models will become a mainstream trend.
