Forget ChatGPT: Why Researchers Now Run Small AIs On Their Laptops

A bioinformatician uses AI to generate readable summaries for his database of immune-system protein structures by running the AI on his Mac, rather than using web-based LLMs like ChatGPT. Two recent trends have emerged: organizations making 'open weights' versions of LLMs publicly available for local use, and technology firms creating scaled-down versions that can run on consumer hardware. These tools allow researchers to save money, protect confidentiality, and ensure reproducibility. As computers get faster and models become more efficient, people will increasingly have AIs running on their laptops or mobile devices. Several small open-weights models are available, including Meta's Llama, Google DeepMind's Gemma, and Microsoft's Phi models. Microsoft has released multiple versions of its Phi models, which have between 3.8 billion and 14 billion active parameters and can handle images. Even the smallest Phi model outperforms OpenAI's GPT-3.5 Turbo in some benchmarks. Running LLMs locally also allows researchers to control reproducibility, as they don't have to worry about companies updating their models. Researchers can build on these tools to create custom applications, and local LLMs should soon be good enough for most applications. The rate of progress in local LLMs has been astounding, and users are encouraged to experiment and explore the possibilities.