VentureBeat
Follow
Self-improving language models are becoming reality with MIT's updated SEAL technique
MIT researchers developed SEAL, a novel technique allowing large language models like those used by ChatGPT to improve themselves. SEAL enables LLMs to generate synthetic data and formulate their own fine-tuning strategies, achieving self-adaptation. This method, unlike traditional models, doesn't rely solely on external data and human-crafted processes. The expanded paper and open-source code released last month have garnered significant attention in the AI community. SEAL is structured with two loops: an inner loop fine-tunes using self-generated edits, and an outer loop uses reinforcement learning to optimize the edit-generation policy. Performance was assessed across knowledge incorporation and few-shot learning tasks, showing significant improvements in accuracy. The technology helps models to restructure knowledge before assimilation, similar to human learning processes. While achieving strong results, challenges include potential catastrophic forgetting and computational overhead during fine-tuning. Nevertheless, SEAL’s ability to create high-utility training data and generalize across different scenarios is promising. The researchers envision applications in self-pretraining and developing more agentic AI systems. This work represents a step toward autonomous LLM evolution, potentially addressing data limitations and leading to improvements.
