Fast Company
Follow
The AI industry’s massive bet on transformer models may not be enough for true AGI
Major AI companies are heavily investing in transformer models, believing they will lead to human-level general intelligence. However, some experts, like Ben Goertzel, question this approach, arguing that current large language models are too similar and that scale alone may not be sufficient. Transformer models require immense computational resources for training and operation, and the increasing costs might soon outweigh the benefits.Goertzel also points out that these models lack the ability for real-time, continuous learning like humans, reverting to baseline parameters with each new interaction. While some researchers are exploring alternative neural network architectures for continuous learning, the industry's focus remains largely on refining existing methods. Despite this, Goertzel is optimistic that AGI could emerge soon, but it will likely require advancements beyond simply scaling current models.In other AI news, startup Sakana AI has launched Sakana Fugu, a system that orchestrates multiple frontier AI models to work collaboratively on complex tasks. This multi-agent system aims to autonomously assign subtasks to the most suitable models and includes a looping mechanism to overcome impasses. Sakana AI claims Fugu outperforms comparable systems on benchmarks for software engineering and scientific reasoning.Meanwhile, Peter Thiel is backing Objection AI, a startup that offers AI-assisted investigations to fact-check media claims. For a fee, the company analyzes information and produces a judgment that, while not legally binding, can be used as a reputational defense tool. Critics, however, worry that Objection AI could stifle investigative reporting by pressuring journalists and discouraging whistleblowers, potentially serving as a tool to discredit reporters and intimidate sources.