“The Big Short” investor Michael Burry has argued that the growing use of AI-generated training data could amplify errors while exposing a deeper limitation of large language models on the path toward artificial general intelligence.
Michael Burry Warns of AI Model Collapse
On Sunday, Burry took to social media platform X to warn about the consequences of increasingly recursive AI-generated content.
"This is to my point about compression being inevitable as human knowledge is too small for what we are building," Burry wrote, arguing that humans’ repetitive needs and questions could further compress the information available to AI systems.
He added that AI-generated content could contain "propagation errors," similar to errors that have accumulated throughout human knowledge, but said LLMs could reproduce them "infinitely faster" and with less ability to self-correct.
Burry made the comments while sharing an X post discussing a 2024 Nature study on model collapse.
Nature Study Raises Synthetic Data Concerns
Researchers from the University of Oxford and University of Cambridge found that repeatedly training generative AI models on data produced by earlier generations can cause "model collapse," with models progressively losing information from the tails of the original data distribution.
The researchers observed the phenomenon across LLMs, variational autoencoders and Gaussian mixture models.
The study found that preserving access to genuine human-generated data becomes increasingly important as AI-generated material spreads across the internet.
Burry Questions LLMs’ Path to AGI
Burry also made a broader argument about artificial general intelligence, writing that "LLMs cannot attain understanding (AGI)" because "understanding cannot exist unless reason first exists without language."
"A likely impossibility for a language model," he wrote.
He suggested that research is already exploring ways around this limitation, while acknowledging that many have not accepted his premise.
Jensen Huang’s AGI Claim Draws Pushback
Earlier this month, Nvidia Corp (NASDAQ:NVDA) CEO Jensen Huang said AGI has arrived, pointing to OpenAI’s GPT-6 Astra as evidence. However, he received some pushback.
Stanford’s Institute for Human-Centered AI defines AGI as AI capable of learning, reasoning and applying knowledge across a broad range of tasks at or beyond human-level performance.
Unlike narrow AI, AGI could adapt to unfamiliar situations. However, the concept remains debated, with no universally accepted definition or test, alongside ongoing safety and ethical concerns.
Disclaimer: This content was partially produced with the help of AI tools and was reviewed and published by Benzinga editors.
Photo courtesy: Shutterstock
Login to comment