This story is from The Algorithm, our weekly newsletter on AI. To get stories like this in your inbox first, sign up here.
Ever since last week’s dramatic events at OpenAI, the rumor mill has been in overdrive about why the company’s chief scientific officer Ilya Sutskever and its board decided to oust CEO Sam Altman.
While we still don’t know all the details, there have been reports that researchers at OpenAI had made a “breakthrough” in AI that had alarmed staff members. Reuters and The Information both report that researchers had come up with a new way to make powerful AI systems and had created a new model, called Q* (pronounced Q star), that was able to perform grade-school level math. According to the people who spoke to Reuters, some at OpenAI believe this could be a breakthrough in the company’s quest to build artificial general intelligence, a much-hyped concept of an AI system that is smarter than humans. The company declined to comment on Q*.
Social media is full of speculation and excessive hype, so I called some experts to find out how big of a deal any breakthrough in math and AI would really be.
Researchers have for years tried to get AI models to solve math problems. Language models like ChatGPT and GPT-4 can do some math, but not very well or reliably. We currently don’t have the algorithms or even the right architectures to be able to solve math problems reliably using AI, says Wenda Li, an AI lecturer at the University of Edinburgh. Deep learning and transformers, a kind of neural network, which is what language models use, are excellent at recognizing patterns, but that alone is likely not enough, Li adds.
Math is a benchmark for reasoning, Li says. A machine that is able to reason about mathematics, could, in theory, be able to learn to do other tasks from existing information, such as write computer code, or draw conclusions from a news article. Math is a particularly hard problem to solve because it requires AI models to have the capacity to reason and for models to really understand what they are dealing with.
A generative AI system that could reliably do math would need to have a really firm grasp on concrete definitions of particular concepts that can get very abstract. A lot of math problems also require some level of planning over multiple steps, says Katie Collins, a PhD researcher at the University of Cambridge, who specializes in math and AI. Indeed, Yann LeCunn, Chief AI scientist at Meta, posted on X and Linkedin over the weekend that he thinks Q* is likely to be “OpenAI attempts at planning”.
People who worry about AI posing an existential risk to humans, one of OpenAI’s founding concerns, fear that such capabilities might lead to rogue AI. Safety concerns might arise if such AI systems are allowed to set their own goals and start to interface with a real physical world or digital world in some ways, says Collins.
But while math and AI might take us a step closer to more powerful AI systems, solving these sorts of math problems doesn’t signal the birth of a superintelligence.
“I don’t think it immediately gets us to AGI or scary situations,” says Collins. It’s also very important to underline what kind of math problems AI is solving, she adds.
“Solving elementary school math problems is very, very different from pushing the boundaries of mathematics at the level of something a Fields medalist [a top prize in mathematics] can do,” says Collins.
Machine learning research has focused on solving elementary school problems, but it’s a problem state-of-the-art AI systems haven’t fully cracked yet. Some AI models fail on really simple math problems, but then they can excel at really hard math problems, Collins says. OpenAI has, for example, developed dedicated AI tools that can solve challenging high-school olympiad problems, but these systems outperform humans only occasionally.
Nevertheless, building an AI system that can solve math equations is a cool development, if that is indeed what Q* can do. A deeper understanding of mathematics could open up applications to help scientific research and engineering, for example. The ability to generate mathematical responses could help us develop better personalized tutoring, or help mathematicians do algebra quicker or solve more complicated problems.
This is also not the first time a new model has sparked AGI hype. Just last year, tech folks were saying the same things about Google DeepMind’s Gato, a “generalist” AI model that can play Atari video games, caption images, chat, and stack blocks with a real robot arm. Back then, some AI researchers claimed that DeepMind was “on the verge” of AGI, because of its ability to do so many different things pretty well. Same hype machine, different AI lab.
And while it might be great PR, these hype cycles do more harm than good for the entire field by distracting people from the real, tangible harms and problems around AI. Rumors about a powerful new AI model might also be a massive own goal for the regulation-averse tech sector. The EU, for example, is very close to finalizing its sweeping AI Act. One of the biggest fights right now among lawmakers is whether to give tech companies more power to self-regulate cutting-edge AI models.
OpenAI’s board was designed as the company’s internal kill switch and governance mechanism to prevent the launch of harmful technologies. The past week’s boardroom drama has shown how the bottom line will always prevail at these companies. It will also make it harder to make a case for why these companies should be trusted with self-regulation. Lawmakers, take note.