Quick Take: Dissecting AGI Claims About o3
How Appeals to Vibes Muddle our Perception of Technology
Earlier this week I launched Discerning AI and published my first post about skeptically evaluating AI claims. I wasn't planning to post again until next week, but the release of OpenAI's o3 model and the subsequent discourse provide a perfect real-world opportunity to put these principles into practice. Let's apply our skeptical framework to a specific claim making waves across social media right now.
The Context
In case you aren’t following all the AI headlines, yesterday OpenAI released their much anticipated o3 model, marketed as their smartest reasoning model yet. In response, Economics professor, blogger, and X influencer, Tyler Cowen shared his thoughts on his blog, arguing that o3 is Artificial General Intelligence (AGI). What is meant by the term ‘AGI’ is disputed (we’ll get into that more next week!), but it broadly refers to an AI with broad, human-level intelligence across many domains. His claim has been widely shared across social media, with responses ranging from credulity to disbelief.
Dissecting the Argument
Let’s take a closer look at the reasoning step by step.
Framing Definitions
First Cowen is defines “AGI” on his own terms as “an AI that subjectively seems smart enough to be AGI”.
At this point, I’d like to officially coin the term “Appeal to Vibes” a new logical fallacy.
It’s a pattern that's become increasingly common in AI discussions (the term “feel the AGI” has become so popular that it’s being sold in T-shirt form) This reasoning is particularly problematic in technical domains like AI because it substitutes subjective impressions for scientific understandings and objective measurements. When we're discussing complex systems with specific technical capabilities, relying on how something 'feels' during casual interactions makes it easy to anthropomorphize. It undermines rigorous evaluation and makes it harder to have substantive debates about AI capabilities, as everyone's 'vibes' may differ.
More formally, I think this vibes-based definition combines a few logical fallacies:
Psychologists’ Fallacy: The assumption that your subject experience of something is reflective of objective reality. Here we have the assumption that an AI chatbot has human-level intelligence because its behavior subjectively seems that way.
Hasty Generalization: I covered this one in my post early this week. this is when someone makes a broad statement based on very limited, unrepresentative data. Individual interactions with o3 are not sufficient to make broad claims about it’s capabilities.
Rebutting Anticipated Objections
Next he makes some references to securities prices and benchmarks in response to anticipated objections to his definition. For context, OpenAI defines AGI in terms of its economic impact, while many AI researchers define it based on performance against evaluation metrics (benchmarks).
The lack of evidence for AGI in economic indicators is justified by explaining that anticipation of AGI is already reflected in the current economic data. I’m fine with this particular line of reasoning. The claim doesn’t strike me as extraordinary and economics is his field of expertise, so I’m willing to tentatively accept this part of the argument.
He dismisses benchmarks for…reasons. This is basically the definition of Appeal to the Stone Fallacy, in which an argument is dismissed for no reason or because it seems absurd.
Conclusion
He concludes that o3 is AGI because it fits his definition.
Overall I think this argument is a clear-cut example of Definist Fallacy. Cowen is defining “AGI” as “an AI that subjectively seems like AGI”, rejecting other popular definitions, and expects the reader to accept his conclusion that OpenAI’s o3 is AGI.
Refuting the other definitions is not enough to justify his claim since he skips the step of justifying his own assertion. Given that he’s using a non-standard definition for a disputed term, Cowen needs to justify it on it’s own merits.
Final Thoughts
This case demonstrates why the skeptical framework I introduced earlier is so valuable when navigating AI conversations. Identifying specific fallacies like the Definist Fallacy, shows us how shifting definitions can conceal an extraordinary claim and misrepresent real technical work. When someone redefines a technical term like AGI to match their subjective experience, they're not advancing our understanding of the concept, they’re just moving goalposts. The spread of this type of reasoning can directly impact funding priorities, research directions, public perception, and policy decisions around AI. It also makes it harder to understand the real capabilities of a tool, preventing us from implementing it appropriately in real-world use cases.
As we continue exploring AI developments, paying close attention to definitions becomes crucial. Next week, I'll get back on course and dive deeper into the various definitions of Intelligence and AGI. For now, this real-time example shows how easily logical fallacies can shape our perception of technical progress, even among knowledgeable commentators.