Ben Davis (co-host of one of my favorite new podcasts this year, Nerd Snipe) has a fascinating take on a question I’ve seen many in the Apple community ponder as well: if these frontier models are so good, why do most AI apps from the big labs still kind of suck?
Davis wrote about this in relation to the ChatGPT/Codex desktop app, which has turned into a bit of a kitchen-sink design phenomenon with all kinds of features crammed into a UI that has two left sidebars, multiple modalities, and all sorts of disjointed menus.
To me the most interesting thing here is the implied question of “if the Codex team has unlimited tokens on really powerful unreleased models, why is the product getting worse?”
The lazy answer is “it’s just vibe coded slop, real artisanal code wouldn’t be like this”. Anyone who still says this is a moron and you can ignore them.
“Vibe coding” is not the problem. Frankly, these models are better at writing code than pretty much anyone at this point. You tell the model to make something and it’ll make that thing quite well. And also to be clear, Jira was horrid before LLMs existed. Good and bad products have always existed, and I actually think you can make infinitely better products now when using the agents properly (see T3 Code and Grok Bot).
I’ve been reviewing apps for over 17 years. Without a doubt, the most basic “vibe-coded” app created with Opus 5.5 today is infinitely more stable and nicer than any first app from an amateur developer in 2009. So if we agree that these models can write good code now, why is it that so many apps are still bad?
As we argued in our interview with OpenAI’s own Andrew Ambrosino (the irony is not lost on me), it comes down to human discernment, taste, and other qualities that can’t be turned into “metrics” for a model…at least not yet.
Davis continues:
To put a more precise definition on it, I think this “taste” is the ability to make good decisions in places where there isn’t a programmatic verification function.
For example: “make this function really fast” isn’t a “taste” task it’s a hill climbing task (make changes => test changes => repeat until goal met) which the models excel at right now (I think that any problem which can be expressed in this shape can be solved now by Opus 5.5)
“Which of these color palettes do you prefer?” is a “taste” question. You can’t really put a verifier on it other than your own opinion AKA your own taste. It’s a simple little example, but everything from codebase structure to dashboard layout to how features in chatgpt fit together all fit under here. Their implementations are “correct”, the tests pass, I just hate the choices they made.
“The tests pass, I just hate the choices” sums up my experience with coding agents perfectly. We’re working on our first two apps for MacStories (more in the near future), and while I’m not writing the code, I have to spend so much time thinking about usability decisions, onboarding flows, iOS best practices, and “it just doesn’t feel right” issues that models haven’t been able to abstract yet. I have to wonder if, in a rush to maximize infinite token usage and build everything “because why not?”, perhaps the big labs may have ignored these other aspects of good software design. I strongly believe that there’s more to a great app experience than elegant code that passes tests.
Knowing how these things can change so quickly in this scene, I’m sure this post will be obsolete in a few months. Maybe even weeks. But at this moment in time, Davis’ article is spot-on and human decisions still make a dent in software coded by agents.
Login
