Here's newsletter i got:
“Welcome back!
OpenAI’s development of an “agent” to automate the work of a senior software engineer—which my colleagues scooped yesterday—will raise the stakes in an already competitive market for AI coding tools. But OpenAI still has to overcome a major data problem before such a product can become widely available.
While the internet is full of coding examples for training models, there is a dearth of information on how senior engineers turn a broad idea from managers—say, rewriting how an app collects user data—into a completed project, according to people we’re talking to.
OpenAI’s product is designed to handle complex tasks involving multiple steps, like code refactoring, the process of simplifying code or making it more understandable to human programmers. The company is targeting work done by staff software engineers, also known as Level 6 engineers—technical employees who often have a decade or more of experience.
How they will train the AI to understand what engineers at that level do will take a while—and cost a lot. As AI companies try to get their models to solve increasingly difficult problems, data providers like Turing and Invisible Technologies have been paying experts to solve tasks that take hours and record their processes throughout. Those costs can add up significantly.
Another alternative is for companies to tap their existing ranks of senior engineers for training data. But although the biggest tech companies have private code repositories running millions of lines, even those firms wouldn’t have enough data on the step-by-step thought process senior engineers and their managers take when completing difficult projects.
That’s likely why OpenAI is currently trying to quietly test the agent with a handful of potential customers. By getting the product in the hands of more coders, it might generate the kind of real-world data that would be useful for improving the product—for example, by having engineers pick the better option from two different ways the product might want to execute a coding task.
OpenAI might also lean on o3 and future reasoning models to improve the AI’s ability to plan projects and break them down into more manageable steps. But without more data from senior human programmers, or their bosses, the AI won’t really know what to do.“