Generative AI tools write code quickly, but need constant supervision and correction. They can be more harmful than helpful in the hands of junior engineers.
Generative AI has hit the mainstream with software developers. According to a recent GitHub survey, more than one million developers actively use GitHub Copilot. More importantly, these developers increasingly use AI as “a new building block in developing applications.” In other words, AI is becoming just as important to software development as tools like Visual Studio Code.
There is, however, a catch. Not everyone benefits equally from AI. As Addy Osmani, an engineering leader with Google’s Chrome team, writes, “AI tools help experienced developers more than beginners.” I’ve talked about this before (here and here), but Osmani makes this argument so lucidly it’s worth repeating, particularly with so many developers adding AI-powered coding assistants to their development process.
All the kids are into tab completion
RedMonk’s Kate Holterhoff has combed through online forums and in-person interviews to identify the “Top 10 Things Developers Want from their AI Code Assistants in 2024.” It’s a great post, filled with useful observations like, “Tab completion is the killer feature in AI code assistants.” Notice, however, that Dr. Holterhoff doesn’t need to talk about whether or not developers want AI coding assistants—they do. The question is, how are they using them?
One reason is to increase productivity. As noted in the GitHub report, GitHub Copilot and similar tools lead to more active contributions to code repositories on GitHub. How much more? “We see higher activity (between 12% to 15% among developers who use GitHub five days a week and 8% to 15% among developers who use GitHub once a week) across open source and public projects.” Lowering the bar to development appears to be A Very Good Thing.
To a point.
Although generative AI can serve as a shortcut for development, such shortcuts are helping the more experienced engineers while hurting the more junior engineers. As Honeycomb co-founder and CTO Charity Majors suggests, generative AI has done nothing “to aid in the work of managing, understanding, or operating … code. If anything, it has only made the hard jobs harder.”
Osmani builds on this idea, arguing that senior developers can look like they’re performing magic when using generative AI tools. (“Scaffold[ing] entire features in minutes”!) But it’s critical to observe carefully what they’re not doing: “They’re not just accepting what the AI suggests.” He says experienced developers are constantly doing things like refactoring AI-generated code, adding things like edge-case handling the AI tool may have missed, etc. In short, he concludes, “They’re applying years of hard-won engineering wisdom to shape and constrain the AI’s output. The AI is accelerating their implementation, but their expertise is what keeps the code maintainable.”
In short, you can’t simply let the AI tool do the work for you. Not if you want well-constructed, maintainable code.
Treat AI like an intern
As I’ve argued, generative AI tools should be regarded as high-functioning interns, not fully autonomous engineers. Osmani makes the same point but takes it further: “The reality is that AI is like having a very eager junior developer on your team. They can write code quickly, but they need constant supervision and correction.”
The very thing that makes an experienced engineer more capable with generative AI has the potential to make less experienced engineers less capable: knowledge. A developer can’t reliably offload work to an AI assistant if she doesn’t first know whether the AI is getting it right. Without that, Osmani notes, developers fall into a pattern of one step forward, two steps back. “You try to fix a small bug. The AI suggests a change that seems reasonable. This fix breaks something else. You ask AI to fix the new issue. This creates two more problems. Rinse and repeat.”
AI tools, in other words, can hurt more than help if engineers don’t first learn the debugging and problem-solving skills of good engineers. As Amazon CEO Andy Jassy used to recite repeatedly, “There is no compression algorithm for experience.” AI doesn’t change this fact.
To ensure less experienced developers can still benefit from AI, Osmani points to three patterns he has seen successfully used by a range of teams: using AI for “first drafts,” using AI for “constant conversations” for tightly scoped areas of development, and using AI as part of a trust-but-verify process. For developers of all experience levels, this roughly translates into the following approach, according to Osmani:
- Start small. Review all AI-generated code, and use it for strictly defined tasks.
- Stay modular. Limit the blast radius of AI gone wrong.
- Trust your experience. Use AI to accelerate, not replace, your judgment.
These are excellent, actionable suggestions for getting started with AI-driven development—and you should get started. Just don’t expect AI to replace you, whether you view that as a positive or negative thing. Never has experience been more important, even as we hope to augment, not replace, that experience with generative AI.
