Behind the hype is the 20%
A brilliant engineer I worked with put it to me plainly. AI is great for 80% of the work. That last 20% is where it falls down.
This engineer spent years inside a platform that had served hundreds of customers, and he knew exactly which 20% he meant.
The platform had thin documentation, few people left who understood the full architecture and hidden dependencies from years of customization without oversight. That’s the 20%. It’s unknowable from the code.
I use these tools daily, and watching an idea become a clickable thing in 10 minutes is a little bit magic.
But the 80% was never what made this work expensive.
An article I read put it this way. Code generation is the easy part, trust generation is the hard part. His answer is evaluation as a real engineering discipline, including whether the model exposes its assumptions or quietly invents them when the legacy system is ambiguous.
You can build a suite that catches an invented assumption. You cannot build one that knows the rule exists because of a 2008 contract, or whether that contract is still in force.
For the systems product teams inherit and steward, AI is a bolt-on. Everyone’s watching the 80%. The 20% is the part nobody can see.