Michael Heilemann, The Norseman / The Viking — On four years of AI-assisted projects and where the line between writing code and engineering software actually sits
My post last week was about AI as another programming tool, and writing it made me realize how much the way I work has changed.
Over the last four years, I've used AI on new projects, revived old ones, and tackled software and hardware I hadn't worked with before. The results have varied widely, but one thing has become increasingly clear to me:
AI is good at programming, but software engineering is more than programming.
Producing code is not the same as engineering software. After a career spent doing both, I've increasingly let AI handle the implementation while I remain responsible for the architecture, design, requirements, and ultimately, whether the result is correct.
-=[ The Junior Programmer ]=-
The easiest way I've found to describe what working with AI feels like is having a very fast junior programmer available whenever I need one.
Give it a well-defined task and it can produce a surprising amount of working code. It can create classes, functions, interfaces, data structures and tests. It can look at an existing codebase and figure out how different pieces fit together. It can make repetitive changes across dozens of files much faster than I would be able to do them myself.
It also produces code that does not compile, misunderstands a requirement, omits part of the implementation, makes a false assumption, or disregards an instruction. Sometimes the result looks reasonable until I run it.
When something goes wrong, I still need enough knowledge of the system to recognize the failure, diagnose its cause, and tell the AI what to change. In other words, I haven't stopped programming. I've changed how much of the programming I personally type.
-=[ The Projects ]=-
My projects have included Unreal Engine 5 plugins for AI texture generation and iteration, experiments with AI-generated 3D models, an interface between a Manus haptic glove and a Dexhand robotic hand, a modern version of my old Apple II BBS (www.norsewanderer.com), tools to validate and repair damaged Apple II disk images, a Unity game I originally wrote in 2017 that I updated to Unity 6, and a complete Foundry VTT system for a historical Western RPG.
As you would expect, the AI has been good with some of them and done poorly with others. None of them worked by simply telling the AI what I wanted. I still had to do the engineering.
-=[ Architecture Matters ]=-
Across these projects, I remained responsible for the high-level architecture and requirements: defining the components, their interactions, the technologies, and the approach to each problem. I then gave the AI specific, bounded parts to implement incrementally just as I would have written it from the bottom up.
When I know what I want a system to do, I can take a few minutes explaining the requirements and get a first implementation instead of spending hours writing it myself. If the implementation isn't right, I can point out the problem, and the AI will try to fix it.
I have had plenty of sessions where the generated code didn't compile. Sometimes the problem was obvious. Sometimes the AI had gone off in a completely different direction, and I had to work out what it misunderstood.
Sometimes the AI got stuck, especially in larger Claude Haiku 4.5 projects after a conversation has accumulated a lot of context. At other times, I had to abandon an entire approach after learning that my prompt was unclear or the proposed solution was unworkable. In both cases, the most effective response was to start a new chat and restate the problem with what I had learned. Those are limitations, but they are constraints I have learned to work around. The projects that I have been using Claude Sonnet Medium on have been much better in those regards.
-=[ Not Every Programming Problem Is the Same ]=-
The most interesting thing I've learned is that AI's usefulness depends heavily on the kind of problem you give it.
One of my least successful experiments was generating 3D models with AI, which I would rate 2 out of 10 as a software-development experience. It was not useless, the result I wanted was just beyond what it could provide. I chose the problem precisely because it was difficult and wanted to see how far I could push the technology.
By contrast, updating a Unity game I wrote in 2017 to Unity 6 went remarkably well (about a 9 out of 10). In less than a day, I worked through a decade of upgrades and deprecated code, including a major rendering-engine change, and restored the entire codebase to its original behavior.
That project had a huge advantage from the AI's perspective. There was already a working program, existing source code, and known problems caused by changes between Unity versions. A lot of the work involved identifying obsolete APIs, changing code to use newer ones, fixing compile errors and working through the resulting problems.
That is the kind of programming task AI is good at. It didn't have to invent the game or decide what the game should be, it had to help me get an existing piece of software working in an updated environment. That's a much easier problem to me to describe and AI to execute.
-=[ Old Software Is Surprisingly Good Training ]=-
The Apple II projects have been another interesting example.
I'm rebuilding the experience of an Apple II BBS I worked on decades ago as a modern website. There is a lot of code involved, but there is also a lot of history and context that isn't in the code.
AI helped tremendously with the implementation. It took old data structures and turned it into something suitable for a modern application. It wrote database code, web interfaces and the other pieces needed to make the system work as it is today.
I know what the original system was supposed to do, and the AI had direct access to that code. I know why certain pieces were designed the way they were. That makes it much easier for me to tell whether the code it produces the correct functionality.
I've had a similar experience with an Apple II disk validation and repair tool. The parser and utility code is exactly the sort of work I can hand to AI. It can help analyze filesystem structures, implement repairs, and write code to compare multiple disk captures and work through the data.
But when the data is corrupted, there isn't always a clean specification telling me what the correct answer is. I need to understand the filesystem and how the hardware stored the data, recognize when something looks wrong and, most importantly, when the proposed repair is wrong.
AI can help me do that work, but it isn't good at inventing the solution. It can't take responsibility for deciding what the correct disk should look like without direction.
-=[ Hardware Makes the Difference Even More Obvious ]=-
The Manus glove and Dexhand project was another useful lesson.
This involved taking movement data from a haptic glove and using it to control a robotic hand.
AI was useful for pieces of the software. It could help with APIs, data structures and communication code, and it could write parts of the implementation much faster than I would have wanted to write them manually. But the biggest challenge was one that AI had the most trouble helping me with.
The hardest part was debugging translations between the coordinate spaces of the two systems, including scale, handedness, Z-axis orientation, and the details needed to align their movements.
There is a physical system on the other end. The glove has its own coordinate system and behavior. The robotic hand has its own mechanics and limitations. There are questions about how the movement of one should map onto the other that aren't answered by simply writing more code.
I had to understand what the hardware is doing and test it. When the result behaved differently than expected, I had to figure out whether the problem is the software, the mapping, the hardware, the assumptions I started with, or some combination of them.
-=[ Where AI Shines ]=-
Virtual Dice software I've been working on is another example. The goal is to connect to various virtual tabletop and communication platforms (FoundryVTT, Roll20, Slack, Discord, etc.), which involves a lot of API and integration work.
Once I establish the architecture and requirements, I can have AI implement relatively well-defined pieces, test them, find problems, and send them back for another iteration. That's very different from asking AI to invent the entire product.
The same has been true with the Foundry system for my Western RPG. Once the rules and architecture are established, there is a tremendous amount of implementation work that can be delegated to AI without giving up control of the design.
-=[ I Learned Which AI to Use ]=-
I've also learned that the model matters, my earliest experiment here was the Manus-to-Dexhand project, which used ChatGPT 3. It was useful, but it was also the reason I started looking at other models. Most of my subsequent projects have used Claude, primarily Haiku 4.5 and, more recently, Sonnet Medium. I still use ChatGPT for simple tasks when it makes sense, particularly when I don't need to spend more expensive model credits on something trivial.
The model matters, but it is only one variable. The same model can excel on one project and struggle on another depending on how it was trained, how well I understand the problem, how clearly I describe it, how easily I can test the result, and how much of the task can be expressed in code.
A lot of discussions about which AI model is better miss this distinction. The question isn't simply whether the AI can program. The question is whether it can program this app, under these constraints, while I can reliably determine whether what it produced is correct.
-=[ Software Engineering Is More Than Just Programming ]=-
This is where the argument about AI replacing programmers gets confused.
If by "programmer" you mean the person who sits down and types the implementation of a function, then AI is already doing some of that work, and I expected it. For decades, the software industry has built tools that reduce how much code people must write: libraries prevent us from reimplementing and retesting algorithms, frameworks supply recurring application infrastructure, and visual tools, code generators, IDEs, and plugins accelerate the rest.
AI is another step in that direction, except this time the tool can understand a description of what we want and produce a surprisingly large amount of the implementation.
What remains is the need to understand what we are building. I must define the architecture, interpret the requirements, recognize and diagnose faulty code, decide whether to repair or replace an approach, test the result, and accept responsibility when it fails. That is software engineering.
This isn't an argument that AI will never become capable of doing these things. The models are improving rapidly, and the boundary between programming and engineering will move. I'm describing where that boundary appears to be from the systems I've actually built with AI today.
Next week, I am going to start a series of detailed posts about the Apple ][ Repair_DSK program I am writing for the Apple ][ with the help of Claude Sonnet 5 Medium. I hope you will tune in and check it out on www.norsewanderer.com.
Prepare ye to plunder,
Michael Heilemann - The Norseman/The Viking