Why GenAI will never replace an experienced programmer

As with any revolutionary technology, generative AI generated, along with its AI, a lot of hype :) In this article I give an example of how the technology does not replace an experienced programmer, and likely never will, if only because there are usually many ways of doing things and at some point it is faster to edit the code than to tell a tool like chatgpt how to edit it.

I have been programming since my early teens which is several decades ago now, when commodity PC's were just starting (think Apple IIe, Commodore 64, etc). I have come to value highly clean code, with just enough abstraction to strike a balance between DRY and clarity.

In the latter years I moved from application development, to cloud "devops": there is still a lot of programming, but it is different in nature. Whereas previously I used object-oriented languages like C++, python, now I use several declarative languages like terraform, docker/compose, github actions, and inside or alongside these, a mixture of python and bash to provide automation logic that would be too cumbersome in declarative languages.

So when chatgpt came along, I started using it to accelerate my programming. I have continued to use it for this as it does such an incredible job at it. Well, at least chatgpt 4o. I have stopped using 3.5 a year ago so I don't know what the "free" experience would be like.

Here is a typical workflow:

1. I want to automate the shutdown of a system that is described in a text document. The document consists of instructions, explanations, and actual shell commands to run, eg in bash or python etc.
2. I paste the document in chatgpt and ask it to write the python code that does this
3. Chatgpt converts everything to a function that executes the instructions and the logic such as "if this happens then do that". It does not have any type hints yet. It has a lot of unnecessary comments that say the obvious.
4. I copy the code into my IDE, format it using ruff, and I check EVERY line of code one by one. There are, not surprisingly, many errors but there are several. Some errors are incorrect regular expressions, some invalid values eg using --format '{{json .}}' instead of --format json. I fix those. I can finally run the "draft" version. But there are now some logic errors to fix: a step of the procedure should be done a certain way that was not made clear in the document, some exception gets raised but the code doesn't handle it, etc. Indeed error conditions are often absent from such instructions.
5. In some cases, I fix the code manually but for the logic fixes that are more onerous, I paste the code into a chatgpt prompt and describe how the logic should be fixed. It almost always does a good job at the fix. I repeat this for all logic errors, although there are typically not many, less than 1 per 100 lines of code. In some cases it is not really an error, but is not ideal, eg it defined a complicated string inside a loop instead of just before the loop, even though the string did not depend on the loop parameters. In some situations, this could impact performance, plus it would give the wrong impression to leave them in the loop so I move them out.
6. I then look at the cleanliness of the code. Eg chatgpt will name a varialble "some_specific_host" in one section of code, then in a highly similar piece of code (but too different to refactor into a function) just names it "host". I fix those so that all variables have clear, descriptive names. There are some blocks of code that can be refactored to a function to help break down the code. Sometimes I can ask chatgpt to do this, but sometimes it is faster for me to just use my IDE's "refactor" capability.
7. Once the code is well broken down into functional items, and the flow from the main() entry point down into the lower levels of logic is clear, I ask chatgpt to make specific improvements: convert this particular function to run things in parallel; convert the script to use python's pathlib instead of os functions; use argsparse instead of if/then/else, etc.
8. Some of these conversions will be good as-is. Others will take several iterations to get right. Eg to parallelize code, chatgpt did a great job of using the concurrent.futures module. However there was not opportunity for me to print every minute the status of the processes running in paralel. So I asked it to do that, and it did a decent job, but the concern of printing status heavily intertwined with the goal of running the jobs in parallel. In other words, if I wanted to remove printing status, there were several non-obvious pieces to remove. So I suggested how it could be done different, and this solution was much better at separating concerns. This allowed me to further break down the problem into functions. Finally that goal was achieved: I had a clear set of functions related to running the processes in parallel and printing status, and I could easily keep track of both separately.
9. I added various print statements with the right blank lines etc in the output so that output in the shell would be easier to read, I worded the questions requiring user input a bit better, I moved a few magic values to globals and uppercased them (to indicate they are constant), and a few other tasks.
10. At that point I realize that there are more commands needed, eg I had a created the shutdown command, so I will need a startup command, and a status command that I can run before shutdown or after startup or whenever. This would be tedious so I give chatgpt my code, and tell it that I want everything done so far to be in a "shutdown" command of the CLI, and to add 2 commands, one for which I paste instructions, but to leave the status command empty (I will specify it later). I also ask it to use the existing functions that I have (modified from the code that chatgpt generated earlier). It does a great job of this change major change, able to identify portions of the startup procedure that can use functions called by the shutdown procedure. Amazing.
11. I repeat all the previous steps to ensure that chatgpt hasn't broken anything after this refactoring. It is really good at not breaking things (from what's I've seen over a year of use). I need more options on one command, but can't remember the argsparse syntax for optional flags, I ask chatgpt. Amazing. As I do these passes, I notice how the procedure can be programmatically simplified. Eg instead of doing this ssh multiline command on the host, I can split this into 2 and have better control on what happens. I do this part manually this time because it is simpler than figuring out how to explain to chatgpt how to do it.
12. By now I'm running and testing and checking that output is good and clear and doing more passes of the code to check that everything is nice and tight and concise and robust. I notice that the function that I refactored from its code to ask for user input does not handle ctrl-C so I ask it to do that (although I'm being a bit lazy here, the change is simple). I specify to show me only the change in the function, because sometimes chatgpt spits out the whole script. That's ok, sometimes that is what I want. Ironically, the code for handling this is not great, chatgpt put too much of it in the try block, instead it should consider the keyboard exception as another way of answering "no" and process the answer outside of the block. I ask it to make that improvement, and it comes up with an answer that is slightly cleaner than what I had in mind using 'finally' clause; instead chatgpt raises exception when user does not answer yes, nice.
13. Finally, I gave chatgpt the code which was over 400 lines of code by then, and asked it to add all the type hints and it did a great job at that. But it removed all the comments in my code, so I stopped it from generating code and reworded my question witih "keep the comments!" and it did it properly that time.
14. Before I overwrite my code with what chatgpt produced, I save my code in a git commit. This way, after pasting, I can check every change that it made. EVERY change. Eg it is able to figure out that a statement like `my_var = dict()` is actually a Dict[str, List[Dict[str, str]]] from the loop that follows this statement where the variable is populated.
15. I realize that none of my functions have docstrings yet, so now that the code runs the way I want it, with the output and want, and is clean with a nice CLI, I ask it to generate the docstrings. HOWEVER, I clarify that docstrings should focus on less obvious aspects of the code such as side effects, when the function should be called, etc. ChatGPT does an awesome job at this, even added the arguments names for functions in the docstring. Amazing. Oddly, I had to try this a few times and it used a different style of docstring once, so I rewrote the prompt to specify the format of parameters that I needed.
16. I ask it to generate a docstring for the whole script does and it does an ok job of that, but that is actually a tougher one: a script docstring needs to describe the purpose in a way that makes for the context in which it will be used. I had to ask it several times until it got something that didn't contain misleading statements. Still amazing.

There is a good chance that the free version of chatgpt would not have been able to do all this because it would have run out of "memory".

So although chatgpt was a great starting point, a great tool to extend the capabilities of my original idea, a great way of exploring a couple different approaches to some functions, saved me tons of time writing docstrings, it does require diligence and a solid knowledge of the language to know when to challenge its solutions, how to instruct it to improve on it vs making the change manually, how to make it chose one method/approach over an another, and a good stepwise improvement methodology. Eg if I had starting with everything from the get-go, including all 3 commands I ended up with, docstrings, etc, it would have been a mess and just too much to verify at once. Also the way I did it, allowed me to refactor some code for re-use by other functions that I asked chatgpt to create later using those functions (I gave it the code for those functions).

All in all chatgpt is an amazing development in the realm of software coding, in fact it is revolutionary due to how deeply it understands human language and can break it down into a sequence of steps and convert it to decent code in the target programming language, using idioms of that language. But some aspects of programming are discretionary and there are choices that chatgpt makes that are just not acceptable, or heavily sub-optimal, even sometimes wrong. ChatGPT is like a capable intermediate programmer: it does solid stuff, but you can't entirely trust it and have to check everything it does and polish it so it makes sense given the wider context that can be too onerous to convey to it.

So, I'm not worried about my job, and neither should you. Use it as the tool that it is, because it is just a tool.