Software Development is becoming the most impacted by Gen AI

Earlier this year, I among many others thought that customer services in call centers would be the first heavy impact from the tsunami of generative AI. But it turned out that the software industry is cannibalizing on its own first by creating AI tools that reduce the need for human developers. In recent months, coding assistance tools, such as copilot, have advanced from smart IDEs to powerful code generators and testers, code explainers, and software co-designers. And then, on Nov 8, Microsoft revealed “Copilot assistant that can learn about companies’ private code”,  which is poised to invade the most important strong hold of human developers – maintaining legacy code bases.

The "Impact" is a productivity shock

We are talking about 100+% productivity increase, though the technology has not yet reached the level of replacing human developers, as I analyzed with my own experience in this post. The productivity boost is often achieved through some primary patterns:

• Assisted coding: Developers delegate detailed coding tasks in narrow scopes including functional implementation and automated tests on such functions to AI and review/curate the implementations from AI.
• Assisted design: Developers leverage AI on digesting requirements and mapping them to technical architecture, and then work with AI often in interactive sessions to fill in details at various granularity levels of the design.
• Assisted learning: Software development jobs have been notorious for having to keep learning new technologies and skills. AI based knowledge base and generated code examples are substantially reducing the learning time.
• No-code development thru automation: It may not be mature yet, but we have seen a preview of this in LLMs' prompt engineering and model fine-tuning API, which are conceptually non-programming ways to reprogram a software.

Such productivity increase will reduce the needs for human workers unless businesses also proportionately expand.

The main reasons why software development becomes the first domain to be impacted

• Software professionals being the force behind AI have a better grasp of their own work and how to improve it, compared to other fields.
• Code can be rigorously and semi-formally verified with compilation, testing and monitoring, which gives humans more confidence on controlling the quality of machine outputs.
• Coding is not part of the critical business process that involves direct user/customer interactions. Even for software product companies, coding is an early stage in the product lifecycle and there are many later stages that can fix the problems and errors caused by it, such as integration testing, feature verification, A/B tests, shadow release, Alpha/Beta launches and ultimately customer support and bug fixes.
• Symbolized by one of its notorious company mottos “moving fast and break things”, this industry has a different mindset than other more traditional industries, as it embraces rapid changes and take risks.

How will this trend affect the profession?

The impact will vary across different roles and skills. The differentiating factor is whether a role can expand its job scope with this new-found productivity boost.

• QA engineers who automate tests on APIs are already feeling the pressure. Manual tests could technically be replaced by a combination of gen AI and computer vision in a similar way to automation of playing games, though this seems niche from a ROI perspective. (see AI based test tools)
• Developers who work at the module level in a codebase are seeing much less demand. This has already happened to junior developers and new grads, who use general knowledge to implement code modules. As Copilot learns from legacy code bases, many senior developers who are valued for their deep knowledge of legacy code bases, are also at risk.
• Most if not all of the coding of process automations by DevOps and SREs are ripe for being taken over by LLMs. However, SREs are still seen as the last line of defense for a live service, and thus part of the critical business path.
• Software Architects have been a vague role in many software companies, as tech leads or managers often perform that function part-time. AI will greatly empower this role in both depth and scope, by helping with understanding requirements, co-designing software iteratively and interactively, writing modules, and explaining existing code bases. Architects will rely less on developers for detailed knowledge and skills, and become the center of gravity of the team.
• Data engineers who implement data analytics and machine learning systems are seeing less need. Both fields have been automated away from heavy coding outside SQL even before the disruption of Gen AI (eg: the ecosystems of Redshift and Sagemaker on AWS). They are valued for their expertise on business data models, existing technology stacks and system designs, in addition to SQL skills. But many of those will be automated. For example, we have seen many attempts of using gen AI to automate coding of analytical processing with specific domain knowledge.
• Product Managers will benefit overall. A lot of project management (which may or may not be their core responsibility) will be further automated away from their daily routine. Tools are getting better at digesting customer requirements, generating UI mockups from text descriptions, and so on. Ultimately, AI will enable them to create prototypes of products with no-code tools, to communicate with both customers and developers, instead of still using PRDs and mockups.
• Data scientists is a tricky case. On one hand, applying foundation models of Gen AI to many different tasks reduces the diversity of technologies and the need for training many task-specific models from scratch. On the other hand, there seems to be a fast push towards Artificial General Intelligence (AGI) that could potentially expand the field dramatically. But it is safe to say that the jobs that manually tune hyperparameters of an existing model training algorithm or creating variants of an existing model architecture are in less demand, as more and more AI workloads are moved to AI-as-a-service (that can be fine-tuned with no-code and low-data) run by OpenAI and other large companies.

How will software companies adapt to this trend?

Many companies have seen their head count grow along with their business. In theory, most of these companies, except for outsourcing, are not really producing code as their main output, but rather code-based services, from Internet services to SaaS. The large employment growth in the industry in the past 20 years can be explained by the following factors:

• In practice, a company hires to implement services for a new business line or adding new features, and then keeps the workers to maintain the new service or the added feature and incrementally adapt to changing customer requirements.
• Companies have had high revenue growth and/or comfortable margins to afford the head count growth.
• There has been a strong perception of people being the most valuable assets of high tech companies, which actually originated from the fact that a small number of superstar developers deliver disproportionately significant part of the technology, but generalized to the rest of the workforce.

But the software industry is entering a period when technology is temporarily outpacing business expansions. We have seen Gen AI being quickly applied to optimize many existing software solutions, from chatbot, document AI to content creation tools, etc. But its business expansion, such as automating complete customer service, is still slow and gradual. During this period, the industry is optimizing its internal workings as well as its outputs using Gen AI.

• A lot of efficiency and cost reduction will come from automation of software development, test and production.
• Such automation will outweigh all other cost reduction measures, such as outsourcing to low-cost regions, as the latter are challenging in terms of managing remote collaborations, cultural differences, complex corporate politics and geo-political risks.

Moreover, the efficiency of the industry will improve greatly beyond cost cutting. The life cycle of software product development will be significantly shortened, actually with smaller team. We will see rapid prototyping and iterative development in response to customer requirements. In particular, we may see the roles of product managers, pre-sale engineers and software architects merge into single one that:

• Interacts with customers to understand their pain points and changing requirements.
• Uses gen AI to produce high level diagrams of workflows, data models and realistic UIs to formalize the ideas and communicate to both customers and developers.
• Quickly develops prototypes to demo and get feedback.
• Uses gen AI to manage development process for more efficiency. And supervise all technical aspects of code development with assistance of Gen AI.