Complicated and complex are different, and complex things can be made simple

In everyday conversation across technology, business, finance, markets, politics, there is over usage of word 'complicated'. It is baffling to see how a man-made thing can be complicated. Complication arises due to lack of information about it.

• When we talk about nature or universe or human body or relationships for that matter, our understanding of them is limited and so these systems/ things can be complicated.
• When it comes to man-made systems and processes, it cannot be complicated. However, they may be complex mainly due to poor engineering or poor documentation or lack of modularity, but it can be made simple.

A complicated relationship .............., but a complex equation can be solved with mathematics.

Let me start this topic, with few experiences from my past in manufacturing industry and then bring it into the IT industry, where these terms are abused and point to the right way of making systems "simple"

Situation 1:

3 decades back, I was responsible for a shop floor with more than 700 machines involved in production of automobile gear boxes. I do not recollect any incident in which we were not able to identify the reason of a machine fault in less than couple of hours. [Resolving might have taken longer due to spare availability] But there was one thing that was done extremely well there, there was clear documentation for complete electrical circuit kept the separate compartment in the electrical panel (which could be even 15-20 feet long and 6 feet tall for each machine). There was never a discrepancy between drawing and the actual machine, and the drawing had all diagnostics instructions and test point in the large circuitry.

[Learning 1: System was built as per the design, no deviations or the diagram was updated should there be a change during the build process and maintainability was core design principle]

Situation 2:

In the same job, for some time I was doing the PLC programming (ladder diagram) for similar set of machine tools. My first design was outright rejected by lead then. (I was in the learning phase, and he was willing to coach & guide me) He gave two reasons (1) I had not build the boundary conditions and fail-safe mechanism first (2) the logic diagram was too complex; hence execution time would be higher, testability was poor, and maintainability was extremely bad.

[Learning 2: Understand the boundaries and ensure operations with safety limits, simplicity delivers performance, reliability and ease of maintenance]

Let me take those 3 key words, I started with on why man-made systems and processes become complex, I will dwindle deeper in the context of IT systems in enterprises to look what can be done

[I am not talking about digital natives like Spotify or Google or product companies like Microsoft or SAP]

Modularity

Though in the IT world, component-based architecture, SOA, domain driven design and more have been discussed for many years and decades, adoption and maturity are quite low.

1. Take a macro view of the system, decompose it into modules and define every module with singular responsibility with clear input and output. Define the information exchange mechanism between modules, that is acceptable to both provider and consumer. [Right modularity]
2. As businesses grow, the features will be added to the application in an organic fashion and but at periodic intervals, invest to refactor the applications and rearrange the modules to keep the core principle of singular functionality. [Ensure maintainability]

Engineering

Engineer is there to solve the problem with the available tools and within the constraints. But understand the problem space clearly before getting into the solution space. Of course, there will be iterations of the solution, as the confidence in implementation (due to better understanding of the problem, the technology and the initial usage feedback) grows

1. Start in the solution space with focus on simplicity; essentially wear a designer hat and not an engineer hat. As Jony Ive says “I think there is a profound and enduring beauty in simplicity; in clarity, in efficiency. True simplicity is derived from so much more than just the absence of clutter and ornamentation. It’s about bringing order to complexity.” [Ensure usability]
2. Relentlessly focus on decomposition till each component/ module is simple but never lose the focus on how all components/ module will fit together. [Ensure composability]
3. Pushing the limits to deliver excellence, every technology, every component has its limitations, but it is about how creatively they are combined to build something extraordinary. “Scientists investigate that which already is; engineers create that which has never been.” - Albert Einstein
4. Build observation points both from the business process and application & system maintainability perspectives as part of design, this cannot be an afterthought. [Ensure manageability]

Documentation

Start with the notion; as a creator, you are responsible for its lifetime usage. I am not referring to documentation as traditional documents (like a deployment diagram in Visio that has to perfectly align with the deployed application), but a means to understand why certain design choices were made, how was it build, how to understand its working, etc.

1. Right and relevant documentation provides confidence to next person handling the system to work with it. As Henry ford says, "Failure is only the opportunity to begin again more intelligently." So do not let next person commit the same mistake as yours, pass on the knowledge in easily consumable manner. [Ease of consumption]
2. Good and consistent documentation facilitates good communication within the team with common taxonomy, reducing the clutter, and the communication overhead. [Clarity emerges instead of confusion]

Bringing it all together, take modular approach, adopt best engineering processes and create adequate documentation, this would take away the clutter for any system, and make it simple.

Should one encounter a complex system, creating better understanding document, and progressively reengineer for modularity and make the system simple.