Data On Air Meeting Minutes Episode 10 – Microservices

From a Data perspective, an easy way to understand Microservices is an application designed as a group of services. Despite this definition being not technical and simplistic, it is possible to know how this distributed application development correlates with data.

Our discussion started with the basic development concepts of cohesion and coupling. Cohesion is welcome in microservices, but the coupling is not the case. The microservices architecture premise is that you will work with loose coupling. This characteristic allows the implementation of continuous integration and deployment (CI/CD) and is crucial to enable the isolation of microservices. It happens because each application piece – or service – is built independently from the other.

To better understand this architecture, it is necessary to talk about the other one. When you have a monolithic application, a single, long, and unified programming code, you will build each part of the process one after the other. For example, consider an e-commerce application where a consumer will choose a product, put it into the cart, select the payment method, and confirm the sale. In the monolithic approach, you would put all the steps above in a single programming code. With microservices, you would create one (or more) programming code for each process step.

We did not say that one is better than the other in our discussion. Depending on the business case, a monolithic approach can address reasonable solutions. And for others, especially if you are working in a Cloud environment, with a high number of concurrent users and using distributed databases and processing power, microservices will perform better. Each service is a small distributed code in different servers that you can shrink and grow depending on app usage.

Naturally, whenever you are working in a distributed environment, you add more complexity. You can work with small and big teams using the proper framework, adding resiliency and security to the development process. And because you are working with small pieces of programming code, it is (in theory) easier to build and maintain. Considering loosely coupled as one of the main characteristics of microservices, putting one service in production will not affect or crash another piece of code.

There are different ways to implement microservices. Synchronous or Asynchronous. Using a shared database/tables or, the closest to its concept, keeping all data into the service. According to our discussion, there is no correct or wrong way to implement it. As with any technology, the teams involved need to evolve and adapt, finding the suitable business case that will better use the technology.

Naturally, there are some drawbacks related to the complexity of distributed systems. Communication of failed services, tests, integration between services, memory consumption, and the promotion to the production environment can be challenging.

From the data analytics perspective, microservices can support the streaming data from diverse sources, like sensors / IoT or defined sources, like ERP, POS, etc. It creates a data pipeline to feed data within the required velocity and project latency. Because we do not need to work with transaction controls in data analytics like transactional systems do, microservices will make it possible to implement ELT (Extract, Load, and Transform) instead of ETL (Extract, Transform, and Load). It will benefit near real-time analysis to speed up business decision-making.

We summarized that Data Analytics is getting great benefit from microservices technology and can get much more with the maturity of the teams and the technology. In very competitive marketing as we currently live, having the data you need to decide before your competitor can make the difference between the leaders and the laggards.

See you in our next Data On Air - Meeting Minutes