Microservice Architecture NFR Considerations

Most of the new modern applications are now built using microservices architecture. This is obvious as microservice architecture makes business entities separation with loosely coupled design and allow them to independently evolve/ change or scale. 

As we start implementing this architecture we often concentrate only on the functional part of the service. There are a lot of non-functional requirements which are not considered from day 1 and then gets realized after minimum viable product go-live. This then sits on technical debt backlog and never gets picked up for development again. As soon as business rolls out this system to a wider audience everybody then realizes the compromise they made earlier. 

Here are some best practices which I have experienced and there could be many but worth considering for designing a microservice architecture pattern.

API Resouce Designing 

Most of the time microservices are developed with a gateway on the top(such as 3scale, APIGGE). To manage standards; Enterprises should have guidance about how API's need to designed by Solution Architects. This also has to be governed by the Gateway team before they accept any new request for proxy creation. Parameters needed to be considered such as:

1. HTTP Verb Used
2. Version support
3. Clean URL with the purpose of the operation. Mostly chain of collections and resource.

API Contract

API contract is key in any microservices development and consumer uses that to integrate with microservice. This contract should have only required information and unnecessary or duplicate data fields need to be relooked. This generally happens through Design Review sessions. The enterprise should have clean contract definition rules(not strict as sometimes deviations are needed) and those again need to enforced by API Gateway team. Parameters needed to be considered such as:

1. Camel Casing attribute names
2. Remove unnecessary attributes
3. Arrays should have meaningful names such as items, categories. Avoid names like categorylist, itemArray.

API Throttling

Microservices are designed for high traffic requirements and mostly they are multi-tenant. They serve different consumers for different requirements and those requirements will have different usage type. Based on usage type throttling policies should be handled and they should be ideally handled at Gateway layer/ Load balancer as a cross-cutting concern. Keeping throttling at microservice level could kill the entire application if too much load is thrown intentionally/ unintentionally by the caller. 

Authentication/ Authorization

Most important point while exposing API's to third parties is to manage scopes and credentials. Most gateways support out of the box IAM layer and implementation of Oauth2 which manages API keys and their scopes for each API's. Incorrect implementation of multi-tenant API's could leak sensitive information. URL paths should drive the file system pattern to allow the gateway to extract that information when the request comes in and match against IAM scopes. Some of the critical Auth considerations should be:

1. Third parties should always granted with Bearer token and token should have a limited lifetime to access the resource. This minimizes the risk of exposure to the system, If somebody gets hold of temporary token it will be valid for a minimal amount of time. 
2. Internal systems can be granted with Basic tokens but each service should take their own(for traceability and lifecycle management).

Unit of work responsibility

Microservices service endpoints or standalone jobs must follow the principle of a unit of work. Below are various benefits attached by a deligating only unit of work to each service endpoint:

1. Easy to change by not impacting other functionalities. 
2. Easy to test as change should only impose regression testing around single functionality input/ output parameters.
3. Too much work on single resource could lead to timeouts. 

Security/ Vulnerabilities

This is supremely critical aspect and mostly managed by Gateway layer/ Firewall services. If an enterprise is implementing a gateway on top API's then most gateways are managed services and all latest security patches are managed by providers. Some of the security considerations needed to be considered by the designer should be:

• Denial of Service Attacks
• IP spoofing

Load Balancing

One the important aspect while designing microservice is spreading the load across nodes. Microservice should sit behind load balancer(mostly private) to spread load across nodes/ pods(kubernetes) / containers. Load balancing algorithm can be decided as per business requirement but at least 2 or more nodes should serve incoming traffic.

High Availability

Mostly load balancing and HA concepts are overlapping as nodes should distribute the traffic and they should be hosted ideally on different physical machines/ availability zones(Cloud) to provide resilience. Skipping this consideration can majorly impact business operations if service is available due to hardware/ server failure.

Caching

Caching plays an important role in terms of improving response times and helps to avoid unnecessary database roundtrips. There are a lot of ways caching can be implemented but purely application designer should consider below caching strategies along with some other factors which will impact the decision.

Caching strategies:

1. Gateway level caching
2. Caching Engine such as Redis, Memcache
3. Application local cache. (for small memory footprints) e.g. hibernate eh-cache

Design Considerations:

1. Time to Live (when caching need to invalidated)
2. Memory footprint
3. Cache cluster
4. Cache region replication for HA and DR scenarios to make solution resilient.

Data Retention Policies:

Data retention is most sidetracked design decision, not only in microservice design but it's applicable for almost every software solution. This critical step should be evaluated in the early stages of design and especially in microservices development as a lot of other decisions could possibly change based on this input. Simple data retention strategies could be something like below:

1. Shadow tables for data older than X days. Separate API to serve such traffic. These tables should be ideally versioned with version numbers to allow easy restoration.
2. After X days data can be archived on the cloud as data-dump/ CSV along with version number. This data should be easy to retrieve and restore in database tables by referring to simple documentation steps.

SSL

Although SSL sits in the same section as security; I have captured separately as its one of the important key consideration while designing. API's should provide end-end to SSL especially if they are sharing a lot of sensitive data over the wire. API gateway's or HTTP proxy servers should have dedicated SSL certificates with SNI implementation. Some of the key considerations should include:

1. Enabling Termination server and policy. This should be ideally terminated on load balancer level if load balancer and microservices are communicating on the private link. 

Logging & Traceability

Application Logging and taking insight out of it is a key factor in software solutions nowadays. Logging should be enough but not too much exhaustive which could add additional performance penalty. A business process gets completed when all microservices work together in the desired manner. These then impose traceability of a single request between various microservices and that can be achieved through correlation ids and logging. Co-relation ids are constant reference moving across all microservices and microservices then uses the same to log complete trace using the id. In case any issue is observed co-relation id will be alerted to support/ application monitoring team and they can trace any microservice using the same.

Exception Handling

Exception handling and returning standard REST reason codes are important from DAY 1 of microservice implementation. All API calls or data processing microservices need to handle the exception at the appropriate level and LOG with standard ALERTING patterns to allow monitoring tools to inform relevant team according to that. Exception handling, logging, and traceability work hands in hand and architects must come up with standard libraries to allow developers to use those for logging, exception handling and alerting requirements. Unit and Integration must need to have acceptance criteria to test whether exception scenarios are working as expected.

Health Checks

Health checks in microservice are like a heartbeat for a human. Healthchecks lets load balancer or microservice controller know about overall request serving capability of the node. Ideally, a method or wrapper method should be written in microservice to implement health checks and exposed as an endpoint for the load balancer to call. If the service status is not healthy corresponding status code will be returned and the load balancer will replace such node. Some of the important factors to include in health check should be:

1. Database connectivity by firing simple queries to the database.
2. Health checks of another service if API composition pattern involved.

Connection Caching/ Pooling

The key critical aspect of improving microservice response time is to implemented connections pooling mechanism. Polling helps to avoid unnecessary database/HTTP trips to request physical connection of the socket instead it keeps in open. Connection pool size is purely based on requirement and will change based on no of workers involved/ scaling factor. These pools of connections parameter should be part of the config and easy to change if required. Some of the keys factors include:

1. HTTP Connection Pooling(If API calls are involved)
2. Database Connection Pooling(Max pool size should be carefully selected considering worker size/ autoscaling factor/ database instance size)

Async Processing

Although the unit of work should be part of the microservice design principle but sometimes requirement is complex and does not fall under the category unit of work. Such scenarios can be implemented using Async behavior. Async implementations design should include status management into the repository and those statuses should be queryable using microservice endpoints. e.g. implementation could be initiating complex job(/jobs/@initiate) which will return handle(jobId) and then callback endpoint for tracking it to its closure(/jobs/{jobId}/status). 

Indexing/ Partitioning/ CQRS

Depending on data and throughput requirements microservice database design gets influenced. With slightly moderate load having proper indexes on database improves response time. With the medium of high load, database partition gives great performance and physical database tables avoid table scan for unnecessary items. With high load scenarios, CQRS can be implemented where read and write loads are separated and can be scaled up or down / scaled out depending on the requirement. CRQS for small requirement could lead to unnecessary cost spending. Some of the key considerations are,

1. Selecting proper indexes based on filter criteria available for microservice. Do not create over-indexing which will degrade the response time of the endpoint.
2. Partitioning can be implemented at date/ collection or data type level based on the requirement. 
3. CQRS should consider eventual/ strong consistency needs of the application requirement.

Conclusion

These are some of the key critical things which often gets missed while implementing a software solution. Missing these aspects while implementing microservice architecture could lead to major rework/ impact on the overall ecosystem and will impose compromises for the business. Ideally, Solution/ Technical architect should keep these and other criteria as a checklist while designing so as to make sure they are implemented for each new development.