The Evolution of Enterprise Integration
Purchasing a camera from Amazon sounds like a trivial task; you simply place the order and receive a receipt almost immediately. Funds are deducted from your bank account, the item is placed on some kind of shipping process, and soon you end up with a box at your doorstep.
From an enterprise IT viewpoint, this order, which is simple to the consumer, actually flows through several business applications; there’s probably one for generating a purchase order, checking order inventory, invoicing and billing, taxation, shipment, and so on. An enterprise typically would have a hundred to a thousand of these business applications that run on legacy platforms, on disparate on-premise operating platforms systems, or in the cloud. To deliver an efficient, secure, and reliable customer experience these disparate systems, services, and applications must be seamlessly integrated.
This task is fraught with technical and managerial challenges. Business applications are designed for specific requirements of functional areas - for example, one will specialize in accounting and another on procurement. In the end, your business will have a host of applications working in their own silos, which makes data sharing difficult. The ease of purchasing these same applications on cloud services will further complicate this as your data will now be on another server halfway around the world.
Even if they are connected, data transformation (the mapping of data types and formats), establishing connections, and controlling transactions between disparate applications are additional technical challenges that occur in these scenarios.
The Early Approach to Enterprise Integration
Initially, enterprises attempted the do-it-yourself (DIY) approach by developing custom integrations. The DIY approach was more of a point-to-point type of integration. Adapters, which function very much like their name suggests, would connect one application to another and handle the connection and data transformation of that relationship.
The disadvantage of this approach was that it led to a spaghetti-like bowl of complex, costly, ad hoc, point-to-point solutions with limited reusability. DIY was suited for enterprises with smaller infrastructure. The complexities and tight coupling involved with this approach caused higher costs in terms of maintenance as soon as the enterprise started to grow.
Thereafter, enterprise application integration (EAI) emerged to overcome these challenges. EAI provides a loosely coupled, central integration system where applications need not be aware of each other; the central integration system handles the tasks needed to provide communication among applications. This approach provides the advantage of a flexible architecture where components can be added or removed as needed. It also promotes re-usability of services across applications.
These initial EAI solutions were implemented as a ‘hub-spoke’ architecture. What this means is that central integration ‘engine’ acted as the hub where all the applications connected to the hub via spokes. There were advantages to this approach like asynchronous message communication for applications (meaning that they don't need to wait for a response to continuing their work) and having a central configuration repository.
However, the ‘hub’ being a single point of failure introduces performance limitations, especially as the number of applications and transactions increase. This, in the past, caused many EAI solution implementation failures, arising in the need for an alternative architecture.
The alternative solution was the bus architecture, a centralized messaging backbone with distributed integration tasks. This eliminates the dependency on a core integration point and enables horizontal scaling, overcoming another limitation in the ‘broker architecture’.
The advent of the ESB coincided with the then-popular service-oriented architecture (SOA) strategy. SOA encourages businesses to implement business functionality as services with an ESB being the infrastructure. The ESB would act as the messaging backbone and perform tasks, such as controlling and coordinating service interactions, transforming message data, routing, protocol conversion, and so on. This proved to be a success in implementing enterprise integration and was subsequently adopted as the best possible option yet.
A Fresh Era With New Challenges
Today, the disruptive forces of data, mobile, social, and cloud are forcing enterprises to rethink enterprise architecture. Apps are now everywhere. Enterprises are required to expose their business capabilities externally for mobile access in a managed and secure way. The EAI and the SOA/ESB-only approach, which were designed for internal enterprise integration, limit the ability to expose services externally. There are many limitations though with non-mobile friendly data formats and complex service contracts being some of them.
Then enters application programming interface (API). APIs expose functionalities and data in an accessible, secure, managed manner. For many enterprises, it becomes a challenge in exposing low-demand internal services as high-demand business APIs in a secure and accessible way.
One approach to this problem is to use an API management solution as a 'facade' in front of an ESB, thereby hiding the internal technical complexities. Leveraging API management this way enables enterprises to expose business functionalities externally without adding drastically to the underlying technical integration complexities.
Future Integration Needs
Similar to the disruption caused by mobile, enterprises are increasingly moving their application and data to the cloud. This, coupled with the availability of a plethora of cloud services (SaaS), makes the need for integration platform as services (iPaaS) for ‘cloud to cloud’ or/and ‘cloud to on-premise’ service integration.
Coupled with the explosion of the internet of things (IoT), these paradigms will likely drive future enterprise integration needs. These demands will require implementation of complex service orchestration logic with the need for agility and ever-increasing ease of integration. The expected growth in traffic will drive demand for higher performance and on-demand container based scalability.
We must also take into account the architecture paradigms, or (as some would look at it) fashions that are currently in vogue. From an architectural perspective, the wide adoption of microservices architecture and container architecture begs the question - is a new service integration approach required?
A product offering supporting micro-integration and service choreography, visual modeling, and an extremely light-weight, container-deployable runtime could very well be the next generation enterprise integration solution you could be looking in the near future.
References: