MongoDB .local Toronto 2019: MongoDB – Powering the new age data demands

About me ..
Vijay Bandaru
Vijay Bandaru -
Principal Enterprise
Architect – Open
Source Practice
2

MongoDB…
Powering the new age data demands
3

Where are we seeing the usage of new age databases?
4
…and all of these keeping in view of the volume, velocity and variety of data
Building
highly available
applications
Enabling
Event-driven
architectures
Implement
distributed
query capability

MongoDB has been solving many of these new age
demands
Let us take a peek into some of these real world use cases
5

Client Scenario # 1
6
1
Who is the client?
Client is an American multinational technology conglomerate. They have embarked on a project to
automate one of their business processes to enhance customer experience and reduce customer churn.
The existing process is manual and touches multiple systems and applications across various lines of
business. A project was initiated to design a target architecture that will completely automate the process
and reduce the elapsed time from month to minutes.
What were we trying to solve?
The objective of this project was to design and implement a solution that will seamlessly integrate with
heterogeneous applications thereby reducing or eliminating manual touch points. The target architecture
was designed following an event driven architecture with Kafka as the integration broker. It was required
to develop a mechanism to have a real-time view on the progress of the transaction as it moves from one
stage to the other.

How is MongoDB leveraged in this use case?
7
1
A transaction is complete when it has gone through all the stages as shown
here. It was important to have a visibility into the process and also there was
a need to recreate the entire transaction in case of a failure.
Each time a stage event used to occur, that event gets also pushed to
MongoDB which forces a refresh on Angular. To persist the transaction
state, one collection has been created with the below data model
{
“transcationId”:”12345”,
“currentState”:” Quote Creation”,
“noOfCompletedStage”:3
“completedStages”: [
{“stageName”:” Client Onboarding”,
“stageStartTime”:” 19:08:00”,
” stageEndTime”:” 19:10:00”},
{“stageName”:” Opportunity Creation”,
”stageEndTime”:” 19:12:00”},
{“stageName”:” Quote Creation”,
” stageEndTime”:” 19:15:00”}]
}
Architecture

What were the salient features of MongoDB used in the use case?
8
1
Schema Free
Design
• The framework need to support multiple business process
• Each of them may have a different payload to process
Aggregation
framework
• Enable useful insights and optimization recommendations
• Ability to aggregate the transactions by multiple dimensions
Change
Streams
• Enable auto refresh of the transaction monitoring dashboard
• Any change or modification to the transaction must push it to the dashboard

Client Scenario # 2
9
2
Who is the client?
Client is an American multinational technology conglomerate. Their current return merchandise process is
manual and error prone. If a customer returns a defective item, it takes manual effort and time to
determine whether the item should be fixed or a new item needs to be shipped
The objective of this project was to design a framework that will orchestrate the entire return merchandise
process. The orchestration metadata must be abstracted from the process so that it can support any
change in the process with minimal changes. The orchestration metadata store was chosen as MongoDB

10
2
{
“eventName”:XXXX,
“eventType” : Action
“eventAPI”:
{ “endpoint”:xxxx
“oauthpurl”:xxxx
“payload”:XXXX
}
}
{
“eventName”:XXXX,
“sourceTopic”:XXXX,
“targetTopic” :XXXX
}
Event 1 Event 2 Event n
Source
topic
lookup
Processing
Layer
Sink
Topic
Source
topic
lookup
Processing
Layer
Sink
Topic
Source
topic
lookup
Processing
Layer
Sink
Topic
Metadata Cache

Client Scenario # 3
11
3
Who is the client?
Client is an American Healthcare retail giant. They have embarked on a project to improve the productivity
of their store employees. As a part of a day’s schedule of the store employee, they have to execute many
workflows. A project was initiated to design a target architecture that will automate part of their workflow
steps with the help of Artificial Intelligence and Machine Learning.
Objective of this project is to design and implement a solution that would integrate with machine learning
algorithms that learn from historical data. The machine learning algorithms would execute not only of data
being sent on the transactional data but would also need reference and master data. As the algorithm
would retrain and mature over time, there would be need for additional reference and master data. This
would mandate a need of flexible data model for different master and reference data entities.

Target State Architecture
12
3
A document based data model was
a perfect choice to solve this use
case and MongoDB was chosen
as the source of reference to
render this master and reference
data entities to the Machine
learning algorithms.
It is also a perfect choice to log the
output of the different algorithms
and orchestration engine. This data
would be used for transaction
processing, operational reporting
and production support.

13
3
{“_id”: {“patient_id” : Number},
“demographics”: { “dob” : String
“gender”: String
“zip_code”: string},
…
}
As more data attributes are needed for Machine
learning algorithms, the schema free nature of
MongoDB provides support for rapid development and
deployment of changes
Support for 250+ TPS across 50000 Terminals that support the store employee workflow. This is
supported by the Active-Active deployment of the application.
MongoDB sharding and replication across data centers supports the workflow. Reads from secondary to
support performance and High Availability requirements
Multiple processes are supported by Change Streams
-Chain State Store level Settings and Roll out of Machine learning algorithms
-Audit and event functions
Operational Data to be persisted for 10 days and then the data gets purged.
TTL Index defined to be only present for 10 days.
Schema Free
Design
Sharding and
Replication to
achieve HA
Change
Streams
Purge through
TTL

Client Scenario # 4
14
4
Who is the client?
Client is American insurance firm. They decided to embark on a new journey to modernize customer’s IT
applications & infrastructure design. The target architecture will allow new businesses and vendors to
collaborate on centralized web portal. The new system will be moving away from the existing Oracle
environment and will be capable of handling unstructured data streams from various agents and third
party businesses.
Objective of this project is to maintain insurance information whose effective dates are in the past,
present, or future. The system also need to turn back the clock and see agreement data as it was at a
specific time in the past. This is useful for retrieving the historical agreement documents, for meeting
statutory data retention requirements, and for understanding how your data changes as it moves from
creation through obsolescence.

Target State Architecture
15
A document store was a perfect choice
for this requirement and MongoDB was
chosen as it offers full ACID transaction
support.
The Persistence Engine will be designed
to have the maximum flexibility to handle
transactions for current and future
business needs and implement
separation of abstract data transactions
from the business logic. The bi-temporal
data transactions need to be transparent
to the application without complicating
the development and maintenance.
4

Schema Less
Design
Data
transactions
Scaling through
Sharding
16
The insurance data is inherently rich data with a wide variety of ever-changing data attributes. MongoDB
is the leading document-store, which is designed to offer a rich experience with support for modern
programming languages and rapid development techniques.
MongoDB is the only database that fully combines the power of the document model and a distributed
systems architecture with ACID guarantees. Through snapshot isolation, transactions provide a consistent
view of data, and enforce all-or-nothing execution to maintain data integrity, even across sharded clusters.
Customer deals with volumes of data that requires handling ever-changing insurance information at
scales. MongoDB dataset can linearly scale across multiple servers without the need of knowing when
the physical horizontal scale comes into play.
4
Document
Oriented DB
MongoDB is the most mature document-oriented database in the database industry today. MongoDB
inherently offers all the features and techniques suitable for storing and managing big data-sized
collections of literal documents like text documents, email messages, XML documents, etc.

Client Scenario # 5
17
5
Who is the client?
Client is major international online e-commerce platform provider. They are planning multi-year journey
with multiple strategic objectives, 1. Replacing monolithic application using micro-services architecture 2.
Considerations of European & Asian legal data residency requirements 3. Respond to highly elastic
business demand on a short notice 4. Products quick to market
Part of the objective was to use right data storage technology and architecture to deliver a global scale
application, converting services from a legacy application using distributed computing capable of rapid
development, elastic capacity, data caging, high performance & fault tolerant.

18
5
{
name: "Midhuna",
spouse: {
name: "Akash",
age: 25
}
}
Flexible schemas are effective to use with micro-services
architecture, quick data models using complex data types,
rapid development, easy adoption for development and
design changes.
Data model –
Quick to market
Supports a hybrid load of 5000 writes and 15000 reads per second. Data reads, writes can be highly
distributes using data-sharding and replication features. Individual queries can be tuned using indexes
based on data access patterns.
High Performance
& HA
Using data-sharding and shard tags, data storage can be ensured at a specified location. This
option can significantly help with country specific legal requirements and helps with performance by
colocation of data with the users.
Data
Residency
Ops Manager is very sophisticated tool from MongoDB, improves ease & efficiency of building and
managing clusters significantly. In additional to build & operate clusters, Ops Manager also provides all
administration capabilities to monitoring/notification, backups/recovery, auto failovers, performance
metrics visibility etc.
Easier to build
and operate

Architecture
19
5
MongoDB was the right choice with native
capabilities like quick data modelling,
shard tags for data residency, high
performance and high availability, easy to
add/remove capacity.
MongoDB is also one of most matured
data storage option in NoSQL/Document
database domain for machine critical
applications and a reliable partner for
enterprise strategy towards open source
adoption.

© 2019 Infosys Limited, Bengaluru, India. All Rights Reserved. Infosys believes the information in this document is accurate as of its publication date; such information is subject to change without notice. Infosys
acknowledges the proprietary rights of other companies to the trademarks, product names and such other intellectual property rights mentioned in this document. Except as expressly permitted, neither this
documentation nor any part of it may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, electronic, mechanical, printing, photocopying, recording or otherwise, without the
prior permission of Infosys Limited and/ or any named intellectual property rights holders under this document.
THANK YOU

MongoDB .local Toronto 2019: MongoDB – Powering the new age data demands

Recommended

More Related Content

What's hot (20)

Similar to MongoDB .local Toronto 2019: MongoDB – Powering the new age data demands (20)

More from MongoDB (20)

Recently uploaded (20)

MongoDB .local Toronto 2019: MongoDB – Powering the new age data demands