DATA MIGRATION IN CLOUD

Cloud migration is the process of moving data, applications or other business elements to a cloud computing environment. There are various types of cloud migrations an enterprise can perform. One common model is the transfer of data and applications from a local, on-premises data center to the public cloud. However, a cloud migration could also entail moving data and applications from one cloud platform or provider to another -- a model known as cloud-to-cloud migration. A third type of migration is to uncloud -- also known as a reverse cloud migration or declouding -- where data or applications are moved off of the cloud and back to a local data center.

When companies move their data and applications from their premises to the cloud, this process is called cloud migration. The process may involve moving all your applications and services, or it may take a slower approach where some applications are moved to the cloud, while others remain on-premise. This approach is called a hybrid migration. In either case, a cloud data warehouse can provide you many benefits over your on-premise solution.

Key advantages to moving to the cloud

Companies move to the cloud for many reasons. Below are just a few of the ways in which moving to the cloud can be a winning decision for your organization.

• Cost-effective. When you maintain your own data warehouse onsite, you are responsible for maintaining the infrastructure and workforce to support every aspect of your warehouse. But when you move to the cloud, you can get the benefit of world-class infrastructure while saving money
• Simplified. Much of the complexity of a data warehouse can be handled by a team of product experts at a cloud data warehouse provider. This means your employees can focus on the things that really matter, like growing your business.
• Scalable. One of the biggest advantages of the cloud is scalability. Whether you are scaling up quickly or slowly, a cloud data warehouse can accommodate you.
• Secure. Cloud data warehouses are responsible for providing services that meet common compliance requirements, including certifications like SOC 2, ISO27001, HIPAA, and PCI. When you move to the cloud, a team of security experts is part of the bargain.

What is AWS Migration?

·        2.5x quintillion new bytes of data generated each day. With so much data around, cloud migration is an ideal solution today.

·        It is the process of moving data, applications, or other business components from an organization’s on-premises infrastructure to the cloud, or moving them from one cloud service to another.

 Cloud Data Migration Challenges

1. Amazon S3 Bucket Name Restrictions

An Amazon S3 bucket name has certain restrictions. The biggest of these Amazon S3 bucket name restrictions is that every bucket name used on AWS has to be unique. For hosting a static website, it is mandatory for a bucket name to be the same as the DNS.

2. AWS SSL Certificates and Amazon CloudFront

It is recommended to have an AWS SSL certificate for a website. It will help for better indexing for search engines as well as for establishing the reliability of your site. Unfortunately, Amazon S3 does not provide an option for SSL with static website hosting. Although an individual object can be accessed using “https,” when it is hosted as a website you cannot access it with SSL. The best solution to this problem is to use Amazon CloudFront.

3. Defining Cache Policy with Amazon CloudFront

Although Amazon CloudFront helps implement https protocols, it also comes with one consideration. Amazon CloudFront is ultimately a caching service that caches the content at edge locations. It is very important that you define cache policy the best way possible. 

In some cases when a cache is not expired, your new updates might not be visible on the site. In this case you might need to invalidate all of the cache to make the updated page available to site visitors. Be specific when invalidating: if you provide generic criteria it will clear all objects including those not modified, too.

4. Data Consistency One thing to remember while uploading objects to Amazon S3 is its data consistency models. When you upload new objects, they are available immediately but when you update or delete objects that data might not be available due to Amazon S3’s eventual consistency model. 

When updated pages do not load, don’t assume that it is a cache issue with Amazon CloudFront, as mentioned above: it can also be an Amazon S3 data consistency issue.

5. AWS S3 Security With cloud and publicly-hosted websites, security is a prime concern. In an

AWS migration, the SSL part will be taken care of with AWS Certificate Manager and Amazon CloudFront, however accessing buckets can be restricted with Origin Access Identity (OAI). 

When using Amazon CloudFront you have the added advantage of being able to implement the AWS firewall, AWS WAF. AWS WAF allows you to control distribution requests based on conditions you set. 

One more security option that works with Amazon S3 is setting proper bucket policy using Amazon S3 security access control lists (ACL). With OAI, you can always restrict bucket access, however it is also important that you use proper IAM policies to give access only to internal users. Refer to these AWS guidelines for more on securing Amazon S3 access.

The five phases of migrating to Google Cloud Platform

Migrating to Google Cloud Platform isn’t hard per se, but there’s a lot to think about. And thankfully, it doesn’t need to happen all at once.

Phase One: Assess before you move a single bit, take stock of your applications and how suitable they are for the cloud. Things to think about include (but are not limited to) hardware and performance requirements, users, licensing, compliance needs and application dependencies.

Phase Two: Pilot this is the point where you take one or two applications, and try moving them. Learn about Cloud Platform and its design patterns, take the time to validate performance, consider your licensing options and establish how to perform a rollback. Don’t skip this step, and don’t be tempted to try and migrate too many apps all at once!

Phase Three: Move Data Some people will tell you to move your applications first, then move your data, but we beg to differ. Most applications have a lot of data with a lot of dependencies. Properly moving data to the cloud sets the stage for a successful application migration later on.

Phase Four: Move Applications Now that your data is in the cloud, you’re ready to move the actual apps. Here too, there are decisions to make. We recommend keeping things simple, and doing the minimum necessary to get the application up and running in the cloud, for example doing a straight lift-and-shift. Or perhaps you can get an app into the cloud by way of backing it up there? That way, in the event of an outage, there’s a full copy of your environment in GCP waiting to take over.

Phase Five: Optimize this is where the fun begins. Once an application and its data have been migrated to GCP, you can start thinking about all the cool ways to make it better. For example, this might be a time to add redundancy in the form of availability zones, elasticity with auto scaling groups, or enhanced monitoring with Stack driver. You might want to offload static assets off of your application tier into Cloud Storage, or decouple tiers by using Pub/Sub. Google’s Deployment Manager can make it easier to launch and scale new instances, and copying your configuration into a second region can insulate you from a regional outage.

CONCLUSION

Overall, Amazon S3 is a great tool for hosting static content, but as we just saw, there are a number of things to keep in mind when migrating your files there. Additional considerations are data persistence (making sure all files are migrated), security, and the migration performance when moving a large number of files. 

Taking all those considerations together, it seems as if carrying out a cloud data migration on your own and configuring all the security, AWS migration, coding, and more can amount to a lot of time-consuming and costly effort. 

To make migrations faster and to save expenses, there are tools available to help ease the process of migrating cloud data, namely Cloud Sync, NetApp’s solution for data migration to Amazon S3.

Author: Tanvish Tuplondhe

(B.tech integrated, MPSTME, NMIMS)

Under guidance of LEENA NADKAR