We built on AWS, what's next?

As a business owner, taking the leap of moving to the public cloud is a big challenge. Maybe you started by completing a proof of concept, migrating some non-core applications whilst your team started to learn the new platform.

Usually, this early stage generates some uncertainty. After some time you begin to feel confident and authorise the big jump.

Maybe you’ve never heard about the cloud governance framework concept before - perhaps it is something that only the “enterprise” organisations need. Or maybe someone in a conference mentioned the term Cloud Centre of Excellence (CCoE) - It sounded interesting, but on your way home you did some maths and determined it would be very expensive to put in place something like that within your organisation.

So, for the past months or even years, you’ve been operating your digital business using a sole  AWS account, and unless you have been extremely careful and dedicated to your AWS infrastructure, you probably left some EC2 instances running for months, S3 buckets full of data (potentially exposed with public access), EBS snapshots or even AMIs generating costs each month. But, how much is that? In terms of cost, leaving one EC2 instance running for a year could range from $200 to maybe $10,000 depending on the size. Having 10 snapshots of 50GB each, could cost around $300 a year. If you continue adding these small costs up, you might end up with thousands. I don’t blame you. You were just learning and experimenting with the platform.

One of the questions that we need to answer is, when are we ready to take the second big jump? Building a governance framework which allows your business to rely on the Cloud platform, but which also gives you the peace of mind that you will have cost, security and operational excellence under control.

The Governance Framework

How can I create this governance framework? What is it? As with the monolith-to-microservices journey that many applications go through, creating a Cloud governance framework is much the same. Break that AWS monolithic account into a far more distributed structure. And here comes the question about cost. Is this going to cost me more? The answer is always no. When you calculate the costs of your cloud infrastructure, you need to remember to account for how much it would cost if someone gained unauthorised access to your customer database? or if you are not tracking your AWS usage and suddenly you receive a bill that is 200% more expensive than normal.

What’s the base structure of your ideal governance framework? Initially you need to make sure you create an AWS Organization.  AWS Organizations helps you centrally govern your environment as you grow and scale your workloads on AWS. Whether you are a growing startup or a large enterprise, AWS Organizations help you to centrally manage billing; control access, compliance, and security; and share resources across your AWS accounts.

Using AWS Organizations, you can automate account creation, create groups of accounts to reflect your business needs, and apply policies for these groups for governance of security and costs. You can also simplify billing by setting up a single payment method for all of your AWS accounts. 

What’s next? Having formed an AWS Organization, you can easily create the accounts you need to start decentralising your environments and begin collecting monitoring information in a central place for auditing purposes.

Let's build AWS Accounts

The first account we will create is Log-Archive, where we aim to build a central Amazon S3 bucket to place all logs generated by CloudTrail, and while we are at it, why not also add our network or load balancer logs. What is AWS CloudTrail? AWS CloudTrail is a service that enables governance, compliance, operational auditing, and risk auditing of your AWS accounts. With CloudTrail, you can log, continuously monitor, and retain account activity related to actions across your AWS infrastructure. CloudTrail provides event history of your AWS account activity, including actions taken through the AWS Management Console, AWS SDKs, command line tools, and other AWS services. Essentially, CloudTrail is our auditing tool that can capture and report on unauthorised, or unintentional changes to your infrastructure.

Next, let’s build our Security account. In our Security account, we will build a centralised space for our AWS Config and GuardDuty tools. AWS Config is a service that enables you to assess, audit, and evaluate the configuration of your AWS resources. Config continuously monitors and records your AWS resource settings and allows you to automate the evaluation of recorded configurations against desired configurations. With Config, you can review changes in configurations and relationships between AWS resources, dive into detailed resource configuration histories, and determine your overall compliance against the configurations specified in your internal guidelines.

Once all our logging and security accounts are ready, our next challenge is to build a shared-services account. This account will serve as a central hub for common applications used across the organisation. Depending on the size of your CCoE, you can use it for a central CI/CD platform, or for adding your configuration management systems, maybe your DNS zones using Amazon Route53.

Now that we have all our core accounts in place, it’s time to build our Project/Environment accounts. The decision here is how do we want to tackle this? One account per Project-Environment usually is the recommended way, simply because it gives you the flexibility of creating/terminating accounts once the Project or Environment isn’t needed anymore. Also, from a security standpoint it helps to reduce the blast of radius in case something goes wrong.  What you need to consider when creating all these accounts, is how are you going to manage access management, monitoring and governance.

What about IAM?

For identity access management, a solution could be either using AWS SSO (free service) or integrate your own Identity Provider (e.g. Okta, JumpCloud, OneLogin). You can also integrate your own Active Directory if you already have one. Then, you define what level of access you want to provide to each individual who needs to interact with services in your accounts.

What about my services or EC2 instances? As a best practice, I don’t recommend using IAM Access and Secret Access keys, the best option here is to create an IAM Role that can be assumed by your applications wherever they are running (EC2, Lambda, ECS, EKS, etc).

The last piece of the puzzle here is governance. How do you ensure that all your accounts are following the same best practices?. By adopting AWS Organizations you can define Service Control Policies at the root level of your accounts, which means, every new account created will adhere to these policies. For instance, you can declare that all your accounts are prevented from running EC2 Instances if they are not tagged properly. This will give you a greater understanding of what’s running and who owns each asset in your cloud environment. Another example would be limiting the regions you want people to deploy assets on; this will keep your environment under control and save money.

In a second post, I will go deeper into how you can understand the state of your environments from a security and compliance standpoint and more importantly, how to remediate deviations.