AWS DevOps automation for STATIC WEBSITES.

If you manage a single team working on a single project you can go through placing a nice DevOps pipeline by using Jenkins (or similar) and ask your infrastructure operators the required integration. The problem may arise when you have multiple teams using your infrastructure (IaaS) and don't want to manually set up everything every time a new project comes.

To get some ideas on how to get a solution, I will automate the DevOps pipeline setup for a simple scenario.

DevOps automation for STATIC WEBSITES.

The good point about AWS is that everything is API oriented, and that means you can programmatically do anything. We can set up, for example, a DevOps automation architecture by using a Lambda function to catch new project requests and a CloudFormation template to create the required pipeline:

Using this architecture, when a team wants to work on a new website, a single function call will set up all the required resources without the need for manual configurations from the cloud ops team (you can take a look at the CloudFormation template here).

"Building" stage.

To build your static website you only need to "move the files" from the project repository to the S3 buckets where your content is published. To complete this work you can use a Lambda function (as proposed by Eric Hammond) or use simple AWS CLI commands included in a CodeBuild project (I used that method to unify with more complex scenarios to come):

# Copy GitRepo source to S3 TEST Environment bucket 
# and invalidates the related cloudfront distribution.
  
CodeBuildDeployTestSite:
      Type: AWS::CodeBuild::Project
      DeletionPolicy: Delete
      DependsOn:
        - CodeBuildRole
        - SiteTestBucket
      Properties:
        Name: !Sub ${ProjectCode}-codebuild-deploy-test
        Description: Deploy site to TEST Environment.
        ServiceRole: !GetAtt CodeBuildRole.Arn
        Artifacts:
          Type: CODEPIPELINE
        Environment:
          Type: LINUX_CONTAINER
          ComputeType: BUILD_GENERAL1_SMALL 
          Image: aws/codebuild/ubuntu-base:14.04 
        Source:
          Type: CODEPIPELINE
          BuildSpec: !Sub |
            version: 0.1
            phases:
              post_build:
                commands:
                  - aws s3 cp --recursive --acl public-read ./httpdocs s3://${SiteTestBucket}/
                  - aws cloudfront create-invalidation --distribution-id ${CloudFrontTestDistribution} --paths /*
            artifacts:
              type: zip
              files:
                - ./httpdocs/*
        TimeoutInMinutes: 10
        Tags:
          -
           Key: "project"
           Value: !Ref ProjectCode

IAM Security.

As we are just configuring a website development pipeline, the project team will just need to push the website content files to the repository (no need for devs to access a server or any other resource).

# Cloudformation resource to create an
# IAM User to access TEAM repository

ProjectUser:
   Type: "AWS::IAM::User"
    Properties:
      UserName: !Ref ProjectCode
      Policies:
        - PolicyName: !Sub "${ProjectCode}-codecommit"
          PolicyDocument:
            Version: "2012-10-17"
            Statement:
              - Effect: "Deny"
                Action:
                  - "codecommit:CreateRepository"
                  - "codecommit:DeleteRepository"
                  - "codecommit:UpdateRepositoryName"
                Resource: !Sub "arn:aws:codecommit:${AWS::Region}:${AWS::AccountId}:${GitRepo}"
              - Effect: "Allow"
                Action: "codecommit:*"
                Resource: !Sub "arn:aws:codecommit:${AWS::Region}:${AWS::AccountId}:${GitRepo}"
              - Effect: "Allow"
                Action:
                  - "iam:DeleteSSHPublicKey"
                  - "iam:GetSSHPublicKey"
                  - "iam:ListSSHPublicKeys"
                  - "iam:UpdateSSHPublicKey"
                  - "iam:UploadSSHPublicKey"
                Resource: "arn:aws:iam::*:user/${aws:username}"
              - Effect: "Allow"
                Action:
                  - "iam:CreateServiceSpecificCredential"
                  - "iam:UpdateServiceSpecificCredential"
                  - "iam:DeleteServiceSpecificCredential"
                  - "iam:ResetServiceSpecificCredential"
                Resource: "arn:aws:iam::*:user/${aws:username}"

Note that you still need to create the SSH credentials to access the project repository (no way to create SSH files by using CloudFormation, but I will solve that in my next post!).

Pipeline Outputs.

After initializing the DevOps pipeline you have to pass to developers how to proceed. In this case, you will just need to send them the repository URL (to clone and push website content) and the related access credentials.

Outputs:

  GitUrl:
    Value: !GetAtt GitRepo.CloneUrlHttp
    Description: The URL to use for cloning the repository over HTTPS.

  WebsiteTestUrl:
    Value: !Sub "https://test.${DomainName}"
    Description: Website Test Environment URL.
    
  WebsiteUrl:
    Value: !Sub "https://${DomainName}"
    Description: Website URL.

Approval Action

Moving from test environment to production environment must always include some approval action to validate everything is ok. In this simple scenario, we add a manual approval step to check the website for final publication.

When this validation is required, a simple email notification is sent, including the website link to check, and the related instructions to accept or deny.

Conclusions

We now have some ideas on how to progress with DevOps automation by defining a Lambda function to catch the setup requests and a Cloudformation template to build a new stack for each project.

It's time to try more complex scenarios!