Configuring LoadBalancer Using Ansible

This article is about creating a loadbalancer using ec2 instances from Ansible

Have you ever wondered that, even millions of clients hit google server, still there would be no downtime for any one.

The IT companies use a concept called loadbalancer to balance the client requests.

But how to configure thousands of webservers manually for the above. Can it be done? It maybe done but there will a huge loss to the company as it requires huge man power, enormous time, etc. So now can it be automated.

Yes the whole process can be automated using many tools, one of it is configuration management tool. In this task we use ansible as the tool.

Ansible

Ansible is an open-source software for provisioning, configuration management, and application-deployment tool enabling infrastructure as code. It runs on many Unix-like systems, and can configure both Unix-like systems as well as Microsoft Windows.

Problem Statement..

♦️Provisioning EC2 instances through ansible.

♦️Retrieve the IP Address of instance using dynamic inventory concept.

♦️Configure the web server in webserver instance through ansible!

♦️Configure the load balancer in loadbalancer instance through the ansible!

♦️Assign the websevers to the loadbalancer

To know about provisioning, configuration-management, visit the below link.

LoadBalancer: Load balancing is defined as the methodical and efficient distribution of network or application traffic across multiple servers in a server farm. Each load balancer sits between client devices and backend servers, receiving and then distributing incoming requests to any available server capable of fulfilling them.

In our Project Haproxy is used for loadbalancing

Haxproxy: HAProxy is free, open source software that provides a high availability load balancer and proxy server for TCP and HTTP-based applications that spreads requests across multiple servers. It is written in C and has a reputation for being fast and efficient.

HAProxy is used by a number of high-profile websites including GoDaddy, GitHub, Stack Overflow, Reddit, Twitter, etc.

Starting from Scratch

In ansible there is a concept of roles, which is like a package with fully configured tasks for specific configuration. These can be used by many systems. By running it we can automatically configure the system, for ex: configuring webserver.

Roles: Roles are ways of automatically loading certain vars_files, tasks, and handlers based on a known file structure. Grouping content by roles also allows easy sharing of roles with other users.

Creating Role for Launching ec2 instance

To create a role, first create a folder with a name, in future it includes the roles created

#bash

mkdir /etc/ansible_roles

Updating ansible.cfg file for roles

Command to create ansible role

#bash

ansible-galaxy init "folder_name"

Updating the vars/main.yml file

region: "ap-south-1"
img: "ami-0ebc1ac48dfd14136"
ins_type: "t2.micro"

sgid: "sg-08ed3eda09e8edc13"

Next we have to create another file in any other folder which contains aws credentials.

# AWS_ACCESS_KEY_ID
access_key: "AKIA2GH*************"
# AWS_SECRET_ACCESS_KEY

secret_key: "0HS9K5u***************************"

As the file contains aws access key and secret key id which are confidential, we have to secure it with a password, so that it cannot be easily accessible.

# to secure file
ansible-vault encrypt --vault-id keys@prompt aws_cred.yml

# to view file
ansible-vault view --vault-id keys@prompt aws_cred.yml

# to edit file

ansible-vault edit --vault-id keys@prompt aws_cred.yml

For the first command it asks for the password to be set. The vault-id name can be anything with @prompt like '*****@prompt'.

For the other two commands it asks for password that has been set.

The file is been encrypted.

Updating tasks/main.yml file in role.

- name: launching ec2 instance
  ec2:
          region: "{{ region }}"
          image: "{{ img }}"
          instance_type: "{{ ins_type }}"
          vpc_subnet_id: "subnet-b88ae1f4"
          assign_public_ip: yes
          group_id: "{{ sgid }}"
          instance_tags:
                  # for webserver
                  name: webserver
          count: 1
          wait: yes
          key_name: "key2"
          state: present
          aws_access_key: "{{ access_key }}"

          aws_secret_key: "{{ secret_key }}"

Note: Change the instance_tags name for webserver and loadbalancer.

Now the role is configured.

For running the ec2 module ansible requires some sdk. As ansible is built on python we have to download the sdk from pip

#bash
pip install boto 

pip install boto3

Next we have to write a playbook for running this role.

Here we include the aws_cred.yml file . Both the playbook and cred files are in the same folder.

Now we are all set to run the playbook. To run

#bash

ansible-playbook --vault-id keys@prompt "filename"

Similarly we provision another ec2 instance for loadbalancer

While launching hundreds of ec2 instances we cannot manually update the ansible inventory with all the instance details. So to overcome this limitation ansible provides dynamic inventory.

For setting up of the dynamic inventory in ansible for ec2-instance, visit below link

After setting up dynamic inventory, to list the hosts:

Command to list the hosts

#bash

ansible ec2 --list-hosts

Now for ansible connectivity we have to add some more keywords in ansible config file.

[defaults]
inventory = /etc/ansiblehosts/ec2
host_key_checking = False
roles_path = /etc/ansible_roles

# disabling password asking during login
ask_pass = False

# private key for the instance
private_key_file = /etc/ansible_keys/key2.pem

remote_user = ec2-user


[privilege_escalation]
become = true

# to enable sudo for ansible
become_method = sudo
become_user = root

# disabling for asking sudo password

become_ask_pass =False

Run the below in the folder where key is present

chmod 400 key2.pem

To check the connectivity run

ansible ec2 -m ping

Provisioning of ec2 instance for webserver and loadbalancer is done.

Configuring LoadBalancer

For configuring the loadbalancer we create a role.

Updating tasks/main.yml file

# tasks file for lbserver


- name: haproxy install
  package:
          name: "haproxy"
          state: present


- name: copy haproxy.cfg
  template:
          src: "haproxy.cfg"
          dest: "/etc/haproxy/haproxy.cfg"
  notify: haproxy restart


- name: haproxy start
  service:
          name: "haproxy"
          state: started

Here we use haproxy for loadbalancer

The haproxy.cfg file is pre-downloaded and saved in templates folder

We have to edit some of the contents in the file.

Updating handlers/main.yml file

Handlers are the files which run only when notified by the task/tasks , which when gets changed.

In our role the handler runs only when haxproxy.cfg file changes, i.e the haxproxy service restarts.

- name: haproxy restart
  service:
          name: "haproxy"
          state: restarted

Role for loadbalancer is created.

Creating Role for WebSever

Updating tasks/main.yml file

# tasks file for webserver


- name: httpd install
  package:
          name: "httpd"
          state: present


- name: copy content
  copy:
          content: "Hello from {{ ansible_hostname }}"
          dest: /var/www/html/web.html


- name: httpd service start
  service:
          name: "httpd"
          state: started

Role for webserver is created.

At last we have to create a playbook for running both the roles

- hosts: "{{ groups.tag_name_webserver }}"
  roles:
          - webserver


- hosts: "{{ groups.tag_name_loadbalancer }}"
  roles:
          - lbserver

Running the playbook setup.yml

#bash
ansible-playbook setup.yml

Output

Haproxy is installed and started.

haproxy.cfg updated with webserver ip.

httpd installed, webpage is copied and service has started.

But here only one webserver is up and there is no point of load balancing.

We can see that while running curl for the loadbalancer ip, there is only webserver.

Lets scale it up.

Two more webservers are launched and are up.

Now lets configure them.

haproxy.cfg is updated with new webserver's ip.

Checking the working of loadbalancer

All the webservers are balancing the load.

Finally the requirement of our project is satisfied

All the files are available in the github link

Thank You