Automating Memory Usage Monitoring and Alerts on Linux: A Simple Solution for System Administrators

Introduction

Monitoring server resources is essential for preserving peak performance in the fast-paced field of system management. Memory is one important resource that can cause system slowdowns or crashes if improperly handled. In this post, I'll show you how I created a straightforward but powerful script that tracks memory utilisation on a Linux machine automatically and notifies you via email when it goes above a predetermined limit. This solution can be a helpful addition to the toolkit of any system administrator and is lightweight.

The Problem:

Memory use tends to increase when servers execute numerous apps and services. Critical apps may lag or even crash due to excessive memory usage. Although the majority of Linux systems use commands like free to provide memory usage statistics, manually verifying these statistics can be laborious and prone to human error. In order to prevent performance problems, we require an automatic solution that can notify us when memory usage beyond a predetermined level.

The Solution:

To address this, I created a script that periodically checks memory use while operating in the background. The purpose of the script is to:

1- Check the memory use on a regular basis.

2- Determine the percentage of memory used. Keep a record of this data for further examination.

3- In the event that memory use surpasses 70%, send an email warning.

In-depth Analysis:

1. Writing the Script:

The script is a Bash script, which leverages the free command to fetch memory statistics. Here’s a breakdown of the logic:

# Email Details
TO="recipient1@example.com, recipient2@example.com"
SUBJECT="Memory Usage Alert"
FROM="your-email@example.com"
LOG_FILE="/path/to/Memory_report.tsv"

while :
do
    # Get memory usage details
    memused=`free -m | head -2 | tail -1 | awk '{ print $3 }'`
    totalmem=`free -m | head -2 | tail -1 | awk '{ print $2 }'`
    memused_percentage=`expr $memused \* 100 / $totalmem`

    # Log memory usage data
    echo -e "$(date)\t\n The Memory_usage is: $memused_percentage%" >> $LOG_FILE

    # Check if memory usage exceeds threshold
    if [ $memused_percentage -gt 70 ]; then
        echo -e "Subject: $SUBJECT\n\n!!Alert!!\nMemory usage has exceeded 70%.            Current usage is $memused_percentage%." | msmtp $TO
    fi

    # Wait for the next check
    sleep 10
done        

2. Explanation of the Script:

• Memory Usage Calculation: The script uses free -m to get memory statistics and calculates the percentage of used memory by dividing the used memory by the total memory.
• Logging: The script logs the memory usage data into a file (Memory_report.tsv) so that it can be analysed later.
• Email Alerts: If memory usage exceeds 70%, an email is sent using the msmtp tool to notify the team.
• Loop and Sleep: The script runs in an infinite loop (while :) and checks memory usage every 10 seconds.

3. Setting Up Email Alerts:

To send email alerts, I used msmtp, a simple and lightweight SMTP client for Linux. Here’s how I set it up:

• Install msmtp using the package manager

 yum install msmtp        

• Configure msmtp to work with your email provider (e.g., Gmail). You can do this by editing the configuration file at /etc/msmtprc.

Example configuration:

account default
host smtp.gmail.com
port 587
auth on
user your-email@gmail.com
password your-app-password
from your-email@gmail.com
tls on
tls_starttls on
        

4. Automating the Script with Cron:

To ensure the script runs automatically when the system starts, I added it to the crontab file. Here’s how:

1. Open the crontab for editing:

crontab -e        

2. Add the following line to run the script at startup:

@reboot /path/to/your/memory_usage.sh        

This ensures that the script will execute each time the server reboots.

Demonstrating the Memory Usage Alert: Script in Action

In this section, we will demonstrate how the memory usage monitoring script works by simulating an increase in memory usage. This will trigger the script to send an alert when the memory usage exceeds the 70% threshold.

1. Simulate Memory Usage Increase

To simulate high memory usage, we will use the stress tool, which is a command-line utility that can simulate system load by consuming CPU, memory, and other resources. In this case, we will focus on increasing the memory usage to trigger our alert.

To use stress, you first need to install it. If it's not already installed, you can install it with the following command:

yum install stress        

Once stress is installed, you can use it to allocate a specific amount of memory for a short period. For this example, we'll allocate 3 GB of memory for 60 seconds:

stress --vm 1 --vm-bytes 3072M --timeout 40s        

Explanation of the command:

• --vm 1: This option tells stress to use 1 worker to allocate memory.
• --vm-bytes 3072M: This specifies the amount of memory to allocate. In this case, it’s 3 GB (3072M).
• --timeout 60s: The stress command will run for 60 seconds, during which time it will consume 3 GB of memory.

This command will cause the system to use 3 GB of memory for 40 seconds, simulating a scenario where memory usage increases significantly. The script we implemented earlier is designed to monitor memory usage continuously, and if the usage exceeds 70%, it will automatically send an email alert.

2. Trigger the Email Alert

Once you execute the stress command and the system consumes 3 GB of memory, the memory usage will likely exceed the threshold of 70%, depending on your system's total memory. At this point, the script will detect the high memory usage and send an email to the designated recipients (such as your team members).

For example, if the memory usage crosses 70%, the email alert will look like this:

We can also check Logs by seeing the content of our Memory_report.tsv file.

cat   Memory_report.tsv         

Final Tips:

• Optimizing the Script: You can further enhance the script by adding logging for CPU usage, disk space, or even setting up thresholds for different alerts (e.g., for CPU usage).
• Scalability: If you have multiple systems to monitor, you can extend this script by setting up central logging or using a tool like Nagios or Prometheus for more comprehensive monitoring.

Conclusion:

This simple yet effective approach to memory monitoring allows system administrators to stay on top of memory usage without manually checking it. By automating the process and sending real-time alerts, we can address potential issues before they affect system performance. This project not only demonstrates the power of Bash scripting but also emphasizes the importance of proactive monitoring in system administration.