WittCode💻

How to Send User Activity Email with Cron Nginx and Docker Compose

By

Learn how to send an email from a Gmail account that contains user activity and errors from Nginx using Cron and Docker Compose. We will also learn about best practices when it comes to multiple processes in a Docker container, how Cron loads environment variables, and more.

Table of Contents 📖

Multiple Processes in a Container

Before we start the article, we need to talk about running multiple processes in a Docker container. It is best practice to separate any process into its own Docker container, or one service per container. This is so each service is separated into its own logical component, can be stopped and started independently, and more. This is what we will do with Nginx and Cron, we will have a separate container for each.

INFO: A container's main running process is what is supplied to the ENTRYPOINT and/or CMD commands.

Environment Variable Setup

To start this project, lets set our environment variables. These variables will be loaded into the Docker images with Docker Compose.

NGINX_HOST=nginx-c
NGINX_PORT=80

CRON_HOST=cron-c

EMAIL_FROM=<EMAIL_TO_SEND_FROM>
EMAIL_TO=<EMAIL_TO_SEND_TO>
EMAIL_HOST=smtp.gmail.com
EMAIL_PORT=587
APP_PASSWORD=<GMAIL_APP_PASSWORD>

We will go over the email related environment variables when we write our cron job to send emails.

Creating a Cronjob to Send Emails

Now lets work with Cron. The first thing we will do is create a cron job to send an email every minute. To send emails we will be using SendEmail, a lightweight SMTP email client. We will also use a Gmail account for this demonstration.

* * * * * root sendemail -f $EMAIL_FROM -t $EMAIL_TO -a /var/log/nginx/access.log -a /var/log/nginx/error.log -u "Nginx Logs" -m "Hey there! Here are the Nginx logs." -s $EMAIL_HOST:$EMAIL_PORT -o tls=yes -xu $EMAIL_TO -xp "$APP_PASSWORD" >> /var/log/cron.log
# A new line is required for a valid cron job!

INFO: Make sure there is a new line character at the end of the cron job or it won't run!

  • -f - The email address of the sender.
  • -t - The email address of the recipient.
  • -a - The email attachment, will be our Nginx error and access logs.
  • -u - The subject of the email.
  • -m - The body of the email.
  • -s - The smtp mail relay. An SMTP mail relay is the process of transferring an email from one server to another. The default is localhost:25.
  • -o - Advanced options.
  • tls - Enable/disable TLS. Setting it to yes requires TLS, which is required when using Gmail.
  • -xu - The username for the email account. Alias for -o username=USERNAME.
  • -xp - The password for the email account. For Gmail this is the app password. Alias for -o password=PASSWORD.
  • Output the results of the command to the file /var/log/cron.log.

Note that the majority of these values will be environment variables loaded into the image with Docker Compose.

Gmail Security and App Passwords

Sending an email with a Gmail account using a third party email client isn't as easy as other email providers as Google doesn't allow us to use a regular password for 3rd party applications. Instead, we have to generate an app password for our Gmail account. An app password is a 16 digit passcode that gives less secure apps permission to access our Google account.

INFO: Note that an app password can only be generated for accounts that have two factor authentication enabled. So if you are using an account without two factor authentication, you will not be able to generate an app password.

To generate an app password, follow these steps:

  • Navigate to your Google account.
  • In the search bar type "app password" and click on it in the drop down.
  • Create an app name and click "Create".
  • After clicking "Create" an app password should be displayed on the screen.
  • Place the app password in the .env file.

INFO: Note that if we lose the app password we will have to generate a new one.

Creating the Cron Docker Image

Now lets create our Cron image. To do this, we will use Ubuntu version 24.04 as the base image.

FROM ubuntu:24.04

Now lets import our cron job into our Docker image and change its permissions.

COPY my-cron-job /etc/cron.d/my-cron-job
RUN chmod 644 /etc/cron.d/my-cron-job
  • Copy over the cron job into the /etc/cron.d directory. Cron reads the cron jobs placed in this directory.
  • Make the cron job readable and writable by the owner while read only to the group and others.

INFO: Every user on the system has their own set of cron jobs. A list of cron jobs for a user is their crontab.

Now, it is important to note that cron isn't installed on the Ubuntu Docker image by default. Therefore, we need to install it in our Dockerfile.

RUN apt update
RUN apt install -y cron

INFO: Note that the -y flag provided to apt install is for non-interactive mode.

We also need to install sendemail and some SSL libraries it depends on to work with SMTPS.

RUN apt install -y sendemail
RUN apt install -y libio-socket-ssl-perl

Now lets create our entrypoint instruction. For this, we will make our environment variables accessible to Cron and then run Cron in the foreground.

ENTRYPOINT printenv > /etc/environment && cron -f

INFO: Both the ENTRYPOINT and CMD instructions are executed when a Docker container is created from an image. However, the instructions supplied to CMD can be overwritten while the instructions supplied to ENTRYPOINT can not.

  • Docker Compose loads the environment variables into the container but doesn't place them inside /etc/environment. We need to place the environment variables in the /etc/environment file because it is where Cron will look for environment variables.
  • We run Cron in the foreground in the Docker container with the -f flag so the Docker container keeps running while the cron service is running. Also, if the cron service ever stopped so would the Docker container.

Creating the Nginx Docker Image

Now lets create our Nginx Docker image. We won't have a Dockerfile for Nginx but rather just use the image on Dockerhub directly.

version: '3.9'
services:

  nginx:
    image: nginx:1.25-alpine
    container_name: ${NGINX_HOST}
    restart: always
    env_file: .env
    ports:
      - ${NGINX_PORT}:${NGINX_PORT}
    volumes:
      - ./nginx/access.log:/var/log/nginx/access.log
      - ./nginx/error.log:/var/log/nginx/error.log
  • Create an Nginx image from nginx:1.25-alpine.
  • Set the container name to the NGINX_HOST environment variable.
  • Set the container to restart whenever it exits.
  • Load the environment variables from the .env file.
  • Map the host port to the container port with the NGINX_PORT environment variable.
  • Create a volume for the access.log and error.log files in Nginx. These host volumes will be shared with the cron image to send them in the email.

Nginx Error and Acess Logs

When Nginx faces any issues, whether it be a configuration file error or Nginx is abruptly stopped, details of the error will be written to an error log. These details consist of the error's severity level and information. Below is an example of an error log.

2024/05/08 19:55:15 [error] 29#29: *1 open() "/usr/share/nginx/html/nuts" failed (2: No such file or directory), client: 172.17.0.1, server: localhost, request: "GET /nuts HTTP/1.1", host: "localhost:6464"

When Nginx processes a request, it will log information about the request to an access log. For example, we can log the IP address of the client, their user agent, status code of the response, etc. Below is an example of an access_log.

172.17.0.1 - - [08/May/2024:18:32:52 +0000] "GET / HTTP/1.1" 304 0 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/124.0.0.0 Safari/537.36"

The format and location of these logs are all specified in the nginx.conf file, Nginx's main configuration file using the error_log and access_log directives.

user  nginx;
worker_processes  auto;

error_log  /var/log/nginx/error.log notice;
...


http {
    ...

    log_format  main  '$remote_addr - $remote_user [$time_local] "$request" '
                      '$status $body_bytes_sent "$http_referer" '
                      '"$http_user_agent" "$http_x_forwarded_for"';

    access_log  /var/log/nginx/access.log  main;

    ... 
    server {
      listen       80;
      listen  [::]:80;
      server_name  localhost;

      root /usr/share/nginx/html;

      location / {
        index  index.html;
      }
    }
}

Cron Docker Compose Service

Now lets create our Cron Docker Compose service.

cron:
  image: cron-i
  build: 
    context: ./cron
    dockerfile: Dockerfile
  container_name: ${CRON_HOST}
  restart: always
  env_file: .env
  volumes:
    - ./cron/cron.log:/var/log/cron.log
    - ./nginx/access.log:/var/log/nginx/access.log:ro
    - ./nginx/error.log:/var/log/nginx/error.log:ro
  depends_on:
    - nginx
  • Create the cron service using our Dockerfile.
  • Name the container with the environment variable CRON_HOST.
  • Set the container to restart whenever it exits.
  • Load the environment variables from the .env file.
  • Create a volume for our cron.log file so we can see the output on our host machine.
  • Create readonly volumes for the access.log and error.log files in Nginx. Nginx will fill these files with access and error logs while Cron will read from them.

Running the Program

To run the program, all we need to do is run the command docker compose up at the top level of the directory.

docker compose --env-file .env up

We also supply our environment variable file. This will load the environment variables into our docker-compose.yaml file. Nginx will now be running on port 80. Send a cURL to get the default index.html file.

curl localhost
<!DOCTYPE html>
<html>
<head>
<title>Welcome to nginx!</title>
<style>
html { color-scheme: light dark; }
body { width: 35em; margin: 0 auto;
font-family: Tahoma, Verdana, Arial, sans-serif; }
</style>
</head>
<body>
<h1>Welcome to nginx!</h1>
<p>If you see this page, the nginx web server is successfully installed and
working. Further configuration is required.</p>

<p>For online documentation and support please refer to
<a href="http://nginx.org/">nginx.org</a>.<br/>
Commercial support is available at
<a href="http://nginx.com/">nginx.com</a>.</p>

<p><em>Thank you for using nginx.</em></p>
</body>
</html>

This will add an access log to our access.log file. Now send a request to a 404 route.

curl localhost/wittcepter
<html>
<head><title>404 Not Found</title></head>
<body>
<center><h1>404 Not Found</h1></center>
<hr><center>nginx/1.25.5</center>
</body>
</html>

This will add an error log to our error.log file. Now, at the minute, check the cron.log file for output similar to the following.

May 09 19:22:03 cd237acc615a sendemail[10]: Email was sent successfully!
May 09 19:23:02 cd237acc615a sendemail[13]: Email was sent successfully!

Now we just need to check our email account for the presence of the email containing the access and error log attachments.