This article goes through the steps on how to egress logs from a MedStack cluster to Logz.io using Filebeat. There are multiple ways of running Filebeat on MedStack, however this article will focus on running it as a Docker container inside a MedStack cluster.

Running Filebeat as a Docker container

Filebeat running as a container runs with an assumption that the service running the application is logging to a file that is stored in a volume attached to the application service. In order to read the log file, Filebeat container also mounts the volume to the same folder path specified in the log paths of the filebeat.yml file.

Note: Due to the way volumes work on Docker, the Filebeat service will need to run containers on all nodes that are running application containers. This is because volume data are localized to nodes. You may use (1) node labels, (2) service placement constraints, and (3) redundant service configurations to ensure Filebeat is deployed on each node running application containers.

Prerequisites

Create a custom docker image using a Dockerfile

Use the following Dockerfile to create a custom Filebeat Docker container with SSL certificate required for sending logs to Logz.io.

FROM docker.elastic.co/beats/filebeat:7.15.1
USER root
RUN curl https://raw.githubusercontent.com/logzio/public-certificates/master/AAACertificateServices.crt --create-dirs -o /etc/pki/tls/certs/COMODORSADomainValidationSecureServerCA.crt

Generate filebeat.yml config file in Logz.io

Use Logz.io to generate a filebeat.yml file by using the "Configuration Wizard"

Example filebeat.yml

############################# Filebeat #####################################
filebeat.inputs:
- type: log
 paths:
 - /usr/src/app/logs/record.log
 fields:
 logzio_codec: plain
 token: {Logz.io token}
 type: flask
 fields_under_root: true
 encoding: utf-8
 ignore_older: 3h

#For version 6.x and lower
#filebeat.registry_file: /var/lib/filebeat/registry

#For version 7 and higher
filebeat.registry.path: /var/lib/filebeat

#The following processors are to ensure compatibility with version 7
processors:
- rename:
 fields:
 - from: "agent"
 to: "beat_agent"
 ignore_missing: true
- rename:
 fields:
 - from: "log.file.path"
 to: "source"
 ignore_missing: true

############################# Output ##########################################
output:
 logstash:
 hosts: ["listener.logz.io:5015"] 
 ssl:
 certificate_authorities: ['/etc/pki/tls/certs/COMODORSADomainValidationSecureServerCA.crt']

Create a "logs" volume for filebeat

Click on "New volume" in the Volumes section
Enter the volume name "logs" in the name field
Click "Create volume"

The result of the log volume and services mounting the volume...

Deploying Filebeat in a cluster

In the cluster that you'd like to deploy Filebeat, navigate the "Manage Docker" to make the following configurations.

Step 0 - Add unique labels to each node where application is running

Click on "Actions" button
From the dropdown click on "Label"
Click on "Add label" button
Fill the "Name" and "Value" fields to uniquely represent this node
- For Example: Name = Worker and Value = 1
Click Update to save the labels
You should see the label for the node on the main Control dashboard.
Repeat for each node running the application

Adding a Label...

Step 1 – Create a Filebeat config

Click on "New Config" in the Configs section
Enter config name "filebeat.yml" in the field
Copy and past the "Example filebeat.yml" data above in the config "Data" field
Click "Create Config"

Creating the config...

Adding data to the config...

Step 2 – Create the Filebeat service

1. Click on "New Service" button
2. Enter the service name "filebeat_1"
3. Enter the customer image's name
4. Click on "Add Config" button
  1. From the dropdown "filebeat.yml" config from the dropdown
  2. Enter the path "/usr/share/filebeat/filebeat.yml" in the file path field
5. Click on "Add Volume" button
  1. Add the path to the logs directory specified in the filebeat.yml config file
  2. Select the "logs" volume from the dropdown (Note: this is the volume with the application log files are written to)
6. Click on "Add constraint"
  1. Enter "node.labels.{label_name} == {label_value}". For example "node.labels.worker==1"
7. Click "Create Service" button