Installing Malcolm Network Traffic Analysis Tools on Debian 11 using the Command Line Interface, CLI

Building a small business or at-home security operations center – starting with network analysis tools by Malcolm.

Malcolm Image Courtesy of https://inl.gov/ics-malcolm/

Our friends at Idaho National Labs and the Cybersecurity and Infrastructure Security Agency have developed a network analysis suite that is easy to install and use. These tools perform network packet capture and analysis while providing easy-to-use, feature full intuitive dashboards that even a novice can figure out. Malcolm was designed for all critical infrastructure sectors and can be used as a security information and events management server at home and at work. The only scale and performance limitations I can see are due to the hardware you install it on. For an overview of Malcolm, go here: Network-Traffic-Analysis-with-Malcolm.pdf

This article provides instructions to install a working Malcolm network traffic analysis suite in a Debian 11 virtual machine running on a VMWare ESXi server. Seth Grover, the lead developer of Malcolm at INL, suggests these specs: 16+GB RAM, 4+ cores, “enough” disk for PCAP, and logs suggested. Those are the bare minimum, and I did run into some memory usage issues with only 16GB of RAM.

Here are the exact specifications for my installation.

· Debian 11.6 virtual machine
· 8 vCPUs
· 26 GB RAM
· 500GB Hard disk
· 2 Virtual network adapters
· 1 PCI device mapped to an ethernet port

I monitor 3 different networks, and one is a mirrored port on my router utilizing a direct mapping to the ethernet hardware using a PCI device. You only need one network connection for this to function. I suggest you have an interface with a working Internet connection while you complete the installation to ensure you do not run into any issues downloading the necessary packages.

Installing on Debian instead of Ubuntu

The ISOs available on Malcolm from GitHub and the HowtoForge Malcolm installation tutorial use Ubuntu 22.04. The prebuilt ISO on GitHub has been built using Department of Defense security standards and is highly customized. That version had pre-configured ports that did not work properly for me once I modified the settings. Also, the HowtoForge installation instructions were a little outdated, leading to my docker containers not installing and running properly. No matter how often I ran the installation scripts, they just wouldn’t work. I got tired of troubleshooting and decided to use an easier OS to work with. Enter Debian 11, bullseye!

Debian provides a supported Linux distribution that works out of the box. Ubuntu is essentially a different version of Debian, so why not just go straight to the source? This also means that all of the commands in Ubuntu are native to Debian; it’s a no-brainer.

These instructions are very similar to the HowtoForge Malcolm and Malcolm.fyi Ubuntu 22.04 LTS installation instructions with minor updates to make things work in Debian. This is a full set of instructions, from start to finish, leveraging parts from both Sites and additional notes to help along the way.

Create a system user account

Create a Malcolm account to use to manage the Malcolm docker and scripts.

me@malcolm:/$ sudo useradd -m -d /opt/malcolm -s /bin/bash -G sudo malcolm

Set a password for the Malcolm account.

sudo passwd malcolm

Check the UID for that malcolm account to ensure your setting took.

id malcolm

Your UID should look like this:

uid=1000(malcolm) gid=1000(malcolm) groups=1000(malcolm),27(sudo)

Switch the user to your malcolm system account

me@malcolm:/$ su malcolm

Your output should look something like this.

Installing package installer for Python

Before you can run the python scripts provided by the Malcolm project (./scripts/install.py), you will need to install the pip or you will see the following errors.

The pyyaml module is part of pip, so we have to install pip first.

 malcolm@malcolm:~$ sudo apt-get install python3-pip

Now reboot your server and log back on with the Malcolm account.

malcolm@malcolm:~$ sudo reboot now

Obtain a local working copy of Malcolm from GitHub

The easiest way to install is to clone the Malcolm GitHub working files.

malcolm@malcolm:~$ sudo git clone https://github.com/idaholab/Malcolm

[sudo] password for malcolm:

Cloning into 'Malcolm'...

remote: Enumerating objects: 28406, done.

remote: Counting objects: 100% (3982/3982), done.

remote: Compressing objects: 100% (1212/1212), done.

remote: Total 28406 (delta 2830), reused 3752 (delta 2657), pack-reused 24424

Receiving objects: 100% (28406/28406), 126.95 MiB | 1.73 MiB/s, done.

Resolving deltas: 100% (20541/20541), done.

Change to the Malcolm directory.

malcolm@malcolm:~$ cd Malcolm

Ensure that everything is copied over.

malcolm@malcolm:~/Malcolm$ ls

Running the Malcolm installation script

Run the install.py and follow the prompts as displayed below.

malcolm@malcolm:~/Malcolm$ sudo ./scripts/install.py

The pyyaml module is required under Python 3.9.2 (/usr/bin/python3)

Importing the pyyaml module failed. Attempt to install via pip3?: yes

Installation of pyyaml module apparently succeeded

The python-dotenv module is required under Python 3.9.2 (/usr/bin/python3)

Importing the python-dotenv module failed. Attempt to install via pip3?: yes

Installation of python-dotenv module apparently succeeded

Installing required packages: ['apache2-utils', 'make', 'openssl', 'python3-dialog']

"docker info" failed, attempt to install Docker? (Y/n): Y

Attempt to install Docker using official repositories? (Y/n): Y

Installing required packages: ['apt-transport-https', 'ca-certificates', 'curl', 'gnupg2', 'software-properties-common']

Installing docker packages: ['docker-ce', 'docker-ce-cli', 'docker-compose-plugin', 'containerd.io']

Installation of docker packages apparently succeeded

Add a non-root user to the "docker" group?: Y

Enter user account: malcolm

Add another non-root user to the "docker" group?: n

fs.file-max increases allowed maximum for file handles

fs.file-max= appears to be missing from /etc/sysctl.conf, append it? (Y/n): Y

fs.inotify.max_user_watches increases allowed maximum for monitored files

fs.inotify.max_user_watches= appears to be missing from /etc/sysctl.conf, append it? (Y/n): Y

fs.inotify.max_queued_events increases queue size for monitored files

fs.inotify.max_queued_events= appears to be missing from /etc/sysctl.conf, append it? (Y/n): Y

fs.inotify.max_user_instances increases allowed maximum monitor file watchers

fs.inotify.max_user_instances= appears to be missing from /etc/sysctl.conf, append it? (Y/n): Y

vm.max_map_count increases allowed maximum for memory segments

vm.max_map_count= appears to be missing from /etc/sysctl.conf, append it? (Y/n): Y

net.core.somaxconn increases allowed maximum for socket connections

net.core.somaxconn= appears to be missing from /etc/sysctl.conf, append it? (Y/n): Y

vm.swappiness adjusts the preference of the system to swap vs. drop runtime memory pages

vm.swappiness= appears to be missing from /etc/sysctl.conf, append it? (Y/n): Y

vm.dirty_background_ratio defines the percentage of system memory fillable with "dirty" pages before flushing

vm.dirty_background_ratio= appears to be missing from /etc/sysctl.conf, append it? (Y/n): Y

vm.dirty_ratio defines the maximum percentage of dirty system memory before committing everything

vm.dirty_ratio= appears to be missing from /etc/sysctl.conf, append it? (Y/n): Y

/etc/security/limits.d/limits.conf increases the allowed maximums for file handles and memlocked segments

/etc/security/limits.d/limits.conf does not exist, create it? (Y/n): Y

Malcolm has been installed to /Malcolm. See README.md for more information.

Scripts for starting and stopping Malcolm and changing authentication-related settings can be found in /Malcolm/scripts.

In the latest version of Malcolm, you will be presented with a full configuration during the installation. I didn’t paste all of that text here to conserve space. Choose the defaults and we will reconfigure it later.

Now Malcom is installed, but it’s not running yet.

A quick note about docker and docker compose

Before moving forward, test the docker and docker compose versions.

Both the HowtoForge Malcolm and Malcolm.fyi Ubuntu 22.04 LTS installation instructions state that you will run a command called “docker-compose pull” with a hyphen. This is invalid syntax for both Ubuntu and Debian, and will cause heartache if this is your first container experience.

Docker is installed, as seen here.

Running docker-compose doesn’t work.

The correct syntax does not have the hyphen.

Running the Malcolm configuration script

Now we are going to rerun the configuration.

Before you proceed, be sure to have your network interfaces written down. One way to look that up is like this:

malcolm@malcolm:~/Malcolm$ nmcli device status | grep ens

Run through the configuration as shown below.

You might get a graphical experience that presents the options like this:

…it’s ok; just follow the settings as listed in the text below.

malcolm@malcolm:~/Malcolm$ ./scripts/configure

Malcolm processes will run as UID 1000 and GID 1000. Is this OK? (Y/n): y
Should Malcolm use and maintain its own OpenSearch instance? (Y/n): y
Compress OpenSearch index snapshots? (y/N): n
Forward Logstash logs to a secondary remote OpenSearch instance? (y/N): n
Setting 10g for OpenSearch and 3g for Logstash. Is this OK? (Y/n): y
Setting 3 workers for Logstash pipelines. Is this OK? (Y/n): y
Restart Malcolm upon system or Docker daemon restart? (y/N): y
- 1: no
- 2: on-failure
- 3: always
- 4: unless-stopped
- Select Malcolm restart behavior (unless-stopped): 4
Require encrypted HTTPS connections? (Y/n): y
Will Malcolm be running behind another reverse proxy (Traefik, Caddy, etc.)? (y/N): n
Specify external Docker network name (or leave blank for default networking) ():
- 1: Basic
- 2: Lightweight Directory Access Protocol (LDAP)
- 3: None
- Select authentication method (Basic): 1
Store PCAP, log and index files locally under /home/user/Malcolm? (Y/n): y
Delete the oldest indices when the database exceeds a certain size? (y/N): n
Should Arkime delete PCAP files based on available storage (see https://arkime.com/faq#pcap-deletion)? (y/N): y
Automatically analyze all PCAP files with Suricata? (Y/n): y
Download updated Suricata signatures periodically? (y/N): y
Automatically analyze all PCAP files with Zeek? (Y/n): y
Should Malcolm use "best guess" to identify potential OT/ICS traffic with Zeek? (y/N): n
Perform reverse DNS lookup locally for source and destination IP addresses in logs? (y/N): n
Perform hardware vendor OUI lookups for MAC addresses? (Y/n): y
Perform string randomness scoring on some fields? (Y/n): y
Expose OpenSearch port to external hosts? (y/N): n
Expose Logstash port to external hosts? (y/N): n
Expose Filebeat TCP port to external hosts? (y/N): y
Use default field values for Filebeat TCP listener? (Y/n): y
Expose SFTP server (for PCAP upload) to external hosts? (y/N): n
Enable file extraction with Zeek? (y/N): y
- 1: none
- 2: known
- 3: mapped
- 4: all
- 5: interesting
- Select file extraction behavior (none): 5
  - 1: quarantined
  - 2: all
  - 3: none
  - Select file preservation behavior (quarantined): 1
Expose web interface for downloading preserved files? (y/N): y
Enter AES-256-CBC encryption password for downloaded preserved files (or leave blank for unencrypted): decryptme
Scan extracted files with ClamAV? (y/N): y
Scan extracted files with Yara? (y/N): y
Scan extracted PE files with Capa? (y/N): y
Lookup extracted file hashes with VirusTotal? (y/N): n
Download updated file scanner signatures periodically? (n/Y): y
Should Malcolm run and maintain an instance of NetBox, an infrastructure resource modeling tool? (y/N): n
Should Malcolm capture live network traffic to PCAP files for analysis with Arkime? (y/N): y
Capture packets using netsniff-ng? (Y/n): y
Should Malcolm analyze live network traffic with Suricata? (y/N): y
Should Malcolm analyze live network traffic with Zeek? (y/N): y
Specify capture interface(s) (comma-separated): eth0 ENTER YOUR INTERFACES HERE
Capture filter (tcpdump-like filter expression; leave blank to capture all traffic) (): not port 5044 and not port 8005 and not port 9200
Disable capture interface hardware offloading and adjust ring buffer sizes? (y/N): n
Enable dark mode for OpenSearch Dashboards? (n/Y): y
Malcolm has been installed to /home/user/Malcolm. See README.md for more information.
Scripts for starting and stopping Malcolm and changing authentication-related settings can be found in /home/user/Malcolm/scripts.

Setting up your Malcolm authentication

Reboot your Malcolm server before you proceed.

malcolm@malcolm:~/Malcolm$ sudo reboot now

Log back in, change to the Malcolm directory, and run the following command to set up your authentication.

malcolm@malcolm:~/Malcolm$ ./scripts/auth_setup

Store administrator username/password for local Malcolm access? (Y/n): y
Enter an administrator name (i.e., analyst) and set the password
Additional local accounts can be created at https://localhost/auth/ when Malcolm is running
(Re)generate self-signed certificates for HTTPS access (Y/n): y
(Re)generate self-signed certificates for a remote log forwarder (Y/n): y
Will Malcolm be using an existing remote primary or secondary OpenSearch instance? (y/N): n
Store username/password for email alert sender account? (y/N): n
(Re)generate internal passwords for NetBox (Y/n): y

Pull in the docker images for Malcolm from GitHub

This is where we use the correct syntax, as shown below.

malcolm@malcolm:~/Malcolm$ sudo docker compose pull

Pulling api ... done

Pulling arkime ... done

Pulling dashboards ... done

Pulling dashboards-helper ... done

Pulling file-monitor ... done

Pulling filebeat ... done

Pulling freq ... done

Pulling htadmin ... done

Pulling logstash ... done

Pulling netbox ... done

Pulling netbox-postgresql ... done

Pulling netbox-redis ... done

Pulling nginx-proxy ... done

Pulling opensearch ... done

Pulling pcap-capture ... done

Pulling pcap-monitor ... done

Pulling suricata ... done

Pulling upload ... done

Pulling zeek ... done

Once you see that all of the pulls are done, you should double check the docker images.

malcolm@malcolm:~/Malcolm$ docker images

REPOSITORY TAG IMAGE ID CREATED SIZE

ghcr.io/idaholab/malcolm/dashboards 23.04.0 xxxxxxxxxxxx 1 days ago 2.01GB

ghcr.io/idaholab/malcolm/dashboards-helper 23.04.0 xxxxxxxxxxxx 1 days ago 179MB

ghcr.io/idaholab/malcolm/nginx-proxy 23.04.0 xxxxxxxxxxxx 1 days ago 168MB

ghcr.io/idaholab/malcolm/zeek 23.04.0 xxxxxxxxxxxx 1 days ago 1.37GB

ghcr.io/idaholab/malcolm/arkime 23.04.0 xxxxxxxxxxxx 1 days ago 833MB

ghcr.io/idaholab/malcolm/logstash-oss 23.04.0 xxxxxxxxxxxx 1 days ago 1.52GB

ghcr.io/idaholab/malcolm/file-monitor 23.04.0 xxxxxxxxxxxx 1 days ago 593MB

ghcr.io/idaholab/malcolm/file-upload 23.04.0 xxxxxxxxxxxx 1 days ago 267MB

ghcr.io/idaholab/malcolm/filebeat-oss 23.04.0 xxxxxxxxxxxx 1 days ago 343MB

ghcr.io/idaholab/malcolm/api 23.04.0 xxxxxxxxxxxx 1 days ago 174MB

ghcr.io/idaholab/malcolm/netbox 23.04.0 xxxxxxxxxxxx 1 days ago 760MB

ghcr.io/idaholab/malcolm/htadmin 23.04.0 xxxxxxxxxxxx 1 days ago 245MB

ghcr.io/idaholab/malcolm/suricata 23.04.0 xxxxxxxxxxxx 1 days ago 282MB

ghcr.io/idaholab/malcolm/opensearch 23.04.0 xxxxxxxxxxxx 1 days ago 1.56GB

ghcr.io/idaholab/malcolm/pcap-monitor 23.04.0 xxxxxxxxxxxx 1 days ago 218MB

ghcr.io/idaholab/malcolm/pcap-capture 23.04.0 xxxxxxxxxxxx 1 days ago 126MB

ghcr.io/idaholab/malcolm/freq 23.04.0 xxxxxxxxxxxx 1 days ago 133MB

ghcr.io/idaholab/malcolm/name-map-ui 23.04.0 xxxxxxxxxxxx 1 days ago 139MB

ghcr.io/idaholab/malcolm/postgresql 23.04.0 xxxxxxxxxxxx 1 days ago 295MB

ghcr.io/idaholab/malcolm/redis 23.04.0 xxxxxxxxxxxx 1 days ago 37.6MB

Starting your Malcolm Network Analysis tools

Run the following command and start the Malcolm servers

malcolm@malcolm:~/Malcolm$ ./scripts/start

In a few minutes, Malcolm services will be accessible via the following URLs:

------------------------------------------------------------------------------

- Arkime: https://localhost/

- OpenSearch Dashboards: https://localhost/dashboards/

- PCAP upload (web): https://localhost/upload/

- PCAP upload (sftp): sftp://username@127.0.0.1:8022/files/

- NetBox: https://localhost/netbox/

- Account management: https://localhost/auth/

- Documentation: https://localhost/readme/

NAME COMMAND SERVICE STATUS PORTS

malcolm-api-1 "/usr/local/bin/dock…" api running (starting) …

malcolm-arkime-1 "/usr/local/bin/dock…" arkime running (starting) …

malcolm-dashboards-1 "/usr/local/bin/dock…" dashboards running (starting) …

malcolm-dashboards-helper-1 "/usr/local/bin/dock…" dashboards-helper running (starting) …

malcolm-file-monitor-1 "/usr/local/bin/dock…" file-monitor running (starting) …

malcolm-filebeat-1 "/usr/local/bin/dock…" filebeat running (starting) …

malcolm-freq-1 "/usr/local/bin/dock…" freq running (starting) …

malcolm-htadmin-1 "/usr/local/bin/dock…" htadmin running (starting) …

malcolm-logstash-1 "/usr/local/bin/dock…" logstash running (starting) …

malcolm-netbox-1 "/usr/bin/tini -- /u…" netbox running (starting) …

malcolm-netbox-postgres-1 "/usr/bin/docker-uid…" netbox-postgres running (starting) …

malcolm-netbox-redis-1 "/sbin/tini -- /usr/…" netbox-redis running (starting) …

malcolm-netbox-redis-cache-1 "/sbin/tini -- /usr/…" netbox-redis-cache running (starting) …

malcolm-nginx-proxy-1 "/usr/local/bin/dock…" nginx-proxy running (starting) …

malcolm-opensearch-1 "/usr/local/bin/dock…" opensearch running (starting) …

malcolm-pcap-capture-1 "/usr/local/bin/dock…" pcap-capture running …

malcolm-pcap-monitor-1 "/usr/local/bin/dock…" pcap-monitor running (starting) …

malcolm-suricata-1 "/usr/local/bin/dock…" suricata running (starting) …

malcolm-suricata-live-1 "/usr/local/bin/dock…" suricata-live running …

malcolm-upload-1 "/usr/local/bin/dock…" upload running (starting) …

malcolm-zeek-1 "/usr/local/bin/dock…" zeek running (starting) …

malcolm-zeek-live-1 "/usr/local/bin/dock…" zeek-live running …

Give it a few minutes for it to start.

Log into your Malcolm

As shown above, a list of addresses are shown for you to gain access.

· Arkime: https://localhost/
· OpenSearch Dashboards: https://localhost/dashboards/
· PCAP upload (web): https://localhost/upload/
· PCAP upload (sftp): sftp://username@127.0.0.1:8022/files/
· NetBox: https://localhost/netbox/
· Account management: https://localhost/auth/
· Documentation: https://localhost/readme/

You can substitute “localhost” with the Malcolm server’s IP address or FQDN.

Arkime looks like this:

Malcolm dashboards main Site look like this:

That’s it! You now have a working Malcolm network traffic analysis suite running on a Debian server.

Now get to work learning the intuitive interfaces and defending your networks!