Install Graylog 3 on Ubuntu 18.04 (Free SIEM Part 2)

Hello all, this is the first of a new series of posts which will show you how to setup a free centralised logging solution for any environment.

After much trial and error I think I’m set on using Graylog, Windows Event forwarding, Sysmon, and OSSEC/Wazuh.

All the official documentation for Graylog can be found here: Graylog Docs

Ubuntu is still my favourite flavour of Linux so we will be starting with the base install of Server version 18.04.

Let’s get started, as always we start by updating the repository

sudo apt-get update

And if required upgrade your install. (If you are starting with a fresh install  but didn’t tick “download updates from the internet” you will need to do this)

sudo apt-get upgrade

Now we are running up to date let’s start with installing the dependencies. First up are these 4 packages, make sure you do all these steps in order or it will not work.

 sudo apt-get install apt-transport-https openjdk-8-jre-headless uuid-runtime pwgen 

If you get no errors when installing we move on to installing mongodb from the official repository.

 sudo apt-key adv --keyserver hkp:// --recv 9DA31620334BD75D9DCB49F368818C72E52529D4 
 echo "deb [ arch=amd64 ] bionic/mongodb-org/4.0 multiverse" | sudo tee /etc/apt/sources.list.d/mongodb-org-4.0.list 
 sudo apt-get update
sudo apt-get install -y mongodb-org 

If again you receive no errors, we move on to enabling it on start up.

sudo systemctl daemon-reload
sudo systemctl enable mongod.service
sudo systemctl restart mongod.service

Graylog recommends using Elasticsearch version 6. You can find the installation guide here if you need to refer to it, but you can install using the following. (This is not the latest version, which is not supported so don’t be tempted to try it)

 wget -qO - | sudo apt-key add - 
 echo "deb stable main" | sudo tee -a /etc/apt/sources.list.d/elastic-6.x.list 
 sudo apt-get update && sudo apt-get install elasticsearch-oss 

Before we can configure and start Elasticsearch we need to edit the configuration file which is located at “/etc/elasticsearch/elasticsearch.yml”

We cd to the correct directory

cd /etc/elasticsearch

Then open the file

sudo nano elasticsearch.yml

then find the following line, remove the ‘#’ to uncomment the line and set the property to “graylog” as shown below. graylog

You also need to add the below to the config file.

 action.auto_create_index: false 

Now start Elasticsearch, and enable it at startup.

sudo systemctl daemon-reload 
sudo systemctl enable elasticsearch.service 
sudo systemctl restart elasticsearch.service

Now we are ready to install Graylog, cd into your download or tmp directory and download the latest repo config.


First we unpack the download and then install graylog using apt.

  sudo dpkg -i graylog-3.0-repository_latest.deb
  sudo apt-get update && sudo apt-get install graylog-server  

Now don’t get carried away, because there is still a bit of work to do before graylog will start.

All the instructions we are contained in the following file “/etc/graylog/server/server.conf”

we can open it directly using the following;

sudo nano /etc/graylog/server/server.conf

Take the time to read through the instructions, it will help you to understand a little of what you are doing. With that in mind, let’s continue. Exit nano using CTL and X.

First we create our “password_secret” from the cmd line. using the below cmd to create the hash.

 pwgen -N 1 -s 96

Then open and save the config again and paste the resulting hash into the config file after “password_secret = ”

 sudo nano /etc/graylog/server/server.conf 

Save and exit, then we create our “root_password_sha2” (Remember this as you will need it to login to graylog later on) in a similar way from the cmd line so save your change and exit the config file.

You could run “echo -n yourpasswordhere | shasum -a 256” as suggested in the config file however the online guidance is to use the below.

 echo -n "Enter Password: " && head -1 </dev/stdin | tr -d '\n' | sha256sum | cut -d" " -f1 

Copy and paste this new hash value into the server.conf file after “root_password_sha2”

OK, so now we will be connecting to graylog over http, to be able to use https we need to configure a proxy server which wont be covered here, so always connect over a vpn if in production and you are not using https. Don’t make the web interface externally available. To configure https have a look at the docs here

Also you should enable the host firewall to only allow ports 22, 9000, and 8514, however don’t enable it yet. Get it setup and confirmed as working, then enable your firewall, as we will show later.

To configure the web interface we need to set two further options in the same server.conf file. These options are; “rest_listen_uri” and “web_listen_uri”

Get the IP of your server with the ifconfig cmd, then paste it into the location shown below and make sure the the line doesn’t have a ‘#’ at the start of the line meaning they are commented out. If the ‘#’ is there remove it. this sets both the Web interface: and REST API: options.

http_bind_address =  yourIPaddress:9000/

Save and close the file. If you want more information on configuring the web interface see the documentation here

All that’s left to do is start and configure graylog to enable at startup

sudo systemctl daemon-reload
sudo systemctl enable graylog-server.service
sudo systemctl start graylog-server.service

That’s it, give your server a restart with the following

sudo shutdown now -r

Browse to “yourIPaddress:9000/” and you should be greeted with the following login box. If not, try manually restarting all the services (mongobd, graylog and elasticsearch) using the steps through this guide and see if that resolves it. If not, you’ve done something else wrong!

Now we we know we can connect let’s enable the firewall

sudo ufw enable

And open the 2 ports we need for connecting to it

sudo ufw allow 22
sudo ufw allow 9000

You can check status as below

sudo ufw status

You can also check the status of graylog as shown below

sudo systemctl status graylog-server.service

If you have any issues you can use the following command to view the logs and look for clues.

 sudo tail -f /var/log/graylog-server/server.log 

Come back for the next part as we setup a complete SIEM and logging system.

Add Linux endpoint to existing OSSEC monitoring Server.

If you have an existing OSSEC server this tutorial will show you how to add a linux endpoint which we want to monitor as an agent.

Now on this new server (also ubuntu) we run very similar commands as for the OSSEC monitoring Server; We need to update our repo and install required dependency’s.

sudo apt-get update
sudo apt-get install build-essential
 sudo apt install libpcre2-dev zlib1g-dev 

Download the latest build to the tmp folder

wget -P /tmp


sudo tar -zxvf 3.3.0.tar.gz

Go to the directory, then install

cd /tmp/ossec-hids-3.3.0
sudo  PCRE2_SYSTEM=yes ./

Now you will need to answer the questions:

  • Installation type – agent
  • Where to install – use the default (just hit enter)
  • Server IP address (this is the IP address of your monitoring server)
  • Run Integrity Check – y
  • Run rootkit detection – y
  • Enable firewall drop – y (you can add your IP address to the whitelist, just in case)
  • Hit enter – If you get any errors then most likely your build-essential has not installed correctly)

Now back to the OSSEC Server so we can add the new agent allowing the two to communicate.

sudo /var/ossec/bin/manage_agents

Select ‘a’ from the options and complete the details for the agent.

Now the agent is added we need to extract the unique key and import it to the agent server.

Select option ‘e’ then make a note of the key or paste it into a file.

When finished select ‘q’ to quit.

Now we return the the agent Server and run

sudo /var/ossec/manage_agents

This time select ‘i’ to import, then copy or paste your key as instructed.

If the key is correct you should get a success message.

Now we need to restart our agent server, then log back in and check that OSSEC is running.

sudo /var/ossec/bin/ossec-control status

If it is not running then use

sudo /var/ossec/bin/ossec-control start

Back on the monitoring server we need to restart the services like so.

sudo /var/ossec/bin/ossec-control restart

That’s it. If you setup email alerts you will already have some notifying you of logins and agents being added.

If the agent is not reporting you may need to open the host based firewall to allow 1514 which is the port OSSEC uses.

ufw allow 1514/udp

In a future blog we will look at adding our own alerts.

Install OSSEC 3.3.0 on Ubuntu 16.04 To Monitor Your IT Infrastructure

I’ve been using OSSEC for a few years now and really like it. Recently while migrating my infrastructure I managed to ruin the install and so decided it would be quicker to reinstall the new version from scratch rather than repair then upgrade the existing install. This was all performed on a fresh install of ubuntu 16.04

Just note that if you wish to monitor other assets on your network you will need to set a static IP for this server. I do this using my router to set static arp entries however you can just set the server to use a static IP in the network config, but that is not covered here. Have a “duckduckgo”, there are thousands of articles telling you how to do that!

Jump onto our box, and update our repository as always.

sudo apt get update

Then we need to get the prerequesites before installing OSSEC.

sudo apt-get install build-essential

Now download the latest version to our preferred destination.

wget -P /tmp

You will want to verify the checksum hash if this is going into a production environment. (We’ll do tutorial on verifying hashes in the future)

Now we ‘cd’ to the location and extract the tar file we just downloaded.

cd /tmp
sudo tar -zxvf 3.3.0.tar.gz

This gave me a folder named ossec-hids, so we cd into it

cd ossec-hids-3.3.0

Then run the install script.

sudo ./

Now you will need to answer the questions:

  • Installation type – server
  • Where to install – use the default (just hit enter)
  • Email notification – y (then enter your email address and smtp details if you want to receive emails)
  • Run Integrity Check – y
  • Run rootkit detection – y
  • Enable firewall drop – y (you can add your IP address to the whitelist, just in case)
  • Hit enter – If you get any errors then most likely your build-essential has not installed correctly)

Initially I received an error and a message that the install could not continue.

The error stated “can’t cd to external/pcre210.32/ && \ ” as shown below.

cd external/pcre2-10.32/ && \
./configure \
	--prefix=/Downloads/ossec-hids/src/external/pcre2-10.32//install \
	--enable-jit \
	--disable-shared \
	--enable-static && \
make install-libLTLIBRARIES install-nodist_includeHEADERS
/bin/sh: 1: cd: can't cd to external/pcre2-10.32/
Makefile:766: recipe for target 'external/pcre2-10.32//install/lib/libpcre2-8.a' failed
make: *** [external/pcre2-10.32//install/lib/libpcre2-8.a] Error 2

 Error 0x5.
 Building error. Unable to finish the installation.

I’m no linux genius but I guessed that I was missing a dependency, namely libpcre2-8.a. I had a look for anyone else who had similar issues and found this thread;

This also seemed to confirm what I suspected by di show I would need to run a cmd I’d not seen before. The solution if you’re installing as shown here is below.

sudo apt install libpcre2-dev zlib1g-dev

Once this had completed (with no further errors) I fired up the install script again, but this time included the new directive in the install script. (PCRE2_SYSTEM=yes)

sudo PCRE2_SYSTEM=yes  ./

Now run through the install questions again;

  • Installation type – server
  • Where to install – use the default (just hit enter)
  • Email notification – y (then enter your email address and smtp details if you want to receive emails)
  • Run Integrity Check – y
  • Run rootkit detection – y
  • Enable firewall drop – y (you can add your IP address to the whitelist, just in case)
  • Hit enter – If you get any errors then most likely your build-essential has not installed correctly or you need to run through the previous section again)

Now were ready to fire up OSSEC

sudo /var/ossec/bin/ossec-control start

or check the status like this

sudo /var/ossec/bin/ossec-control status

The other thing we should do at this point is enabled the firewall and only allow the required ports. 1514 is to allow it to communicate with agents on the network that it is monitoring. 22 is to allow ssh so you can connect to the server using putty. If you don’t connect over ssh then don’t allow port 22.

sudo ufw enable
sudo ufw allow 1514/udp
sudo ufw allow 22/tcp 

I also use Graylog so I need to open an additional port to allow the logs to be ingested

ufw enable allow XXXX/udp

If you want to use Graylog with OSSEC the tutorial is here;

To add agents for Windows machines is here:

I’ll do a new blog for adding linux agents soon. Now up here;

Until next time!

How to remove old Kernel and header versions.

While trying to update a server today I was receiving a dpkg error warning that the disk was full.

A simple cmd will show disk usage

df -l

This showed that my \boot partition was full!

We need to browse to the partition

cd /boot

Then list the contents


This showed it was full of old kernels and header versions, so how do we remove these without breaking everything? This is how I did it.

First we need to find out which version we are using.

uname -r

whatever version you are using, DO NOT REMOVE IT!

Let’s see what other versions dpkg thinks we have.

 dpkg -l | tail -n +6 | grep -E 'linux-image-[0-9]+' | grep -Fv $(uname -r) 

Now we can see what we have we can remove them, but note you should only remove those labelled ‘ii’ in the output. If they are labelled ‘iU’ DO NOT remove them.

ALWAYS make a backup/snapshot before making major changes!!!!!

To remove a kernel image we use;

 sudo dpkg --purge linux-image-4.4.0-142-generic 

In my example I state version “4.4.0-143” but yours will be dependent on which versions show in the output in the previous step. I however received an error stating a dpkg error.

No worries we know we can remove the version as it is not in use so we should be able to remove the dependency

We run the same cmd but this time against the dependency.

 sudo dpkg --purge linux-image-extra-4.4.0-142-generic 

Excellent, let’s try and remove the kernel again.

Excellent, so we just repeat the steps for all the versions and their dependencies

I tried to run ‘apt-get update’ but it shows the headers are still installed.

So I used the same cmd to remove the headers

sudo dpkg --purge linux-headers-4.4.0-142-generic 

Then tried running “apt-get update” again, but when trying to install updates I received the same output that headers are still installed. I tried to purge again but get a dpkg error that the header is not installed. Hmmmm.

I’m not a fan of “autoremove” as I have broken many an installation using this cmd incorrectly. However as I have manually removed these I’m pretty confident it’ll be OK. I created another snapshot just to be sure, then went for it.

sudo apt autoremove 

No errors, so I updated Grub next

sudo update grub 

Again no errors and able to update without issues. Phew. No to carry on with what I was trying to do in the first place.

Upgrading To Graylog Enterprise. (Free SIEM Part 4)

We have covered Graylog a fair bit, but to make the most of all it’s functionality we need to upgrade to an Enterprise license. Now before you start screaming “I want a FREE solution” Graylog Enterprise is free for up to 5GB of data a day, and if you are using more than that then you should be paying for it. You can use Graylog for logging without a license, however you won’t be able to make use of the Enterprise Plugins. It depends how much functionality you want from Graylog.

First we need to request a license by going to and completing the form shown below.

NOTE: To get your Cluster ID log in to your Graylog instance, go to the “System” Tab and select “Overview”. 

Now while we are waiting for our license key to arrive we need to install the Enterprise Plugin Package to our Graylog Server. As we are running the latest version it’s as simple as below

sudo apt-get install graylog-enterprise-plugins -y

Now we need to restart the graylog server

sudo shutdown now -r

While the server is rebooting hopefully you should have received an email which contains instructions for getting your license key. Once you have your key you need to head back to Graylog and import our key. Go to the “Systems” Tab and select “License”. Then select the “Import New License” button and paste in your license.

Paste your license key here.

You will then receive a message that your instance is activated, and your license will show under installed licenses on the same page.

That’s it, we are now licensed and ready to make full use of Graylog. Check back for the next steps of our free SIEM, and eventually adding threat feeds, and custom alerts to our Graylog server.

How To Install Nessus Vulnerability Scanner

In our previous post we showed how to download and verify the hash of the Home edition of Nessus, and here we will show how to install, setup and run your first scan. This is a very basic setup to get you up and running quickly with the free version of Nessus. If you are going to be using in a live production environment, don’t use this guide.

If you have not read the initial post go here then come back.

You should be where we left off which is just after checking our hash and confirming against the checksum as below

Get used to running the “ls” cmd to check the directory you are in and that you have access to the correct files.

We “ls” to check as mentioned above then use “dpkg -i” which will de-package and then install Nessus.

While that runs and installs we need to go back to and get an activation code.

Fill in the details and wait for your email with the activation code.

Go back to your Terminal where the install should now have completed. Check the Window for errors and if there are none we are good to continue.

Run the following “/etc/init.d/nessusd start”

Nessus is now running, so open a browser on the same machine and go to https://localhost:8834 and you should get the login screen.

Create a Username and a Password to login  for the first time (don’t forget these!) and you will get the activation page.

Leave the scanner type, and enter your activation key which you will have received by email.

If correct you will see the next screen, now is the time to make a coffee as this stage may take some time as Nessus sets up.

Once completed you will see the “New Scan” screen.

Select New Scan as shown, then select “Basic Network Scan”. This will allow us to do a basic scan of our internal home network.

We will need to find out the IP of our network for the next step, the easiest way to do this is to use either “ifconfig” on a Linux box or “ipconfig” on a Windows box. Run this from a Terminal Window and make a note of your ip address. (This is a simple step so if unsure how to do it you really shouldn’t be installing Nessus to be honest!)

Name and description can be whatever you want, but the IP Address in the targets box needs to be the IP address you want to scan. In the example it shows “” which means that we are going to scan every address on our network. The scan an individual host you would use a single address, for example “”. Now save as shown below and you’ll go back to the main page.

To start your scan click the chevron, as outlined below, then wait for your scan to complete.

Once the scan completes, the real fun starts!

How to verify a file hash in Linux

We have recently shown how to do this in Windows so we will now show how to do this in Linux. Here we will be using Kali but it will work with most Linux distros.

We want to download the free Home Version of Nessus but want to make sure the file has not been tampered with before we install.

We browse to the download site and download the version we need but also copy the hash checksums to simple text files for comparison later. You can do this by simply copying to your clipboard and then paste into a blank text file.

We will download everything to our download folder to make things simple. Once everything is done you should have 3 files in your download folder as shown below.

Now off to the cmd line so open a Terminal and “cd” to the Downloads folder as shown, then use “ls” to list the directory to also confirm you are in the correct location and the correct files are there.

Now we run “sha256sum Nessus-7.2.0-debian6_amd64.deb”. The cmd part is “sha256sum” and the next part is just the file name you want to hash.

You should see the output of the cmd which is your file hash to compare to the one from the site that you had copied earlier.

Now copy that hash output and paste underneath the one you have from the site. We used sha256sum and so will need to compare against the sha256 checksum. 

As you can see, the highlighted one is our output, and they are a perfect match. Excellent, we can now install Nessus with confidence that it has not been tampered with or had malicious code added.

Our next post will see us install Nessus.


How to add SCSI drive to Ubuntu Server

While messing about in my home lab I found I needed to add a SCSI drive to a virtual machine and had no idea how to do it. Although this doesn’t have anything to do with security I’ve posted it here as I know I will definitely need to do this again at some point. Here we have already attached the new VHD to a SCSI connection in the Virtual machine management console, so the disk is physically connected but needs to be formatted, labelled, and initialised.

To list all connected disks run

ls /dev/sd*

this will return the following

If you are unsure which is the new disk, then the easiest way to check is to disconnect the new SCSI drive and run this command, then reconnect the disk and run the same command again and compare the results. The disk that wasn’t there the first time is obviously your new disk. The disks should be labelled “sd” then with a corresponding letter. The primary disk is usually “sda” with the partitions numbered – so sda1 would be the first partition on the first disk. In our example sdb is our new disk so we will be applying changes to this, if your disk is named differently then you will need top replace “sdb” with the name of your own disk in the following instructions.

Once we have identified the new disk we are ready to launch “fdisk” utility.

sudo fdisk /dev/sdb

Always look at the help menus when using new tools as it will help you gain an understanding of it rather than just blindly following instructions. Help menu is shown below

From here we type “n” for a new partition, then “p” for primary and accept all the defaults.

You must then select “w” to write and save the changes, if you don’t do this then the partition will not be created.

Now that we’ve created our partition we need to create the file system

sudo mkfs.ext3 -L DATA /dev/sdb1

The “-L” switch sets the partition label, here we have used “DATA” but you can chose anything, and we are setting it on our new partition which is “/dev/sdb1”

Now we mount the filesystem

sudo mount /dev/sdb1/DATA

To check the location is mounted we can run

Df -l

Next we make a directory

sudo mkdir /Documents

The final thing we need to do is change a configuration so that this new location is mounted every time the server is restarted.

We do this by editing the following file.

sudo nano /etc/fstab

Then add the following line as shown below. (If you named your partition differently then use your label name instead

/dev/sdb1          /Data        auto       defaults    0  0

That’s it. Check this config by restarting your machine then running the list disk, and show mounted commands again and you should see that your new partition is already mounted as shown in the screen shots below.




OSSEC Logs into Graylog

As you know I’m a fan of Trend Micros free HIDS (Host Intrusion Detection System) OSSEC, and that after flirting with Splunk briefly we are now using Graylog for centralised logging.

The question. can we pull our OSSEC logs into Graylog? Course we can.

In previous versions of Graylog you need to install the CEF plugin, but as we are running the latest version, the CEF input plugin is included with the install.

So first let’s login to Graylog, and select inputs

Then from the drop down menu select “CEF UDP” then click “Launch new input”

Select your node from the drop-down menu and complete the other settings as shown. (Unless you already have something running on port 5555! In which case use a different port)

Save then start the input and check that it is running.

If you are running a firewall on the graylog server you will still need to open the port on the host firewall. If you are using ufw the command will be

sudo ufw allow 5555/udp

That’s it for the Graylog server, now over to our OSSEC master Server.

Basically all we need to do is configure OSSEC to forward a copy of it’s alerts to Graylog on the port we chose earlier.

First we cd to the correct location. (This is the default location)

cd /var/ossec/etc/

This directory contains the file we need so use nano to open it.

sudo nano ossec.conf

Then inside the following tags  <ossec_config></ossec_config>

You insert the following new section (put the IP address of the Graylog server where it says “putyouriphere” although if you have DNS configured you can use its FQDN)


Then we need to enable the OSSEC syslog subsystem which is not running by default.

First we need to move to the bin directory of the OSSEC install

cd /var/ossec/bin/

The execute the following command

./ossec-control enable client-syslog

Then restart OSSEC.

./ossec-control restart

If all is working you should see  “csyslogd” start with the other processes

Started ossec-csyslogd........

You can also check in


by running the following

tail -n 1000 /var/ossec/logs/ossec.log | grep csyslogd

and you should be able to see an INFO entry which shows a “Forwarding alerts” message.

If you have a firewall running and have not opened the port you may need an error.

In our case we are using ufw so would run the following.

sudo ufw allow 5555/udp

That’s it. Don’t forget depending on how you have OSSEC set up, you may not have any messages immediately. If you know what alerts you have configured then trigger one of them and then check Graylog.


Graylog Setup Linux Endpoint

The last post showed how to install graylog server, but what good is a log server with no logs?!

Let’s get some data into our servers, we are going to start with a Ubuntu server version 16.04 which is using rsyslog. (Which is installed by default)

Throughout this post the graylog server will be referred to as “graylog”, the server which is being configured to forward it’s logs will be referred to as Ubuntu.

First we need to go to the Ubuntu server login and cd to where we configure rsyslog. For a full explanation, or if you are using syslog-ng look here

cd /etc/rsyslog.d/

using the “ls” command we can see two files in this directory.

We are going to create a new config file for graylog

sudo nano 60-graylog.conf

A blank file will open, as we are running the latest version of Ubuntu we will be running the newer version of rsyslog so we will enter the following into our file

*.* @yourGraylogServerIP:8514;RSYSLOG_SyslogProtocol23Format

If you are running an older version you will need the following.

*.* @yourGraylogServerIP:8514;GRAYLOGRFC5424

If you want to use the most modern approach you would use the following

action(type="omfwd" target="yourGraylogServerIP" port="8514" template="RSYSLOG_SyslogProtocol23Format")

I have not fully tested this latest approach, so if you have any issues with this revert back to the first example.

The eagle-eyed will notice that the port number used is 8514, whereas syslog typically runs over 514. This is due to permission issues when setting up ports in graylog which are below 1000. You can chose any port you wish as long as it starts above 1000.

Save and close the file, then restart rsyslog

sudo service rsyslog restart

We will also need to open the port on the firewall. If you are using Ubuntu and ufw the command will be.

sudo ufw allow 8514/udp

Now we go over to graylog and login, and go to the “Systems/Input” menu as shown.

Then we Select “Launch New Input”

And fill in as shown.  You only have one node, so select your server from the drop down menu.

If you get a green box saying “running” as below, that’s it.

If you think it’s not working then restart the Ubuntu Server (the forwarding server), and login as root, or create a new file so some logs are created, it’s common to think that the logging process is not working when in actual fact there  just hasn’t been any new logs created!

If you are still getting a failed message and are running a firewall on the graylog server you will need to open port 8514.

If using ufw you would type;

sudo ufw allow 8514/udp

Or if you have already done this then check it’s OK by checking the status.

sudo ufw status

Now go to the “Search” tab and select all logs and have a look through your logs. Happy threat hunting. In a later post will look at some further configuration, and setup a Windows Server to forward to graylog.