Upgrading To Graylog Enterprise

We have covered Graylog a fair bit, but to make the most of all it’s functionality we need to upgrade to an Enterprise license. Now before you start screaming “I want a FREE solution” Graylog Enterprise is free for up to 5GB of data a day, and if you are using more than that then you should be paying for it. 

First we need to create an account by going to https://www.graylog.org/downloads and completing the form shown below.

NOTE: To get your Cluster ID log in to your Graylog instance, go to the “System” Tab and select “Overview”. 

You will receive an email with contains instructions for getting your license key. Once you have your key you need to head back to Graylog and import our key. Go to the “Systems” Tab and select “License”. Then select the “Import New License” button and paste in your license.

Paste your license key here.

You will then receive a message that your instance is activated, and your license will show under installed licenses on the same page.

That’s it, we are now licensed and ready to make use of other tools. Check back for a tutorial for adding thread feeds to our Graylog server.

Forward Windows Firewall log to Graylog (Using nxlog)

Was just about to finish for the day when I got this working so wanted to post before I forgot lol!

We have covered how to enable Windows Firewall logging, and how to enable and install Nxlog to allow Event Logs to be forwarded into Graylog, however up to this point I hadn’t figured out how to forward Windows log files into Graylog.

It always ends up being ridiculously simple, but hey, sometimes we have to learn the hard way!

If you haven’t setup Nxlog, or enabled Windows Firewall logging then go to the relevant blogs here and here first. as we will be assuming you already have Nxlog installed and forwarding to our Graylog server.

We just have to make a simple addition to the “nxlog.conf” file located by default at “C:\Program Files (x86)\nxlog\conf”

In the image below you will see the original “<input in>” field, and below that you can see the new “Input FileWatch” field.

Complete this as shown (unless of course you have changed the default location of your filewall log file. Mine is named “FSfirewall.log” but yours may be named differently. Browse to that location and check the name of your file before trying to configure this.

Also as you can see above, you need to add the “FileWatch” directive into the Route section of the config file. This will forward the logs through our “output” directive, which points to our Graylog logging server.

That’s it. Go to “services.msc” from powershell or cmd window and restart the nxlog service.

Now head over to Graylog and you should start seeing your Windows Firewall logs. Don’t forget if this is too noisy then you can reduce the logging conditions from within Windows Firewall Console.

Forwarding Windows Event Logs into Graylog (Nxlog)

In previous posts we have covered using rsyslog to forward logs from Linux servers into Graylog, and also how to use Trend Micro’s OSSEC to forward alert logs to Graylog from both Linux and Windows, but here we will show you how to forward Windows logs into Graylog.

We won’t be covering the use of IPSEC in this tutorial, but we will cover that in the future. If you haven’t installed Graylog yet then see the guide here:

First you’ll need to download Nxlog community edition from here:

Nxlog will facilitate the sending of your Windows logs to a logging server, which in this case is Graylog.

Once you have downloaded Nxlog it’s a one click install.

Accept the license agreement and install.

Once installed check the location of the root folder as described in the README file

If when you come to start Nxlog the service doesn’t start then this is the first thing to check.

Now we have to modify the config file located as in the README file above, named nxlog.conf.

In our test environment our Graylog server is on IP 10.1.1.57 however you will need to put the IP address of your Graylog Server instead. All the other settings can be left the same. GELF is simply the format we are telling Nxlog to use when sending the data to Graylog.

That’s it for configuring Nxlog, next is to allow Nxlog through the Windows firewall.

I hope you are using host-based firewall (security in layers right?) If you don’t know how to add a rule to Windows Firewall we will run through it very quickly here.

Open “Windows Firewall with Advanced Security” and right click “Outbound Rules” and select “New Rule”

Choose “Port”

Then “UDP”, “Specific remote ports” and type in “12201” (This is also the port specified in the Nxlog config file earlier) (CORRECTION: Image shows 11201 but this is incorrect. Should show 12201)

Allow the connection (We will cover the IPSEC connection at a later date)

Select the network Profile. (The profile your network is using. If you’re not sure then Windows Firewall will say which Firewall profile is active and that relates to your network profile. If Domain Firewall profile is active, then your network profile is domain).

Then give it a name and finish the Wizard. We still need to right click our new rule from the list and adjust a few settings.

The settings below tightens the port control a little more by us explicitly specifying the local port.

 

This locks things down a little further as we are specifying our Graylog Server by IP.

That’s it for Windows, go to “Services” , and restart Windows Firewall and then start the nxlog service.

Now we head over to Graylog to add our new input and accept the messages.

Before we do we need to open the port on our Graylog Server.

If you have been following the previous tutorials Graylog is installed on Ubuntu Server and is using ufw. The commands to open the port in this configuration is as follows. (Don’t forget that if you have chosen a different port you will need to specify that port number instead)

sudo ufw allow 12201/udp

Then check the status

sudo ufw status

Then login to Graylog’s web interface and go to “Inputs” as shown below

Select  “GELF UDP” from the dropdown and then “Launch new Input”

Select the correct node (If you only have one server then you will only have one to choose from), and complete as below except for the IP address which will be the IP of your Graylog Server.

Save then start the new input. If you receive no errors head over to the Nxlog Log file (on your Windows machine) and check for errors. (Check the README file mentioned earlier for the location).

There you have it. All done.

At this point if you have not received any messages into Graylog yet then go over to your Windows Server and restart the nxlog service. This should create a message. If not then you have either setup something wrong so retrace your steps and check through this tutorial, or if you are not receiving any errors in Graylog then it’s likely that issue is with Windows firewall. Check the nxlog file for clues. Fixing issues is where you really start to learn so don’t give up if you have issues!

 

 

 

 

OSSEC Logs into Graylog

As you know I’m a fan of Trend Micros free HIDS (Host Intrusion Detection System) OSSEC, and that after flirting with Splunk briefly we are now using Graylog for centralised logging.

The question. can we pull our OSSEC logs into Graylog? Course we can.

In previous versions of Graylog you need to install the CEF plugin, but as we are running the latest version, the CEF input plugin is included with the install.

So first let’s login to Graylog, and select inputs

Then from the drop down menu select “CEF UDP” then click “Launch new input”

Select your node from the drop-down menu and complete the other settings as shown. (Unless you already have something running on port 5555! In which case use a different port)

Save then start the input and check that it is running.

If you are running a firewall on the graylog server you will still need to open the port on the host firewall. If you are using ufw the command will be

sudo ufw allow 5555/udp

That’s it for the Graylog server, now over to our OSSEC master Server.

Basically all we need to do is configure OSSEC to forward a copy of it’s alerts to Graylog on the port we chose earlier.

First we cd to the correct location. (This is the default location)

cd /var/ossec/etc/

This directory contains the file we need so use nano to open it.

sudo nano ossec.conf

Then inside the following tags  <ossec_config></ossec_config>

You insert the following new section (put the IP address of the Graylog server where it says “putyouriphere” although if you have DNS configured you can use its FQDN)

<syslog_output>
    <server>putyouriphere</server>
    <port>5555</port>
    <format>cef</format>
</syslog_output>

Then we need to enable the OSSEC syslog subsystem which is not running by default.

First we need to move to the bin directory of the OSSEC install

cd /var/ossec/bin/

The execute the following command

./ossec-control enable client-syslog

Then restart OSSEC.

./ossec-control restart

If all is working you should see  “csyslogd” start with the other processes

Started........
.......
Started ossec-csyslogd........
.......
........

You can also check in

/var/ossec/logs/ossec.log

by running the following

tail -n 1000 /var/ossec/logs/ossec.log | grep csyslogd

and you should be able to see an INFO entry which shows a “Forwarding alerts” message.

If you have a firewall running and have not opened the port you may need an error.

In our case we are using ufw so would run the following.

sudo ufw allow 5555/udp

That’s it. Don’t forget depending on how you have OSSEC set up, you may not have any messages immediately. If you know what alerts you have configured then trigger one of them and then check Graylog.

 

Graylog Setup First Input

The last post showed how to install graylog server, but what good is a log server with no logs?!

Let’s get some data into our servers, we are going to start with a Ubuntu server version 16.04 which is using rsyslog. (Which is installed by default)

Throughout this post the graylog server will be referred to as “graylog”, the server which is being configured to forward it’s logs will be referred to as Ubuntu.

First we need to go to the Ubuntu server login and cd to where we configure rsyslog. For a full explanation, or if you are using syslog-ng look here

cd /etc/rsyslog.d/

using the “ls” command we can see two files in this directory.

We are going to create a new config file for graylog

sudo nano 60-graylog.conf

A blank file will open, as we are running the latest version of Ubuntu we will be running the newer version of rsyslog so we will enter the following into our file

*.* @yourGraylogServerIP:8514;RSYSLOG_SyslogProtocol23Format

If you are running an older version you will need the following.

$template GRAYLOGRFC5424,"<%PRI%>%PROTOCOL-VERSION% %TIMESTAMP:::date-rfc3339% %HOSTNAME% %APP-NAME% %PROCID% %MSGID% %STRUCTURED-DATA% %msg%\n"
*.* @yourGraylogServerIP:8514;GRAYLOGRFC5424

If you want to use the most modern approach you would use the following

action(type="omfwd" target="yourGraylogServerIP" port="8514" template="RSYSLOG_SyslogProtocol23Format")

I have not fully tested this latest approach, so if you have any issues with this revert back to the first example.

The eagle-eyed will notice that the port number used is 8514, whereas syslog typically runs over 514. This is due to permission issues when setting up ports in graylog which are below 1000. You can chose any port you wish as long as it starts above 1000.

Save and close the file, then restart rsyslog

sudo service rsyslog restart

We will also need to open the port on the firewall. If you are using Ubuntu and ufw the command will be.

sudo ufw allow 8514/udp

Now we go over to graylog and login, and go to the “Systems/Input” menu as shown.

Then we Select “Launch New Input”

And fill in as shown.  You only have one node, so select your server from the drop down menu.

If you get a green box saying “running” as below, that’s it.

If you think it’s not working then restart the Ubuntu Server (the forwarding server), and login as root, or create a new file so some logs are created, it’s common to think that the logging process is not working when in actual fact there  just hasn’t been any new logs created!

If you are still getting a failed message and are running a firewall on the graylog server you will need to open port 8514.

If using ufw you would type;

sudo ufw allow 8514/udp

Or if you have already done this then check it’s OK by checking the status.

sudo ufw status

Now go to the “Search” tab and select all logs and have a look through your logs. Happy threat hunting. In a later post will look at some further configuration, and setup a Windows Server to forward to graylog.

 

Graylog Ubuntu Install

Hello all, I know it been a while (and I am aware I am mainly talking to myself here!) what with life and work, it’s been over 2 months since I posted. I also had a server die on me which meant quite a lengthy process of server replacement and data retrieval, but enough about that!

I’ve been trying to find a good logging solution to run along side Security Onion to give as much visibility as possible, and the two I chose were Splunk and Graylog, with Graylog install and setup being covered here.

All the official documentation for Graylog can be found here: Graylog Docs

Ubuntu is still my favourite flavour of Linux so we will be starting with the base install of Server version 16.04.

Let’s get started, as always we start by updating the repository

sudo apt-get update

And if required upgrade your install. (If you are starting with a fresh install  but didn’t tick “download updates from the internet” you will need to do this)

sudo apt-get upgrade

Now we are running up to date let’s start with installing the dependencies. First up are these 4 packages, make sure you do all these steps in order or it will not work.

sudo apt-get install apt-transport-https openjdk-8-jre-headless uuid-runtime pwgen

If you get no errors when installing we move on to installing mongodb from the official repository.

sudo apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv 2930ADAE8CAF5059EE73BB4B58712A2291FA4AD5
echo "deb [ arch=amd64,arm64 ] https://repo.mongodb.org/apt/ubuntu xenial/mongodb-org/3.6 multiverse" | sudo tee /etc/apt/sources.list.d/mongodb-org-3.6.list
sudo apt-get update
sudo apt-get install -y mongodb-org

If again you receive no errors, we move on to enabling it on start up.

sudo systemctl daemon-reload
sudo systemctl enable mongod.service
sudo systemctl restart mongod.service

Graylog recommends using Elasticsearch version 5. You can find the installation guide here if you need to refer to it, but you can install using the following. (This is not the latest version)

wget -qO - https://artifacts.elastic.co/GPG-KEY-elasticsearch | sudo apt-key add -
echo "deb https://artifacts.elastic.co/packages/5.x/apt stable main" | sudo tee -a /etc/apt/sources.list.d/elastic-5.x.list
sudo apt-get update && sudo apt-get install elasticsearch

Before we can configure and start Elasticsearch we need to edit the configuration file which is located at “/etc/elasticsearch/elasticsearch.xml”

We cd to the correct directory

cd /etc/elasticsearch

Then open the file

sudo nano elasticsearch.xml

then find the following line, remove the ‘#’ to uncomment the line and set the cluster.name property to “graylog” as shown below.

cluster.name: graylog

Now start Elasticsearch, and enable it at startup.

sudo systemctl daemon-reload 
sudo systemctl enable elasticsearch.service 
sudo systemctl restart elasticsearch.service

Now we are ready to install Graylog. First we install the repository.

wget https://packages.graylog2.org/repo/packages/graylog-2.4-repository_latest.deb

Then we unpack and install graylog

sudo dpkg -i graylog-2.4-repository_latest.deb
sudo apt-get update && sudo apt-get install graylog-server

Now don’t get carried away, because there is still a bit of work to do before graylog will start.

All the instructions we are contained in the following file “/etc/graylog/server/server.conf”

we can open it directly using the following;

sudo nano /etc/graylog/server/server.conf

Take the time to read through the instructions, it will help you to understand a little of what you are doing. With that in mind, let’s continue. Close the server.conf file and run the following from the command line, copy them into a text file and then paste them once you have generated both hashes.

Firstly to create our “password_secret”

secret pwgen -N 1 -s 96

then we create our “root_password_sha2” (Remember this as you will need it to login to graylog later on)

echo -n yourpasswordhere | shasum -a 256

Copy and paste these into the server.conf file after the “password_secret” , and “root_password_sha2” entries.

OK, so now we will be connecting to graylog over http, to be able to use https we need to configure a proxy server which wont be covered here, so always connect over a vpn if in production and you are not using https. Don’t make the web interface externally available. To configure https have a look at the docs here

Also you should enable the host firewall to only allow ports 22, 9000, and 8514, however don’t enable it yet. Get it setup and confirmed as working, then enable your firewall.

To configure the web interface we need to set two further options in the server.conf file. These options are; “rest_listen_uri” and “web_listen_uri”

Get the IP of your server with the ifconfig cmd, then paste it into the two options as previously mentioned, and make sure the two lines don’t have a ‘#’ at the start of the line meaning they are commented out. If the ‘#’ is there remove it.

rest_listen_uri = http://yourIPaddress:9000/api

(text removed.....)

web_listen_uri =  http://yourIPaddress:9000/

Save and close the file. If you want more information on configuring the web interface see the documentation here

All that’s left to do is start and configure graylog to enable at startup

sudo systemctl daemon-reload
sudo systemctl enable graylog-server.service
sudo systemctl start graylog-server.service

That’s it, give your server a restart with the following

sudo shutdown now -r

Browse to “yourIPaddress:9000/” and you should be greeted with the following login box. If not, try manually restarting all the services (mongobd, graylog and elasticsearch) using the steps through this guide and see if that resolves it. If not, you’ve done something else wrong!

 

The next blog will show how to configure your first input into graylog.