laneolson

Splunk is probably the greatest tool that I’ve found for managing pretty much any kind of data. Firewalls, switches, windows servers, software logs… the list goes on! The creators of Splunk generously offer a free version with a few limitations which is perfect for those of us who aren’t huge enterprises. Unfortunately, one of the limitations is the lack of some type of authentication which is pretty crucial if your logs contain sensitive data. I’ve configured my Splunk installation so that it requires authetication through an Apache proxy. I’ve also enabled SSL to allow for encryption of web traffic. The following guide outlines the steps I followed in order to do so on Ubuntu 9.04. This is mostly just a combination of the following guides into one:

• Splunk htaccess Authentication
• SSL Setup
• IPtables setup

I recommend viewing them for a more detailed explanation.

This guide assumes you have a fresh installation of Splunk but should work fine with an existing one. If you don’t have a Splunk installation yet, you can install it quite easily:

cd /opt
sudo wget 'http://www.splunk.com/index.php/download_track?file=3.4.10/linux/splunk-3.4.10-60883-Linux-i686.tgz&ac=&wget=true&name=wget&typed=releases'
sudo tar xvfz splunk-3.4.10-60883-Linux-i686.tgz
sudo splunk/bin/splunk start
sudo /opt/splunk/bin/splunk enable boot-start

Note: Make sure you obtain the latest release. The address in the above wget is probably not the most recent version.

You will also want to enable https on the Splunk installation. You can do this bygoing to http://127.0.0.1:8000/ and clicking the Admin link on the top right part of Splunk Web. Then in the Server: View Settings page under Splunk Web set Enable SSL (HTTPS) in Splunk Web? to Yes. Restart Splunk and you should now be able to access it with an https prefix.

For this guide you are also going to need apache2, openssl.

sudo apt-get install apache2 openssl

Once apache has installed you have to enable the required modules:

sudo a2enmod proxy
sudo a2enmod proxy_http
sudo a2enmod ssl

The proxy modules are required because we are going to be taking requests on port 80 and those requests will be forwarded to the Splunk web interface on port 8000.

Now I am not going to go into extreme detail about generating certificates as this could be a whole topic in itself. Check out Paul Bramscher’s page on generating self signed certificates if you want more details.

The first step is to go to your home directory and generate a server key:

cd ~
sudo openssl genrsa -des3 -out server.key 4096

Next create a certificate signing request with the server key. You will be asked a series of questions, answer them as you see fit:

sudo openssl req -new -key server.key -out server.csr

Now sign the certificate signing request. This one is valid for 1000 days, you may use whatever you like:

sudo openssl x509 -req -days 1000 -in server.csr -signkey server.key -out server.crt

Now make a version that does not require a password:

sudo openssl rsa -in server.key -out server.key.insecure
mv server.key server.key.secure
mv server.key.insecure server.key

Lastly copy the key and certificate that do not require a password to the designated apache folder:

sudo mkdir /etc/apache2/ssl
sudo cp server.key /etc/apache2/ssl
sudo cp server.crt /etc/apache2/ssl

That’s it for the certificates! Now we can start configuring apache. Create a configuration file for the ssl setup:

sudo vim /etc/apache2/sites-available/ssl

You will want to enter something similar to this:

<VirtualHost *:443>
        ServerAdmin webmaster@domain.com
        ServerAlias machine.domain.com
        ProxyPass / https://127.0.0.1:8000/
        ProxyPassReverse / https://127.0.0.1:8000/
        ErrorLog /var/log/apache2/error.log
        CustomLog /var/log/apache2/access.log combined
        SSLEngine On
        SSLCertificateFile /etc/apache2/ssl/server.crt
        SSLCertificateKeyFile /etc/apache2/ssl/server.key
        SSLProxyEngine on
</VirtualHost>
<Proxy https://127.0.0.1:8000/*>
        Order deny,allow
        Deny from all
        Allow from all
        AuthName "machine.domain.com"
        AuthType Basic
        AuthUserFile /var/www/.htpasswd
        Require valid-user
</Proxy>

The ServerAlias will be whatever your DNS name for your Splunk server is.

Now enable the site we just set up:

sudo a2ensite ssl

You should also make sure that Apache is listening on port 443. Do this:

sudo vim /etc/apache2/ports.conf

Add this (if it does not already exist):

<IfModule mod_ssl.c>
    Listen 443
</IfModule>

Now the last step is setting up your user account in the .htpasswd file. This can be done with the following:

sudo htpasswd -c /var/www/.htpasswd username

Where username is replaced with the username you wish to use. The -c flag creates the file, if you need to add multiple users do not use the -c flag after the first command or it will overwrite the file.

Now restart Apache:

sudo /etc/init.d/apache2 restart

If all goes well you should be able to type in the https://machine.domain.com/ and you will receive a password prompt. Enter the username/password you just created and you will be at Splunk Web!

Now, try typing in https://machine.domain.com:8000/ and see what happens. You will probably be directed to Splunk web without being prompted for a password. Uh oh… did you just do all this for nothing?! Nope! We can set up some ip table rules to prevent access to port 8000 to force users to use the proxy. I won’t go into depth on this but we first allow access to port 8000 from localhost with the following command:

sudo iptables -A INPUT -s 127.0.0.1 -p tcp --dport 8000 -j ACCEPT

Now we want to drop all other hosts from accessing port 8000:

sudo iptables -A INPUT -p tcp --dport 8000 -j DROP

Now any attempts to access https://machine.domain.com:8000/ will be blocked, but access attempts to the password protected interface on port 443 will still be allowed.

You can also modify the above rules to limit access to the interface by IP address. Just replace 127.0.0.1 with the IP address you wish to allow acces and change the dport to 443. Enter one of these commands for each IP address to allow. Then enter the second rule (with dport 443) once all the allowed hosts have been added.

Note: The iptables will not be reloaded when you reboot the Splunk machine. I recommend following this firewall guide in order to have a configurable iptables script that loads on startup