Simple and lightweight website analytics using goaccess

Super simple website analytics without cookies or 3rd party services using goaccess

GoAccess analytics web page showing only crawler traffic for a website

Getting some insights into the visitors to your website is nice, and there are a lot of options for analytics, and you they include hosted free or paid services and self-hostable versions too. Most notable privacy-respecting service I know of is Plausible Analytics.

What if you just want to see how much traffic there is, without wasting server resources or even running a separate server for analytics, and with some light analyzing of the data?

Meet GoAccess! It’s a simple program that for example parses webserver logs and outputs a static web page. You can also use GoAccess from the cli, but in this post I’m going to focus on looking at the analytics in the browser, parsing NGINX logs.

The best part - this is all fully local on the webserver, based on only your webserver logs. No cookies, no extra scripts on your site.

The method depicted here is just one way to use goaccess, it’s also possible to parse logs real-time. For my sites with very low traffic, periodic updates are plenty, and the data is more useful when parsed from a longer period of time.

Requirements

Setting this up doesn’t require much:

  • Webserver
  • Read access to the webserver’s log directory
  • zcat installed (we’ll be parsing rotated logs too, likely by default)
  • goaccess
    • available on most distributions’ repositories
    • also available as a container image, but for this purpose I think containers are unnecessary. Depends on your setup.
    • possible to build from source too (see the project’s GitHub page)

Usage

Install goaccess.

Make your webserver ready to serve the output HTML from GoAccess. Here’s an example NGINX server block, which binds to a private IP address so your analytics are not visible to everyone on the internet:

server {
  listen 100.1.2.3:80;

  server_name mywebserver;

  root /srv/goaccess;
  index index.html;
}

This could actually be a Tailscale node named mywebserver, you can now access your analytics page by typing http://myserver to the address bar.

The block serves files from /srv/goaccess directory, and assumes index.html is the main page.

Now let’s generate the statistics, this assumes your webserver writes logs of a site to /var/log/nginx/mysite_access.log, change the filename as you see fit:

goaccess /var/log/nginx/mysite_access.log -o /srv/goaccess/index.html -a --log-format=COMBINED

So what’s going on here? You’re running goaccess, and

  • reading /var/log/nginx/mysite_access.log (first argument)
  • outputting html to /srv/goaccess/index.html (-o)
  • enabling a list of user agents by host (-a)
  • interpreting the logs in NGINX’s format (--log-format=COMBINED)

If you have customized NGINX’s log format, check the man page for GoAccess to interpret them correctly.

That’s the most simple form of it! Restart your webserver to serve the output HTML and navigate to http://myserver

If you’re not using a VPN with integrated DNS, add a host entry to /etc/hosts: 100.1.2.3 mywebserver

This can be done on Windows too, the hosts file is somewhere.

These statistics are a one time deal, they won’t update with latest data.

Refining

The refinements here are focused on functionality, please excuse the crudeness.

Separating humans from bots a bit

There’s a log of stuff you can do with GoAccess, one useful thing is ignoring either all crawler traffic (--ignore-crawlers), or showing only crawler traffic (--crawlers-only).

Let’s add a very simple “Analytics landing page”, with links to all traffic, presumably human-only traffic, and only crawler traffic.

Replace the contents of index.html with the following:

<DOCTYPE html>
<html>
    <p>
        <a href="/all.html">All</a>
    </p>
    <p>
       <a href="/human.html">No crawlers</a>
    </p>
    <p>
        <a href="/crawlers.html">Only crawlers</a>
    </p>
</html>

Then let’s generate the actual statistics. The first one is already a familiar command, the next too generate human.html and crawlers.html respectively:

goaccess /var/log/nginx/mysite_access.log -a --log-format=COMBINED -o /srv/goaccess/index.html
goaccess /var/log/nginx/mysite_access.log -a --log-format=COMBINED -o /srv/goaccess/human.html --ignore-crawlers
goaccess /var/log/nginx/mysite_access.log -a --log-format=COMBINED -o /srv/goaccess/crawlers.html --crawlers-only

Parsing rotated logs

Depending on your logrotate setup and the amount of traffic, this won’t give statistics from a very long period of time. What we’ll do next is add rotated logs to the mix.

This is simplest (subjective) to do with a small shell script, create it for example in /usr/local/bin/update_goaccess.sh and make the file executable.

The script will check if a log file is compressed (based on filename), and use either zcat or cat to output the log files, then pipe the output to goaccess (notice the - in the end of the goaccess command):

#!/bin/bash

for f in /var/log/nginx/mysite_access.log* ; do
	[[ $f =~ gz$ ]] && zcat $f || cat $f
done | goaccess -o /srv/goaccess/all.html -a --log-format=COMBINED -


for f in /var/log/nginx/mysite_access.log* ; do
	[[ $f =~ gz$ ]] && zcat $f || cat $f
done | goaccess -o /srv/goaccess/human.html -a --ignore-crawlers --log-format=COMBINED -


for f in /var/log/nginx/mysite_access.log* ; do
	[[ $f =~ gz$ ]] && zcat $f || cat $f
done | goaccess -o /srv/goaccess/crawlers.html -a --crawlers-only --log-format=COMBINED -

Run the script, and you’ll now have statistics parsed over a longer period of time, depending on your rotation settings.

Automation

You likely don’t want to be running the script manually every time you want to look at statistics. The absolute simplest “automation” is to use cron.

Add the following to the webserver user’s crontab (here the user is nginx). This will run the script 1 minute past every hour:

01 *  *  *  *  /usr/local/bin/update_goaccess.sh