GoAccess Real-time Web Log Analyzer
GoAccess Real-time Web Log Analyzer

GoAccess: Real-time Web Log Analyzer

GoAccess is an open source real-time web log analyzer and interactive viewer that runs in a terminal on *nix systems or through your browser. It provides fast and valuable HTTP statistics for system administrators that require a visual server report on the fly.

Example of use in terminal
Example of use in terminal
Real-time Demo GoAccess
Real-time Demo GoAccess

Features

GoAccess parses the specified web log file and outputs the data to the X terminal. Features include:

  • Completely Real Time
    All panels and metrics are timed to be updated every 200 ms on the terminal output and every second on the HTML output.
  • Minimal Configuration needed
    You can just run it against your access log file, pick the log format and let GoAccess parse the access log and show you the stats.
  • Track Application Response Time
    Track the time taken to serve the request. Extremely useful if you want to track pages that are slowing down your site.
  • Nearly All Web Log Formats
    GoAccess allows any custom log format string. Predefined options include, Apache, Nginx, Amazon S3, Elastic Load Balancing, CloudFront, etc.
  • Incremental Log Processing
    Need data persistence? GoAccess has the ability to process logs incrementally through the on-disk persistence options.
  • Only one dependency
    GoAccess is written in C. To run it, you only need ncurses as a dependency. That’s it. It even features its own Web Socket server — https://gwsocket.io/.
  • Visitors
    Determine the amount of hits, visitors, bandwidth, and metrics for slowest running requests by the hour, or date.
  • Metrics per Virtual Host
    Have multiple Virtual Hosts (Server Blocks)? It features a panel that displays which virtual host is consuming most of the web server resources.
  • Color Scheme Customizable
    Tailor GoAccess to suit your own color taste/schemes. Either through the terminal, or by simply applying the stylesheet on the HTML output.
  • Support for Large Datasets
    GoAccess features the ability to parse large logs due to its optimized in-memory hash tables. It has very good memory usage and pretty good performance. This storage has support for on-disk persistence as well.
  • Docker Support
    Ability to build GoAccess’ Docker image from upstream. You can still fully configure it, by using Volume mapping and editing goaccess.conf.

Nearly all web log formats…

GoAccess allows any custom log format string. Predefined options include, but not limited to:

  • Amazon CloudFront (Download Distribution).
  • Amazon Simple Storage Service (S3)
  • AWS Elastic Load Balancing
  • Combined Log Format (XLF/ELF) Apache | Nginx
  • Common Log Format (CLF) Apache
  • Google Cloud Storage.
  • Apache virtual hosts
  • Squid Native Format.
  • W3C format (IIS).
  • Caddy’s JSON Structured format.

Why GoAccess?

GoAccess was designed to be a fast, terminal-based log analyzer. Its core idea is to quickly analyze and view web server statistics in real time without needing to use your browser (great if you want to do a quick analysis of your access log via SSH, or if you simply love working in the terminal).

While the terminal output is the default output, it has the capability to generate a complete, self-contained, real-time HTML report, as well as a JSON, and CSV report.

You can see it more of a monitor command tool than anything else.

Installation

Build from release

GoAccess can be compiled and used on *nix systems.

Download, extract and compile GoAccess with:

wget https://tar.goaccess.io/goaccess-1.4.6.tar.gz
tar -xzvf goaccess-1.4.6.tar.gz
cd goaccess-1.4.6/
./configure --enable-utf8 --enable-geoip=mmdb
make
make install

Build from GitHub (Development)

git clone https://github.com/allinurl/goaccess.git
cd goaccess
autoreconf -fiv
./configure --enable-utf8 --enable-geoip=mmdb
make
make install

Distributions

It is easiest to install GoAccess on GNU+Linux using the preferred package manager of your GNU+Linux distribution. Please note that not all distributions will have the latest version of GoAccess available.

  • Debian/Ubuntu
apt-get install goaccess

Note

It is likely this will install an outdated version of GoAccess. To make sure that you’re running the latest stable version of GoAccess see alternative option below.

Official GoAccess Debian & Ubuntu repository

echo "deb https://deb.goaccess.io/ $(lsb_release -cs) main" | sudo tee -a /etc/apt/sources.list.d/goaccess.list
wget -O - https://deb.goaccess.io/gnugpg.key | sudo apt-key --keyring /etc/apt/trusted.gpg.d/goaccess.gpg add -
sudo apt-get update
sudo apt-get install goaccess

Note

.deb packages in the official repo are available through HTTPS as well. You may need to install apt-transport-https.

  • Fedora
yum install goaccess
  • Arch
pacman -S goaccess
  • Gentoo
emerge net-analyzer/goaccess
  • OS X / Homebrew
brew install goaccess
  • FreeBSD
cd /usr/ports/sysutils/goaccess/ && make install clean
pkg install sysutils/goaccess
  • OpenBSD
cd /usr/ports/www/goaccess && make install clean
pkg_add goaccess
  • openSUSE
zypper ar -f obs://server:http http
zypper in goaccess
  • OpenIndiana
pkg install goaccess
  • pkgsrc (NetBSD, Solaris, SmartOS, …)
pkgin install goaccess

Windows

CowAxess is a GoAccess implementation for Windows systems. It is a packaging of GoAccess, Cygwin and many other related tools to make it a complete and ready-to-use solution for real-time web log analysis, all in a 4 MB package.

If you prefer to go the more tedious route, GoAccess can be used in Windows through Cygwin. See Cygwin’s packages. Or through the GNU+Linux Subsystem on Windows 10.

Usage / Examples

Note

Piping data into GoAccess won’t prompt a log/date/time configuration dialog, you will need to previously define it in your configuration file or in the command line.

Getting Started

  • To output to a terminal and generate an interactive report:
goaccess access.log
  • To generate an HTML report:
goaccess access.log -a > report.html
  • To generate a JSON report:
goaccess access.log -a -d -o json > report.json
  • To generate a CSV file:
goaccess access.log --no-csv-summary -o csv > report.csv
  • GoAccess also allows great flexibility for real-time filtering and parsing. For instance, to quickly diagnose issues by monitoring logs since goaccess was started:
tail -f access.log | goaccess -
  • And even better, to filter while maintaining opened a pipe to preserve real-time analysis, we can make use of tail -f and a matching pattern tool such as grepawksed, etc:
tail -f access.log | grep -i --line-buffered 'firefox' | goaccess --log-format=COMBINED -
  • or to parse from the beginning of the file while maintaining the pipe opened and applying a filter
tail -f -n +0 access.log | grep -i --line-buffered 'firefox' | goaccess -o report.html --real-time-html -

Multiple Log files

  • There are several ways to parse multiple logs with GoAccess. The simplest is to pass multiple log files to the command line:
goaccess access.log access.log.1
  • It’s even possible to parse files from a pipe while reading regular files:
cat access.log.2 | goaccess access.log access.log.1 -

Note

the single dash is appended to the command line to let GoAccess know that it should read from the pipe.

  • Now if we want to add more flexibility to GoAccess, we can use zcat --force to read compressed and uncompressed files. For instance, if we would like to process all log files access.log*, we can do:
# zcat --force access.log* | goaccess -

Note: On Mac OS X, use gunzip -c instead of zcat.

Tips

  • Also, it is worth pointing out that if we want to run GoAccess at lower priority, we can run it as:
nice -n 19 goaccess -f access.log -a

and if you don’t want to install it on your server, you can still run it from your local machine!

ssh -n root@server 'tail -f /var/log/apache2/access.log' | goaccess -

Note

SSH requires -n so GoAccess can read from stdin. Also, make sure to use SSH keys for authentication as it won’t work if a passphrase is required.

Dark Mode

goaccess (this link opens in a new window) by allinurl (this link opens in a new window)

GoAccess is a real-time web log analyzer and interactive viewer that runs in a terminal in *nix systems or through your browser.

More Stories
APK Infector EmbedB meterpreter APK to any APK
APK Infector: Embed/Bind meterpreter APK to any Legitimate APK