Spring Sale! Save 30% on all books w/ code: PLANET24
Web Dev + WordPress + Security

Enable Logging for nG Firewall

This tutorial is for users of my nG Firewall, version 8G or better. It explains how to enable logging for all blocked requests. This is useful for testing, debugging, and keeping an eye on things. Takes only a few minutes to set up, and of course it’s all open source and 100% free for everyone :)

Note: The following article is generalized for any supported version of nG Firewall, 8G or better. For 7G Firewall, check out logging for 7G.

Contents

Requirements

Here are the requirements for logging nG Firewall.

To implement logging, you need the ability to use/modify the site’s public root .htaccess file. Alternately you can implement directly via server configuration. Also need to be able to change permissions of the log file.

Quick start

Here is a quick-start guide for anyone already familiar with logging nG Firewall.

  1. Add nG Firewall
  2. Upload nG_log.php and nG_log.txt to root web directory
  3. Set writable permissions for nG_log.txt
  4. Add protection for nG_log.txt

Done! Remember to test well and report any bugs via my contact form. Please do not report bugs in the comment section, thank you. That’s all there is to it. The remainder of this tutorial provides more detailed instructions and notes.

Important: nG Logging is recommended for testing/development purposes only. Not recommended for live/production sites.

How it works

The nG Firewall (version 8G or better) includes built-in logging directives. Once logging is enabled (as explained below), mod_rewrite will send request data to the PHP logging script. The logging script then parses the data and writes it to the log file. The result is a log file that looks very similar to Apache defaults.

Under the hood, here is what happens for any URI requests that are blocked by nG Firewall when logging is enabled:

  1. nG checks the request
  2. If not blocked, the request continues normally
  3. If blocked, the request data is sent to the logging script
  4. The logging script then writes the data to the log file
  5. The script exits with a simple message (configurable)

So regular visitors and legit traffic will continue normally, while any blocked requests are redirected to the logging script. Conceptually simple.

Log Example

Here is an example showing what the log entries look like:

012.012.012.01 - 2024/01/24 01:47:12 - GET - HTTP/1.0 - /www.php - boot.ini [boot.ini] - - - Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_6)
187.255.215.91 - 2024/01/24 21:19:01 - GET - HTTP/1.0 - / - - - - Mozilla/5.0 (compatible; Exabot/3.0; +http://www.exabot.com/go/robot) [Exabot] 
222.333.444.00 - 2024/01/24 00:15:33 - POST - HTTP/1.0 - /xertive.php [xertive] - - - - curl/7.54.0

In each log entry, matching firewall patterns are indicated via brackets like [this]. In the above example, notice the following blocked patterns:

  • boot.ini — from QUERY STRING rules
  • Exabot — from USER AGENT rules
  • xertive — from REQUEST URI rules

So for each request, the location of the bracketed/match string tells us the related section of nG. More details on this later in the post.

Configure nG for logging

Depending on the version, nG Firewall comprises the following sections:

  • nG:[CORE]
  • nG:[QUERY STRING]
  • nG:[REQUEST URI]
  • nG:[USER AGENT]
  • nG:[REMOTE HOST]
  • nG:[HTTP REFERRER]
  • nG:[REQUEST METHOD]
  • nG:[HTTP COOKIE]

Each of these sections contains a set of mod_rewrite rules. For each section (except CORE rules), you want to disable (comment out) the default RewriteRule, and enable (un-comment) the logging RewriteRule. For example, in the QUERY STRING section:

# nG:[QUERY STRING]
<IfModule mod_rewrite.c>
	
	.
	.
	.
	
	RewriteRule . - [F,L]
	
	# RewriteRule .* /nG_log.php?log [END,NE,E=nG_REQUEST_URI:%1___%2___%3]
	
</IfModule>

We want to change that to this:

# nG:[QUERY STRING]
<IfModule mod_rewrite.c>
	
	.
	.
	.
	
	# RewriteRule . - [F,L]
	
	RewriteRule .* /nG_log.php?log [END,NE,E=nG_REQUEST_URI:%1___%2___%3]
	
</IfModule>

So now the QUERY STRING rules are sending request data to our nG_log.php log file. Repeat this same “rule swap” for each section (except CORE rules) in nG Firewall. Once this is done, nG is configured for logging and will send all blocked requests and data to the nG Logging script.

Preparing the Log file

To prepare the log file, you want to make sure it is writable by the server (e.g., change CHMOD permissions on the file). Then you also want to make sure the file is protected from outside access. This is prevent sensitive information from falling into public domain. As we’re already working with .htaccess, here is a simple code snippet that protects our log file from all outside access:

# Apache < 2.4
<IfModule !mod_authz_core.c>
	<Files ~ "nG_log\.txt">
		Deny from all
	</Files>
</IfModule>

# Apache 2.4+
<IfModule mod_authz_core.c>
	<Files ~ "nG_log\.txt">
		Require all denied
	</Files>
</IfModule>

Add that to your root .htaccess file and done. No changes need to be made. Note however that both sections are not necessary; if you know your version of Apache, it is safe to use only the relevant code. More details over at htaccessbook.com: Access Control for Apache 2.4 (and 2.2).

Reading the log file

For each log entry, the following request data is recorded (if available):

  • IP Address
  • Date/Time
  • Request Method
  • Request Protocol
  • Request URI
  • Query String
  • Remote Host
  • Referrer
  • User Agent
  • Cookie

So each log entry/line records these fields in the following order:

IP Address - Date/Time - Request Method - Request Protocol - Request URI - Query String - Remote Host - Referrer - User Agent - Cookie

Note that empty/blank values are simply left as-is. So when you see stuff like this in your log file it’s totally normal:

- / - - - - User agent 1234..

It just means that no data was available for each of the blank items. It is common especially for Remote Host and Referrer fields to be blank.

Customize nG Logging script

Once everything is in place and ready, nG logging should be enabled on your site. Note that the nG_log.php script includes several variables/options that you can modify if desired:

Variable Name Description
SEVENGLOGPATH Path to the log file
SEVENGSTATUS Status Code for blocked requests
SEVENGUALENGTH Length of User Agent log entry
NGFIREWALL_COOKIE Include cookie data in log file
SEVENGLOGFILE Name of the log file
SEVENGEXIT Message displayed to blocked visitors
date_default_timezone_set('UTC') Default Timezone

These variables are predefined to work according to this tutorial, out of the box. So no changes need to be made to the logging script, but there are few things that you can customize as desired.

Download nG Logging script

Here is the latest version of the nG Logging script. License: GPL v3 or later.

Download nG Logging scriptVersion 2.0 ( 2.68 KB ZIP )
Show support: If the logging script is useful for you, please make a donation »

Troubleshooting nG Firewall

Once logging is enabled on your site, troubleshooting and testing the nG Firewall becomes quick and easy. This section gives some ideas and tips that should be useful for anyone wanting to debug, test, and so forth.

While using nG, if any legitimate page or resource fails to load, or if some feature stops working, simply take a look at the nG log file. Locate the most recent (and/or relevant) entries. For each related entry, the offending firewall pattern will be indicated with [brackets]. So you will know immediately which nG rules/pattern(s) are responsible for any issues.

If you discover any offending patterns, simply disable (comment out) or remove. Also please report any bugs or patterns that are causing problems. That way I can update the firewall and keep things as error-free as possible.

Note: Capturing HTTP requests via mod_rewrite can be challenging, because each host/server/setup/config is unique. The nG Logging script is developed in several different Apache environments, but it is not guaranteed to work out of the box for everyone. Further configuration may be necessary.

Troubleshooting Tips

Here is a working list of things to check when troubleshooting nG.

Locating patterns
Some sections of nG contain a lot of “tightly packed” rules. This can make locating offending patterns difficult. In such cases, the quickest way to identify the correct pattern is to use the the halving method.
Code placement
In your .htaccess file, Make sure that nG code before any existing mod_rewrite rules (e.g., WordPress Permalinks).
Server error
If you get a server error after installing nG, double-check that your site meets the requirements. Also check the site’s error log for any related entries.
Encoded Characters
By default, special characters are passed to the log file unencoded. To instead pass the encoded equivalents, remove the “No Escape” flag NE from each of the nG rewrite rules.
Live Demo: Check out a live demo of nG Firewall here at Perishable Press.

More tips will be added as they arrive :)

Bonus: IP Logging with nG

Users familiar with nG Firewall may have noticed that IP blocking has been removed. As explained in previous posts, there are better, more effective ways to prevent site access rather than blocking based on IP address. Nonetheless, blocking by IP remains useful in a variety of scenarios. So with that in mind, here is a bonus IP ADDRESS section that may be added to nG (or any .htaccess file):

# nG:[IP ADDRESS]
<IfModule mod_rewrite.c>

	# RewriteCond %{REMOTE_ADDR} ^(000\.000\.000\.000)$ [OR]
	# RewriteCond %{REMOTE_ADDR} ^(100\.100\.100\.100)$ [OR]
	# RewriteCond %{REMOTE_ADDR} ^(200\.200\.200\.200)$ 

	# RewriteRule .* /nG_log.php?log [END,NE,E=nG_IP_ADDRESS:%1]

</IfModule>

As written, this code does nothing, because each directive is disabled (commented out with a hash/pound sign #). To enable these rules, remove the hash from each line located inside the <IfModule>. Then replace the “dummy” placeholder IP values with any that you would like to block and log. Once implemented, these rules will result in any blocked IP addresses indicated in the log file with [brackets].

Tip: For the above IP ADDRESS section, each RewriteCond should end with an [OR] flag, except the last RewriteCond. If you only want to block one IP address, use only one RewriteCond and omit the [OR] flag.

Faster Way to Block IP Addresses

One last thing came to mind: for anyone who may be new to all of this, a much better way of blocking some IP address, is to just use core Apache functionality:

# Apache < 2.4
<IfModule !mod_authz_core.c>
	Deny from 123.123.123.123
</IfModule>

# Apache 2.4+
<IfModule mod_authz_core.c>
	Require not ip 123.123.123.123
</IfModule>

You can drop that puppy in wholesale, or just use whichever snippet applies to your Apache version. Going this route is much faster than invoking the majesty of mod_rewrite. And if you need to add more IPs to block just add more lines:

# Apache < 2.4
<IfModule !mod_authz_core.c>
	Deny from 123.123.123.123
	Deny from 111.111.111.111
	Deny from 222.222.222.222
</IfModule>

# Apache 2.4+
<IfModule mod_authz_core.c>
	Require not ip 123.123.123.123
	Require not ip 111.111.111.111
	Require not ip 222.222.222.222
</IfModule>

For more information, check out How to Block IPs with 6G Firewall.

May the log files be with you!

About the Author
Jeff Starr = Web Developer. Security Specialist. WordPress Buff.
WP Themes In Depth: Build and sell awesome WordPress themes.

2 responses to “Enable Logging for nG Firewall”

  1. I have a couple of websites running with akamai service as a proxy. Is there a way to log the users’ real IP and not the one from akamai?

    • You would need to examine the headers sent with the proxy requests. Then adapt the script accordingly.

Comments are closed for this post. Something to add? Let me know.
Welcome
Perishable Press is operated by Jeff Starr, a professional web developer and book author with two decades of experience. Here you will find posts about web development, WordPress, security, and more »
USP Pro: Unlimited front-end forms for user-submitted posts and more.
Thoughts
I live right next door to the absolute loudest car in town. And the owner loves to drive it.
8G Firewall now out of beta testing, ready for use on production sites.
It's all about that ad revenue baby.
Note to self: encrypting 500 GB of data on my iMac takes around 8 hours.
Getting back into things after a bit of a break. Currently 7° F outside. Chillz.
2024 is going to make 2020 look like a vacation. Prepare accordingly.
First snow of the year :)
Newsletter
Get news, updates, deals & tips via email.
Email kept private. Easy unsubscribe anytime.