Enable Logging for nG Firewall

This tutorial is for users of my nG Firewall, version 8G or better. It explains how to enable logging for all blocked requests. This is useful for testing, debugging, and keeping an eye on things. Takes only a few minutes to set up, and of course it’s all open source and 100% free for everyone :)

Contents

• Requirements
• Quick start
• How it works
• Log example
• Configure nG Firewall
• Preparing the log file
• Reading the log file
• Customize nG Logging script
• Download nG Logging script
• Troubleshooting nG Firewall
• Troubleshooting Tips
• Bonus: IP logging with nG

Requirements

Here are the requirements for logging nG Firewall.

• Apache version 2.2 or better
• PHP version 5.4.0 or better
• nG Firewall version 8G or better

To implement logging, you need the ability to use/modify the site’s public root .htaccess file. Alternately you can implement directly via server configuration. Also need to be able to change permissions of the log file.

Quick start

Here is a quick-start guide for anyone already familiar with logging nG Firewall.

1. Add nG Firewall
2. Upload nG_log.php and nG_log.txt to root web directory
3. Set writable permissions for nG_log.txt
4. Add protection for nG_log.txt

Done! Remember to test well and report any bugs via my contact form. Please do not report bugs in the comment section, thank you. That’s all there is to it. The remainder of this tutorial provides more detailed instructions and notes.

How it works

The nG Firewall (version 8G or better) includes built-in logging directives. Once logging is enabled (as explained below), mod_rewrite will send request data to the PHP logging script. The logging script then parses the data and writes it to the log file. The result is a log file that looks very similar to Apache defaults.

Under the hood, here is what happens for any URI requests that are blocked by nG Firewall when logging is enabled:

1. nG checks the request
2. If not blocked, the request continues normally
3. If blocked, the request data is sent to the logging script
4. The logging script then writes the data to the log file
5. The script exits with a simple message (configurable)

So regular visitors and legit traffic will continue normally, while any blocked requests are redirected to the logging script. Conceptually simple.

Log Example

Here is an example showing what the log entries look like:

012.012.012.01 - 2024/01/24 01:47:12 - GET - HTTP/1.0 - /www.php - boot.ini [boot.ini] - - - Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_6)
187.255.215.91 - 2024/01/24 21:19:01 - GET - HTTP/1.0 - / - - - - Mozilla/5.0 (compatible; Exabot/3.0; +http://www.exabot.com/go/robot) [Exabot] 
222.333.444.00 - 2024/01/24 00:15:33 - POST - HTTP/1.0 - /xertive.php [xertive] - - - - curl/7.54.0

In each log entry, matching firewall patterns are indicated via brackets like [this]. In the above example, notice the following blocked patterns:

• boot.ini — from QUERY STRING rules
• Exabot — from USER AGENT rules
• xertive — from REQUEST URI rules

So for each request, the location of the bracketed/match string tells us the related section of nG. More details on this later in the post.

Configure nG for logging

Depending on the version, nG Firewall comprises the following sections:

• nG:[CORE]
• nG:[QUERY STRING]
• nG:[REQUEST URI]
• nG:[USER AGENT]
• nG:[REMOTE HOST]
• nG:[HTTP REFERRER]
• nG:[REQUEST METHOD]
• nG:[HTTP COOKIE]

Each of these sections contains a set of mod_rewrite rules. For each section (except CORE rules), you want to disable (comment out) the default RewriteRule, and enable (un-comment) the logging RewriteRule. For example, in the QUERY STRING section:

# nG:[QUERY STRING]
<IfModule mod_rewrite.c>
	
	.
	.
	.
	
	RewriteRule . - [F,L]
	
	# RewriteRule .* /nG_log.php?log [END,NE,E=nG_REQUEST_URI:%1___%2___%3]
	
</IfModule>

We want to change that to this:

# nG:[QUERY STRING]
<IfModule mod_rewrite.c>
	
	.
	.
	.
	
	# RewriteRule . - [F,L]
	
	RewriteRule .* /nG_log.php?log [END,NE,E=nG_REQUEST_URI:%1___%2___%3]
	
</IfModule>

So now the QUERY STRING rules are sending request data to our nG_log.php log file. Repeat this same “rule swap” for each section (except CORE rules) in nG Firewall. Once this is done, nG is configured for logging and will send all blocked requests and data to the nG Logging script.

Preparing the Log file

To prepare the log file, you want to make sure it is writable by the server (e.g., change CHMOD permissions on the file). Then you also want to make sure the file is protected from outside access. This is prevent sensitive information from falling into public domain. As we’re already working with .htaccess, here is a simple code snippet that protects our log file from all outside access:

# Apache < 2.4
<IfModule !mod_authz_core.c>
	<Files ~ "nG_log\.txt">
		Deny from all
	</Files>
</IfModule>

# Apache 2.4+
<IfModule mod_authz_core.c>
	<Files ~ "nG_log\.txt">
		Require all denied
	</Files>
</IfModule>

Add that to your root .htaccess file and done. No changes need to be made. Note however that both sections are not necessary; if you know your version of Apache, it is safe to use only the relevant code. More details over at htaccessbook.com: Access Control for Apache 2.4 (and 2.2).

Reading the log file

For each log entry, the following request data is recorded (if available):

• IP Address
• Date/Time
• Request Method
• Request Protocol
• Request URI
• Query String
• Remote Host
• Referrer
• User Agent
• Cookie

So each log entry/line records these fields in the following order:

IP Address - Date/Time - Request Method - Request Protocol - Request URI - Query String - Remote Host - Referrer - User Agent - Cookie

Note that empty/blank values are simply left as-is. So when you see stuff like this in your log file it’s totally normal:

- / - - - - User agent 1234..

It just means that no data was available for each of the blank items. It is common especially for Remote Host and Referrer fields to be blank.

Customize nG Logging script

Once everything is in place and ready, nG logging should be enabled on your site. Note that the nG_log.php script includes several variables/options that you can modify if desired:

These variables are predefined to work according to this tutorial, out of the box. So no changes need to be made to the logging script, but there are few things that you can customize as desired.

Download nG Logging script

Here is the latest version of the nG Logging script. License: GPL v3 or later.

Troubleshooting nG Firewall

Once logging is enabled on your site, troubleshooting and testing the nG Firewall becomes quick and easy. This section gives some ideas and tips that should be useful for anyone wanting to debug, test, and so forth.

While using nG, if any legitimate page or resource fails to load, or if some feature stops working, simply take a look at the nG log file. Locate the most recent (and/or relevant) entries. For each related entry, the offending firewall pattern will be indicated with [brackets]. So you will know immediately which nG rules/pattern(s) are responsible for any issues.

If you discover any offending patterns, simply disable (comment out) or remove. Also please report any bugs or patterns that are causing problems. That way I can update the firewall and keep things as error-free as possible.

Troubleshooting Tips

Here is a working list of things to check when troubleshooting nG.

Locating patterns

Some sections of nG contain a lot of “tightly packed” rules. This can make locating offending patterns difficult. In such cases, the quickest way to identify the correct pattern is to use the the halving method.

Code placement

In your .htaccess file, Make sure that nG code before any existing mod_rewrite rules (e.g., WordPress Permalinks).

Server error

If you get a server error after installing nG, double-check that your site meets the requirements. Also check the site’s error log for any related entries.

Encoded Characters

By default, special characters are passed to the log file unencoded. To instead pass the encoded equivalents, remove the “No Escape” flag NE from each of the nG rewrite rules.

More tips will be added as they arrive :)

Bonus: IP Logging with nG

Users familiar with nG Firewall may have noticed that IP blocking has been removed. As explained in previous posts, there are better, more effective ways to prevent site access rather than blocking based on IP address. Nonetheless, blocking by IP remains useful in a variety of scenarios. So with that in mind, here is a bonus IP ADDRESS section that may be added to nG (or any .htaccess file):

# nG:[IP ADDRESS]
<IfModule mod_rewrite.c>

	# RewriteCond %{REMOTE_ADDR} ^(000\.000\.000\.000)$ [OR]
	# RewriteCond %{REMOTE_ADDR} ^(100\.100\.100\.100)$ [OR]
	# RewriteCond %{REMOTE_ADDR} ^(200\.200\.200\.200)$ 

	# RewriteRule .* /nG_log.php?log [END,NE,E=nG_IP_ADDRESS:%1]

</IfModule>

As written, this code does nothing, because each directive is disabled (commented out with a hash/pound sign #). To enable these rules, remove the hash from each line located inside the <IfModule>. Then replace the “dummy” placeholder IP values with any that you would like to block and log. Once implemented, these rules will result in any blocked IP addresses indicated in the log file with [brackets].

Faster Way to Block IP Addresses

One last thing came to mind: for anyone who may be new to all of this, a much better way of blocking some IP address, is to just use core Apache functionality:

# Apache < 2.4
<IfModule !mod_authz_core.c>
	Deny from 123.123.123.123
</IfModule>

# Apache 2.4+
<IfModule mod_authz_core.c>
	Require not ip 123.123.123.123
</IfModule>

You can drop that puppy in wholesale, or just use whichever snippet applies to your Apache version. Going this route is much faster than invoking the majesty of mod_rewrite. And if you need to add more IPs to block just add more lines:

# Apache < 2.4
<IfModule !mod_authz_core.c>
	Deny from 123.123.123.123
	Deny from 111.111.111.111
	Deny from 222.222.222.222
</IfModule>

# Apache 2.4+
<IfModule mod_authz_core.c>
	Require not ip 123.123.123.123
	Require not ip 111.111.111.111
	Require not ip 222.222.222.222
</IfModule>

For more information, check out How to Block IPs with 6G Firewall.

May the log files be with you!

About the Author

Jeff Starr = Designer. Developer. Producer. Writer. Editor. Etc.