Tech Manager
01-21-2008, 05:42 PM
As an IT Manager a large amount of my daily time is spent on examining server logs looking for security issues, examining trends and tracking various forms and types of other data. Logs are a very important component of managing all types of servers. They are also an excellent tool to help you gather traffic data on your website(s).
This topic is not meant to be an in depth study of server and web logs, but merely an informational tool for those of you who may not have access to your web server logs or those who wish to create some quick statistical data related to your site visitors and data your visitors may input into forms. I am limiting this topic to gathering, storing and displaying a very small amount of data just to give you some ideas on how you might use an SSI language like PHP to dynamically develop some logging tools for your own personal use. If this topic garners enough interest I will expand it into some more advanced logging procedures including security responses.
In the early stages of the internet explosion many websites displayed a small hit counter at the bottom of the homepage. These displays were usually unreliable and often used to imply high volumes of traffic on a low volume website. As a webmaster, it will help you to get a good idea of the traffic (including origin & volume) coming to your website. For our scenario we will assume you are using PHP and are looking to gather data on three important visitor variables: IP Address, Date & Time, and a Search Term the visitor may enter into a search field on a web form located on your website. We will store this data in a text file located in a protected directory we will call site_logs.
Here are some simple steps for you to follow:
On your web server create a writeable directory called site_logs (located in your root directory).
In this directory add an .htaccess file that looks something like this:
<Limit GET HEAD POST>
Order Allow,Deny
</LIMIT>
The site_logs directory will be used to store the specific log file we create and the .htaccess file will be used to prevent prying eyes from viewing the contents over the web. The .htaccess file above is just a very simple example. Please feel free to modify this file to fit your needs.
Create a file called gather_data.php (or anything else you want to call this file).
The gather_data.php file will collect specific data and write it to a file in your site_logs directory.
The gather_data.php file will look like this:
<?php
$handle = fopen("site_logs/web_log.txt", "a+");
// Set File Permissions to 644
chmod("site_logs/web_log.txt", 0644);
// Get date & time
$date_time = date("l, F j, Y g:i:s", time());
// Get GMT
$gmt = date("O", time());
$date_time_gmt = "$date_time GMT$gmt";
$fp = fopen("site_logs/web_log.txt", "a");
$s_remote= $date_time_gmt . " " . 'REMOTE_ADDR: ' . " " . $_SERVER['REMOTE_ADDR'] . "\n";
fwrite($fp, $s_remote);
$s_remote= 'Search Term: ' . $_POST['search_term'] . "\n";
fwrite($fp, $s_remote);
fclose($handle);
?>
If you are a newbie, don't be overwhelmed by the above code. I'll provide you with a brief overview:
Within the PHP tags (<?php ?>) the first thing we want to do is to create a filehandler for the text file we will be using to store the data. We want this file to be appendable (we want it to continue to add data instead of overwriting existing data).
$handle = fopen("site_logs/web_log.txt", "a+");
We create a file handling variable called $handle and use it to open (fopen function) the site_logs/ directory and the web_log.txt for appending (a+). If the web_log.txt file does not exist it will be created the first time this script is successfully used.
The next thing we do is set the file permissions (assuming a Linux server) to 644, Using the chmod function.
// Set File Permissions to 644 (this is just a coding comment)
chmod("site_logs/web_log.txt", 0644);
Now we will get the date/time & GMT for the webserver using three variables (We are using three variables just so you can see everything as it happens).
// Get date & time
$date_time = date("l, F j, Y g:i:s", time());
Using the date(), time() functions we get the date and time and format it so it will appear something like this:
Monday, January 21, 2008 3:34:19
Then, using the date(), time() functions we get the Greenwich Mean Time (GMT) for the local webserver (yes, we could've combined date/time/GMT into one simple function).
// Get GMT
$gmt = date("O", time());
This produces an output, depending on your server location/timezone, something like this:
-0600
Then we combine these results into one variable: $date_time_gmt = "$date_time GMT$gmt";
and prepare to append the results into our web_log.txt file:
$fp = fopen("site_logs/web_log.txt", "a");
Next, we append the data, including the IP address (using the environmental variable: $_SERVER['REMOTE_ADDR']) of the site visitor to the log file using the fwrite() function:
$s_remote= $date_time_gmt . " " . 'REMOTE_ADDR: ' . " " . $_SERVER['REMOTE_ADDR'] . "\n";
fwrite($fp, $s_remote);
Then we append the $_POST['search_term'] variable to the file:
$s_remote= 'Search Term: ' . $_POST['search_term''] . "\n";
fwrite($fp, $s_remote);
and close the file until it is needed again:
fclose($handle);
...and that's all there is to it.
Using an include statement, such as:
include("gather_data.php");
We will insert the above script into the page(s) where we want to collect the specific data.
I hope this simple script gives you enough background to play with logging specific data on your website. Please give it a try using some additional variables.
Best of luck.
This topic is not meant to be an in depth study of server and web logs, but merely an informational tool for those of you who may not have access to your web server logs or those who wish to create some quick statistical data related to your site visitors and data your visitors may input into forms. I am limiting this topic to gathering, storing and displaying a very small amount of data just to give you some ideas on how you might use an SSI language like PHP to dynamically develop some logging tools for your own personal use. If this topic garners enough interest I will expand it into some more advanced logging procedures including security responses.
In the early stages of the internet explosion many websites displayed a small hit counter at the bottom of the homepage. These displays were usually unreliable and often used to imply high volumes of traffic on a low volume website. As a webmaster, it will help you to get a good idea of the traffic (including origin & volume) coming to your website. For our scenario we will assume you are using PHP and are looking to gather data on three important visitor variables: IP Address, Date & Time, and a Search Term the visitor may enter into a search field on a web form located on your website. We will store this data in a text file located in a protected directory we will call site_logs.
Here are some simple steps for you to follow:
On your web server create a writeable directory called site_logs (located in your root directory).
In this directory add an .htaccess file that looks something like this:
<Limit GET HEAD POST>
Order Allow,Deny
</LIMIT>
The site_logs directory will be used to store the specific log file we create and the .htaccess file will be used to prevent prying eyes from viewing the contents over the web. The .htaccess file above is just a very simple example. Please feel free to modify this file to fit your needs.
Create a file called gather_data.php (or anything else you want to call this file).
The gather_data.php file will collect specific data and write it to a file in your site_logs directory.
The gather_data.php file will look like this:
<?php
$handle = fopen("site_logs/web_log.txt", "a+");
// Set File Permissions to 644
chmod("site_logs/web_log.txt", 0644);
// Get date & time
$date_time = date("l, F j, Y g:i:s", time());
// Get GMT
$gmt = date("O", time());
$date_time_gmt = "$date_time GMT$gmt";
$fp = fopen("site_logs/web_log.txt", "a");
$s_remote= $date_time_gmt . " " . 'REMOTE_ADDR: ' . " " . $_SERVER['REMOTE_ADDR'] . "\n";
fwrite($fp, $s_remote);
$s_remote= 'Search Term: ' . $_POST['search_term'] . "\n";
fwrite($fp, $s_remote);
fclose($handle);
?>
If you are a newbie, don't be overwhelmed by the above code. I'll provide you with a brief overview:
Within the PHP tags (<?php ?>) the first thing we want to do is to create a filehandler for the text file we will be using to store the data. We want this file to be appendable (we want it to continue to add data instead of overwriting existing data).
$handle = fopen("site_logs/web_log.txt", "a+");
We create a file handling variable called $handle and use it to open (fopen function) the site_logs/ directory and the web_log.txt for appending (a+). If the web_log.txt file does not exist it will be created the first time this script is successfully used.
The next thing we do is set the file permissions (assuming a Linux server) to 644, Using the chmod function.
// Set File Permissions to 644 (this is just a coding comment)
chmod("site_logs/web_log.txt", 0644);
Now we will get the date/time & GMT for the webserver using three variables (We are using three variables just so you can see everything as it happens).
// Get date & time
$date_time = date("l, F j, Y g:i:s", time());
Using the date(), time() functions we get the date and time and format it so it will appear something like this:
Monday, January 21, 2008 3:34:19
Then, using the date(), time() functions we get the Greenwich Mean Time (GMT) for the local webserver (yes, we could've combined date/time/GMT into one simple function).
// Get GMT
$gmt = date("O", time());
This produces an output, depending on your server location/timezone, something like this:
-0600
Then we combine these results into one variable: $date_time_gmt = "$date_time GMT$gmt";
and prepare to append the results into our web_log.txt file:
$fp = fopen("site_logs/web_log.txt", "a");
Next, we append the data, including the IP address (using the environmental variable: $_SERVER['REMOTE_ADDR']) of the site visitor to the log file using the fwrite() function:
$s_remote= $date_time_gmt . " " . 'REMOTE_ADDR: ' . " " . $_SERVER['REMOTE_ADDR'] . "\n";
fwrite($fp, $s_remote);
Then we append the $_POST['search_term'] variable to the file:
$s_remote= 'Search Term: ' . $_POST['search_term''] . "\n";
fwrite($fp, $s_remote);
and close the file until it is needed again:
fclose($handle);
...and that's all there is to it.
Using an include statement, such as:
include("gather_data.php");
We will insert the above script into the page(s) where we want to collect the specific data.
I hope this simple script gives you enough background to play with logging specific data on your website. Please give it a try using some additional variables.
Best of luck.