shintara . net - keepin` it real!

UNIX Shell and the Web
Server Side Includes are commonly used on the web to manipulate and control information on websites. I decided to venture one step further and used UNIX shell scripting to achieve something that would have been a little to complex to do using SSI's like php or asp.

On shintara I have a log system that gives me some statistics about the users that visit the site, its like a counter but a little more advanced. The problem with logs is that they grow in size and need to be rolled over - how can you automate the process of rolling over logs automatically? You probably could write a php or asp script that may achieve a simular effect, but for me I wanted something simple and light that didnt rely on SSI's. Writing a UNIX shell script was the perfect choice (unfortunatly those of you that use asp dont have this alternative - you could try writing a batch script which I have no idea on how to do).

Before I get into the script I will explain what you need to do. I use php to gather the data to a file - we will call that file 'data.txt' - in that file I collect the users IP address, thier browser information, the referring website and the time they visit.
Second step is to create a file used to store and update a number - we will call that file 'number.txt' - you will understand this a little later on.
Third step is to create a file that will be used to store links to view the rolled over logs - we will call that file 'rollover.txt'.
Finally you need to create a php file to display these results and links, here is my example:

‹?php
printf "‹p›"
include ('rollover.txt');
printf "‹/p›"
printf "‹p›"
include ('data.txt');
printf "‹/p›"
?›

This is a very simple format - you can choose to make something a little more prettier to the eye, but for this example its suffice. Name this file what ever you like, for this article we will call it 'data.php'.

The Shell Script
Now we can get down to the heart of this excercise, the shell script itself: (call the script whatever you like - for this we will call it 'visitor')

#!/bin/sh
dadate=`date`
exec < number.txt
read string
vnumber=`expr $string + 1`
printf "${vnumber}\n" > number.txt
touch data${vnumber}.txt
cat data.txt > data${vnumber}.txt
rm data.txt
touch data.txt
chmod 666 data.txt
printf "‹a href=\"data${vnumber}.txt\" target=\"_blank\"›data${vnumber}.txt log ${dadate}‹/a›‹br /›\n" >> rollover.txt
exit 0

Please explain...?
The first line is compulsory, it tells the operating system that its a shell script
The second line sets the date variable
Third line opens the number.txt file and extract the information, the line following it, 'read string' reads the information from number.txt and sets it as the variable $string
Add the value 1 to $string and make it a variable called vnumber
The sixth line writes the new number to the file number.txt (overwrites the old number by using the '>')
Line seven creates a new file to hold the old data from the visitor data we collected in data.txt
Pass the data from data.txt to the new data file that is rolled over
Lines nine ten and eleven remove the data.txt file, then create a new data.txt file that will hold the new site visitor information and then change the permissions on that file so it can be written to and read from
We then create a new line in the rollover.txt file - this will be used to link to the older rolled over files to view at a later stage
Note the '>>' in the above line - this causes the information to be appended to the file
And finally exit the script cleanly

A few things to do on your webhost
Next step is to create the relevant files, so ssh (or telnet but this isnt secure so use ssh if available) to your webhost and do the following:

touch number.txt && echo 0 > number.txt
touch rollover.txt

All these files should be saved in one directory - like my below example /home/yourname/public_html/visitor

The final step is to make it run, you can type 'chmod +x visitor' (which makes the file executable - thus all you need to type is './visitor') but this is quite insecure on a website unless you password protect the directory or make it hidden somewhere in a folder, the command to make it work without making the file executable is 'sh visitor'.

Making it all work
We have now put this all together, but how does this make it automatic, you still need to execute the command to make it work?

This is where unix crontab comes into play. Crontab will automatically execute the command when we program it to. Just about every unix (or linux) webhost will have crontab available to its users. For this example we can set crontab to execute the command once a week on Sunday night at 12 midnight. Without going into detail about cron here is what the syntax would look like:

For those of you that set the file to executable:
0 0 * * 0 /home/yourname/public_html/visitor/visitor

For those of you that didnt make it executable:
0 0 * * 0 sh /home/yourname/public_html/visitor/visitor

(Replace /home/yourname/public_html/visitor/visitor with the path to your script.)

Thats it, your automated!

Summary of what we have done
You need to create a php script that writes the user information to a file - data.txt. We created a small php script that we saved as data.php which will read all the relevant files. We then created a small shell script called 'visitor' that will be used to rollover the log files of data.txt, we then chose whether to make it executable or not. We then created two files - number.txt and rollover.txt and added the number 0 to number.txt. Finally we automated the script by using the unix crontab - this ran the visitor shell script once a week.

I hope this will give you an alternative to using php completely for a website, a simple shell script that automatically rolls over logs. Please note that its your choice to use the above script and I take no responsibility for the outcomes of the way you use it.

Information that may interest you
Shintara.net's content both visually and written is copyright 1997-2006 and has rights reserved under guidance of the Creative Commons license. Contact me for more information.