Most high-volume web sites use a heavy-duty monitoring service such as those from Keynote Systems, while smaller ones rely on the "free" monitoring services available. I wanted something with more features or a shorter monitoring interval than a free service, but I didn't have the budget for something like Keynote Red Alert (which I've used in the past for commercial work).

When you use a hosting service to serve your site, you're basically paying for bandwidth served, support, and reliability. While its easy to judge the first two, usually all you know about reliability is the service's marketing claims. A site monitor like this one lets you track downtime and response time.

Given that I had just installed a PC in my office that I would be leaving on all the time, I decided to write my own web site monitor that would check my site and send an email page to my cell phone if the site went down. I decided this would also be a good opportunity to try out Perl, since it has good support for network and web programming. On a Win2000 PC, I would have had to do a lot more work in Visual C++ that Perl's packages provide off-the-shelf. So, if you don't like my programming style, keep in mind this was my second or third program in Perl.

This program is only as good as the machine it runs on. If the monitoring machine crashes, you won't get an alert. Also, if this machine has a network failure, you'll get a log entry, but it won't be possible to send the alert. A commercial service would have multiple machines and network connections to keep this from happening.

The program works by requesting a page from your specified web server and checking for the presence of a text string you specify in the page. If the web server doesn't return the page in 30 seconds, or the string is not in the page or it's HTML code, it assumes the site is down and sends an email to the account or device you specify.

The program also records the web server's response time each time the site is checked. If your site's server is slow due to load or maintenance at certain times of the day, you'll be able to see it.

This program is a Perl script, so to run it, you need a Perl interpreter on your machine. On a Unix/Linux machine, you probably already have one. On a Windows machine, you will need to download a free copy of ActiveState ActivePerl. On a Win 2K or XP machine, use the "MSI" version download. I suggest using the 5.6.1 version, as the 5.8 versions don't yet have one of the packages required. Once you have installed ActivePerl, you should test it with the provided example.pl program. Drag the C:\Perl\eg\example.pl file into a command prompt window and hit Enter. Once you see "Hello from ActivePerl", you can proceed.

This script also uses some Perl modules that aren't in the main ActivePerl distribution. You need to get them from the ActiveState web site using the PPM utility. Start the PPM utility from the command prompt window, then type the following at the PPM> prompt:

PPM> install Net-DNS
Install package 'Net-DNS?' (y/N): y

PPM> install Time-HiRes
Install package 'Time-HiRes?' (y/N): y

PPM> quit

On my system, I have version 0.33 of Net-DNS and 1.42 of Time-HiRes. Once these modules are installed, you need to use a text editor (like Notepad) to add your site information. Basically, you need to know the IP address of your DNS server, the URL of the web page you want to monitor, and the domain name for your site. These items go in the section labeled "stuff you have to fill in:" at the top of the file.

You can find out the DNS server address on a Win2K machine from your network connections properties dialog box or by entering "ipconfig /all" in a Command Prompt window.

Finally, you will need to add the address information for the device or email account you want to receive the alert information. This goes in the section labeled "the other stuff to fill in:"

If you are on a Unix/Linux machine, you may need to include the path to your perl interpreter on the first line of the script - replace the "#" with "#!/usr/bin/perl". On a PC, active state will register itself to handle ".pl" files automatically.

Once the program runs without complaining, you should test it by either changing the check phrase to something that will not be found or by taking the check phrase out of the web page temporarily. Note that the check phrase can be an HTML comment instead of text that will show up in the browser.

On a failure, you will see several lines of debugging messages when an email is sent to the alerted device. This is intended, and can be helpful in troubleshooting a problem communicating with the mail agent receiving the message.

To stop the program, just close the Command Prompt box the program runs in, or press ctrl-c while that window is selected. The log files are flushed and closed after every check, so I didn't add any explicit exit command.

Once you're satisfied the program is working, open Windows Explorer, and right click to create a shortcut to the webMonitor3pub.pl file. Then, left click and drag this shortcut to the Start button, and continue dragging as you navigate to the Programs | Startup menu, where if you release the mouse button, the shortcut will drop into the Startup menu. This will start the program every time the machine is booted.

Usability experts like the Nielsen-Norman group and the Don't Make Me Think! author recommend that response times for a web site be under four seconds, preferably under two. The webMonitorStats.txt file contains a line for each day the monitor is running, which looks like this:

[Wed Jan 29 00:53:50 2003] 0.59 0.63 0.59 0.56 0.58 0.55 0.61 0.64 0.58 0.56 0.59 0.58 0.61 0.64 0.59 0.58 0.55 0.55 0.58 0.56 0.58 0.58 0.55 0.53 (0.581333333333333)

[Thu Jan 30 00:54:04 2003] 0.55 0.55 0.58 0.56 0.55 0.55 0.55 0.53 0.56 0.61 0.55 0.64 0.53 0.53 0.55 0.56 0.66 0.56 0.73 0.69 0.53 3.59 1.25 0.98 (0.747458333333333)

The numbers following the date are the time in seconds for the monitored web page to load each time the monitor checks it. The number in parentheses at the end is the average response time for the day. The monitor checks every hour (you can change the interval by substituting a new value for $normalPoll in the program), so there are 24 times on each line. You can see that on Thursday, the hosting provider was probably doing some late-night maintenance (this would have been at 12:54 their time) that cause the page to take 3.59 seconds to load.

One caution about checking more frequently - it can drive up the bandwidth usage on a personal or low-volume site if you have large pages. Like a dripping faucet, each drop is small, but they add up over a month's time. A 10KB page polled every hour is 0.008 GB/month (nothing really), but a 40 KB page polled every five minutes is 0.35 GB/month (probably close to the actual bandwidth limit of a $5 site).

A -1 in the file indicates the site request timed out (the site is down), while a -2 indicates the network is down. If you have to restart your machine, the script will continue the day's logging on a new line.

There's been no attempt to synchronize the polling with any clock time, nor to pretty up the log files. They've worked so far for me, but some post processing could be done to make a response graph, etc.

The webMonitorLog.txt file is self-explanatory.

This program has been tested on two Win2K machines for about six months (as of Feb. 2003). If you install ActivePerl and are able to install the two packages using PPM, it should work with the correct parameters entered in the program. As mentioned below, it's not really designed for a Unix/Linux machine, but in theory should work there.

This program is free - you shouldn't even expect it to work! If you have a severe knowledge gap (you have never written a computer program before, for example) emailed questions will be futile. On the other hand, you will eventually get my attention if you report a bug at the address below.

The biggest problem areas are likely getting the extra packages to install using PPM and troubleshooting email connections. As a Perl novice, PPM is still magic to me, so if it doesn't work, I can't help much.

It personally took me some time to understand Perl, as it's quite different from the C I'm used to. I would suggest the "camel book", Programming Perl, from O'Reilly. I have several of their books, but would suggest this one as the best compromise between a readable tutorial and enough detail to document the stuff used here.

While I like to carefully understand what is happening when I use a program, strictly speaking, you don't need any knowledge of Perl, email, or networking to use this program. If you understand how to install software, and can use the PPM utility and a text editor, you should be able to run the program. Good luck.

Unlike a Unix/Linux system, a non-server Windows PC doesn't have a program running in the background to send and receive mail. The PC is supposed to connect to a server, usually with the POP and SMTP protocols, to transfer mail.

Since most of the Perl code literature evolved from a Unix platform, you'll see example scripts where mail is sent with a call to "sendmail" or another Mail Transfer Agent. On a PC, there is no such thing. So, this program emulates the actions of sendmail by directly sending the alert message to the device's or user's SMTP server. A peek at the first chapter of Sendmail will show how the SMTP protocol works.

Unlike a real MTA, this program doesn't queue or retry a message. If the SMTP servers at your pager company are down, it just doesn't send the message.

The routine getMX looks up the MX DNS records for the domain you want to send the mail to. This is the standard way of learning the name of the SMTP mail server for a domain. Then the routine mailMsg sends the message. The foreach loop in mailMsg tries the server in each MX record (many domains have a backup SMTP server) until one takes the message.

checkWebsite uses the LWP HTTP routines to request the web page from the monitored site. The HTML code of the web page is in $res->content and Perl's regular expressions are used to search for the check phrase.

After writing this program, I became aware of the timefetch script from CPAN, which does a more thorough job of timing a web page download. I have not tried using it, but if you're more interested in the exact time to serve your page, you might want to take a look at it. My program doesn't request included links, so if you have gif or jpg images on your page, they will not be downloaded or timed. If you are aware of something that does a better job, let me know. I've thought of looking into this, as a true usability measurement should include the time to execute DHTML or onLoad() scripts, etc.

Another resource you may want to investigate is webperf.org, which has a tool that shows how long each component of your page takes to download. They are working on a network to monitor response times from distributed locations on the net.

The program senses that your network connection is down by checking the home pages of Google and Yahoo. If neither responds, it assumes the network is down. If either of these sites go out of business or change their home page, you may want to modify this check.

Send mail on technical problems with this page to: [email protected]