Q. What is a server log?

A. Server logs are very useful in helping you undestand the impact of your web

site, and in better positioning your web site to attract new and repeat visitors.

Each log records an individual request of a file on a web site (be it HTML,

GIF, JPG, PDF, etc). On first glance a log looks like a bunch of numbers and

letters. Depending on how your server logs are recorded, a collection of logs

may tell you such things as:

Q. I've heard people describe their site's popularity in terms of "hits". What

does that mean?

A. The number of "hits" a site receives can be a misleading way to judge its popularity.

It is important that you understand each of the following terms:

Q. How do I locate my server's logs?

A. You must first find out if your server is dedicated to your web site, or if

it is a multi-homed domain (hosts other web sites with other "root" web addresses

-- or www.yourname.com vs. www.theirname.com). The answer will determine whether

all the logs you'll find on the server represent your web site (you're in luck),

or might contain logs for multiple sites (your logs _might_ be mixed in). Also,

if your web site is part of a larger whole, your logs are likely to be interspersed

with those of the whole site.

If you don't run the server yourself (and/or know the answer), you need to

ask the person running the server (through your ISP, employer etc.) about the

state of your logs. In the worst case scenario - you have no server manager

to ask - you should be able to tell by looking at the logs themselves whether

your logs are separate or mixed with other sites.

Q. Okay, I've determined my situation (or will once I see the logs). What

next?

A. Next you'll have to find the actual folder the logs reside in on the server.

While I can't tell you the exact location (every server has the possibility

of being organized differently), I can suggest the following strategies:

You may have to look around a bit to find them, but once you do the folder

is likely to contain dozens or even hundreds of files. The files are typically

organized by date. For example, our logs for the beginning of July 1999 are

called:

in990701.log

in990702.log

in990703.log

in990704.log (etc.)

Yours may be called something entirely different. When you open the file, you

will see a collection of individual logs for that date, one per line, containing

all the data you need. An individual log file will look something like:

a-xix.wincom.net, -, 7/11/99, 9:12:03, W3SVC, AGNR, www.agnr.umd.edu,

10453, 328, 5354, 200, 0, GET, /users/CMREC/2-6ART4.HTM, Mozilla/4.03 [en] (Win95;

I), http://www.lycos.com/cgi- bin/pursuit?query=grub+control&cat=dir, -, -,

Q. Some of that looks familiar, but what does it all mean?

A. Let's take it piece by piece. For starters, depending on your system each

tidbit is separated by commas, or simply by a blank space. The log above came

from a Windows NT server, and yours may look significantly different. A dash

(-) indicates no data for that category.

There are seven basic (or common) fields in most logs, and the server software

may be configured to collect additional data, resulting (naturally) in additional

fields. I've included a large number of the possible additional fields for further

clarification (for those of you who have them). There may be other fields not

covered below.

I'll be up front about the fact that your log fields may be in a completely

different order than represented above. This can really become a headache when

trying to decipher which field is which. If you have access to server documentation,

it should help (though in the case of MS IIS, the documentation I have really

doesn't help...big surprise there). I should add that the WebTrends site has

a great glossary

of log report related terms (but it's still not comprehensive).

  1. Clients IP address: a-xix.wincom.net

    The host is the user's server that requested the data (page). In our case

    the host's IP address has been resolved into a server name. In your case,

    the host may be represented by a server name or an IP number (such as 186.255.255.1).

    Whichever you get reflects whether your server is set up to "resolve" (look

    up) IP addresses.
  2. Clients Username: - (in our case no data was recorded)

    This field is reserved for the identification of the person's user name. This

    field is rarely used, and data which appears in it can be faked, so in most

    cases it's worth ignoring.
  3. Date (mm/dd/yy): 7/11/99

    This field can record the date in various formats.
  4. Time: 9:12:03

    This field can record the time in various formats, and can include the offset

    from GMT (Greenwich Mean Time) at the end. Ours doesn't include the GMT offset,

    but if it did, it would say something like -0400 (for US Eastern Standard

    Time, 4 hours behind GMT in the summer) or +0700.
  5. Service: W3SVC

    Um, I think this indicates the kind of service the server is configured to

    offer (?)
  6. Computer name: AGNR

    Name given the host server. In our case, AGNR stands for the College of Agriculture

    and Natural Resources.
  7. Server IP address (Multihome domain field): www.agnr.umd.edu

    As indicated, this is either the IP address of the server or in its resolved

    form (the actual URL).
  8. Processing time: 10453

    How long it took the server to process the request in milliseconds.
  9. Bytes recieved: 328

    Data received from the client.
  10. Bytes sent: 5354

    Size of file sent to client. If the field contained a "-" or a "0", this probably

    means that header information only was requested (most often used by spiders

    and bots).
  11. Status Code: 200

    200 in particular reflects a successful file transfer. This code could be

    anything from 1xx to 5xx, depending on the action resulting from the file

    request. In brief, actions are:
  12. 1xx - continue

    2xx - success

    3xx - redirect (also a success)

    4xx - client error (failure)

    5xx - server error (failure)

    (for information on specific numbers, please see this

    page
    .)

  13. Windows NT status Code: 0

    Okay, I'm clueless, and the documentation is of no help!
  14. Operation: GET

    This records the type of request from the client's browser to the server.

    Types of requests can include:
  15. GET - requests the file in its entirety

    HEAD - requests the header information of the file

    POST - places a file on the server

  16. Target file: /users/CMREC/2-6ART4.HTM

    This indicates the path to the requested file.
  17. Browser/Platform: Mozilla/4.03 [en] (Win95; I)

    Indicates which browser and platform the visitor was using. Mozilla is the

    same as Netscape Navigator. When there is additional info such as in "Mozilla/4.0

    (compatible; MSIE 4.01; AOL 4.0; Windows 95)
    " this usually indicates

    that it was MSIE masquerading as Netscape (happens sometimes). Alternatively,

    this field could record a visiting spider.
  18. Referring URL: http://www.lycos.com/cgi- bin/pursuit?query=grub+control&cat=dir

    This indicates who referred the visitor or bot to the web page, thus telling

    you from where your visitors are coming. In this case, the visitor came from

    a search engine (Lycos), and you have the added benefit of knowing which words

    they were using for the search ("grub control" - yummy!).
  19. Script or dll variables: - (in our case no data was

    recorded)

Q. Do I have to go through each and every log to figure that stuff out and

calculate my statistics?

A. I certainly hope not! You could try using software to analyze your server

logs and create reports for you. I've used freeware including the popular Analog

and the less known WWWStat.

Netscape has a comprehensive

list
of links to similar software. I've also had the great pleasure of using

WebTrends Log Analyzer, a top-of-the-line

tool that has a wide variety of capabilities and functions. I have found it

worth the money, especially because my employer paid for it.


If you want to use a good utility for finding and counting your log information,

I hear "grep"

is a great tool built into UNIX boxes, and downloadable for the Mac. I've not

used it, so I can't personally vouch for it. A final alternative would be to

use the search or find feature around various parameters. I'm not going to go

into the specifics of any of these options, in the hopes that any supporting

documentation that comes with the software/your system will be sufficient in

helping you perform your chosen task. When all else fails, call tech support.

Conclusion

I hope this brief introduction to server logs has been helpful. I've acquired

this information from working with logs, from server and software manuals, a

wonderful tech-support worker, and from the various software packages mentioned

above. If you're interested in learning more about server logs and how to use

them to boost the number of visitors to your web site, I would highly recommend

that you read "Increase Your Web Traffic

in a Weekend
" by Willliam R. Stanek. Compared with the wealth of information

he provides, not just on server logs but on numerous ways to greatly enhance

your site's performance, I have barely scratched the surface.

If you have any questions or comments, please don't hesitate to contact

me.