[TriLUG] Web Usage Tracking...

[Knowles, Christopher]

netstat -Mnc  OK, now I understand.  two more questions, and then I'll
wander off and mull things over.

you say hourly log files... do you start netstat and then kill it before
starting the next hour, or is there some special and more elegant way?

Um, while thinking over my second question, I thought of an answer... so
I'll leave it at one!

Thanks again.  

CJK

----------  Forwarded Message  ----------

Subject: Re: [TriLUG] Web Usage Tracking...
Date: Fri, 28 Sep 2001 01:58:26 -0500
From: "Jon Carnes" <jonc at nc.rr.com>
To: <trilug at trilug.org>

> On Thursday 27 September 2001 06:24 pm, you wrote:
> > > You could set ipchains to log all outgoing packets destined for ports
> > > 80, 443, and 8008.  This would give you basic traffic analysis.  Not
> > > perfect or elegant, but it's quick and easy.
> >
> > Heck, go ahead and log everything, but make sure you put in a scsi disk
> > sub-system for the firewall or your performance is going to take a hit.
> >
> > I have a couple of firewalls based on RH6.2 and they work great for

logging

> > and I can even tell in real time what is going on:
> >   netstat -Mn |grep 80
> > tells me who is using the web and for what.  I have cron jobs pull the

top

> > 10's out of my logs nightly and then flush the logs after seven days.
>
> <SNIP>
>
> OK, can you give me an example of the cron job you use for that?  (I

assume

> the logs to which you refer are the packet logs...)

Actually for logs, I pump netstat -Mnc into hourly log files that end with
.txt.  To process them I use the following script:
===
echo Top Ten http users
cat *.txt |grep " 80 "|cut -c15-32 |sort |uniq -c |sort -g |tail -10

echo Top Ten http destinations
cat *.txt |grep " 80 "|cut -c36-55 |sort |uniq -c |sort -g |tail -10

===
If you don't like IP's then you can pipe the IP's through:
... |nslookup |grep "Name"

I also like to look at just the records from midnight to 6am.  This gives me
a good idea of the general Background traffic on the network.  This traffic
is mostly from folks who have left automated processes running on their
boxes.  It is amazing how many connections we have that go to weatherbug...

Most folks just don't realize the cumulative consequences of their actions.
I had one lady here that had subscribed to over 16 automated services.  80%
of my night-time traffic was being generated from her box.  Oddly enough,
she also complained about how slow her box was compared to everyone else...
Well daaa. (sorry, its late.)

Jon

_______________________________________________
TriLUG mailing list
http://www.trilug.org/mailman/listinfo/trilug

-------------------------------------------------------