[Dev] Benchmarking your web app

[Andrew C. Oliver]

Hi all,

I have a question.  What method do you use to benchmark your webapps?
I'm currently using "ab" (apache benchmark) but I'd like to be able to
do comparative analysis.  

Specifically, I'm testing POI::HSSF (big shock) Serializer in Cocoon. 
I'd like comparative numbers for the HSSF:Serializer against the HTML
serializer.  

The numbers "ab" gives me are not shall we say believable.  One moment
it says the HSSF::Serializer (Excel file format) is FASTER then the HTML
Serializer.  Which I find very doubtful (no offense to our
serializer)...sometimes it doesn't.  The numbers vary greatly.  

Now I believe the thought that the numbers should very some->a lot but a
rational test based on controlled parameters should give me some idea of
a speed benchmark with some level of variance.  

The idea is to use this as a general point of reference for proving
future builds/releases and concepts.  While we do this easily with the
APIs via junit tests and other "known" situations, this doesn't help us
much with the serializer in a concurrent light/medium and heavy load
situation.  

I'm not getting paid for this so neither should my benchmarking tool
;-).

Thanks,

Andy
-- 
www.superlinksoftware.com
www.sourceforge.net/projects/poi - port of Excel format to java
http://developer.java.sun.com/developer/bugParade/bugs/4487555.html 
			- fix java generics!


The avalanche has already started. It is too late for the pebbles to
vote.
-Ambassador Kosh