Benchmarking client performance

Post Reply
User avatar
Steve Rowe
Site Admin
Posts: 2416
Joined: Wed May 14, 2008 4:25 pm
OLAP Product: TM1
Version: TM1 v6,v7,v8,v9,v10,v11+PAW
Excel Version: Nearly all of them

Benchmarking client performance

Post by Steve Rowe »

I've built a little workbook that benchmarks client response time, it's very simple piece of VBA that recalculates the workbook and records on the amount of time it takes for the workbook to calculate.

There are about 120,000 DBRWs in the workbook split across 4 sheets.

When I run the test "close" to the TM1 server I get very consistent response times (25s - 28s, mean 26s). The hardware this test is running on is old TM1 server hardware (STARST1P in the graph) so it is much more powerful than my laptop.

When I run the same test on my laptop I get a very different picture. When I initially ran the tests I was getting a response time that was just a couple of seconds slower than the base line test. This is to be expected as there is a lot more network involved. As I ran more and more tests the difference in performance between the laptop and the old TM1 server got more and more pronounced. The obvious explanation was that the there was more network traffic in the afternoon and so this caused the TM1 performance to degrade. The IT people are looking at this for me.

I’m a little nervous about this explanation since it seems a big coincidence that when I first started the testing the results were closely aligned with the results on STARST1P so implying that I happened to start the testing at exactly the moment the network performance started to degrade. If it's not a network issue this seems to imply a correlation between the number of queries issued and the response time.
Unfortunately the client version on the STARST1P machine is 845 and my laptop is 842 so I can't rule out it being a "bug" in the client version I'm on and of course my laptop is also running a different OS and outlook and so on too.

Anyway for interest’s sake I thought I would through this out there for other people to comment on. I would be especially interested if anyone else has noticed tm1 performance degrading as more and more queries are issued

Image

The solo or paired reference on the graph is just if I was running the test on both machines at once or at different times. The tests were run on a dev server with only me logged on to it.

My laptop specs – Compaq nx8220, 2GB Ram and 1 2GHz processor. OS=XP SP2, excel 2003SP3
Cheers
Technical Director
www.infocat.co.uk
Post Reply