Web Stress Test Sample Performance Numbers, Part 2
Realistic Web StressTest
The following measurements are the results from a stress test that was designed to accurately simulate virtual users. Because the number of virtual users that can be generated varies by the bandwidth of the simulated connection from the virtual users to the web server, several bandwidths will be tested, as well as the difference in performance between operating systems. It is a best case scenario meant to determine the absolute maximum amount of transactions that a user can expect to generate from a single computer. It does not take into account features that require additional processing resources such as session tracking, or parameter replacement, so the performance you see on your particular situation is likely to be different.
The test hardware and test configuration are shown below. The software used is Web Performance Trainer™ 2.7 Beta 5.
|
Hardware |
Test Parameters |
||
|---|---|---|---|
|
Manufacturer |
Dell PowerEdge 300 Server |
Number Web Pages |
68 |
|
Operating System |
Windows 2000 Server |
Number Hits |
755 |
|
CPU |
2.4 GHz P6 |
Simulated Network |
56K-1Mbit/s |
|
Network |
100Mbit/s Ethernet (on board) |
Pacing |
Recorded |
|
Memory |
512 |
Sample Period |
10 Seconds |
|
Memory allocated |
250MB |
||
|
Repeat Delay |
100ms |
||
|
Random Start |
On |
||
|
Virtual Users |
Start with 50, add 50/min |
||
Performance v.s. CPU Speed
The first question people ask is how does performance vary with CPU speed. The tables below compare the performance in terms of the numbers of users and hits per second that can be generated on different speed computers for a typical test running at a relatively high simulated bandwidth of 1Mbit/s per each virtual user . Notice that the performance difference is largest on SSL tests.|
Parameters |
800MHz |
800MHz (SSL) |
2.4 GHz |
2.4 GHz (SSL) |
|---|---|---|---|---|
|
Hits per Second |
714 |
327 |
936 |
873 |
|
KBytes per Second |
3,615 |
2,092 |
5,129 |
3,256 |
|
Virtual Users |
500 |
327 |
775 |
500 |
Performance v.s. Bandwidth
The following chart shows a comparison of how performance varies with the bandwidth of the virtual users. Note that the important information here is the number of virtual users; the hits/sec and Kbytes/sec values are the result of simulating the given number of virtual users. In order to make sure the simulated users are realistic, Web Performance Trainer uses dynamic load balancing technology to limit the number of virtual users to what can be supported on the hardware.One common question is why doesn't the hits/sec increase more when the bandwidth increases? The reason is that in a realistic user simulation, the majority of the time is spent with the user reading the page or filling out forms when compared to page download time. The extra bandwidth is only used during the actual download of the page and images, so that the overall hits/sec increases at a rate which is disproportionate to the increase in bandwidth.
Bandwidth v.s. Capacity
|
Parameters |
128Kbit/s |
256Kbits/s |
1Mbit/s |
10Mbits/s |
|---|---|---|---|---|
|
virtual users |
822 |
800 |
795 |
550 |
|
hits/sec |
728 |
760 |
936 |
760 |
|
Kbytes/sec |
3,137 |
4,000 |
4,129 |
4,328 |
The use of SSL greatly affects load generating capacity because the task of encrypting and decrypting the browser communication has significant overhead. As you can see from the following chart, the SSL performance numbers are significantly smaller than the non-SSL case. Because there is so much math involved, multiple CPU machines tend to show a much higher performance increase on SSL tests than they do on non-SSL tests. The other characteristic of SSL tests is they require memory at least twice the rate of non-SSL tests. The following tests were done with 150MB of RAM.
Bandwidth v.s. Capacity with SSL|
Parameters |
128Kbit/s |
256Kbits/s |
1Mbit/s |
10Mbits/s |
|---|---|---|---|---|
|
virtual users |
556 | 528 | 496 | 382 |
|
hits/sec |
612 |
793 |
873 |
664 |
|
Kbytes/sec |
3,393 |
3,923 |
3,256 |
3,671 |
Operating System Comparison: (NOT UPDATED FOR 2.7)
Under some test conditions different operating systems show different capacities to generate load on your web server. This is especially true in the case of a stress test, shown in the next section, but can also be true in the case of a realistic test configuration. The following tests show the differences when running a 10Mbit bandwidth SSL simulation using the same test configuration as in the previous tables.|
Parameters |
Win 2000 |
Win 2000 |
RedHat 7 |
RedHat 7 |
|---|---|---|---|---|
|
virtual users |
170 |
298 | 130 | 363 |
|
hits/sec |
444 |
788 |
444 | 848 |
|
Kbytes/sec |
2618 |
4602 |
2503 | 4233 |