Load Engine simulates 3500 users on a single PC in performance test
We are frequently asked how many load engines will be needed to run a load test. Some general guidelines are provided here. Those guidelines were established using a fairly typical test scenario. With some tuning, we were able to tune the OS and Load Engine to simulate 3500 users on a single computer.
Many testcases are much sparser than the testcase used to create our guidelines – that is, they generate fewer pages requests per user than a more typical scenario. When testing systems for our customers where the testcases are sparse, we have found that the number of virtual users that can be generated will be limited not by the processing power of the load engine, but rather the threading capacity of the OS and the tuning parameters that affect it. If the customer needs to run very large tests (e.g. tests in the 10,000 to 100,000 users range), this can present infrastructure problems — we would need a very large number of machines to execute these tests even though the total processing power on each machine is not fully utilized.
So we looked for improvements that would allow better utilization of the available resources to simulate more users on the same hardware. The baseline estimates on Linux were generated using our Instant Load Engine boot disk. This is based on Red Hat Fedora 8 – which is a desktop-oriented distribution. So we first switched to an enterprise server OS – CentOS 5, which is based on Red Hat Enterprise Server 5. Then we increased one of the limiting factors, the number of available file descriptors, to a much larger number than the default (50,000 – by setting the hard and soft nofiles limits in /etc/security/limits.conf). Note that file descriptors are also used by the Linux OS when new threads are allocated. We then increased the memory available to the Load Engine to 800M (in the Load_Engine.lax file). We also raised the Load Engines internal limit to 3500 users (by adding a system.properties file to the Load Engine installation folder containing one line: “EngineUserCapacity=3500”).
After making these changes, we were able to simulate 3500 users on a single machine (2.8GHz Xeon with 1G RAM and 1Ge NIC). On a small testcase (7 pages,4000 hits/sec, >450 pages/sec and >200 Mbps.
Note that we have tried different combinations of file descriptor limits, memory and user capacity settings with varying results. For instance, the above configuration could run with more users but would sometimes malfunction and abort the test. Some experimentation may be required to find optimal reliable performance.
When his dad brought home a Commodore PET computer, Chris was drawn into computers. 7 years later, after finishing his degree in Computer and Electrical Engineering at Purdue University, he found himself writing software for industrial control systems. His first foray into testing software resulted in an innovative control system for testing lubricants in automotive engines. The Internet grabbed his attention and he became one of the first Sun Certified Java Developers. His focus then locked on performance testing of websites. As Chief Engineer for Web Performance since 2001, Chris now spends his time turning real-world testing challenges into new features for the Load Tester product.