Website Performance Testing Services
How many users can your website handle?
Load Testing Methodology
The key to using a testing methodology is in adapting it to the unique needs of a particular project. We take the experience from hundreds of consulting clients and use it to streamline the testing process while adapting it to each particular client. The diagram below shows the various steps through each part of the testing procedure. While it may look like there are a lot of boxes, for simple web sites the process goes quickly and can be turned around in as little as a day. The color coding signifies whether each step is to be done by Web Performance, our customer, or both.
Phase One: Requirements
The Statement of Work
The process starts by collecting customer requirements, including performance goals, schedule, system architecture, and use cases, and incorporating those into a statement of work containing a binding quote. The interview process usually takes about an hour, and we use desktop sharing to co-browse through the website and establish typical use cases.
Kickoff Meeting
Load testing is a team process, and the kickoff meeting is a chance for everyone who's participating to meet (via teleconferencing) and hammer out the last-minute details that weren't included in the Statement of Work.
Phase Two: Preparation
Server Monitor Installation
During a test performance is monitored from both the user's and server's perspectives. In order to collect detailed information about what's going on at the servers our Server Monitoring Software is installed. Note that we designed the software for installation on production hardware, and so it has a zero footprint, modifying no files on the servers, and thus uninstalls completely. This process can be done in parallel with test case development.
Test Case Development
Its crucial for the simulated server load to be as close as possible to what the server will see during production, and thus a series of test cases are developed that run in parallel during the test. For example, on a typical e-commerce system different users may be searching, placing items in a cart, updating their address, contacting customer service, all at the same time. Each of these different scenarios require a different test case so that the performance can be tracked separately.
Phase Three: Test Iterations
A test iteration takes you through the entire process of running a single test. In a single engagement we may run one test or several depending on how happy the customer is with the site's performance. The length of time it takes to complete an iteration depends on the complexity of the testing and the amount of changes needed. For a simple website that's almost at peak performance an iteration could take a couple of hours, while one in development that needed extensive re-work could take weeks between iterations.
How many test iterations will it take to reach your performance goals? Usually it comes down to two factors: the experience of the team and the complexity of the application. With an experienced team and a simple application usually it just takes one or two iterations: the system is measured performing to specification and no further tests are necessary. With a complex system or an inexperienced team or both any number of problems can be detected, including mis-configured load balancers or routers, dropped packets, or any number of code-related bugs.
Test Case Development
Its crucial for the simulated server load to be as close as possible to what the server will see during production, and thus a series of test cases are developed that run in parallel during the test. For example, on a typical e-commerce system different users may be searching, placing items in a cart, updating their address, contacting customer service, all at the same time. Each of these different scenarios require a different test case so that the performance can be tracked separately.
Test Case Verification
Test results are only as good as the test case design. In this step each test case and its associated data are thoroughly tested so that everyone has confidence that they are working correctly.
Load Simulation
In most cases the load simulation consists of a ramping load. Simulated concurrent users are added gradually until the load step is reached, and then held there until enough statistics are collected about each part of the application. By measuring the end-user performance and server-side statistics at each load level, and relationship between load and performance is established.
The test itself can last anywhere from an hour to three hours depending on the length of the test cases. Optionally, longer tests can be run, usually referred to as "soak tests" or "stress tests" can run up to a day, although those usually come at the end of the testing process when the site is shown to be performing up to specifications over shorter periods.
Configuration and Code Changes
After a test has been run Load Tester™ automatically calculates how many users the system can handle, but that is only the beginning of the analysis. The harder question is why? In a load testing consulting services engagement an experienced engineer will comb through the vast amounts of data to use his or her experience to determine the likely cause of any performance issues, or even if the given architecture is working up to its potential. The result is a written analysis that can be fairly detailed as it shows in these load testing report examples.
Make Changes
In this option step the customer makes any changes required to their code or configurations. Usually this is just fixing bugs or changing the network configuration, but in extreme cases it can include changing the code's session tracking framework or moving the entire application to new hardware. When all the changes are ready another iteration can begin to verify that the changes had the intended effect.