How Much Bandwidth Do You Need To Do a Load Test?
One of the common questions people are interested in is finding out how much a test is going to cost. There are a lot of factors that go into this equation, such as getting an appropriately sized testing license, configuring a test server environment, reserving hardware for Load Engines, and bandwidth costs between the Load Engines and the content delivery servers (such as a CDN or origin servers). Let’s take a look at just how we might calculate the bandwidth charges that are involved in a single test.
If your site is only needs to support a few hundred users, then chances are your bandwidth costs for a load test will be small, with test development and preparation being a larger focus. But if you measure you capacity in the tens or hundreds of thousands, then the cost becomes more significant. For those that go further, supporting hundreds of thousands or even millions of users, may find that their users can consume hundreds of petabytes of data over the duration of the test, leading to 5 figure charges from their content provider.
Obviously, bandwidth charges are applied by the provider serving your bandwidth. Thus, if you have an internal system, you most likely need read no further. However, if you pay for access to your site through a hosting and/or content provider, it’s important to understand how bandwidth will be charged be it for a Load Test that runs for a few hours, or for a constant stream of users returning to your site for months. To illustrate the costs for a single testcase, we’ve added a handy calculator.
Step 1: Define your Load Test
This part is specific to your Load Configuration in Load Tester, and helps plan for how many times will my testcase be run. You will need to be able to answer some basic questions:
- How long will the test run?
- What is the peak number of users the test will reach?
Step 2: Define your Testcase
To determine how many times the testcase will run, we also need to know how long it would normally take a user to complete their scenario (and thus the period between scenarios for each Virtual User being simulated). Additionally, we also need to know how much bandwidth is consumed by the scenario. Both of these factors are easily visible in Testcase Editor (improved in Load Tester 5.3).
By clicking the Up/Down arrows, we can see a baseline for the Upload & Download consumption each time this scenario will be run.
The size of the testcase will have a great influence on how much bandwidth your users consume – both during a test, and after you go live. For example, with 3 MB of download content for each user, it will only take 1,000 total visits to the site to consume 3 GB from your hosting provider. If your provider were to charge 50¢ per GB, then you are effectively paying your hosting provider $0.0015 for each visit to your site.
Step 3: Pricing rate for your provider
To start converting bytes into dollars, you’ll need to know how much your hosting provider is charging you. If you use a CDN, you’ll want to start with the CDN’s data transfer rates (more on CDNs below).
Pricing for data transfer to the load engines is often more modest: most tests will use the engines to receive bandwidth which many providers do not charge for. If you are using Load Tester’s built in Amazon EC2 integration, then the data transfer charges are currently included. However, if you are using EC2 for other services, you can find their pricing at: http://aws.amazon.com/ec2/pricing/#DataTransfer .
Disclaimers
Obviously, this estimate should be verified against your environment, as providers can have very different pricing models. It is also important to be aware these estimates are constructed around the test running smoothly. If your system becomes unresponsive, then your bandwidth may drop during the test (and your customers will leave you in production). However, we’ve also seen systems that generate unpredictably enormous error pages when the system begins to fail, causing massive surges in bandwidth as the system struggles with load.
If you are connecting to your content through a CDN hosted domain, then you will also want to be sure to estimate the data transfer charges between your origin servers and the various CDN edge points.
Lastly, this calculator also displays an estimated cost per visit, based on a visitor following the workflow modeled by your testcase. This number may be helpful in making projections of monthly bandwidth costs of a production system, if such data is not already available.
To get started using the calculator now, just navigate to: https://www.webperformance.com/library/tutorials/CalculateNumberOfLoadtestUsers/#bandwidthbill
We hope this calculator will be useful. If you have any questions, please feel free to use our contact form, and our friendly staff will be happy to answer!
Happy Testing!
Frank is an engineer for Web Performance. He is also an advocate for correct fire safety procedures whenever applying massive load to production test rigs.