Testing Auto-Scaling Servers with Load Tester 5.1
One of the easiest ways to create a system that keeps up with sudden increases in demand is to use auto-scaling systems offered by many cloud providers. The concept of bringing new systems online to handle demand is nothing new: this has also been used by CDNs for years to replicate high demand data to edge locations. Testing this type of system, however, can become more challenging when the auto-scaling relies on updating DNS records in order to route new users to new servers.
In Load Tester 5.0 and earlier, Load Tester could be set to automatically simulate DNS Load Balancing for hostnames with multiple IP addresses. This allows a small number of Load Engines to apply round-robin Load Balancing of IP addresses to all their Virtual Users, rather than directing traffic to a single IP address because of engine OS or DNS server caching a single result. This feature works well when Load Tester is able to resolve all the IP addresses for a given name at the beginning of a test, but would prevent Load Tester 5.0 and earlier from detecting when new IP addresses came online for the user’s hostname.
For an example of how this works, we’ll try a test system using Amazon ELB (Elastic Load Balancing). For the sake of demonstration, we will scale this system up manually during a load test, rather than using the automatic scaling. In our tests, Amazon’s ELB creates an IP address within each availability zone which has instances running. The traffic to each IP address is then distributed among the instances in the same zone. As instances come online in new availability zones, new IP addresses are added. Let’s start a test with 2 web servers (each in different availability zones) and add them to a single ELB Load Balancer. We see that Load Tester will distribute the load among both IP addresses for the web servers, just as if hundreds of real users resolved the IP address with a wide distribution of DNS servers.
Note that our Network usage graphs show that both web servers are being nearly equally utilized.
Next, we’ll try adding a 3rd instance while the test is running with Load Tester 5.1. We’ll add this instance to a previously unused availability zone, in order to create a new IP address. Given some time for the test to add more virtual users, and reset those users that completed their workflow, we can re-examine the traffic pattern.
Notice the network charts (Max Network In and Out). When the new server comes online (blue), the network usage for that server starts out low. As new web sessions are created (either from new virtual users being added to the load test, or existing virtual users restarting their testcase), Load Tester will re-query the DNS entries to hand off to users, and some users will be directed to the new server. The bandwidth usage gradually equalizes between the three servers as a some of the users move from the first two servers to the new server.
If the system had a single Load Balancer with a single public facing IP address, we would expect the Load Balancer to shape traffic from new sessions in a similar fashion. However, by pushing the load balancing to the DNS level, our test system has relied on the DNS system to distribute IP addresses to clients.
Here, we’ve shown that Load Tester 5.1 can respond to dynamically changing test environments, and emulate traffic patterns accordingly to provide improved test accuracy and reliability, and all out of the box without complicated configuration scripts. We hope that you will find this feature improves your testing process with cloud and application hosting providers. As always, if you have any questions, our friendly support staff will be happy to answer them.
Happy Testing!
-Frank
Engineer at Web Performance
Frank is an engineer for Web Performance. He is also an advocate for correct fire safety procedures whenever applying massive load to production test rigs.