Major Software as Service Vendor Tests with 20,000 Concurrent Users
Software as a Service
Software as a service (SAS) has come to the forefront of IT trends with success stories such as salesforce.com, but it is also making waves in other corporate domains. A major trend in human resources for medium and large companies is the outsourcing of benefits management, replacing time consuming and slow paper forms with immediate access via web-based applications. An example of this is Long Term Care Partners, LLC (LTCP), which runs a secure website that enables Federal employees and family members to manage their FEDVIP (Federal Employees Dental and Vision Insurance Program) coverage that is provided with the oversight of the U.S. Office of Personnel Management (OPM). The BENEFEDS website (www.BENEFEDS.com) launched in the fall of 2006 with an initial 700,000 enrollees, and that number has since swollen to 1.2 million! The system also provides a convenient interface to manage the payroll and billing systems and customer service functions necessary to administer FEDVIP.
The Problem: Peak Usage Periods
Besides handling very large numbers of individuals, BENEFEDS.com had the disadvantage of concentrated peak user 's design, code defects, and web container configurations.
Internal Testing
LTCP realized that to meet their service level goals the website needed to support a minimum of 5,000 concurrent web sessions, and from there drastically scale for the accompanying volume of customer service calls. And so they began working with Web Performance Inc’s Web Performance Load Tester™ in their own testing lab, using a series of verbose internal load simulation tests that were designed to mimic how their production systems would perform in real world conditions. By analyzing the system metrics returned from the tests, they were able to identify and correct for the root causes of the application scalability issues.
External Testing
But internal tests only tell part of the story. LTCP coupled their own internal testing workbench results with a series of external tests conducted in a cooperative effort with Web Performance. The external tests were conducted by simulating users accessing the website from 3 different time zones simultaneously to model a true real world scenario. Load testing with users being simulated from outside the network also enabled LTCP to verify their Internet Acceleration and Security (ISA) Server configurations and network infrastructure. Mitch Hall, Web and Application Architect at Long Term Care Partners, said that working with Web Performance Inc.’s software the LTCP team was able to finally get a true picture of the system’s weaknesses and its capabilities. “Without this tool,” he said, “we would have been totally in the dark with respect on how to best design our applications to meet our current and future needs”.
As a result of both internal and external load testing, LTCP was pleased to report that during their 2007 open season things went incredibly smoothly. The BENEFEDS application was comfortably able to handle peaks of 15,000 concurrent sessions, and their customer service reps were able to accommodate an astounding call load that peaked at 30,000 calls in a single day. In preparation for the upcoming 2008 open season LTCP was able to successfully ramp up to 20,000 virtual sessions during their load simulation testing. The maximum value recorded for the Average Page Duration was 1:05 minutes. This coupled with high sustained CPU utilization suggests that the upper limits of their hardware had been reached. This information in and of itself is invaluable in that it indicates that larger sustained throughput will require additional hardware and or system upgrades in the future.
Web Performance Load Tester is a registered trademark of Web Performance, Inc. All the other trademarks are trademarks or registered trademarks of their respective owners. All rights reserved - 2008