AS/400 Load Testing

Large Scale AS/400 Load Testing

In November of 1995, Network Pathways received a request from a large utility looking for help to reduce the risk in implementing its first AS/400 application. Our client was concerned the application might be too slow or the AS/400 might not "fit" in its existing TCP/IP environment. The utility’s executive had to know with certainty that the new system would work when their 500 users signed on for the first time! The project implementation strategy was unforgiving in this regard -- a 20 year old legacy system would to be "turned off" on a Friday evening and replaced by an AS/400 system that would "go live" two days latter. Added to this, the utility was about to become the application vendor’s largest installed site by a factor of two.

Network Pathways was asked to lead the acceptance test effort for this project and immediately began looking for automated test tools. After an extensive search of available products, three software vendors were selected for proof-of-concept demonstrations and, based on these evaluations, Performix/TTY™ from Rational Software Corporation was selected.†

Over the next 6 months Network Pathways developed a test capability which at that time was likely unparalleled anywhere. Today we believe it may only be equaled by IBM's Rochester facility.

Using Rational Software's Performix/TTY test tools and StarQuest Connectivity's TN5250 terminal emulator, Network Pathways was able to generate accurate workloads consisting of 100s of "virtual" users all running telnet 5250 sessions on the target AS/400 system. The Performix/TTY test harness was found to be extremely scalable and capable of generating 1000's of virtual users. Users that operate real AS/400 applications and that look and feel exactly like real users to the AS/400 system under test.

The Test Environment

The test bed used for this project consisted of three Pentium-166Mhz test drivers all running SCO UNIX. Each test driver was used to generate between 140 and 170 virtual users depending on the requirements of the specific workload. The Performix/TTY tools made it easy to coordinate complex workloads across multiple test drivers and provided the ability to scale workloads to 1000’s of users.

Performance

Workloads were developed to model the business activity expected with 300 users operating the new system under different seasonal conditions. These models included 5 different classes of users (call center, field staff, etc.) and 13 different business events (service request, trouble report, account inquiry, etc.). Classes and events were easily combined to create a variety of workloads which simulated the expected worse case scenarios at different times of the year.

Capacity

These same workloads were then scaled from 50 to 500 users to develop capacity curves, which predicted the performance brick walls!

Stress

Stress tests were devised to subject the application to unrealistic workloads to determine the weakest points. In these tests, users performed the same operations in lock step and committed their transactions at the precisely the same time. Tests were run with 5 to 100 users, adding 5 more each time, to see when/if the system would break. In each test one virtual user was appointed to "supervise" the others. Each time the users reached a rendezvous point, the "supervisor" would signal them to all click [OK] at exactly the same time. This is a punishing test that could not have been duplicated manually.

Also, Sign-On tests were run to ensure the system would be able to handle all 500 users signing on to the application during a half-hour period in the morning.

Test Results

Performix/TTY allowed us to measure the AS/400 application’s response times with pinpoint accuracy. It provided detailed statistics and identified application bottlenecks. The test results were easy to understand, well organized, and easily summarized by class of user, business event, individual screen, and groups of screens.

Technical Information

For technical information on how Network Pathways’ AS-400 Test Library extents the impressive capabilities of Rational Software Corporation's Performix/TTY software to include AS/400 testing, click here.

Footnotes:

†

In 1995, Performix/TTY was known as Empower/TTY. The product was originally developed by Performix, Inc. of McLean, Virginia. In 1996, Performix, Inc. was acquired by Pure Software, Inc. which latter merged with Atria Software Inc. to form Pure Atria, Inc.. In 1997, Pure Atria was acquired by Rational Software Corporation.