I’m not usually a fan of vendor lock-in when it comes to performance test integration. I like being able to use the best product available to meet the need at hand. Were it otherwise; I’d still be driving around in a Ford, the car I drove right out of college; today I own a Kia. I’ve also driven a Saab, Toyota, Acura, Cadillac, Dodge, Pontiac, Mercury and Chevy pickup. As times changed, so did my needs. For me, it’s the car, not the manufacturer. The same is right for computer technology. I am not a Microsoft guy, any more than I am an Adobe guy, a Computer Associates guy or an Apple guy. Whether it’s software or cars, I use products that allow doing what I need to do, when I need to do it, in the best way possible. I need versatility.
The Value Proposition: Versatility Counts
My need for versatility extends to the tools that I use for testing. It’s rare to come across a vendor-centric, one-stop-shopping solution for testing, particularly performance test integration, which meets every need defined in a particular testing scenario. Yes, I’ve had episodes where a single testing package can do it all. But, those episodes were short-lived, typically brought to end when a new testing requirement came up, and the response from the vendor’s customer support line was the often heard, “sorry the product can’t do that, yet.”
In such cases I’ve been left high and dry, at the vendor’s beckon call, waiting for the next version to release. The problem was that I couldn’t wait a few months. I needed to solve my problem immediately.
I’ve learned my lesson. These days I avoid vendor lock-in and take an agnostic approach to performance test integration. My testing needs require that my tools play well together. Thus, the most important thing I look at when deciding the particular technology to use is how well it lends itself to integration. Broad support for integration is key to versatility. Versatility gives my testing solutions a long shelf-life. Hence, I end up doing a lot less work and saving my company a lot more money!
Building an Ecosystem that Supports Versatility and Integration
It’s been my experience that taking an agnostic approach to any software integration is key to understanding the ecosystem in which development takes place. Typically frameworks that support plugins or publish APIs lend themselves well to an agnostic approach.
Plugins Enhance Power and Extend Versatility
Software designers understand the importance of promoting third-party development to support feature integration. No one vendor has the staff and wherewithal to imagine every use a given product needs to help. This is why plugin architectures are so important. Plugins allow developers to extend a product to meet new needs by adding features far beyond the imagination of the original developer. Support for plugins is why products such as Jenkins, Eclipse, and WordPress have become so popular. Providing plugin capability has created a whole ecosystem of community developers that offer unique solutions well beyond what the product manufacturer could have done on its own. Plugin support is a win-win for all involved. A product with a large developer community that is writing a broad set of plugins provides the versatility, adaptability, and long-term viability a framework needs to stay relevant in the world of modern software development. Simply put, a plugin ecosystem without a third-party developer, ain’t.
APIs Provide a Standard for Integration
Plugins aren’t the only way to extend the power of a general solution framework. There’s also APIs. Most modern testing solutions provide an API that allows testers to manipulate test execution by way of code. Some APIs can be accessed directly on a local machine using a client library. Selenium Driver is one of the more well known local libraries. Software Development Engineers in Test (SDETs) use Selenium Driver libraries to execute selenium tests automatically on a local machine. Also, the driver can be used to perform tests that run on remote computers. One script can be used locally or remotely. The API is the same no matter where it is run. This consistency makes integration easy.
Many web-based testing products, running localhost or remote, provide RESTful APIs that can be used to integrate services on an as-needed basis. The beauty of a REST API is that all interaction is standardized under the HTTP protocol. No special libraries are needed. A REST-based API provides a one size fits all approach to testing and development integration. For example, it’s entirely possible for a test designer to integrate a system’s monitoring service and a separate performance testing service. All that’s required is that both services are accessible via a REST API. (See Figure 1.)
Figure 1: The REST standard allows a test runner to observe application behavior easily using a variety of data gathering resources
Whether integration is achieved using a plugin architecture or working with an API, the most important thing is that there is no vendor lock-in play. Yes, there are standards that must be supported, but these standards are not exclusionary. IBM can write a Jenkins plugin. Git can write a Jenkins plugin. Neotys can publish an API. I can create a WordPress plugin. No person or company is prohibiting development. All that’s required for entry into the ecosystem is adopting a standard open to all. Public access is essential for an agnostic approach to be viable.
| NeoLoad allows testers to take an agnostic approach to performance testing.
NeoLoad was designed from the start to allow testers to take an agonistic approach to performance test integration. NeoLoad integrates easily with commonly used tools such as Selenium and Eclipse. (Here’s a link to a step-by-step video that shows you how to execute Selenium scripts under NeoLoad using Eclipse). Also, NeoLoad Web provides a powerful REST API that allows testers to observe ongoing test and system behavior over a variety of systems as well as compare real-time and historic data. Testers can use NeoLoad’s APIs to get low-level system performance information or integrate with other RESTful system monitors in an agnostic fashion |
Putting it All Together
Modern software development is fast and furious. Feature releases that used to take months now take days, if not hours. Automation and integration allow more software to get to users at faster rates. Testing is no exception. Modern test designers cannot afford to risk slowing down their efforts by dedicating themselves to one vendor. Instead, they need to be able to pivot on a dime to meet release demands. Using a new plugin or adopting a new API immediately instead of waiting for a single vendor to release a feature to achieve a mission-critical need provides the versatility required to be competitive in today’s economy. Taking an agnostic approach to performance test integration is not an option that’s nice to have. It’s essential for ensuring that your company is making the quality software your customers want.
Learn More about Performance Test Integration
Discover more load testing and performance test integration content on the Neotys Resources pages, or download the latest version of NeoLoad and start testing today.
