A new paper from CIC is now available for download with the kind permission of Microsoft where we examine the benefits and challenges of early lifecycle testing for a cluster of technical test types, commonly known as performance testing. The aim is to bring managers up to date on the latest thinking on the subject and practice in the field, with a view to cutting away the myths and removing obstacles blocking adoption.
Here is a summary of the paper’s central points:
Early testing reserved for functional testing
Early testing delivers business benefits by enabling IT organizations to deliver more at lower cost. However, organizations have concentrated on introducing early functional testing and have largely ignored the potential of early performance testing.
The dynamics of early testing apply equally or more to performance testing
Early testing delivers benefits in broadly two ways:
• It saves money on rework
• It mitigates the risks of surprises late in a project.
These dynamics are extremely relevant to performance issues, which perhaps even more than functional errors hold the potential of sending a project ‘back to the drawing board’ if components or infrastructure don’t perform as required.
Practitioners believe there are two unsolvable challenges to early performance testing
There are a whole host of potential difficulties to introducing early performance testing. Most of them are the same as those that impede the adoption of early functional testing. They are largely soft, people issues, centered around the roles and psychology of software coders. There are fairly well understood change management techniques for dealing with them.
However, to most testers and QA managers, there are two unique aspects of performance testing that render it unsuitable for early testing. The beliefs are:
• You can only test performance on a fully built system which doesn’t exist early on.
• You don’t have suitable component level performance requirements.
The solutions to these objections require a finer look at what performance testing consists of and a pragmatic approach to achieving your goals without necessarily achieving textbook perfection.
Introducing early performance testing is a change management project
Starting down the road towards early performance testing will involve developing motivation among those who will be responsible for doing it. You will need to make controlled changes to all your existing development processes, enlist the help of specialists – including your existing late cycle performance testers – and adopt tools that give you the kind of automated support that works best alongside existing development tools. You should only undertake this within the context of a properly conceived change management project.
The full paper is available on our Creative Research page.