Illusions in Software Testing Schedules
In a typical software development life cycle (SDLC) it is not unusual for the software testing phase of the project to be compressed.
As the last stage in the cycle the time scales for the software testing tend to get reduced as the project team attempts to maintain the commitment to meet the delivery date.
It is usually the software test schedule that gets reduced because reducing the development time either means removing features or increasing the amount of developers.
Neither of these options appear very attractive.
One means an uncomfortable discussion with the customer and the other means increased costs.
So any tough decisions around adjusting the development phase are usually postponed and the call deferred until the product enters the software test phase.
At this point the discussions become more interesting.
The main options at this point include adding more software testers, reducing the amount of software testing or changing the delivery date.
Changing the delivery date is usually dismissed initially as not acceptable to the customer.
Adding more testers is usually dismissed as it requires additional expenditure.
This only leaves reducing the amount of software testing.
The interesting thing to note here is that the first two (delivery date and more testers) are both quantifiable and visible.
Changing the delivery date requires agreement with the customer and adding more testers will increases costs (and decrease profit).
Reducing the amount of time spent testing isn't as visible and is difficult to quantify.
The only point at which reducing the amount of software testing becomes visible is after the product is released.
So, unless you've got a very good case, forget about trying to convince the project team that reducing the amount of software testing is the wrong solution.
The project team are unlikely to listen when the other options are so visible and quantifiable.
The option to reduce the amount of software testing is likely to be selected and as a result this will create the illusion that the project is on track.
But it's an illusion that can't be maintained indefinitely.
Usually this illusion is shattered just after the software product is released.
The aim is to shatter that illusion before the product ships so that the customer maintains their faith in your ability to deliver a quality product.
There is only really one strategy at the test team's disposal that combats this demand to reduce the software testing schedule.
This strategy is as follows: 1.
Initially accept the reduction Don't waste your time trying to argue against the reduction in test time unless you have a very strong case, which you know you can win.
You are better off directing your efforts into more constructive tasks and following the next 3 steps.
2.
Deploy the best testers you have on the project Good software testers find good defects.
With the best testers you have on the project you stand a better chance of finding more defects in a shorter time frame.
From the project managers point of view this will look like you are playing ball with their requirements to reduce the testing time.
In reality you'll be using your best testers to help you prove your case.
The case you are aiming to prove is that the product will not reach the required level of quality by the time this software product is ready to ship.
3.
Prepare your evidence This step is all about turning the testing from something that isn't quantifiable in to something that is quantifiable.
Start tracking the defect find rate (number and severity of defects against the date) on a graph.
If your best testers are doing the job they are employed for then this graph is going to show a ramping up and steady increase in the defect detection rate.
4.
Use your evidence Your defect detection rate should be used as evidence to break the illusion that the project is on track.
If your testers are continuing to find defects at a steady rate, with significant defect priorities, then this can be projected forward.
With projected estimates you can demonstrate that the software application won't be of sufficient quality to release on the agreed date.
This isn't an exact science in software testing but it will give you something concrete to fight your corner with.
If you combine this with the rate at which the development team are fixing defects you have quite a powerful set of statistics to work with.
You could even go as far as estimating the number serious defects that the software product is likely to be released with.
You need to start collecting this evidence as quickly as possible and present the evidence to the project team as soon as you have enough data.
This then gives everyone three tangible options to choose from; change the delivery date, add more resources (to development and test) or release with poor quality.
Of course the option to release with poor quality will still be high on the list of choices here.
So you may want to support your defect find rate evidence with details about the types of likely defects.
For example, if your team are finding high priority defects, like issues that would block a release, on a daily basis then you are in a stronger position.
In a strong position you are better able to counter people using the software testing reduction solution as a credible option.
It is important to remember that the aim isn't to work against the development and projects team here.
The aim is to work with them.
After all the software test team have as much to gain from a high quality release, delivered on time, as much as anyone else.
The goal should always be to present as much information as possible so that the whole team (test, development and projects) can make the right call.
The right call in balancing the release date, features and quality is best made with all the relevant data.
Sometimes, much to the disdain of the test team, that right call is to release with slightly below par quality.
So long as that call is made with the best intentions and is based on the right data then you stand a much higher chance of getting it right.
As always, in the end, it comes down to getting the balance of delivery date, features and quality right for the customer.
As the last stage in the cycle the time scales for the software testing tend to get reduced as the project team attempts to maintain the commitment to meet the delivery date.
It is usually the software test schedule that gets reduced because reducing the development time either means removing features or increasing the amount of developers.
Neither of these options appear very attractive.
One means an uncomfortable discussion with the customer and the other means increased costs.
So any tough decisions around adjusting the development phase are usually postponed and the call deferred until the product enters the software test phase.
At this point the discussions become more interesting.
The main options at this point include adding more software testers, reducing the amount of software testing or changing the delivery date.
Changing the delivery date is usually dismissed initially as not acceptable to the customer.
Adding more testers is usually dismissed as it requires additional expenditure.
This only leaves reducing the amount of software testing.
The interesting thing to note here is that the first two (delivery date and more testers) are both quantifiable and visible.
Changing the delivery date requires agreement with the customer and adding more testers will increases costs (and decrease profit).
Reducing the amount of time spent testing isn't as visible and is difficult to quantify.
The only point at which reducing the amount of software testing becomes visible is after the product is released.
So, unless you've got a very good case, forget about trying to convince the project team that reducing the amount of software testing is the wrong solution.
The project team are unlikely to listen when the other options are so visible and quantifiable.
The option to reduce the amount of software testing is likely to be selected and as a result this will create the illusion that the project is on track.
But it's an illusion that can't be maintained indefinitely.
Usually this illusion is shattered just after the software product is released.
The aim is to shatter that illusion before the product ships so that the customer maintains their faith in your ability to deliver a quality product.
There is only really one strategy at the test team's disposal that combats this demand to reduce the software testing schedule.
This strategy is as follows: 1.
Initially accept the reduction Don't waste your time trying to argue against the reduction in test time unless you have a very strong case, which you know you can win.
You are better off directing your efforts into more constructive tasks and following the next 3 steps.
2.
Deploy the best testers you have on the project Good software testers find good defects.
With the best testers you have on the project you stand a better chance of finding more defects in a shorter time frame.
From the project managers point of view this will look like you are playing ball with their requirements to reduce the testing time.
In reality you'll be using your best testers to help you prove your case.
The case you are aiming to prove is that the product will not reach the required level of quality by the time this software product is ready to ship.
3.
Prepare your evidence This step is all about turning the testing from something that isn't quantifiable in to something that is quantifiable.
Start tracking the defect find rate (number and severity of defects against the date) on a graph.
If your best testers are doing the job they are employed for then this graph is going to show a ramping up and steady increase in the defect detection rate.
4.
Use your evidence Your defect detection rate should be used as evidence to break the illusion that the project is on track.
If your testers are continuing to find defects at a steady rate, with significant defect priorities, then this can be projected forward.
With projected estimates you can demonstrate that the software application won't be of sufficient quality to release on the agreed date.
This isn't an exact science in software testing but it will give you something concrete to fight your corner with.
If you combine this with the rate at which the development team are fixing defects you have quite a powerful set of statistics to work with.
You could even go as far as estimating the number serious defects that the software product is likely to be released with.
You need to start collecting this evidence as quickly as possible and present the evidence to the project team as soon as you have enough data.
This then gives everyone three tangible options to choose from; change the delivery date, add more resources (to development and test) or release with poor quality.
Of course the option to release with poor quality will still be high on the list of choices here.
So you may want to support your defect find rate evidence with details about the types of likely defects.
For example, if your team are finding high priority defects, like issues that would block a release, on a daily basis then you are in a stronger position.
In a strong position you are better able to counter people using the software testing reduction solution as a credible option.
It is important to remember that the aim isn't to work against the development and projects team here.
The aim is to work with them.
After all the software test team have as much to gain from a high quality release, delivered on time, as much as anyone else.
The goal should always be to present as much information as possible so that the whole team (test, development and projects) can make the right call.
The right call in balancing the release date, features and quality is best made with all the relevant data.
Sometimes, much to the disdain of the test team, that right call is to release with slightly below par quality.
So long as that call is made with the best intentions and is based on the right data then you stand a much higher chance of getting it right.
As always, in the end, it comes down to getting the balance of delivery date, features and quality right for the customer.
Source...