What is Confirmation Testing?
Confirmation testing is a type of software testing technique in which the software-under-test is run through a set of previously run tests, just to make sure that the results are consistent & accurate. The intent is to ferret out any remaining bugs and check that all once-found bugs have been genuinely eliminated from the software components.
All tests run earlier are run once again after devs have fixed the bugs found in the first tests. Confirmation testing is also called re-testing because it runs the same test twice – one before finding bugs and one after.
Generally, when testers find a bug they report it to the dev team that actually created the code. After looking through the issue, the devs fixed the issue and pushed another version of the feature. Once QAs receive this scrubbed version of the software, they run tests to check that the new code is, indeed, bug-free.
When to do Confirmation Testing?
Right after a bug fix: Once a bug has been reported and fixed, confirmation tests should be run to check that the bug has been resolved.
Right before regression tests: Regression tests verify that code changes have not affected existing software functionalities. So, for regression to go smoothly, confirmation tests should be run beforehand. That way, the regression tests won’t be disrupted by existing bugs.
Right after bugs have been rejected: If a bug has been reported and is then dismissed by the dev team, these tests must be run to reproduce the bug….and say it again in greater detail.
How to run Confirmation Testing
Unlike most other software tests, confirmation testing has no specific techniques. You just run the same tests twice. As soon as a bug has been resolved, put the software module through the same tests that led to the discovery of the bug in the first place.
If the same bug (or new ones) do not emerge and all confirmation tests pass, you’re done. If not, testers must re-examine emerging bugs, reproduce them, and provide more profound, detailed reports to devs. Reappearing bugs can indicate more profound flaws in the underlying system.
What to do after Confirmation Testing
Once confirmation tests have confirmed that no live bugs exist in the application, the software can be moved further along the development pipeline. You push it to the next stage of testing/deployment.
The whole point of confirmation testing is to ensure the accuracy of bug elimination, thus making the software more reliable and worthy of customers’ positive attention.
However, often, confirmation tests are immediately followed by regression tests. Since one or more bug fixes have been implemented on the software, the regression tests check that these changes haven’t negatively impacted any of the software functions that were working perfectly well before debugging took place.
Features
- It requires planning and effort from the development and the testing team.
- It is not necessary to create new test cases, as the same ones that led to finding bugs will be rerun after debugging.
- It is often performed before regression testing to check that any existing modules have not been affected when fixing the bug
- Aims to increase software success rate by ensuring that an assuredly bug-free piece of software goes through the funnel.
- Confirm if the bug(s) still exist in any shape or form after the debugging exercise.
Advantages
- Confirmation tests ensure that an application works at the highest possible performance levels. These tests double-check if any existing bugs remain or if new bugs have appeared after debugging. Without the proper confirmation tests, bugs will not get through to deployment.
- These tests ensure that the software is bug-free and that users never disrupt its functionality.
- Confirmation tests are easy to execute because you run them twice. There is no need to create specific test cases or scripts.
- These tests are often executed right before regression tests. This ensures that specific modules are bug-free before the regression test checks for disruptions to the more extensive system.
- It also helps with the earlier detection of major or minor bugs.
Disadvantages
- Confirmation tests must be run manually since they cannot usually be automated due to uncertainties with defect resolution. Manual testing inevitably stretches timelines and can delay product releases.
- Generally, it is a best practice to run regression tests after confirmation tests to check the application’s overall viability. But running regression tests takes a fair amount of time, often overnight.
Challenges in Confirmation Testing
- These tests are change-oriented. For every confirmation test, testers have to run regression tests, which is a natural sink of time. As mentioned above, this can lengthen time-to-market, which is never a good thing.
- Confirmation tests usually check if pre-existing bugs are still plaguing the software. They hardly ever find new bugs. Identifying new bugs requires a separate set of tests, which pushes more workload onto tests, and the devs have to fix those new bugs. This is quite a gap in the scope of testing.
- Once again, as mentioned above, confirmation tests can usually only be manually executed. Every time a bug is fixed, these tests must verify that they are gone. Given that most modern software is pretty complex and layered, multiple bugs will show up at any given time. Imagine running confirmation tests for all those bugs, and you’ll see the problem.
Conclusion
Confirmation tests are a necessary (if somewhat inconvenient) fail-safe that testing cycles need to push out truly bug-free products. The challenge lies in designing and scheduling these tests without stretching the timelines to unacceptable levels. However, the efficacy of these tests is beyond question, and they absolutely deserve a place of pride in your test suites.
