Testing and Traceability
Testing is an important part of the Software Development Life Cycle (SDLC) and many teams strive to include testing into the way they build their products.
The implementation of testing can range from a completely manual process carried out after development, to developers writing automated checks as part of writing their code and a team of testers carrying out exploratory testing alongside them.
While this is a great way to ensure the product is of a high quality, this only benefits the development team. The business side of the team (Business Analysts, Product Owners etc) don’t have a means of understanding these quality metrics in the context of the user stories and other artefacts they use in their roles.
Often if teams do share information about the testing activities it will be in the form of a test report that will cover which of the tests were run and the status of those tests.
This report may provide some information to allow the business facing team members to determine the quality of the product’s functionality, but it is a one-way information flow, with limited means of reversing the link between test case and user story (or other artefact).
Bi-Directional Traceability
This lack of two-way linking, (called Bi-Directional Traceability) can lead to the testing output losing value and may even lead to testing being seen as a siloed activity that the development team do with a check at the end to ensure they’ve completed the activity.
If the testing activity is seen as a siloed activity (even if it’s fully integrated into the development process) then it’s going to be hard for the business team to realise the value that testing brings to product development.
With Bi-Directional Traceability both the business-facing and development teams can see which user stories, test cases and test results are linked from their respective work items (user stories, automated tests, exploratory sessions etc).
This traceability becomes even more useful when testing is carried across a matrix of different configurations such as different devices or environments, or the test results are ‘bubbled up’ from the user stories and into the epics as this allows for the team to see a high level view of the quality of the product easily and make decisions to tackle problematic areas.
Tools for enabling Bi-Directional Traceability
Bi-Directional Traceability can be achieved in a number of ways, the simplest being links to test reports for different tests added to user stories and links to user stories being added to tests. This approach it hard to maintain though due to the manual effort needed to keep the links updated.
BDD tools such as Gherkin and Spec offer a nicer way of linking test cases to user stories as they use human language to define the tests, meaning that as long as the user story has acceptance criteria using the wording and the tests use the same wording then it’s easier to find the matching stories and test cases.
Linked to BDD is another technique called Living Documentation, where the human language of the BDD tests is used to produce documentation to describe the features of the product and the test results are used to show the quality of those features.
The Living Documentation approach is really great when you’re building a product for an external client who isn’t integrated into your development process but if you’re working closely with the business team then you should look to move this traceability to the artefacts so they can use the testing data to make informed decisions.
If you’re using an issue tracker (such as Jira) to capture user need there are a number of apps you can install which provide a means for test result data files (such as those generated by an automated test framework) to be uploaded and have those tests represented in the issue tracker where they can be linked to the user stories.
The issue tracker traceability is especially powerful when the product’s issues are linked to other metadata such as release version, environmental configuration and more, as this then allows for the quality of the product to be calculated based on these properties and the team can query and get answers quickly.
Cultural changes for embracing Bi-Directional Traceability
Implementing Bi-Directional Traceability via the tools above isn’t going to transform the way the business and development teams use test data to make informed decisions overnight — there needs to be cultural changes to the way the team carry out their analysis and development activities.
As a lot of the Bi-Directional Traceability tooling requires using shared language, there needs to be time given for the business and development teams to define this language and you’re using one of the more manual linking methods that this language doesn’t get changed in the time between the business needs being written and the test being executed.
Once the test data is linked to the business artefacts (user stories etc) then the team needs to change the way they carry out analysis to actually use this data in their discussions.
This can be as simple as reviewing which test cases need to be updated based on changes that are going to be made or advanced usages, such as using the test results from the different environmental permutations during a backlog grooming session to determine the priority of a bug.
The test cases linked to business artefacts also allow the developers on the team to understand which code needs to be changed as they can run those linked tests and see which functions are called, which allows them to raise concerns or suggest refactoring to be completed during development of new functionality.
The additional time taken during analysis will ultimately save the team time, in that it’s less likely there will be a need for bugs to be fixed or the work will need to be redone, but it can be hard to find the time to do these activities.
Summary
Testing is an important activity to do but if that activity is carried out in a silo, regardless of how integrated with the development team, then you’ll never realise the full value of testing.
Bi-Directional Traceability is a powerful technique that empowers the business side of the team to make informed decisions about the new features they are building and how they should prioritise the new features over fixing issues in the product.
In order to really gain value from Bi-Directional Traceability there needs to be cultural changes to allow the team as a whole to use the data to build a shared understanding about the impact of the change and the value it will give the customer.