Wednesday 27 August 2014

How to create a visual test coverage model

Creating a visual test coverage model to show test ideas in a mind map format is not a new idea. However it can be a challenging change in paradigm for people who are used to writing test cases that contain linear test steps. Through teaching visual modelling I have had the opportunity to observe how some people struggle when attempting to use a mind map for the first time.

Though there is no single right way to create a visual test coverage model, I teach a simple approach to help those testers who want to try using mind maps but aren't sure where to begin. I hope that from this seed, as people become confident in using a mind map to visualise their test ideas, they would then adapt this process to suit their own project environment.

Function First

The first step when considering what to test for a given function is to try and understand what it does. A good place to start a mind map is with the written requirements or acceptance criteria.

Imagine a story that includes developing the add, edit, view, and delete operations for a simple database table. The first iteration of the visual test coverage model might look like this:


Collaborate

Next consider whether all the behaviour of this function is captured in the written requirements. There are likely to be items that have not been explicitly listed. The UI may provide a richer experience than was originally requested. The business analyst may think that "some things just go without saying". There may be application level requirements that apply to this particular function.

Collaboration is the key to discovering what else this function can do. Ask a business analyst and a developer to review the mind map to be sure that every behaviour is captured. This review generally doesn't take long, and a quick conversation early in the process can prevent a lot of frustration later on.

Imagine that the developer tells us that the default design for view includes sort, filter, and pagination. Imagine that the business analyst mentions that we always ask our users to confirm before we delete data. The second iteration of the visual test coverage model might look like this:


Think Testing

With a rounded understanding of what the function does the next thing to consider is how to test it.

For people that are brand new to using a mind map, my suggestion is to start by thinking of the names of the test cases that they would traditionally scope. Instead of writing down the whole test case name, just note the key word or phrase that differentiates that case from others. This is a test idea.

Test ideas are written against the behaviour to which they apply. This means that tests and requirements are directly associated, which supports audit requirements.

Imagine that the tester scopes a basic set of test ideas. The third iteration of the visual test coverage model might look like this:


Expand your horizons

When inspiration evaporates, the next challenge is to consider whether the test ideas captured in the model are sufficient. There are some excellent resources to help answer this question.

The Test Heuristics Cheat Sheet by Elisabeth Hendrickson is a quick document to scan through, and there is almost always a Data Type Attack that I want to add to my model. The Heuristic Test Strategy Model by James Bach is longer, but I particularly like the Quality Criteria Categories that prompt me to think of non-functional test ideas that may apply. Considering common test heuristics can help achieve better test coverage than when we think alone.

Similarly, if there are other testers working in the project team ask them to review the model. A group of testers with shared domain knowledge and varied thinking are an incredibly valuable resource.

Imagine that referring to test heuristic resources and completing a peer review provides plenty of new test ideas. The fourth iteration of the visual test coverage model would have a lot of extra branches!

Lift Off!

From this point the visual test coverage model can be used in a number of ways; a base for structured exploratory testing using session based testing, a visual representation of a test case suite, a tool to demonstrate whether test ideas are covered by automated checks or testing, or as a radar to report progress and status of test execution. Regardless of use, the model is likely to evolve over time.

I hope that this process encourages those who are new to visual test coverage modelling to try it out.

8 comments:

  1. Great post Katrina - nice to share the methods! Cheers Sharon

    ReplyDelete
  2. Loved it. Sharing on my company handle.

    ReplyDelete
  3. Katrina, thank you for this clear and concise explanation. I work like this too and find it's a really fast and effective way to capture test ideas (but I don't always get to explain it as nicely as you do).

    A few observations, that might add to what you have written:
    - Your example is of course depicting a simple function, to ease the understanding of the principle. Most testers will start with these basic CRUD functions, and then add layers of 'submit', 'cancel', 'validation of field' etc. As the testing gets going, testers quickly learn all such functions that needs to be checked out, and this layer tends (in my experience) to wander from the mindmap into a checklist - which is good, as it helps to keep the overview in the mindmap, and checklists helps you avoid writing a lot of things several times.
    - Also, over time, these functions tend to get more and more complex. Instead of '[entity]->add->submit/cancel', it becomes more like '[process]->variation'. That's a sign that the test team has been accumulating a lot of knowledge.
    - To many people, this is not 'proper’, until it all has been transferred into detailed test cases. To me, the mind map /is/ very proper and professional indeed. We have splendid documentation of what we want to investigate (in a mindmap form which is easily communicated), and documentation of what happens during testing can be done in various ways.
    - Some of the things you identify as test ideas are not readily possible, most often because noone knows how to do it. These can be marked with a coloured ring, symbol/icon or similar - so that we all know that it will be a stopping point until someone sits down and figure out how to accomplish it (in contrast when transferring ideas into testcases that information is often obscurred and hidden, as it is just 'work' that gets delayed or take longer than expected).
    - In my experience creating such a mind map takes only a few minutes (I have done this for 8-9 user stories in one go in less than 30 minutes). At that point a lot of this testing is /ready to start/! Transferring to test cases is not adding value to these, just wasting time and delaying obtaining test results.
    - During testing the mindmap can be extended. Even if using help from heuristics, fellow testers, architects/business analysts and programmers to create it, the feedback from the system itself often open your eyes to a whole new aspect (which further talks to not making writing test cases the next step after the mindmap, but rather go to testing).

    Thanks for a great post!

    ReplyDelete
  4. Hi Katrina,

    This is useful, thanks.

    What app did you use to create the visual test coverage model?
    I tired Visio, but it was more time consuming than I had hoped.

    ReplyDelete
    Replies
    1. Mike, thanks for your comment. I have used both FreeMind and xMind to create these models. There are a lot of options out there for free mind mapping software. Best of luck :)

      Delete
  5. Thanks Katrina for this great post. I have tried many times to write my test cases using the mind-map but had struggled initially. Your approach is definitely going to help me and others for sure. Cheers.

    ReplyDelete
  6. Nice blog. This is really helpful to trigger the thinking process towards creating better test cases and coverage.

    ReplyDelete