Friday, 18 April 2008

Challenge your assumptions and presuppositions to identify usefulvariation

Any curious tester can find a number of published heuristic documents out there on the web (James Bach, Elisabeth Hendrickson)

'Heuristics' appear regularly on blog posts. (Mike Kelly, Ainars Galvans, Scott Barber, David Gilbert)

In this post I aim to show you an easy way of identifying new test ideas without recourse to heuristics, on a case by case basis, to allow you to add further depth to your own test explorations.

You can apply this approach to an exploratory testing process as well as a scripted testing process.

But first the 'thing' I assume before you implement this approach: when you have 'done' your testing, you know what you did, and why you think you did it (most of the time).

If that assumption seems true to you then you stand a good chance of immediately implementing this approach.

If that assumption seems untrue then the next time you practice your testing, spend some energy writing down the decisions that you make and some 'rationale' for those decisions.

So, the approach:

  • Step back and examine your assumptions underlying what you did,
  • Identify how you could challenge those assumptions.

So... a worked hypothetical example:

I have a system that uploads a spreadsheet file that I created from another application. The spreadsheet imports correctly. What assumptions have I made?

  • The app will have 'closed' the spreadsheet prior to uploading
  • The app will have finished writing the spreadsheet prior to uploading
  • The spreadsheet had valid data -> presupposes -> The spreadsheet had data
  • File size did not matter

I could then go and challenge these assumptions through testing.

How did I identify these assumptions?

Some of them were 'attribute' assumptions. What attributes of the spreadsheet did I make assumptions about? size, contents (what other attributes of a spreadsheet did I make assumptions about or not even consider? versions, file extensions, naming, location... what else?

Some of them were 'state' assumptions. Closed, Written - what other states did I miss?...Deleted, (question - does open = locked?)

Now a real worked example:

(you may have to follow this through at home to make sense of the writing)

Firefox has tabs. I can use the left button to drag the tabs to reorder them. If I do that, what did I just assume?

  • Whenever I click on it with the left button it puts it into reorder mode.

How can I challenge that?

  • Left click and drag it after doing something else? what? a right click?
  • Hmm, that didn't drag it, seems like when I right click and let the popup menu come up, and then left click that I didn't go into left click mode, and even though I still held my finger down on the left mouse button I appear to have moved into "no button down mode". I wonder if that counts as normal...I'll check with the IE tabs.
  • OK, so the IE tabs do a different thing. If I right click on a tab in IE and get the popup menu then left click on the tab then I do go into left click mode and can drag the tab. That may not count as something worth mentioning but the 2 browsers certainly operate differently.

OK, so now I have further interest and questions in tab clicks in Firefox.

When I right click on a tab, a menu comes up. What assumptions have I made about that?

  • That I click down and release the button on the same tab. So what if you don't?
  • Erm, I get a menu up over another tab
  • But which tab does that menu relate to? Does it relate to the tab that you still have selected or does it relate to the tab you hovered over?
  • Erm, I don't know. I'll close it and find out. Oh, it related to the tab I hovered over, I didn't expect that.
  • So how does IE work?
  • Erm, if I right click on one tab and move the mouse to another and then release the right mouse button the menu doesn't appear at all. I only seem to get the right click menu if I stay in the bounds of the tab I clicked on.

OK, so nothing earth shattering there but I did learn a few things that I hadn't noticed before. Some of which I might class as a usability issue.

And I can now see that the popup menu that Firefox uses doesn't stop the app from responding; I still get tooltip popups when I move the mouse around and the little greyed out tab crosses turn red when I hover over them. Suggesting I could click hem even though the popup menu still displays, but when I click on the red buttons nothing happens.

Certainly though it seems like a little mismatch there that might lead to further revelations if I explore further.

No such indications from IE, when the popup menu displays, IE seems unresponsive to my mouse hovering.

And I put this exploration method down to an approach based on assumption challenging.

If you do this already and you ascribe what you do to a different thought process or 'approach' then I'd love to know how you explain what you do, so I can try on that head set as well.

And now, having read all that back, I suspect what I just explained here can come under the classification of 'heuristic' as well.

And all this time I thought of it as an approach too!

Since it can't count as a an algorithmic approach it probably should count as a heuristic approach.

Oh well, I guess I suckered you in here on a false premise and really explained another heuristic - The "What Assumptions I Took" heuristic.

But I offer you this with one benefit, at least it doesn't have a mnemonic associated with it to learn...

...Oh 'WAIT', I guess it could.



  1. Algorithms are heuristics too.

    A heuristic is "a fallible method for solving a problem or making a decision". Algorithms are methods of solving problems or making decisions. Do we know of algorithms that will never fail?

    ---Michael B.

    Thanks for the comment Michael. I wrote "Since it can’t count as a an algorithmic approach" in the post because I read the definition of 'heuristic' when I wrote the post. uses the text "pertaining to a trial-and-error method of problem solving used when an algorithmic approach is impractical". I would certainly not treat what I wrote as algorithmic, therefore I count it as heuristic. I don't know enough mathematics to know if there exist algorithms that people consider infallible.

  2. I was all set to write a comment about how you ARE describing a heuristic process, and then I got to the end. Oh you EVIL tester! Tricking us! :-)

    Ah, that sweet sucker punch of presupposition.

  3. A useful framework for this thought process is the scientific method - I've discussed the scientific method in the context of exploratory testing before.

    On the subject of heuristics... Readers should note that there are other sources of heuristics... they aren't always labelled heuristics. For example, the 'attacks' in the "How to Break Software" book series can be applied as heuristics.

    I have slowly come to believe that 'heuristic' acts as one of those 'global' words that encompasses every other word and we can't get away from it :)

    When I read "How to break ..." books, I found applying the 'challenge assumptions and presuppositions' approach a really useful exercise. By challenging the 'assumptions' or presuppositions listed in the attacks you can start to generalise the attacks listed and apply them in other contexts.

  4. A response and a name for it! (CKA)

    Thanks Erik, I like the notion of multiple labels so I fully support whatever helps people remember the approaches.

  5. [...] mainly used this in team based exercises, and primarily to explore presupposition analysis as a method for [...]

  6. NationalRental Affordability Scheme...

    [...]Challenge your assumptions and presuppositions to identify useful variation[...]...