Friday, 16 January 2009

Notes on the evolution of my exploratory testing documentation style

Looking back at ETA I can see that my current approach to documenting exploratory testing has changed since I last worked on ETA.
Back then I remember my style as:
  • what did I just do?
  • what did I just observe?
  • scribbled thoughts on paper
I kind of scribbled future thoughts on bits of paper off to the side as they occurred to me. Since various people have derogatively described my handwriting as "a scrawl", "unreadable" and "teeny tiny random lines" on a page. And since my handwriting seems to self destruct and become unreadable, even to me, after a day or two - keeping too much in handwriting on paper forced an evolution.

ETA supported that 'post documentation' process. So it lets you add notes about things you did and saw. But not so much about what you think you will do next (except in the form of notes).
My current ET style requires me to document my thoughts about the future too, in some circles we might even call that planning.
So I now write down:
  • what I think I might do
  • my aims and intents for what I plan to do next
  • thoughts of the future
  • possible bugs to look out for
  • the intent behind my current investigations
  • etc.
And I still write down:
  • what did I just do?
  • what did I just observe?
I can see parallels between this approach and my current time management approach.
My current time management approach involves:
  • write down task before doing
  • write down what done on task
  • retrospectives on what done (at different level: immediate, daily, weekly)
  • reviews of what else needs doing (at different level: immediate, daily, weekly)
(excuse my primitive Tarzan speak, I over indulged in ERB this week)
I also engage in retrospective when I construct my test report from my notes to summarise my findings and plan for future sessions.
So both my time management approach and ET approach mirror each other. The more I learn about one, the more I learn about the other. And I continue to change and refine both.
This may not surprise you given that you can view the Session Based Management approach as a Time/Task Management approach.
As such ETA in its current form does not really support everything I do.
I use a few consistent and nameable tools at the moment for the majority of my ET documentation:
How does everyone else document their ETA and what do you use?


  1. In my most recent forays into ET, I used Excel and it worked quite well. I previously used text editors (NoteTab Light is my current fave), but I've learned that I tend to write in a hierarchal, tree-like fashion, and spreadsheets are very flexible for that kind of thing. Excel can also can do groupings that expand/collapse, I can add as many rows for notes/data as I need and filter/sort, multiple tabs for new test areas, cells that can be cross-referenced, etc. It felt more natural to me. I'm not doing ET right now, but I plan to use Excel the next time I pick it back up.

    Hi Mitch, I haven't tried Excel. I might just give it a shot. Thanks.

  2. Interesting post. It resembles my experience - I am getting into writing down sort of everything I think of, including 'Hmmmm's and 'Wow's.. and occassionally 'Woa's. :-)

    When I first get going it's "observations" - "reflections" - "ideas" - "decision of next step". This gives me room for discussion what I've found.

    I do use a note book. My handwriting simply improved by being used more often and by paying a little attention on it. What I've discovered about using a book is, that it takes my focus away from the machine which helps free my thoughts to think creatively. This is a strange effect I cannot explain, but I think it's sort of healthy not looking into the screen 8 hours a day.

    Thanks for adding your insights Carsten.