Thursday, 30 March 2017

How do you interview testers?

TLDR; I received an email asking if I have any good interview questions and the short answer is "No". slightly longer is "I don't know, I do not know the person you are interviewing, or why you are interviewing." and since that could come across as arrogant and unhelpful, I thought I'd explain in a blog post that I audition, rather than interview based on a set of questions.

Friday, 17 March 2017

Top Ten Books For Testers for Huib Schoots

Huib Schoots recently asked for a top 10 books list for testers. I had to think hard when I wrote my list.

Bulleted below you can read the list I created. I added amazon links to the book so you can find them easily.
Only one of my titles had other people listing it as a shared title - Boris Beizer's book. Sadly,
it didn't make the top ten, so if you do read the above books then you'll be joining an exclusive club of Testers in the know.

And if you read the other books you'll be initiating yourself into an even more exclusive group. The group of people who have read all those books.

I think I added some of these books into the related reading section of "Dear Evil Tester", which has additional notes on why the books are important to me.

(This post was originally written around May/June 2013, but for some reason never left draft until 17/3/2017)

Sunday, 12 March 2017

Representation and Meaning: relating Programming, Testing, Coding and Checking

TLDR; older computing books and papers have a lot of really useful information - read them. Programming has an ‘easy to automate’ level called ‘coding’, with a similar relationship to ‘testing’ that ‘checking’ has. Assert as well as Check. Developing includes Testing and Programing and other stuff.

Some quick notes from a reading of the book “Representation and Meaning”, published in 1972, compiles various academic papers from 1960 - 1965.

The first paper in the book “The Heuristic Compiler” by Herbert A. Simon contains the following quote at the start of section 2.2
‘One distinction between the restricted, relatively simple tasks we call “coding”, and the broader, more difficult tasks we call “programming,” is that the latter may encompass the selection or design of an appropriate problem representation, while the former do not.’
Which made me think of the various discussion about “Testing” vs “Checking” and “Automation”, specifically:
  • we can’t automate ‘testing’, we can automate ‘checking’
  • why don’t we talk about “programming automation”?

We can’t automate ‘testing’ we can automate ‘checking’

The quoted statement about coding and programming seems to have a similar relationship to statements about checking and testing.
  • Programming and Testing
    • broader and more difficult than coding and checking
    • selection and design of appropriate problem representations
  • Coding and Checking
    • a representation of the work done when ‘programming’ or ‘testing’

Why don’t we talk about “programming automation”?

We don’t talk about “programming automation” because we automate coding.
  • code completion
  • macro systems
  • annotations for code generation e.g. projectlombok
  • transcompilers
It almost seems abnormal to consider the act of coding without thinking of how we can automate it, we have been doing this to coding since we started coding.

My first major project was the generation of program code from JSP diagrams. In the course of that project I automatically generated ‘C’ and ‘Cobol’ code.

The paper by Herbert A Simon, referenced above describes an exploration of automating the ‘programming’ tasks by treating it as a problem solving exercise. The automating of ‘coding’ was already a given and taken for granted.

A Diagram

I drew a diagram in my notes from reading the book.

I added some extra information to my diagram:
  • “…” to represent the fact that ‘developing’ does not consist of only programming and testing
  • both programming and testing have levels of representation - the stuff we can easily automate and the stuff that humans do (which we find harder to automate)
  • we automate at the ‘easy’ level
And for public consumption I have tidied the diagram, and I added additional information:

  • “-” to represent the fact that every named ‘high level set tasks’ has a lower level (which may or may not have a name) of easy to automate tasks
  • I added ‘asserting’ into the ‘checking’ representation because…
    • we check that a particular condition is met e.g. if(x==2){return false;}
      • we can report on the check in reports and monitoring
    • we assert on it to ‘halt’ execution of an automated process


I try to develop a model of ‘automating’ as part of the Software Development. This means I avoid thinking of “test automation” or “automation” and I think that allows me to think broadly about tool support in the development process than limiting it to ‘testing’ or ‘testers’.

I think this makes it easier to communicate to people who identify with the role of ‘programmer’. Because we no longer talk about ‘automating testing’ we talk about extending the normal process of automating our development approach to include:
  • executing code flows in the application
  • checking results
  • asserting on those checks
And… much value exists in the documentation trail of computing history, seek it out, do not ignore it.


Wednesday, 8 March 2017

Ambiguity Detection and Weaponisation for Software Testers

TLDR; You can learn to detect ambiguity and then weaponize it for your testing. Do you think I meant that? What else could I mean?


Can you identify ambiguity in written, verbal and visual communication? If so then you can apply that skill during your testing to give you ideas of where in an application to test.

In the places that you perceive ambiguity, when you test there, you might find an easy win due to different expectations between project staff e.g.:
  • stakeholders believe they asked for Y
  • development team thought they X
  • managers push for Z
  • etc.


You can also use your skill to improve your own communication and avoid ambiguity:
  • make any issue or defect reporting clearer
  • have concise statements of risk that become harder to ignore
  • avoid ‘triggering’ development staff when you talk about potential system malfeasance
  • etc.
And you can expand on the identification of ambiguity to the harnessing of ambiguity in your testing:
  • Do you perceive the validation rules as ambiguous? Can you create data that validates but should not?
  • Could you traverse the entity states ambiguously? Can you move the entity into states that it should not validly enter ‘yet’?
  • etc.
And if you feel particularly brave then you can harness this in your test process:
  • Write your ‘test strategy’ and ‘test plan’ such that it gives you more flexibility than ‘they’ want you to have.
  • Communicate ‘risks’ and ‘issues’ in such a way that you generate increased fear and concern about the fellow to achieve your aim of having it worked on.
  • etc.

Development & Homework

I estimate that there exist well over 3, and possibly over 11 million, different ways to develop your ambiguity detection and utilization skills.

On the assumption that you read this at home, relaxing in your easy chair, and reflecting on your testing skill development, I suggest that you could:
  • find some politician speeches on the YouTube and listen for ambiguities
    • in speeches, politicians often provide high level generalizations which sound specific in intent, but lack clarity of end results and have no implementation details
  • find some politician interviews on the same or similar video platforms and listen for ambiguities
    • do they answer specific (Yes/No) questions with long answers? Do you think they tried to avoid the question? Did they they avoid ambiguities in the question? What can you read into their answers? What intent might stand behind their answer?
Look for multiple interpretations - when you find something that they ‘might’ have meant, look again at what else they might have meant, and then again, and then again. I suggest you stop when your answer reveals to you that they form part of an alien lizard race conspiring with the Illuminati to achieve world domination, because that model probably does not match reality.

If you want to antagonize your family then play the “did you mean?” game. For their every utterance where you detect ambiguity you reply “Did you mean …?” and supply an outlandish alternative meaning. Advanced players might consider not waiting for a response and instead chain “Or did you mean…?” with additional alternative interpretations in rapid fire. For maximum effect, you should continue to play the game even when faced with slammed and locked doors, in the modern world, you can continue to contact your family via up to date communication channels such as WhatsApp, FaceBook and SMS messaging.

I harbor no doubt that you can find your own ways to practice these skills.

Find your own ways to practice these skills.