Wednesday, 24 May 2017

Quaere, Heuristics, Mnemonics, and Acronyms

Don’t limit yourself to a set of attributes and words, seek more, develop strategies for identifying new concepts and ways of exploring them for then you have manifested the spirit of Quaere.



How might I describe the process of model building?

I was writing some notes on ‘Testing’ and trying to think through how I might describe the process of model building.

And I wrote down a few words:

  • Questioning,
  • Exploration,
  • Experimentation,
  • Analysis.
Useful words methinks.

Marketers ruin everything

And then my marketing brain kicked in.

“What if you made an acronym?”

Q.U.A.E.R.E - An Acronym I could Market

OK, well, “Q” requires a “U”

  • Usage?
And what else?

  • R? for Reasoning
And feed all of that into an online anagram solver:

“Quaere”

“Quaere” a word “to seek”

The word “Quaere” coincidentally maps on very well to the process.

Since the word originates from Latin, to “seek, look for; ask”.

“Quaere” definitions:

Mnemonics and Heuristics

Clearly at this point I should own this and present it as a branded Mnemonic.

“Quaere”, which would lead you to the individual words: Questioning, Usage, Analysis, Exploration, Reasoning, Experimentation.

And how would you use those words?

I see many presentations of lists of words as Heuristics, but I don’t think of individual words as heuristics.

I think of individual words as words.

But I could have a parameterized statement that works as a Heuristic that says “Meditate on an individual word to think of ideas to improve your testing”.

And the parameter is defined as “an individual word”

  • “Meditate on [an individual word] to think of ideas to improve your testing”.
Becomes:

  • “Meditate on Questioning to think of ideas to improve your testing”.
  • “Meditate on Usage to think of ideas to improve your testing”.
  • “Meditate on Analysis to think of ideas to improve your testing”.
  • “Meditate on Exploration to think of ideas to improve your testing”.
  • “Meditate on Reasoning to think of ideas to improve your testing”.
  • “Meditate on Experimentation to think of ideas to improve your testing”.
I might think - what other parameterized heuristicesque sentences could I use?

  • “Have I performed enough [an individual word]?”
  • “Has my [an individual word] been good enough?”
  • “Did my [an individual word] cover everything it could?”

A Springboard for Ideas

As a springboard for ideas, word generation and cogitation can work well, I’m not knocking it, I just don’t think of a word as a Heuristic.

And I get nervous of lists in general because I have a tendency to view them as complete and never see the invisible “etc.” at the bottom which reminds me to expand them.

Don’t limit yourself to this set of attributes, seek more, for then you have manifested the spirit of Quaere.

Related Reading

Newer readers might like to read my earlier mentions of Stichomancy:

Friday, 19 May 2017

How to use JavaScript Bookmarklets to Amend Web Page Example [Tutorial Text and Video]

TLDR; When you learn to manipulate the DOM with JavaScript you can create simple tools and automate from within the browser and use bookmarklets to make the code easy to execute and sync across different machines.





Wednesday, 10 May 2017

Resolutions and Trends in Software Testing Xebia Meetup

TLDR; Map resolutions every day. Evaluate if you are living the purpose you set. Go deep with your current knowledge and it will allow you to adopt the trends when they become strong influences on your work.

I gave a short 20 minute talk (including Q&A) at Xebia in Hilversum in January 2017, the evening before TestBash Netherlands.

The aim was to discuss New Year’s Resolutions and Trends for Software Testing. I filmed the talk on my mobile phone (hence the strange angle).
  • How to keep resolutions
  • Figure out your ‘slogan’
  • Own your definitions
  • Build on what you know
  • Weak Signals and Strong Signals
  • Responsibility
  • Implement

Monday, 8 May 2017

TOTE Model For Testers - Test, Operate, Test, Exit

TLDR; Map the TOTE (Test, Operate, Test, Exit) model on to TDD, Exploratory Testing, Design processes, Analysis, Learning, Decision Making and Problem Solving.

TOTE

In 1960, George Miller, presented a model of problem solving which he called the T.O.T.E model
  • Test, Operate, Test, Exit
The notion being that you loop around a [Test, Operate]* cycle and when your Test is complete, then you have done enough Operations and you can Exit.

It was a model of problem solving, or decision making.

I wrote about it a while back on the blog and in my NLP papers

You can find George Miller’s book Plans and the Structure of Behavior on archive.org. He describes the TOTE model in that book.

TOTE in Action

Showing T.O.T.E in action. I drew this dynamically to make the point that it is a cyclical process and we Test to decide if we continue to Operate, to Exit the process, and to change what we will operate.



You can see the 9 second version on Instagram

Test to build a model.

And sometimes we exit because things are Good Enough, but we still need criteria to determine what Good Enough means. And sometimes we Operate to learn if something is Good Enough.

TOTE for learning

At the time that I explored the TOTE model previously I didn’t make the connection that arc from Operate -> Test was also a feedback process.

In the TOTE model the Test learns from Operate, which we can easily map on to Exploratory Testing.
  • we come up with an idea to explore (Test)
  • we explore (Operate)
  • we learn from that (Operate -> Test)
  • we derive new things to explore (Test)
  • etc.
And we Exit when our time has finished or we have covered our ideas or whatever other ‘exit’ criteria we started our testing with.

TDD TOTE

I’ve also written a lot more code using TDD. And I know that my TDD process very often resembles a TOTE process.
  • write some @Test code (Test)
  • see if fail and write some code to make the test pass (Operate)
  • write more @Test code to flesh out the design (Test)
  • and repeat
Until our design is complete, or our review of our @Test and code can’t come up with anything new, and we have compared it with our statement of intent, etc.

TOTE Learning

We could view this as a completely well defined process of evaluation where at every ‘Test’ point we know exactly what we are deciding upon and use the pre-defined evaluation criteria.

We could also view the Operate process as a learning process which feeds into the Test process and explains the cycle. Each time we ‘do’ something (operate), we learn something which we feed into the Test process.

I missed the learning process inherent in the Operate -> Test arc first time around.

I won’t do that again, and that makes TOTE an even better model for the type of work I do.

A model worth investigating.




See also
PS. If you want your own T.O.T.E model diagram then feed this into Graphviz or WebGraphviz

digraph G { 
  node [shape = "rectangle"];
  Test -> {Operate Exit}
  Operate -> Test

  subgraph { rank = same; Test; Exit}

  Exit [shape = "ellipse"];
}

Thursday, 27 April 2017

Notes on Structured Analysis and System Specification by Tom Demarco

TLDR; Time unfortunately has not been kind to this - it still has moments of well worth reading but it also has sections where you hope no-one follows the instructions lest they doom the project, but the chapter on estimation is well worth reading.


I haven’t read this book since University but I vaguely remembered it as one of the books that taught me system modelling, a skill that I still rely on to this day.


Wednesday, 26 April 2017

Do you know what your framework is doing? A quick use of WebPageTest.

TLDR; Frameworks implement an abstraction layer so we don’t have to bother about it. But, what if the implementation is doing stuff you don’t want? How do you know? Find tools that let you observe inside. WebPageTest.org does that for web pages.


Tuesday, 25 April 2017

Notes from Glenford Myers Advances in Computer Architecture

TLDR; Abstractions are not new, have never been easy, and have always been important when architecting our Systems.



Monday, 24 April 2017

A Quick Intro To BookMarkLets

TLDR; Bookmarklets are an easy way to have custom javascript to support your testing that sync across browsers.



It took me quite a while to start using Bookmarklets but now that I’ve started… ooh, just try and stop me.

Friday, 21 April 2017

Lessons from the making of "Are you Experienced"

TLDR; Learn your skills and techniques. Then learn your tools. Mastery of tooling can lead to new techniques and new ideas. Continue to learn your theory, skills and techniques. Continue to master your tools.



I’m not exactly sure what I was hoping to learn when I started reading “Not necessarily stoned, but beautiful” subtitled “The making of Are You Experienced” by Sean Egan - a book which chronicles the making of the debut album by the Jimi Hendrix Experience.

But I pulled out information about Hendrix guitar playing that I didn’t know.

Thursday, 20 April 2017

Lessons from "Platoon Leader" by James McDonough

TLDR; war is horrible. Lessons can be learned from it. Would a distinction between defensive measures and pursuit, help your testing? I often see many test strategies that are highly defensive, but low on pursuit.



I do turn to books written by people who have fought in the army for lessons on tactics and leadership. I read “Platoon Leader” by James McDonough because it was a first hand account of a rookie leader in the Vietnam war.


Tuesday, 18 April 2017

Normal is the rarest of all states

TLDR; if you blindly copy an expert you do not learn context, you replicate mannerisms and lose their subtlety. Consciously analyse their actions, learn their skills, and apply them individually in a coordinated fashion.

When you learn about ‘context’ what disciplines do you learn from?



Thursday, 13 April 2017

That moment where you should have automated but didn't

TLDR; I migrated blogs over to Hugo and I didn’t automate because I was only doing it once, I should have automated because I actually migrated 450+ times (at least once per post. Find results at testerhq.com




Thursday, 30 March 2017

How do you interview testers?

TLDR; I received an email asking if I have any good interview questions and the short answer is "No". slightly longer is "I don't know, I do not know the person you are interviewing, or why you are interviewing." and since that could come across as arrogant and unhelpful, I thought I'd explain in a blog post that I audition, rather than interview based on a set of questions.


Friday, 17 March 2017

Top Ten Books For Testers for Huib Schoots

Huib Schoots recently asked for a top 10 books list for testers. I had to think hard when I wrote my list.

Bulleted below you can read the list I created. I added amazon links to the book so you can find them easily.
Only one of my titles had other people listing it as a shared title - Boris Beizer's book. Sadly,
it didn't make the top ten, so if you do read the above books then you'll be joining an exclusive club of Testers in the know.

And if you read the other books you'll be initiating yourself into an even more exclusive group. The group of people who have read all those books.

I think I added some of these books into the related reading section of "Dear Evil Tester", which has additional notes on why the books are important to me.

(This post was originally written around May/June 2013, but for some reason never left draft until 17/3/2017)


Sunday, 12 March 2017

Representation and Meaning: relating Programming, Testing, Coding and Checking

TLDR; older computing books and papers have a lot of really useful information - read them. Programming has an ‘easy to automate’ level called ‘coding’, with a similar relationship to ‘testing’ that ‘checking’ has. Assert as well as Check. Developing includes Testing and Programing and other stuff.

Some quick notes from a reading of the book “Representation and Meaning”, published in 1972, compiles various academic papers from 1960 - 1965.

The first paper in the book “The Heuristic Compiler” by Herbert A. Simon contains the following quote at the start of section 2.2
‘One distinction between the restricted, relatively simple tasks we call “coding”, and the broader, more difficult tasks we call “programming,” is that the latter may encompass the selection or design of an appropriate problem representation, while the former do not.’
Which made me think of the various discussion about “Testing” vs “Checking” and “Automation”, specifically:
  • we can’t automate ‘testing’, we can automate ‘checking’
  • why don’t we talk about “programming automation”?

We can’t automate ‘testing’ we can automate ‘checking’

The quoted statement about coding and programming seems to have a similar relationship to statements about checking and testing.
  • Programming and Testing
    • broader and more difficult than coding and checking
    • selection and design of appropriate problem representations
  • Coding and Checking
    • a representation of the work done when ‘programming’ or ‘testing’

Why don’t we talk about “programming automation”?

We don’t talk about “programming automation” because we automate coding.
  • code completion
  • macro systems
  • annotations for code generation e.g. projectlombok
  • transcompilers
It almost seems abnormal to consider the act of coding without thinking of how we can automate it, we have been doing this to coding since we started coding.

My first major project was the generation of program code from JSP diagrams. In the course of that project I automatically generated ‘C’ and ‘Cobol’ code.

The paper by Herbert A Simon, referenced above describes an exploration of automating the ‘programming’ tasks by treating it as a problem solving exercise. The automating of ‘coding’ was already a given and taken for granted.

A Diagram

I drew a diagram in my notes from reading the book.



I added some extra information to my diagram:
  • “…” to represent the fact that ‘developing’ does not consist of only programming and testing
  • both programming and testing have levels of representation - the stuff we can easily automate and the stuff that humans do (which we find harder to automate)
  • we automate at the ‘easy’ level
And for public consumption I have tidied the diagram, and I added additional information:


  • “-” to represent the fact that every named ‘high level set tasks’ has a lower level (which may or may not have a name) of easy to automate tasks
  • I added ‘asserting’ into the ‘checking’ representation because…
    • we check that a particular condition is met e.g. if(x==2){return false;}
      • we can report on the check in reports and monitoring
    • we assert on it to ‘halt’ execution of an automated process

Summary

I try to develop a model of ‘automating’ as part of the Software Development. This means I avoid thinking of “test automation” or “automation” and I think that allows me to think broadly about tool support in the development process than limiting it to ‘testing’ or ‘testers’.

I think this makes it easier to communicate to people who identify with the role of ‘programmer’. Because we no longer talk about ‘automating testing’ we talk about extending the normal process of automating our development approach to include:
  • executing code flows in the application
  • checking results
  • asserting on those checks
And… much value exists in the documentation trail of computing history, seek it out, do not ignore it.

References:

Wednesday, 8 March 2017

Ambiguity Detection and Weaponisation for Software Testers

TLDR; You can learn to detect ambiguity and then weaponize it for your testing. Do you think I meant that? What else could I mean?




Detection

Can you identify ambiguity in written, verbal and visual communication? If so then you can apply that skill during your testing to give you ideas of where in an application to test.

In the places that you perceive ambiguity, when you test there, you might find an easy win due to different expectations between project staff e.g.:
  • stakeholders believe they asked for Y
  • development team thought they X
  • managers push for Z
  • etc.

Weaponization

You can also use your skill to improve your own communication and avoid ambiguity:
  • make any issue or defect reporting clearer
  • have concise statements of risk that become harder to ignore
  • avoid ‘triggering’ development staff when you talk about potential system malfeasance
  • etc.
And you can expand on the identification of ambiguity to the harnessing of ambiguity in your testing:
  • Do you perceive the validation rules as ambiguous? Can you create data that validates but should not?
  • Could you traverse the entity states ambiguously? Can you move the entity into states that it should not validly enter ‘yet’?
  • etc.
And if you feel particularly brave then you can harness this in your test process:
  • Write your ‘test strategy’ and ‘test plan’ such that it gives you more flexibility than ‘they’ want you to have.
  • Communicate ‘risks’ and ‘issues’ in such a way that you generate increased fear and concern about the fellow to achieve your aim of having it worked on.
  • etc.

Development & Homework

I estimate that there exist well over 3, and possibly over 11 million, different ways to develop your ambiguity detection and utilization skills.

On the assumption that you read this at home, relaxing in your easy chair, and reflecting on your testing skill development, I suggest that you could:
  • find some politician speeches on the YouTube and listen for ambiguities
    • in speeches, politicians often provide high level generalizations which sound specific in intent, but lack clarity of end results and have no implementation details
  • find some politician interviews on the same or similar video platforms and listen for ambiguities
    • do they answer specific (Yes/No) questions with long answers? Do you think they tried to avoid the question? Did they they avoid ambiguities in the question? What can you read into their answers? What intent might stand behind their answer?
Look for multiple interpretations - when you find something that they ‘might’ have meant, look again at what else they might have meant, and then again, and then again. I suggest you stop when your answer reveals to you that they form part of an alien lizard race conspiring with the Illuminati to achieve world domination, because that model probably does not match reality.

If you want to antagonize your family then play the “did you mean?” game. For their every utterance where you detect ambiguity you reply “Did you mean …?” and supply an outlandish alternative meaning. Advanced players might consider not waiting for a response and instead chain “Or did you mean…?” with additional alternative interpretations in rapid fire. For maximum effect, you should continue to play the game even when faced with slammed and locked doors, in the modern world, you can continue to contact your family via up to date communication channels such as WhatsApp, FaceBook and SMS messaging.

I harbor no doubt that you can find your own ways to practice these skills.

Find your own ways to practice these skills.

Friday, 17 February 2017

Harness Your Ruthless Efficiency as MVP in testing and development

TLDR; Ruthlessly look at your process and incrementally improve your efficiency. Take the same attitude when testing and developing and harness MVP as often as you can.




In this post I’m going to describe focus and how you can apply that in your work, not just for testing but for software development in general with examples.

On the morning of 17th Feb 2017, I created an Instagram video on ‘focus’ and it was about… how ruthlessly efficient we can be if we focus.

The Monty Python Test Tactics

And it was partly inspired by Monty Python with their Spanish Inquisition sketch. They have three main weapons, four or five, but it’s surprise, fear, and ruthless efficiency.

Wednesday, 15 February 2017

Should I test at the GUI Level or the API Level?

TLDR; Where to test? Can you isolate the functionality? If so, test the isolation most heavily. Then look to see what integrates, and how it integrates. Then test the integration as heavily as the ‘how’ requires.




Question: Is there a rule of thumb when deciding to test at the GUI level or API level? Are any rules to help decide when to test at one level over the other?
Answer: I don’t think I use a simple rule of thumb. But I will try and explore some of the thought processes I use to make the decision.

When I am trying to decide whether to test at the GUI or the API I have to figure out:
  • what am I trying to test?
  • can I isolate the functionality I’m testing to a specific ‘level’?

Tuesday, 31 January 2017

How to test a text adventure game - some notes on Testing RestMud

TLDR; RestMud has JUnit Unit @Test coverage, functional integration testing, REST API testing with Jsoup and Gson, Bots for multi-user and model based testing, Postman and GUI based exploratory testing.




I’m getting RestMud ready for some workshops later in the year and this means:
  • making sure a player can complete the maps I have created
    • I’m OK with some bugs (because they are for testers), but I need them to complete because they are games
  • making sure they can handle enough players
    • I imagine that we will have a max of 20 or so people playing at a time
  • making sure I don’t break existing games
    • with last minute engine changes
    • with new game maps
    • with new game commands and script commands
As most of you reading this will realise - that means as well as developing it, I need to test it.

RestMud Text Adventure Game for Testers Walkthrough

I have created a walkthrough for the RestMud game I released last year.

Why? Because I have had a few people email me asking for tips.

Warning:
  • This is a full walkthrough
  • Start to finish, every command
  • It will not improve your technical skills
  • If you don’t play the game with dev tools on then it might not be obvious why you use some of these commands
  • there is a full map at the end of the walkthrough, I put it at the back because you should only use it in an emergency
I recommend:
  • use this sparingly
  • go as far as you can, and only if you get stuck to you use it
  • search for your location id if you are stuck to jump to the correct hint
  • make your own map
You can find the walkthrough and a hand drawn map on the RestMud game page.

PS. The walkthrough was generated by an automated script that played the game to check that it completes.

Thursday, 12 January 2017

What do you do? As a Tester, when you are asked for ROI calculation

TLDR; push back, ask questions, and if all else fails - plan it as a manageable set of tasks

At the Test Automation Guild I was asked a question about how we calculate ROI for Test Automation. And its a hard question for me to answer in 30 seconds when I don’t know enough about the situation the person asking the question finds themselves in.

Because testers are often asked by managers for information that the manager should really be dealing with and which does not seem to add value to the process and which we are concerned trivializes or views the process from the wrong perspective.

I know this is a difficult position for testers.

And I know testers are asked to do this.

I empathize. I’ve been there.

And I want you to be able to take this less seriously - consider that your only warning for what you are about to receive.

So I will attempt to provide tactics to help with the situation.

Monday, 2 January 2017

Hack the JavaScript Evil Tester Sloganizer to Generate Random New Year's Resolutions



I wrote a blog post for the new year:
And in there I noted that if you go off to the Evil Tester Sloganizer
and type this code

for(var x=0;x<100;x++){
console.log("-" + process_sentence(sentences[30]));}

Then you’ll get a bunch of New Year’s Resolutions printed out to the console:

Lessons Learned from Arnold Schwarzenegger Applied to Software Testing

Lessons Learned from Arnold Schwarzenegger Applied to Software Testing

TLDR; Start emulating people, use your job to learn, keep training,  identify other people's strategies, experiment to see what works for you, make your own tools, harness your uniqueness.



Everyone that is successful in their discipline and is prepared to tell their story, we can probably learn lessons from. Particularly if they are someone who’s really driven toward certain goals.

With Arnold Schwarzenegger, you’ve got the benefit that he has had multiple careers or multiple things that he has done throughout his life, and each one of them he has had to work for and practise hard to achieve.

I read Arnold Schwarzenegger Autobiography “Total Recall” and I made some notes, and I’m going to turn these into applied notes for learning how to improve our testing.