Thursday, 31 March 2016

Everyday Browsing to improve your web testing skills - Why?

Who doesn’t like looking at the innards of a web page and fiddling with it?
  • Inspect Element
  • Find the src attribute of an image on a page
  • Edit it to the url of another, different image
You could, as my son enjoys doing; visit your school website, and replace images of people with blog fish and much hilarity doth ensue.
In my Sigist slides you’ll find some ‘tips’ for improving your web technical skills which cover this type of skill.
Asking and investigating:
  • How is the site doing that?
  • What are the risks of doing that?
  • Could you test that?
  • Do you understand it?
Some people ask: Why would I need to learn this stuff? Why would I use this?
I find that interesting. They have a different core set of beliefs underpinning their approach to testing than I do. They test differently, but it means I have to explain ‘why?’ for something that I do ‘because’.
I have studied the testing domain. I’ve read a lot of ‘testing’ books and have a fairly sound grasp of the testings techniques, principles and the many, varied reasons, why we might test.
None of those books, described the technology of the system.
Very few of those books used the technology of the system as a way of identifying risk.
Risk tends to be presented as something associated with the business. Business Risk. e.g. "Risk of loss of money if the user can’t do X".
I spend a lot of time on projects understanding the technology, to identify risk in how we are using the technology and putting it together.
  • If we are using multiple databases and they are replicating information across to stay in synch, then is there a risk that a user might visit the site and see one set of data, then the next visit see a different set (because now they are connected to a database that hasn’t had the information replicated across to it?).
  • Is there a risk that something goes wrong when we visit the site and it is pulling out information from a database that is currently being synched to?
If I ask questions like that and people don’t know the answers then I think we don’t understand the technology well enough and there might be a risk of that happening. I would need to learn the technology more to find out.
If people do know, and we have ‘strategies’ for coping with it - our load balancer directs the same user to the same database. Are there any risks with our implementation of that strategy? Will our test environment have the same implementation? Could we even encounter a manifestation of this risk when we are testing?
As well as knowing the requirements. I want to understand the pieces, and how they are put together.
Because I know from putting together plastic models as a child, or flatpack furniture as an adult, that there are risks associated with putting things together.
I have to learn about technology to do this. I then have to interpret that technology with my ‘testing mind’ and in terms of the system of the project I’m working on.
I suspect that is a better ‘why?’ answer; for learning the technology, and the associated technical skills, than my more flippant:
  • Q: Why would I use this?
  • A: Well, if you don’t have the skill, you never will. If you learn it right, then you might.
You'll find some simple tasks to help expand this in my Sigist slides 

Monday, 21 March 2016

Behind the Scenes - "Dear Evil Tester" lessons in risk based release management

On Wednesday the 16th a paperback copy of “Dear Evil Tester” arrived through the post. This was my “final” proof copy. Was it ready for release? In this episode we learn that the “Go Live” release decision is always a business risk decision.

Errors found in the staging environment

Upon reading it, I found two tiny errors.
Did I delay publishing?
Heck no.
I’m Agile. I’m lean (with a thin layer of fat for winter).
Did I fix the errors?

Errors found late cost more to fix than early

Heck yes.
What were they?
Two typos.
  • I had written a " instead of a :
  • I missed out one character from the end of a word
I could probably have left the second error. But the first one annoyed me. Especially since I was sure I had fixed it prior to ordering the proof.
I could have hit publish, there and then, but I didn’t I fixed the error.

Integration Testing

But I don’t have any integration tests. How could I check I had fixed it?
I needed to improve my test approach. Previously I had been scanning the changed pages, and flipping through the pages to check for no unexpected changes. Then order a proof. Then wait 3 days. Then repeat.
This time I found a tool to help. diff-pdf
Perhaps, I could generate a ‘print ready pdf’ and ‘diff’ it with the previous one.
It worked. The diff showed me two pages with changes, and both exactly where I expected them.
I was pretty sure this was good enough to de-risk the release process.

Release Process

But I still had to go through a release process.
I had to resubmit the print ready proof to the CreateSpace process.
Then wait for the ‘we check your document’ process to complete.
But that would take too long. I had already started the “Coming Soon” hype machine. I couldn’t dial it back into “Coming a little bit less soon than I wanted, but still Coming Soon” mode. Or I could, but I didn’t want to.
So on Thursday morning. After the CreateSpace approval process was finished I was able to review the digital proof of the book.
I wasn’t so sure this was good enough to de-risk the release process. Because I didn’t gain any information. I didn’t notice any problems. I didn’t think there were any problems. So this just confirmed my world view.
In theory, this should mean that I found no additional risks and issues, and so should be good to go.
As a tester, I’m not used to finding no problems. I must have missed something.
I could order a print copy, wait a few days, then compare the text, and then release. If I did that I probably wouldn’t release the book for another 5 days.
What to do? What to do?

What to do?

On Thursday.
I ordered a print proof.
And I went live.
I thought - if there was a problem then I would receive the proof copy before anyone received there copy.

Friday, Saturday, Sunday

On Friday, people started tweeting that they had received their print copies from Amazon.
HUH! WHAT! My proof still hadn’t arrived!
On Saturday, people tweeted that they had received their print copies from Amazon.
WAIT! But my proof copy still hadn’t arrived!
On Sunday… etc.


And lo! The proof copy arrived.
I read through it and it looks fine.

Lessons learned

  • Release Go Live decisions are always a risk based decision.
  • Sometimes business priorities will take precedence over ‘testing’ risks and concerns.
  • Tools can help.
  • You have to know what you need a tool to do, before you find one.
  • If you self-publish, then try to order your final proof from Amazon rather than CreateSpace. It will arrive faster.

Friday, 18 March 2016

Behind the Scenes - The "Dear Evil Tester" Style Guide and its Impact on my Testing

During the editing stages of "Dear Evil Tester" I started to write and maintain a style guide.
This was to help me track the ‘basic’ errors I found in my writing, and as an attempt to ensure a consistent attitude and formatting in the text.
Most books, and "Dear Evil Tester" is no exception, have a ‘conventions used in this book section’. This is a mini style guide which contains a set of small templates that I can use during the writing process. This section doesn’t go so far as to list the weaknesses of the writer, which is one reason for making it public. Whereas the weaknesses can be inferred from the reading of a style guide, so we keep it secret.
I’ve embedded the style guide below.
But first… a quick reflection since I will carry this approach forward into my testing.
I generally make a lot of notes when I’m testing. I do not always summarise them into a concise guide. I will now.
I’ll write down the:
  • handy tips,
  • obvious errors I commit,
  • reminders of high level aims.
Previously I’ve relied on my archive of notes and ‘searching’ through them, which has worked well. But I think that the additional exercise of reviewing, and summarising, will help embed the knowledge in my head which might even cut down on the searching.
It should also make it possible to share with other people.
I know that we have tried to do this on projects through wiki’s but, they often fall into disuse. Partly because they are written to be ‘read’, rather than ‘remind’, so are often wordier than they need to be. Also they tend to be written for ‘others’ rather than for ‘ourselves’.
Having the guide for my testing, might also help ‘others’ spot weaknesses in my approach. For example, if I haven’t listed a shortcut/feature in the tools then I might be missing out on something that might help my testing. The guide might reveal that to someone more expert than myself.
I’ve had in mind to build these for each of my websites. When I do, this will also have the side-effect of creating a single page that I can quickly review for any obvious CSS or common JavaScript issues.
And now, the style guide…

Evil Tester Writing Style Guide

This is the ‘writing style guide’ to support editors for Dear Evil Tester.
But Alan, that’s you! Yeah, and I forget stuff. OK!

Sublime Tips:

  • ctrl + shift + F - Find across all files
  • F6 - Spellcheck

Publishing Tips

  • Remember to check in to Version Control before starting a preview
  • Remember to deploy to Dropbox before starting a preview
  • Remember to close down PDF viewer, before starting a preview

Editing reminders

If you find an error. “Find across all files” and see if you find it again - if you do, fix it!
When reading from print, to find the file, use “Find across all files”

Writing Style

Worry less about the sentence structure. Worry more about the flow. Think. What would Norvell Page write? What would Lester Dent write? Do not model H. P. Lovecraft.
Is it a sentence? Who cares. Just make sure it looks like one. And reads like one.
But we’re not supposed to start with a conjunction!
And who said that? Was it someone who started a sentence with “however”? What would the author of the Bible write? (Hint: And in paragraph two, they started with “And”! If its good enough for God…)
  • If you have (brackets) and have punctuation outside the brackets (like this), then it is only outside if it started outside. (Having a full stop outside this would be bad.)
  • Its vs it’s (remember: find across all files) - It’s wrong to write its, unless its thing is being described.
  • Don’t worry too much about capitalisation, but at least make it consistent in each letter.
  • PS. PPS. (Don’t make it a dotty abbreviation.)
  • Capitals for Things,
    • at the very least don’t mix it. “exploratory Testing” would be bad.
  • Chapter and Section Titles use Capitals
    • Letter chapters are written like sentences. Unless they have a Thing in it.
  • For example, we will not write For Example. And we will still write e.g.
But what about the readers that know grammar?
Ha. They’re going to be annoyed however you write it. Write it for the masses.

Recurring Jokes

If you ever get the opportunity to make a joke about repetition then you should take it. Copy and paste a previous section of text again. Preferably the previous sentence or paragraph. Some people will get it. Some people won’t notice. Some will call the grammar police. Either way, much hilarity shall ensue. So if you ever get the opportunity to make a joke about repetition then you should take it.

Thursday, 17 March 2016

Behind the scenes of Dear Evil Tester : Hitting Publish

Today became "Dear Evil Tester" launch day.
I suspect ‘professionals’ create a plan and stick to it. I’m more of a kanban, ship it when it is ready, kind of guy.
On the 16th March, I received a final proof copy, and I reviewed that. Then on the 17th I hit publish.
I found the free pdf comparison tool "diff-pdf" very useful for final leanpub print ready pdf comparison during my final proof phase.
The launch process proceeded as follows:
  • start at leanpub
  • check the book details
  • realise you haven’t added a price
  • add price (make it slightly cheaper than amazon kindle because of the difference in royalty rates)
  • click publish
  • check the page looks OK in incognito mode
  • tweet
  • tweet that a sample is available now so people can try before they buy
  • move on to createspace
  • click publish
  • stare at the message that says “this might take 3-4 days” and wonder how people manage to create a co-ordinated launch across all platforms
  • move on to kdp (kindle direct publishing)
  • click publish
  • again stare aghast at the “this might take 3-4 days” message
  • be amazed as you start responding to peeps tweets about buying the book. Thank you all.
  • see that have listed the book for sale
  • tweet that
  • start editing all your web pages
  • notice that the feedjs server you were relying on for your rss feeds has stopped responding, so create a new endpoint for your existing rss caching to work on different sites
  • notice that the kindle version has been published
  • tweet that
  • respond to more peeps tweets. Thank you all again.
  • Write a promotional blog post
I still have more to do in the launch.
  • make sure Amazon notice that the paperback and kindle book are actually the same book and are listed together
  • create a leanpub promo video
  • amend the sales pages
But overall, the ‘publish’ part of ‘self publishing’ has become quite smooth.
Thank you for your support.

Wednesday, 16 March 2016

Sigist Keynote March 2016 Notes

I have updated my resources page for the Sigist March 2016 Keynote to include the slide deck and the social media commentary that Lisa Crispin kindly provided during the talk. (The above photo was taken by Lisa during the talk)

The talk was possibly partially recorded and might partially make its way to YouTube. The Sigist team are experimenting with recording technology and experiencing the normal teething troubles associated with live video recording. I'll let you know if the recording makes its way to live.

During the talk the Sigist kindly provided two copies of "Java For Testers" that we gave away as prizes, and I brought along a unique 'proof' copy of "Dear Evil Tester", which I also gave away as a prize.

In the talk I was basically providing some case studies of using technical knowledge and skills to inform your testing. And gave some examples of the overlap between this style and security testing.

I also mentioned the Usborne computer books from the 80's, many of which Usborne have released as free pdfs. There are a few on the website that I do not own, so I will read those later.

These books were a major influence on my career. I learned to write adventure games using "Write your own Adventure Programs"

A book that I previously mentioned over on Selenium Simplified where I describe the relationship between Keyword Driven Test Frameworks and Text Adventure Verb Noun parsers.

I still use the lessons I learned in this book, to this day. 

I later augmented this information with another 'Dragon' book:

Monday, 14 March 2016

How to draw Evil Tester

"Dear Evil Tester" includes some Evil Tester images and comics.

In this post I will show the basic process I use to go from a scribbled, draft, pencil drawing, to an inked and digital comic.

You'll see (in super blurry vision because of the web cam autofocus) the draft A3 sheets.

Then we move to an A4 pencil sketch. And then to an A4 ink sketch.

And we'll finish with a quick demo of the tools we need to scan it in and polish it up on the computer.

To scan the drawings I use a Canon Lide 220. Small, easy to store, USB powered (minimises cable and setup hassle), portable to different machines in different rooms. Fast enough. (For document scanning with multiple pages I use a ScanSnap ix500)

I scan into Paint .Net and I use Riot to shrink the file sizes.

Friday, 11 March 2016

Behind the scenes: Dear Evil Tester Book editing continues with testing hints on Observation

A proof copy of "Dear Evil Tester" just popped through the door from CreateSpace.

So now, with a different observation tool (the book instead of the printout). I will start to spot different errors.

And we enter the next stage of editing. Hopefully after these edits and fixes, only one more proof copy will be required before we go to print.

Observation is one of the key parts of my simple testing model:

  • Modelling, Observation, Reflection, Intent, Manipulation

This model changes for technical testing to:

  • Modelling, Observation, Reflection (contains intent), Interrogation, Manipulation

Different tools change the way that we observe. Different printouts are no different. So now we enter the 'physical' proof editing stage.

Coming even sooner than before: Dear Evil Tester

Behind the scenes: publishing and editing Dear Evil Tester book on leanpub

A quick behind the scenes look at the pre-proof editing process used when I'm writing a book via

In this video you will see all the tips and tricks of the trade.

The super secret special pen that I use for editing (clue: its a "Fisher Military Space Pen Matt Black").

The hi-tech editing software in use (clue: its a text editor, in this case Sublime Text).

The hi-tech version control software in use (clue: its subversion because I'm like that at home).

The sophisticated deploy system (clue: a windows batch file).

The hi-tech local to remote environment synchronisation software (clue: Dropbox)

And the highly automated preview process (clue: I click on a link in

And other splendid Dear Evil Tester goodness.

Stay tuned for more "Behind The Scenes" videos as we approach launch of "Dear Evil Tester"

Thursday, 10 March 2016

Announcing "Dear Evil Tester" coming soon, and why I wrote it

"Dear Evil Tester" is coming soon.

It is funny.

And that's fairly revolutionary in the testing market although Rob Sabourin's "I am a bug" and Andy Glover's "Cartoon Tester" provided humour in book form before I did. And this book contains the kind of humour that I relate to most.

Huib Schoots said "Alan writes with a dark humour that I like a lot!" Thanks Huib, I like that a lot too.
Richard Bradshaw called it "a warm dark blanket of humour and wit". Thanks Richard.

I wrote "Dear Evil Tester" because I see a gap in the coverage of current testing books. I don't see many testing books which cover the 'Attitude' that we need and want testers to build up.

This is a Question and Answer book.

And the answers are written in a way that I needed/wanted someone to talk to me when I was growing up as a tester. Someone who's answers would force me to think about whether I thought what we were going was a good idea and caused me to take responsibility for my test approach. Not because it was what the process said, or because it was what the 'experts' say, instead because it was what we say.

I learned to do this.

Hopefully with enough experience we learn to do this. So testers who have more experience will be able to relate to the book on this level. And testers with less experience will hopefully have that experience as they read the book.

We can use humour to challenge how we think, just as we use humour as a defense mechanism. In some ways I've drawn upon my readings of Provocative Therapy with a "Devil's Advocate" approach to provide answers which lead to a change of attitude.

Rob Sabourin said "I laughed - then I cried - then I laughed harder - then I cried softer." Thanks Rob, it moved me too :)

I'm working through the final edits now, if you want to be informed when the book is released then you can register your interest on the page.

I hope you'll like it. And stay tuned because I'll be showing you some behind the scenes footage as I complete the editing process.

Tuesday, 8 March 2016

Video Version of Lessons Learned Testing the House of Test contest app

I know that some people will have read, and been able to implement the notes I published after testing the House of Test Data Generation contest application.

I also know that a video can help people overcome issues trying to implement the notes because you'll be able to see my testing in action and the steps I take to create the project and add the jar to the project etc.

So I have turned the case study notes into a set of free videos.

I have uploaded the first video to YouTube, if you want to watch it prior to signing up to watch the videos for free on the training site.