Evil Tester
Friday, 14 June 2013
"Java For Testers" released
I've been working on a lot of stuff in the first 6 months of this year.
Most of this hasn't been converted into usable product form yet, and is still working through my drafting process.
But the first thing has made it to the 'public' stage - a new book called "Java For Testers".
I've released this as beta, the same way I did "Selenium Simplified" when I was writing that.
The price will gradually increase as the book nears 'done' status. So get in early if you want a bargain.
Friday, 31 May 2013
A Bugzilla exploratory testing session from the vaults
Back in 2011, I decided to try recording an exploratory testing session.
I uploaded the results privately to Youtube and essentially forgot about it.
It was the first time I tried to think out loud, and record, and conduct exploratory testing, and make notes, all at the same time.
Despite its rough edges, and horrible editing, I'm going to make it available.
I have long lost the original recording so I can't recreate a full unedited version, and I can't really adjust this video too much, so I'll let it stand as a time bound representation of where I was with the multi tasking approach to exploratory testing back in 2011.
If you want to learn more about Technical Exploratory Testing then I have a free Udemy course that covers more of the basics.
I uploaded the results privately to Youtube and essentially forgot about it.
It was the first time I tried to think out loud, and record, and conduct exploratory testing, and make notes, all at the same time.
Despite its rough edges, and horrible editing, I'm going to make it available.
I have long lost the original recording so I can't recreate a full unedited version, and I can't really adjust this video too much, so I'll let it stand as a time bound representation of where I was with the multi tasking approach to exploratory testing back in 2011.
If you want to learn more about Technical Exploratory Testing then I have a free Udemy course that covers more of the basics.
Thursday, 11 April 2013
"I was a zombie tester" a true story
I was a testing zombie. And I confirm that the details herein conform to the truth as I recall it.
And you can't tell from the film still, but all of us in this picture are actually covered in fake blood and dirt. The whole segment is on screen for about 10 seconds but took about 6 hours to film.
I'm not sure if there are any parallels to testing in this story, but I can at least say "I was a zombie tester, and I have the photographic evidence to support it".
If this film was a project, it would have run massively over budget It was wildly inefficient (6 hours for 10 seconds) and was filled with waste - which you'll hopefully identify as the story goes on.
Being a closet thespian I turned up in a suit, shirt and tie, after all, I've just risen from the grave, and when you are buried you wear a suit. Or at least, I plan to.
I was a little surprised then to see all my fellow extras in jeans and T-shirt.
The make up and costume people were also a tad miffed at me.
I suspect a lesson can be found here regarding the value of tacit domain knowledge. Possibly having more value than the explicit contextual script and staging knowledge that the project team had.
It was a suit that I had purchased at a charity shop for the princely sum of less than 5 pounds, for the express purpose of events such as these, and other theatrical play acting in the woods that we don't need to go into right now.
To recap, the people involved in the film hadn't intended the requirement to be interpreted in the manner that I had.
Consequently, I looked forward to seeing what costume they actually had in mind then.
Actually, they didn't have anything in mind.
Nothing.
Not a thing.
The professionals covered us in fake blood, dirtied us up a bit. But that didn't work.
And at one point they even put sheets over us (remember this is a true story relating to a professional film with a budget in excess of a million dollars (1,400,000 according to IMDB)).
The sheets came off when the writer came across and put his foot down, or started crying, I forget exactly which.
And it plainly described revenants approaching from Highgate cemetery. We were filming in Crouch End, which is some way from Highgate Cemetery, but due to the magic of film making and by summoning the spirit of Ed Wood, that would make no difference. They could edit in Highgate Cemetery, and indeed they did, for you can briefly see a gravestone, which then turns into a bush.
I did meet the writer upstairs. He looked a bit sad. I tried to cheer him up by mentioning that I was wearing a suit because I knew what a revenant was. But it didn't seem to cheer him up.
As it got chillier. And darker. I was quite glad of my suit. Because it has a jacket you see, not just bare arms, which is what you have if you're wearing a T-shirt.
And we zombie/revenants got shuffled off to the back of the garden.
They shouted action. And we just stood there.
Because we weren't real actors you see, and we were expecting to be told "walk forward slowly" or some other instructional phrase.
Instead we had to wing it. But slowly, because we were really dead.
So we all shuffled forward in a 'we don't really know if we are zombies or what" type of fashion.
I thought we must have done well because we were given a round of applause and promised we could turn up for the film premiere.
I assume the film went straight to video because no premiere invitation ever actually arrived.
Its the details that are important. Even if you can't see them on the screen.
Take for instance our triumphant close up at 1 hour, 24 minutes and 8 seconds into the film.
It just looks like a black blob to the uninitiated. But no, there are 5 or 6 zombies in that picture. Can you see them?
This is like a horror version of "Where's Wally?", if "Where's Wally" was inked by someone with a 6 inch emulsion paint brush instead of a 0.5mm pen.
So there you go. I was a zombie tester. The last picture acts as evidence. And testers value evidence.
Well a revenant actually
This is a still from the 2002 film "Mad Dogs", specifically 1 hour, 23 minutes and 44 seconds into the film. And that's me on the left. In full zombie mode.And you can't tell from the film still, but all of us in this picture are actually covered in fake blood and dirt. The whole segment is on screen for about 10 seconds but took about 6 hours to film.
I'm not sure if there are any parallels to testing in this story, but I can at least say "I was a zombie tester, and I have the photographic evidence to support it".
If this film was a project, it would have run massively over budget It was wildly inefficient (6 hours for 10 seconds) and was filled with waste - which you'll hopefully identify as the story goes on.
Wot, no Script!
As an extra - which I was. We didn't receive a script. We were told to turn up, that we would be playing the part of a zombie, and that we should wear clothes that we didn't mind getting messed up.Being a closet thespian I turned up in a suit, shirt and tie, after all, I've just risen from the grave, and when you are buried you wear a suit. Or at least, I plan to.
I was a little surprised then to see all my fellow extras in jeans and T-shirt.
- Didn't they read the requirements?
- Didn't they interpret the requirement?
- What kind of zombies were they?
- Had they really just risen from the grave in their scruffy T-shirts?
The make up and costume people were also a tad miffed at me.
"We told you to wear something you could mess up", they said.
"Yes", said I. "I am", said I.
"But you are wearing a suit", they said.
"And you can mess it up", said I
"Hmmmph", they said.
Regarding Domain Knowledge
Because they were the professionals, who clearly knew what they were doing. What did I - a reader of horror novels and comics from a young age, and watcher of horror films my entire life - know?I suspect a lesson can be found here regarding the value of tacit domain knowledge. Possibly having more value than the explicit contextual script and staging knowledge that the project team had.
It was a suit that I had purchased at a charity shop for the princely sum of less than 5 pounds, for the express purpose of events such as these, and other theatrical play acting in the woods that we don't need to go into right now.
To recap, the people involved in the film hadn't intended the requirement to be interpreted in the manner that I had.
Consequently, I looked forward to seeing what costume they actually had in mind then.
Actually, they didn't have anything in mind.
Nothing.
Not a thing.
Scripts require interpretation
It wasn't in the script. The script just called for some revenants to be scarily visible through some dry ice at the back of a garden (Highgate Cemetery).The professionals covered us in fake blood, dirtied us up a bit. But that didn't work.
And at one point they even put sheets over us (remember this is a true story relating to a professional film with a budget in excess of a million dollars (1,400,000 according to IMDB)).
The sheets came off when the writer came across and put his foot down, or started crying, I forget exactly which.
"Revenants, not ***** zombies or ****** ghosts" I heard him say.Because he had written a script you see.
And it plainly described revenants approaching from Highgate cemetery. We were filming in Crouch End, which is some way from Highgate Cemetery, but due to the magic of film making and by summoning the spirit of Ed Wood, that would make no difference. They could edit in Highgate Cemetery, and indeed they did, for you can briefly see a gravestone, which then turns into a bush.
I did meet the writer upstairs. He looked a bit sad. I tried to cheer him up by mentioning that I was wearing a suit because I knew what a revenant was. But it didn't seem to cheer him up.
As it got chillier. And darker. I was quite glad of my suit. Because it has a jacket you see, not just bare arms, which is what you have if you're wearing a T-shirt.
And we zombie/revenants got shuffled off to the back of the garden.
Action, doesn't mean much without motivation
We didn't actually see the script. We were just told to walk forward slowly when they told us to.They shouted action. And we just stood there.
Because we weren't real actors you see, and we were expecting to be told "walk forward slowly" or some other instructional phrase.
Instead we had to wing it. But slowly, because we were really dead.
So we all shuffled forward in a 'we don't really know if we are zombies or what" type of fashion.
I thought we must have done well because we were given a round of applause and promised we could turn up for the film premiere.
I assume the film went straight to video because no premiere invitation ever actually arrived.
Little things count
But, not to be too down on the professional make-up costume people... when I was walking home I certainly looked particularly gruesome because everyone I passed looked at me in a truly horrified fashion - covered in fake blood and dirt as I was, and dressed in a suit and tie.Its the details that are important. Even if you can't see them on the screen.
Take for instance our triumphant close up at 1 hour, 24 minutes and 8 seconds into the film.
It just looks like a black blob to the uninitiated. But no, there are 5 or 6 zombies in that picture. Can you see them?
This is like a horror version of "Where's Wally?", if "Where's Wally" was inked by someone with a 6 inch emulsion paint brush instead of a 0.5mm pen.
So there you go. I was a zombie tester. The last picture acts as evidence. And testers value evidence.
Thursday, 28 March 2013
Think of a Word - a 99 Second Talk
I've said in various talks that I don't enjoy creating, justifying, or applying, definitions.
I think creating your own definition does work well as an exercise, because you can explore your vocabulary and try and create an encompassing statement of intent to cover what you mean when you use a word. And there exist, people who do the 'definition' thing really well. James Bach and Michael Bolton act as exemplars of this approach and freely share, discuss and debate their definitions via blogs and twitter.
I do not appear to fit into that group, my definitions do not work well, and when I adopt a definition it feels stifling. I find that my definitions change, not because I have changed, or the situation changed, but because I created a definition that didn't encompass everything I needed it to cover.
Fortunately, for me, I found an exercise that words better for me. Using words as symbols, and identifying words that apply to the concept or term I want to explore. These words might act as attributes, or characteristics, or high level abstractions, or symbols.
When used as symbols we deliberately read into them. We deliberately don't try and tie them down. We deliberately explore them from different angles and take from them what we need at the time. The symbol doesn't have a definition. You find and explore the relationship between yourself and the word, at the time and place you find yourself now.
I phrase it slightly differently in the 99 second talk. Different medium. Different message.
I prepared this 99 second talk in advance of TestBash 2.0 but in the end the talk didn't feel right on the day. So I created another one instead. Since I prepared the talk in advance I have a recorded practice session, which I release now.
Next Steps:
Create your own mind maps at MindMeister
I think creating your own definition does work well as an exercise, because you can explore your vocabulary and try and create an encompassing statement of intent to cover what you mean when you use a word. And there exist, people who do the 'definition' thing really well. James Bach and Michael Bolton act as exemplars of this approach and freely share, discuss and debate their definitions via blogs and twitter.
I do not appear to fit into that group, my definitions do not work well, and when I adopt a definition it feels stifling. I find that my definitions change, not because I have changed, or the situation changed, but because I created a definition that didn't encompass everything I needed it to cover.
Fortunately, for me, I found an exercise that words better for me. Using words as symbols, and identifying words that apply to the concept or term I want to explore. These words might act as attributes, or characteristics, or high level abstractions, or symbols.
When used as symbols we deliberately read into them. We deliberately don't try and tie them down. We deliberately explore them from different angles and take from them what we need at the time. The symbol doesn't have a definition. You find and explore the relationship between yourself and the word, at the time and place you find yourself now.
I phrase it slightly differently in the 99 second talk. Different medium. Different message.
I prepared this 99 second talk in advance of TestBash 2.0 but in the end the talk didn't feel right on the day. So I created another one instead. Since I prepared the talk in advance I have a recorded practice session, which I release now.
Next Steps:
- Create your own definitions - see if that works for you, see how you feel about it
- Identify some symbols, explore them - see if that works for you, see how you feel about it
Create your own mind maps at MindMeister
Labels:
Conferences
Monday, 25 March 2013
99 Seconds at TestBash 2.0
I presented a 99 second talk at the TestBash 2.0
I went to the TestBash with a different talk prepared, but it didn't feel right for the TestBash so I created something else when I was there. As a result I forgot 1/4 of it, so I only hit about 70 seconds.
I don't think anyone noticed, but I'll link to the recorded video should it ever find its way online. (The actual video is contained within this vimeo).
So that I have a record of what I meant to say. I recorded the 99 Second talk at home.
The basic theme revolved around the same concepts as the talk I didn't do. About ownership of the words that we use to describe testing. Something that I've talked about and blogged about before. But I say it again because I think the testing world will transform into something more effective when we take responsibility for the words we use and the testing we do.
This talk above came out different from the talk at the Test Bash, which came out different from the one in my head. Because despite having some notes on what I meant to say. I reinterpreted those notes differently each time.
Create your own mind maps at MindMeister
Note:
I went to the TestBash with a different talk prepared, but it didn't feel right for the TestBash so I created something else when I was there. As a result I forgot 1/4 of it, so I only hit about 70 seconds.
I don't think anyone noticed, but I'll link to the recorded video should it ever find its way online. (The actual video is contained within this vimeo).
So that I have a record of what I meant to say. I recorded the 99 Second talk at home.
The basic theme revolved around the same concepts as the talk I didn't do. About ownership of the words that we use to describe testing. Something that I've talked about and blogged about before. But I say it again because I think the testing world will transform into something more effective when we take responsibility for the words we use and the testing we do.
This talk above came out different from the talk at the Test Bash, which came out different from the one in my head. Because despite having some notes on what I meant to say. I reinterpreted those notes differently each time.
Create your own mind maps at MindMeister
Note:
- I used e.ggtimer.com as the timer
- Richard Bradshaw posted his experience and talk description for his 99 Second Test Bash talk
Wednesday, 6 March 2013
What does a technical exploratory test session look like?
As part of "Technical Web Testing 101" I wanted to provide an example of what an exploratory test session with additional "Technical" focus might look like, at the same time demonstrating some of the capabilities of modern browsers whilst comparing them to proxy servers. Phew, a bit of a mouthful, and you can see the resulting video below.
I uploaded the video to Youtube, as well as it forming part of the course, because I think it has interesting elements that can stand alone. And I don't see many examples of exploratory testing on the web. I wanted to try and provide an example of 'doing' exploratory testing, and the type of notes I took.
As testers we can provide harsh criticism but I won't let that stop me sharing. If you don't think this provides a good example then I encourage you to share your own. I do welcome constructive comments and critique.
I'm getting better at thinking aloud as I test, so my verbal narration actually makes sense in this video.
A few things I want to point out.
Regarding the notes
I uploaded the video to Youtube, as well as it forming part of the course, because I think it has interesting elements that can stand alone. And I don't see many examples of exploratory testing on the web. I wanted to try and provide an example of 'doing' exploratory testing, and the type of notes I took.
As testers we can provide harsh criticism but I won't let that stop me sharing. If you don't think this provides a good example then I encourage you to share your own. I do welcome constructive comments and critique.
I'm getting better at thinking aloud as I test, so my verbal narration actually makes sense in this video.
A few things I want to point out.
- I try to explain the thought processes and decisions I'm making
- You can hear me verbally describe risks that I want to investigate
- You see me spin off track a little to investigate some 'interesting' ideas and then get back on track
- Modern browsers have a lot of impressive functionality built in that we used to have to use proxy servers to achieve the same effect
- I'm "Tool Augmented" not "Tool Driven" so the tool helps me do what I identify I want to do, not what the tool allows me to do
- I'm testing http://google-gruyere.appspot.com/
- I'm using the "Edit This Cookie" chrome plugin
- I did this as two sessions. The first was to get my bearings - and I made notes during it, which you can read below. The second was to record the video. The second session was slightly different as you can see if you compare the video with the notes, which shows that even when we repeat sessions, we learn additional things and do the testing differently.
Regarding the notes
- This was an informal session so I didn't timestamp anything - which I would do if I was testing on site professionally.
- The notes were mainly to guide me in replay so aren't formatted with any annotations e.g. @Bug or headings
And here are the notes in all their glory. I used Evernote as my note taking repository.
Testing with Gruyere with Google Chrome
Create a new account
"bob" "bob"
I can see the new account is created with a "GET"?!?
http://google-gruyere.appspot.com/804259209683/saveprofile?action=new&uid=bob&pw=bob&is_author=True
perhaps I can use different actions?
Perhaps I can amend? and change password?
Perhaps is_author has other alternatives
e.g. is_admin?
Having created an account - check storage
And I have a cookie - have I logged in automatically? I have I'd like to amend the cookie and check if the name can allow me to login as someone else, or the permission field can change - but I can't do that out of the box with Chrome
Technique - means I have to look for a tool to do that - fortunately I already have one installed, but if I didn't - this would prompt me to do so.
Try changing the cookie value to admin, refresh, and I no longer appear to be logged in
Perhaps that is a key? to the ID?
Repeat the get request and see what happens
http://google-gruyere.appspot.com/804259209683/saveprofile?action=new&uid=bob&pw=bob&is_author=True
User already exists - ok fine.
Try and use the url for different actions e.g. "amend" gives me an invalid action
Let's see what profile does
I'll inspect the form and I can see an update value and it is a get request again
So instead of "amend" try "update" in the url
incorrect password? But it is the same one?
Ah - perhaps it is looking for the validation password as seen in the profile update form
for update it probably needs oldpw as well
If I take out pw then what happens? request accepted
but presumeably didn't update anything
what about the is_admin risk?
hmm nothing seemed to happen, - what if I logout and login again?
Woohoo admin
Tuesday, 5 March 2013
Technical Web Testing
I have put "Technical Testing" at the top of my agenda this year and have just released a new free Udemy course - Technical Web Testing 101.You can suggest things you want me to cover by filling in this 2 question survey.
- I pretty much 'officially' started this last year with "The Evil Tester's Guide to Technical Testing" webinar for Eurostar.
- I followed this up in February at the Test Management Summit by hosting a session on "Managing Technical Testers".
- In May this year I will provide a tutorial at TestNet on "The Evil Tester's Guide to HTTP Proxies", and also in May I'll present at Let's Test with "The Evil Tester's Guide to Technical Web Testing".
Since I build up a fair bit of material from all of this, I've started putting it into video form, and have setup a new free Udemy course Technical Web Testing 101. to provide introductory and overview material. I'll build longer courses around this material but I like to try and put out some material for free.
Sign up to the free course and you'll get updates when I add new material.
Subscribe to:
Posts (Atom)



