Tuesday, 28 June 2011

How Can I Estimate My Testing?

Have you had anyone ask you a question about estimation? I get asked these types of questions and I suspect that the person really wants answers about how to communicate and justify their guesses.
I think they hope that some process exists which will accurately and objectively give them a set of numbers. And by using these numbers they can disavow responsibility for the production of them. And no-one will hold them responsible if the objectively produced ‘estimate’ does not meet reality.
Well in reality ‘Estimate’ really equals ‘Guess’.
So if you want some strategies:
  • Answer the question you want them to ask
  • Ask how long they want it to take
  • Trust your gut
  • Assumptions, Risks and Issues
  • Track the passage of time
I could add an etc. in there, to alert you to the incompleteness of all of this.
But I won’t.
You should assume incompleteness in my presentation. And map it on to your model to identify the blanks.

What culture do you think you work in?

Do you think you work in a culture where ‘estimates’ and ‘actual time’ have become synonyms?
Do ‘they’ berate you for continuing to work, after the ‘estimated’ length of time has passed?
Well, you have learned that regardless of where the estimate came from ‘they’ will still hold you responsible for it.
So you need to develop some beliefs. You need to believe that “estimate equals guess”.
You need to believe that any supporting documentation for your guess does not provide a justification for it, it provides a partial model of the thinking that led to the guess.
‘They’ don’t need to believe any of this.
You need to.
Then your communication will change.

Estimates equals Guesses

All estimates equate to GUESSES.
We don’t know how long things will take.
I’m a pretty good psychic. But even I don’t know.
Sometimes we put a framework around our GUESSES. And that can reveal some of our ASSUMPTIONS. It can reveal what RISKS we have considered. It can reveal what ISSUES we perceive.
The framework reveals part of the model we think we used when creating the GUESS. It helps us explain how we created the GUESS.
But even when we do that, our estimates remain GUESSES.

So what could you do?

Do you try and ‘educate’ them?
I don’t.
I’m obnoxious like that.
  • So if you get asked “How long will your testing take?”
    • Don’t say “I will provide you with an estimate for testing.”
    • Say “I don’t know.”
    • The conversation might end there.
You might respond with the question you want them to ask.
  • So if you get asked “How long will your testing take?”
    • ask “Do you mean? If I had to guess, right now, how long I think it will take for you to have enough information that you can decide whether or not you want to release this? If so then…”
    • Yup. I say stuff like that.
    • Because then we start talking about what they really want to know.
    • I don’t think I engage in an education process. I try and communicate so we all work from the same assumptions and understandings.
The test managers among you might feel a little nervous with this. As a test manager I used to feel nervous saying that. So you might ask instead:

“How long do you want it to take?”

We live in a world of deadlines. So guesses often don’t help very much.
By asking “How long do you want it to take?” you can build a plan which tries to maximise the value that you can add as quickly as possible before the deadline. Work on the high value items, risks and issues.
Don’t worry about the guesses. Think about the constraints.
When you ask “How long do you want it to take?” then you can think about how much stuff you might fit into that timescale, does it ‘feel’ right?

Trust Your Gut

What you ‘feel’ can feed into your guessing process.
You can get the whole guessing process over really quickly if you just come up with number.
You won’t have stated any assumptions. You won’t have considered any risks. you won’t have any weighted factors. But you can come up with a number quickly.
And your gut will tell you how you feel about it.
Does it feel right?
More importantly – does it feel wrong? If so, you can:
  • Ask yourself what feels wrong about it. Bam. Now you’re starting to come up with assumptions, risks and issues.
  • Come up with a new ‘better’ guess.
When you immediately decide to come up with a new ‘better’ guess. Then you trusted your gut.
If you ask your questions. You distrusted your gut. So you ask those questions to help you regain your trust in your gut.

Assumptions, Risks, Issues

Why bother with the questions?
Why bother with Assumptions, Risks and Issues?
Well, they help you communicate.
Until you develop the force of personality where you can give a number and have it accepted as a best ‘guess’. You might want to engage in a communication process.
If you do want to communicate then you can explain to people the things you have assumed. e.g.
  • I assumed we deliver a release candidate on day X
  • I assumed we have a clean build with no failing unit tests
  • etc.
You can do the same for risks and issues.

We track the passage of time

If your guess gets shot down and ‘they’ convince you into using a shorter guess. Then you can ‘manage’ the on-going process by monitoring for any:
  • negated assumptions
  • change in riskiness of the risks
  • transformation of risks into issues
This may not help in the short term as you simply point out that the correctness of the guess becomes ever less likely.
But you might help build credibility long term by communicating based on your frame of reference.
We track over time, the ‘goodness’ of our progress, and how much distance we appear to have to travel before we reach our end point.
We validate our model over time, by comparing it with the reality that we perceive.

Have you covered everything?

No.
I just want to encourage you to keep it simple.
I think you could waste time on a complicated ‘estimation’ process using spreadsheets and lots of factors.
‘They’ might well think they need those spreadsheets. Their process may require those spreadsheets. If so, use the spreadsheets to communicate your model, don’t use them to create the guess.
Of you could:
  • guess,
  • start to do the work,
  • based on how long it currently takes extrapolate forward and…
  • guess again
If you do this then you need more courage.
Swapping a belief of “objective estimation”, for “guesses and underpinning models” takes time.
I will not try to convince you that you should.
I just want to make you aware that you could.

Thursday, 2 June 2011

How can I learn to automate my testing using Selenium?

In this blog post, I’m going to lay out the Evil Tester free and simple 8 step guide to learning how to automate a Web Browser using Selenium.
I’ll reveal the sources of information you can use.
And yes, I will promote my book.
music by Drongomala @ flyingmountainrecords.com
I see the same questions, and variations on these questions, asked on social network forums every few days.
    • “How can I learn Selenium?”
    • “How do I use Selenium?”
    • “What do I read?”
    • “How do I get started with Selenium?”
    • “I want the answers. Please do the needful”
For some reason the previous answers don’t stick. Or people can’t find them. Or maybe they don’t look for them. Or, something… I don’t know.
But a lot of people have tried very hard to make Selenium easy to get into, and to help get you started. So if you are prepared to put the work in. The Answers are out there. Just don’t expect someone to magically drop the information into your brain, you have to put the work in to follow the learning steps.
Expect to put in some work.
If you are prepared to do that.
Then the pointers in here will help you.
First learn a bit about Selenium
Start with the basics.
Research what the tool sets out to do. Visit the official site. Have a look around.
Need/Want more than the Official Documentation
You can read two books:
I wrote a comparative review of them here:
Both David and I wrote our books when the Official documentation was harder to get into than it is now, so we aimed these books at the beginner. If you consider yourself a beginner and find the official documentation too hard. Then get hold of these books.
The official documentation has improved massively and the Selenium team have done a great job. These books go beyond the official documentation and add additional value. Sometimes people need more in depth tutorials, and more examples, more code samples, and these books provide that.
Selenium is Open-Source, Learning should be free too
OK, if you say so.
Both books have free previews. And these are big previews. Last time I checked mine was 75 pages. At the end of which you will have installed all the basic tools. Recorded a test in the IDE. Converted the test to Java. Run the test in the Eclipse IDE. Learned how to debug tests in an IDE.  Learned how to start and stop the Selenium Server programmatically. And given pointers of where else to look for more information. For some training courses, that would be day 1.
They cover:
  • installing the tools you need,
  • recording your first script in the IDE,
  • converting that into program code – because you do not want to get stuck in the IDE
And all of that in the free previews.
My book leads you through the process of automating some basic web pages and covers automating most of the common HTML features, automating AJAX, and refactoring your tests into Page Objects and building an abstraction layer. And even through the steps needed to put your tests into continuous integration. All in a consistent tutorial guide.
Read this comparative review, follow up the links, read the previews and tables of contents and see which books meets your needs.
More than enough to get you started. Enough to have you install the tools. Enough to get you moving.  In just the free preview.
Search for blog posts on how to start, here are a few searches to try:
And let’s not forget the official site http://seleniumhq.org
A lot of info out there. Some of it will be out of date. They will repeat and overlap. They will not be presented in a consistent style. But you can get all the information you need without buying the books if that is the learning style you prefer. You will learn from your mistakes as you follow the instructions. That learning will help you.
Where can I get help when I start learning?
The official site has a page all about that:
The bigger point though is not “where” but how you use that help.
  • Don’t ask generic questions.
  • Show the source code you were using when you got stuck.
  • Make sure your learning how to debug the scripts and have investigated it as far as your knowledge will take you.
People have put a lot of effort in simplifying the learning of Selenium but it still requires you to do some experimentation and make an effort.
You are learning how to code, and do some complicated stuff. Automation often requires workarounds, so if you don’t get into the habit now of trying different things, of exploring the API, of searching for tips around the area you are automating then you are doing yourself a disservice in your learning phase.
What else can I read?
I’m going to give you one link:
http://seleniumhq.wordpress.com
Adam Goucher does a great job, scanning blogs looking for material on Selenium. So follow up the links. That will take you to more blogs. Read those. Subscribe to those in your rss feed.
But I hate reading!
Fine. There are plenty of videos out there. Lots of Videos out there:

This all sounds like hard work
The learning process requires that you make an effort.
But its your choice. You do not have to learn how to use automation tools. Personally I think that will help your career and provide you with more options in your test approach, but you don’t have to learn it.
If you choose to learn it, then you have to put the effort in.
  • Learning involves Experimentation.
  • Testing involves Experimentation.
You already know how to do both those things.
Well, Automation involves experimentation too.
And remember – if you get stuck
Remember there are two books which aim to help you learn Selenium.
I wrote Selenium Simplified for the very reason that people do get stuck, so it provides step by step instructions for getting you unstuck and productive in Selenium.
Time to start now
The onus is on you. Right Now.
Just start following the 8 step process:
Step 1 - Read the official site and selenium documentation
Step 2 - Start trying to use Selenium using the documentation to help you
Step 3 - Investigate the books available, use the previews to get started
Step 4 – Auto-generate scripts from the IDE
Step 5 - Use Google to find tutorials, blogs and videos
Step 6 - Use stackExchange, linkedin and the Selenium Forums for specific help. Read the official support page
Step 7 - Read through the old issues of "A smattering of Selenium"
Step 8 - If you get stuck, and can't make progress - buy one of the books. You can apply this step at any time.
You can learn Selenium. Start Now.
>