Thursday, 28 February 2008

My Novice Mistakes in Test Driven Development

Since I generally present myself as a tester, coupled with my relatively novice status when coding in Java using TDD. I don't mind writing up my TDD mistakes here.

Wednesday, 20 February 2008

Some Notes on Software Testing Waste - my lean perspective

One key 'lean', or 'TPS', concept covers 'muda', or waste. Over my years in Software Testing I have tried to make my testing processes more 'Agile' or 'Lean', but when I started I didn't have the concept of 'muda', I just had the knowledge that things weren't working, so I changed them and got rid of the things that I thought added no, or little value. Now that I know a little about Agile and a little about Lean, I can try to apply the concepts of 'muda' to testing.

Monday, 11 February 2008

Notes on 'abstraction' and 'equivalence classes' in Software Testing

Abstractions can put your testing in danger if you don't handle them correctly. So some hints and tips on handling the abstraction known as an 'equivalence class' may help.

I use the word 'abstraction' (wikipedia definition) a lot when I discuss Software Testing. 'Abstraction' underpins so much of my modelling approach.
I recently read Glanville Williams' concise description of abstraction, as quoted in Hayakawa's "Symbol, Status and Personality" [][][].
"Abstraction my be defined as the imaginative selection of some one characteristic of a complex situation so that it may be attended to in isolation."
" short, the perception of similarity in spite of differences."
Quoted from the Law Quarterly Review
When I read this, I thought of 'equivalence classes', since I think of 'equivalence classes' as 'abstractions' and this definition reminds me of my approach to equivalence classing.
When I model using equivalence classes I group 'things' together based on a 'perception of similarity' in spite of differences.

Equivalence Class?

e.g. Instead of dealing with, a set of 'ages' from {0-140},
I could create a set of 'male insurance rate calculation and application age' equivalence classes of {young{0-12},teen{13-19},adult{20-35},mature{36-64},retired{65-140}}.

Warnings of danger

I particularly like Glanville Williams' definition because the carefully chosen words help warn of the dangers of abstraction.
  • ...imaginative selection...
  • ...some one characteristic...
  • ...complex situation...
  • ...attended to in isolation...
  • ...perception of similarity...
  • spite of differences...
I try to remain conscious of the differences. I don't want to fool myself that I can always, under all situations, treat as equivalent, each of the 'things' that I placed into these groupings. I need the awareness that I grouped them, for the situation which I currently "attend to", and that I isolated them to aid my understanding.
I impose a grouping. I create an abstraction. I lose information ('in spite of differences') as a result of that abstraction because I want to work with a simpler model ('complex situation'). When I forget this, I may fail to spot errors in the system.

Avoid this mistake:

I have in the past made the mistake of considering an equivalence class to 'always' retain equivalence throughout all functions of the system.
i.e. any value from the 'young' set can stand for any other value - no matter what functional area or process in the system I apply to it.
I have missed errors when I treated my abstractions as 'real' instead 'imaginative'.
When I have had the error of my abstraction pointed out to me I have then made the mistake of trying to maintain a system wide equivalence class and partitioning the classes further e.g. teen{13-16}, driving age {17-19}. Rather than creating a new 'contextualised' equivalence class for the specific situation that I require the abstraction to help me with.
I have learned from these mistakes. But I will also remain mindful of Glanville Williams' definition so that I do not repeat those mistakes again.

Hints and tips:

  • I 'name' my equivalence classes e.g. young, teen, adult etc.
  • I also 'contextualise' them to avoid forgetting the 'differences'  
    • I do this by 'naming' the set, to associate it with the particular 'complex situation' which I consider the set contents will act as equivalent representatives under. e.g. 'uk male insurance rate calculation and application age'.


I don't really want to write a huge post explaining an elegant 5 line excerpt. So I will instead close by recommending that the next time you create an equivalence class; that you re-read Glanville Williams' definition, and use it as a trigger for additional thought and reflection on your modelling process.

Related Reading

  • Equivalence classes received some attention on various blogs late last year [Thinking Tester] [I.M.Testy (1)(2)(3)(4)].
  • James Bach covers some similar ground in his blog post on model-based testing.

Sunday, 10 February 2008

If a demand for testing certification exists then why don'tUniversities meet that demand?

Should ISEB even run a certification scheme?

If Industry really does have a demand for it then perhaps ISEB's role should involve collating the Industry needs and passing them on to the Universities to provide the necessary certification mechanism. Academia generally seems to 'get' certification.

The Games industry has managed to get "Games Development" recognised as a degree level subject.

If industry wants a certification then should academia meet that need?

If we want our testers educated to 'Diploma' level then push the Universities.

Get people doing MSc's in testing.

Have our consultants and trainers do part time lecturing at Universities so that the students receive teaching from Industry professionals.

I'd much rather see on someone's CV that they took a career break to study for an MSc or MPhil in Software Testing, than an "ISEB Practitioner Certificate in Software Testing".

Friday, 8 February 2008

ISEB, and the train that won't stop going

What an easy target ISEB makes, it comes in for a lot of criticism. And I think it should. To an outsider like myself the certification train looks like a money spinning exercise, why else keep cranking out certification levels? I wonder what they could do to change my perception...

ISEB currently promotes 4 testing certification levels - the Foundation, Intermediate, Practitioner Test Manager and Practitioner Test Analyst.
I find it too easy to believe that ISEB has money making intentions rather than the 'profession's best intentions to hand. Given that the names of the writers of the syllabus all supply training towards ISEB Certification. I find it too easy to perceive a conflict of interest.
I don't want to believe that 'It's all about the money' because the BCS has registered as a charitable organisation that states "Its objects are to promote the study and practice of computing and to advance knowledge of and education in IT for the benefit of the public." For the benefit of the public, not for its own monetary gain.
So I'm going to step back a little and say some good things:
  1. Syllabuses freely available on the web, this counts towards the public good.  I won't critique the content here.
  2. The foundation syllabus has matured and represents a very basic, committee produced, introduction to 'software testing'.
Encourage any students using the syllabus for self learning to look for other sources of information on the web.
Bad points:
  1. You have to chain through the exams
  2. The other syllabus do not match the maturity of the foundation and don't really add much to augment the foundation documentation.
Why do you have to chain through the exams? Why can't you just jump ahead to the practitioner if you want to? Even if you have worked as a tester for 25 years, you still have to sit and pass the foundation, and the intermediate. Why? To protect the candidate from their own risky predisposition? Let them take the risk if they choose to.
A tester, a person who deals with risk on a daily basis.
A cynic might think that the chaining related to making money or selling further training.
Suggested Fixes:
  1. Have Industry provide a syllabus which contains the things that 'it' wants the testers to know.
  2. Have ISEB ratify the syllabus and organise the creation and sitting of the exams, and the marking facilities.
  3. Remove the conflict of interest with the trainers.
    1. Do not let the trainers write the questions.
    2. Do not let the trainers define the syllabus.
    3. Do not let the trainers mark the exams.
    4. Do not let the trainers sit on the 'board'.
    5. Do not do anything that looks like a conflict of interest and makes this look like a money spinning exercise.
  4. Allow anyone who wants to, at any point in the chain take the exam. Have no dependency on a previous exam. Just charge them a fair price to cover the cost of sitting and marking the exam.
  5. Make trainer costs and pass rate/fail rates visible on the main web site to allow candidates to choose
  6. Make pass/fail rates for self study candidates visible on the main web site
  7. Provide a simple, visible, online mechanism for taking comments publicly on the syllabus and publishing a summary of disagreements. (this would help prevent duplicate comments coming in)
  8. Make any multiple choice test completely automated and as nominally priced as possible (try and make it free).
Note on the above:
On making multiple choice exams nominally priced (or free). Any fee should cover the costs of writing questions, hosting the site and database, and all related admin. Think of it like a charitable act to the testing 'profession'. This would provide a public service to the testing community. I suspect that the foundation exam has probably paid for itself by now - if so make it completely free.
Trainers can still train.
It seems entirely possible to me that when you remove conflict of interest you may lose all the people required to run the programme. This may indicate a lack of business value and business demand for the certification.
So please ISEB, and everyone involved, just stop now, add no more certifications and remove all the constraints. It really does look like you keep feeding coal into the money train.
"Old Charlie stole the handle and
The train, it won't stop going --
No way to slow down. "
Locomotive Breath, Jethro Tull
Ironically, this post will end up surrounded by Adsense adverts for ISEB certification training. Oh well, I only do this for the money anyway.

Wednesday, 6 February 2008

My notes on how to study for ISEB foundation

While trawling through my hard drive on a deletion frenzy, I found my old notes on what I did to pass the ISEB foundation exam. Bear in mind that I made these notes in 2003 and the ISEB syllabus has moved on since. But I have updated all the links and someone might find my updated notes useful.

Original Notes - with up to date links.

Hints and tips

  • The questions get written from the syllabus - know the syllabus
  • The definitions come from the glossary - forget what you think you know if you want to pass - learn the definitions in the glossary
  • The techniques come from the BS7925, read and understand this
  • At the end of your study, if you feel like doing something adventurous then read the practitioner syllabus for extra depth


  • Study the syllabus in conjunction with the Glossary of terms used in software testing and the BS7925 standard
  • Build a mind map or set of study notes for the syllabus to summarise what you think important
  • Check your understanding and notes against the definitions in the glossary and the standards
  • Learn some memory techniques and general exam techniques. The  multiple choice foundation exam tests your memory skills rather than your testing skills.

Optional with Cost

  • From the Recommended Texts: Hetzel reads at about the right level; Beizer, although admirably included in the list should count as a bit more advanced than the foundation requires (but do read it for your own education and edification); I use Roper as a good summary of test techniques at about the foundation level
  • Get the sample paper from ISEB and try it out

Additional References

Up to Date Notes

  • The BCS web page recommends that you take a course to learn this stuff - frankly I think this massive overkill for a 40 question multiple choice test. Apparently "the overall public exam pass rates are notably lower than for candidates who have attended an accredited training course", but I could not find those stats on the ISEB site. Given the cost of a training course, you could probably sit the foundation exam 6 or 7 times on your own and still not have paid the same as a course would have cost.
  • The massive BCS Reading List seems designed to intimidate you into taking a course or grabbing the "An ISEB Foundation Course"
  • Where did this Intermediate course come from! Cue a post for a later date.
  • Sunday, 3 February 2008

    How To Learn To Program

    • Make it free to Learn – open source implementations of languages.
    • Don't use a traditional programming book – too slow.
    • Learn it to do something – be focused.
    • Ask for help.
    More helpful hints and links follow...

    How To Learn

    • Make it free to Learn
      • Find some open source implementations of the language you want to learn.
      • Check out sourceforgefreecountry
    • Don't use a traditional programming book – too slow.
    • Learn it to do something
      • Find a small project to do in it
      • Find a way to generate Test Data in it
      • Parse a log file in it
      • Work on an actual system
    • Ask for help
      • Developers will help if you are sincerely interested. I've never met a developer that wasn't happy to help me understand a language in more detail if I have made an effort to learn.
    There is so much information out there on the web, all you have to do is look for it.
    Go look. Now.


    Friday, 1 February 2008

    ISEB testing certification, fear, and the hiring dilemma

    At EuroSTAR 2007 Michael Bolton started the conference off with "Why I am not Certified" - Michael had obviously kept more up to date on the ISEB progress than I. I looked at the ISEB site recently, and now I feel uneasy.

    I know that the people who started all this certification had 'industry' and possibly 'profession' best intentions in mind I assume that the people involved still do have those intentions.
    The ISEB web site quotes one of the intentions as:
    To enable software suppliers to hire certified testers and thereby gain commercial advantage over their competitors by advertising their tester recruitment policy
    At EuroSTAR I held the belief that ISEB only had "foundation" and "practitioner". But I just looked and ISEB now has "foundation", "intermediate", "practitioner test manager" and "practitioner test analyst"
    When I act fulfil my obligations as a hiring test manager. I don't think "you know, if only ISEB had more levels of certification then I could more easily distinguish between good candidates and bad candidates."
    I don't need any more levels of certification created to meet my hiring needs. ISEB certification does not currently help my hiring needs.
    Now, people obviously do get trained and certified through ISEB, and I obviously don't know their reason for doing so. This situation suggests, to me, that some sort of demand exists for the certification.
    I have my suspicions that Fear produces this demand.
    Fear from the testers that if they don't get the certification, then their application will not go forward to acceptance for jobs. That seems like a valid fear, because it happened with the foundation certificate in the UK testing market.
    As a contractor, I encountered companies who wanted ISEB certification on my CV. I encountered agents working for companies who, on the instructions from the company, filtered out CVs on the basis of "Certified? Y/N".
    I think that if job adverts start to ask for - "must have ISEB Practitioner Test Analyst" then our industry and our 'profession' will have managed to get itself in a terrible mess.
    I don't just moan though. I now present an alternative for the hiring dilemma...
    How about ISEB throw away the other certifications and create a certification for hiring managers? The "ISEB Foundation Certificate in Recruiting Testers".
    Rather than building up an industry in certification that will eventually find its way into the hiring practices of big companies that don't know any better. Train the people doing the recruiting to recognise good testers. That sounds, to me, like a good move for the industry and in line with the BCS ISEB's charitable status to act in the public good.
    But since that certificate doesn't exist - why not read...