Monday, 7 March 2011

The results are in for the Evil Tester Certification Survey

Did you enter? Probably not. And if not, you lost your chance to save the world.
The survey will remain open as I found many of the answers highly entertaining. So feel free to pop random entertaining snippets into the survey and I shall read them and chuckle.
Dare I draw conclusions from the survey, given some may have viewed it in jest?
Of course.
But first… three serious points.
  1. I take software testing incredibly seriously. I poke fun at it. Because I take it seriously.
  2. If it did not seem like a serious topic to me. I wouldn’t bother poking fun at it.
  3. I consider software testing so serious a topic that I need to think about it, not just coast along with the Status Quo. I can’t help find humour in it if I think about it. We laugh to change. If I don’t find a way to mock it then I probably haven’t thought it through from enough angles.
  4. Do you find testing serious enough to laugh at?
So, on with the statistics and numbers. The closest you will come to metrics on this web site.

We asked “Do you hold a Certificate in software testing?”
Our Survey said…
  • Yes == 56%
  • No == 44%
Proof positive then that the people who replied know what they replied about. I think.

We asked “Did your certification offer value for money?”
Our Survey Said…
  • I'm not certified, didn't you read my answer to question 1? == 41%
  • Yes == 13%
  • No == 31%
  • Definitely not, I got my money back == 0%
  • Other == 16%
Contentious. Some people didn’t think it offered value for money, BUT DIDN’T ASK FOR THEIR MONEY BACK!
Meaning – this could become a profitable business to get into. No-one asks for their money back.
Certification providers take note, you can drop your standards and it won’t make any difference.

We asked “Would you like to self certify yourself in Software Testing?”
Our Survey Said…
  • Yes, clearly that would be better == 50%
  • No, I can't possibly know better, or as much, than the official certification experts. == 3%
  • Never! What new certification scam are you trying to pull? == 47%
It seems as though some people saw through my thinly veiled attempt to see how much appetite exists for self-certification. And 50% think it sounds good. So before the certification crowd jump on this particular bandwagon. This one belongs to me.
And 3% thought they didn’t know as much as the certification experts. Which clearly (in an abuse of the survey process, but in keeping with poor statistical analysis) means that 97% of people doing the survey think they know better than the certification experts. And therefore they consider self-certification a good idea.

We asked “If a self-certification scheme did exist, how much would you pay for the privilege?”
Our Survey Said…
  • Nothing, it should be free - like all good things in life == 63%
  • < £10 == 3%
  • >= £10 && <= £50 == 16%
  • >£50 && <= £100 == 0%
  • > £100 (I only value something if I pay a lot for it) == 3%
  • Other == 16%
I don’t know what you see from these results, but I see:
  1. a free sample to hook in the 63%
  2. an immediate uptake of 35% of respondents
  3. hopeful conversion of the 63% to a total of 50%
Self-certification as a potential goldmine.

We asked “If it did cost money, what would a self-certification scheme pack have to contain so that you felt you were getting value for money?”
At this point I’m giving away the secrets of my future success. Because you too could use the information here to create a money spinning self-certification pack.
Our Survey Said…
  • Sample Certificates to fill in == 25%
  • Sample Answers to help you show off your newly certified knowledge in interviews == 25%
  • Empty promises about industry standard knowledge and future career prospects == 31%
  • Hints and tips on how to fill in your certificates == 22%
  • Affiliate links to lamination services so you could create laminated certificates == 22%
  • A direct debit form so you could continue to pay for your certification every year (and thereby 'prove' that you were still worthy of the certification) == 13%
  • A letter congratulating you on your decision == 28%
  • A document explaining example topics you could certify yourself in == 28%
  • Multiple levels of certification in one pack (e.g. novice, journeyman, master, expert, Nobel Prize Winner) == 25%
  • A Membership card == 38%
  • A Badge == 41%
  • Money off vouchers for additional certification packs == 16%
  • Other == 50%
Clearly people want to experience a sense of belonging – hence the popularity of the badge and membership card. This may explain why the certification sales pitch based on Social Proof (everyone else does it, why don’t you) does so well.
The direct debit, whilst looking good from a money angle, doesn’t seem that popular, so CAT’s compulsory re-certification scheme might not go down too well with the paying masses. The CAT folk might want to consider changing that aspect of the scheme. But since they didn’t get back to me with previous comments I sent them on their scheme, they might not take any notice of this one either.
So quick wins for all certification providers. Supply a badge and a membership card to boost sales.

We Asked “Do you wish this was a prize survey?”
Our Survey said…
  • Yes, I only fill in surveys where I can win a prize == 28%
  • No, I fill in surveys for the fun of clicking buttons and feel sure that the person on the end of the survey has read my 'other' answers carefully, so carefully in fact that I have undoubtedly succeeded in changing their mind about something or other == 50%
  • Other == 22%
Clearly people are opinionated. And want to express themselves. And you don’t need a prize to get people to fill in a survey.
In fact. Surveys have shown that if you do give a prize, you can’t trust the results.
Meaning. You can trust the results of this survey all the more.

And lastly, We Asked “Is Software Testing Certification Evil?”
  • Yes, Obviously == 22%
  • No, Of course not == 22%
  • Maybe, I'm not sure == 31%
  • Undoubtedly, my Holy Book lists certification as one of the main causes of discord in the world == 19%
  • Other (and no I won't explain why) == 6%
Pretty evenly split there. And maybe people just answered joshingly, but only 22% of our respondents claimed absolutely certainty that certification did not have the “Big E” connotation.
So time for a PR blitz from the certification providers me-thinks. And if any want to pay me for this marketing advice, please get in touch.

Favourite moments
Ah, the blooper reel. For as the credits roll, we can partake of the ‘other’ answers.
In terms of cost, there clearly exist people I could milk this scheme for all they had for they threw monetary amounts that no-one in the real world would pay for certification. Almost up to £1000 pounds or more for some people. Quite ridiculous and I’m sure none of the other schemes would charge anywhere close to that.
Some really good ‘stuff I want in the pack’ answers. Which certainly made me want to get one.
  • Hosted online certificates
  • A Propeller hat
  • Doughnuts
  • Test
  • Discounts on framing services
  • Cuddly Certification Toys
And some people saw parts of the page in Dutch which I thought splendid.

Conclusion
So there you have it. Self-certification clearly has legs.
And if you see any certification providers starting to give out Doughnuts on their training courses – you know where they got the idea.

1 comment:

  1. Worse part is when companies filter candidates on the basis of certificate and tester writes a certificates just to clear this stage (sigh)

    That sounds like the perfect strategy for dealing with incompetent filtering. If more people did it, then certification would lose all value as a filtering technique. Where there's a fiddle, people find a way.

    Thanks for the comment Tarun :)

    ReplyDelete

>