Skip to main content
blog title image

10 minute read - Testing ROI

How To Invest In Testing

Aug 23, 2018

TLDR: If you want to invest in yourself, we have books and courses and a patreon page

Finding testing too expensive?

Trying to replace your testers by Automating Testing?

You’re doing it wrong.

Tool vendors want to sell you tools to automate your testing and make testing cheaper - but here’s what they don’t tell you.

Testing is Expensive

You know, I know, and everyone knows; testing is expensive.

You, and everyone else, knows; that tool vendors sell expensive automation tools to make testing cheaper.

You know you want those tools.

We all do.

After all, new and shiny is better than what you have now. It’s natural to want those tools.

But, if you try to buy those tools to replace your testers, you’re going about the whole automation thing incorrectly.

Here’s the top three things that those tool vendors don’t tell you.

  • Automation isn’t free
  • People are not resources
  • 100% Automation isn’t possible

In this handy, “print out and keep” or “copy the URL and send to your friends and promote on Social Media to help this blog post go viral”, guide. We’ll tell you the truth about what those vendors don’t tell you, and what you can do about it instead.


Automation isn’t free

Of course Automation isn’t free. Tools are expensive.

But you know you need to invest if you want to reap rewards.

Return on investment calculations are how you make it appear free to people who ask.

The right way to calculate the return on investment is to:

  • Calculate the tool investment
  • Calculate the cost of a job as if done by humans
  • Assume that automating the job is free
  • Divide the investment by the cost of the job
  • This gives you the number of times you have to run the job to achieve the required return on investment via automation

The simple process above is explained below in more detail; to help complicate the explanation and make you feel that this guide has provided more valuable information than it actually has.

Calculate the tool investment.

First we want to calculate the tool investment.

For example:

Let us assume that Tool X costs £100,000.

Did you spot my deliberate mistake? Don’t worry if you didn’t, it’s natural to use normal language when talking about these things. Particularly if it’s your first automation project. You’ll learn to communicate properly over time.

We never say that a tool “costs” money.

We say that “we invest in the tool”.

Let us assume that we invest £100,000 in Tool X.

Calculate The Cost of the Job

People cost money. The job they do costs money. We want to invest money, not pay for costs.

  • Tool X will automate task Y.
  • We have 10 people doing Task Y
  • Each person costs us £350 per day

10 * £350 = £3500

Currently we have 10 people doing variations of task Y on a daily basis.

Obviously we don’t know exactly what they do, but they call it task Y and there are 10 of them.

Let’s assume those people cost us £350 a day.

Yes they are the lowest paid team members, but a penny saved is a penny earned. And these are people costs remember, not resource costs, that means this money comes out of your Operational or Recurring budget, rather than the company Capital budget.

That means that those 10 people cost us/you £3500 a day.

That’s a lot of money. Particularly since Tool X can automate Task Y.

The nice man at the trade show that gave you the sticker told you that. And it’s written in the brochure so you have that in writing if you ever need evidence to help with blame distribution.

Assume that automating the job is free

Clearly someone will have to setup the tool and automate the job, but that is something that is only done once, so it is effectively free.

And maintenance is easy. Particularly if you go for a tool which uses: AI, ML, Self Healing etc.

Forget about it.

Divide the investment by the cost of the job

Our tool investment is £100,000.

Our job costs £3500 per day (PER DAY!)

In Math this means:

100,000 / 3,500 = 28.5

That’s right. If you ran this job once per day, you’d have paid for your investment in twenty eight and a half days.

Hint: Instead of saying “twenty eight and a half” days, say “less than one month”

Think about that.

Return on investment

But we can do better than that.

When performing Return on Investment calculations, simple arithmetic can really help us boost our investment.

Over time you’ll learn to say “ROI”. Saying “ROI” will endear you to management and make all your meetings easier.

Rather than running the job once per day.

What if we ran it twice per day?

  • A morning job (just prior to everyone coming in to work)
  • An evening job (after everyone has gone home and left work)

(100,000 / 3,500) / 2 = 14.25

Only fourteen and a bit days.

Hint: Say “We could achieve our ROI in a fortnight”

But it gets better than this.

Using a little spreadsheet magic we can check what works best.

What happens if we ran this 10 times a day? After all, we have 10 people doing the work. 10 is a number we understand.

(100,000 / 3,500) / 10 = 2.85

A little over two and three quarter days.

Hint: Say “Less than 3 days” or “Over a weekend, if we start on Friday”

Always make a spreadsheet

Hint: Always make a spreadsheet

One of the keys to success when working with ROI numbers is to make a spreadsheet. This allows you to experiment easily with the repeated execution number to achieve the desired ROI period that you need to sell the idea to your management and relieve the pressure on your budget.

Automation isn’t free ROI Summary

Automation isn’t free. But a good, accurate ROI proposal can make it seem so.

And by “good” we mean one that makes the ROI seem free, fast and painless.

And by “accurate” we mean one which uses a spreadsheet and all the numbers have lots of numbers after the decimal point. This makes them accurate.

Math is your friend. Spreadsheets are your reporting tool of choice. Go ROI.


People are not resources

People are not resources.

Resources have no rights.

People have rights.

Treat People Like People

If you try to replace your test team with automation you may have to go through a costly process of:

  • re-training
  • hiring new staff with new skills
  • redundancy process and payments
  • employment tribunals

We need to treat people like people.

We might not want to, but legally we know we should.

Unless we have someone else to blame.

Since we’re not in goverment, we usually don’t have someone else to blame.

If your role is “Test Manager” then it usually means that you are the one to be blamed.

Legally then, we have to treat people like people.

Learn Lessons from Others

But this doesn’t mean we can’t learn lessons from our government, and those people in power.

Who better to learn from than “Our Betters”:

  • Don’t tell them anything
  • Tell them “Nothing is Changing”
  • Tell them “Everything will be better”
  • Remember: you owe them nothing

Keep people in the dark. Don’t tell them anything.

If you do tell them anything, then tell them “Nothing is changing”.

They may protest that it looks like things are changing.

Tell them it’s not and “Nothing is changing”.

You have to be prepared to sacrifice your people for the greater good, after all, they volunteered, you didn’t force them to work for you.

And you paid them. Something. Probably.

If they were resources, you wouldn’t care. If you move to automation then everything will be easier. Learn to not care early in the automation process and everything will be easier.

How to avoid people problems?

All problems are people problems.

After all, if no-one was there to point out the problem then the problem wouldn’t be a problem. It would be a feature, or possibly a ‘concern’, or possibly you wouldn’t know about it because no-one had told you and you didn’t care enough to look.

Either way. No people, no problems.

A Three Step Process for avoiding all people problems

Here’s a three step process for avoiding all people problems:

  1. Move all Task Y people into “Company 1” that specialises in Task Y
  2. Create a specialist company “Company 2” which automates Task Y
  3. Give all work to “Company 2”, and bankrupt “Company 1”

Step 1: Create Company 1

Step 1: Create an entirely new company “Company 1” to do Task Y.

This separate legal entity will benefit your current “People” because:

  • more autonomy
  • no longer viewed as resources, more like entrepreneurs
  • more scope for interesting and varied work in the future
  • more control over the future revenue potential
  • shares in the new legal entity

Shares are best when they are worthless if the company fails but worth loads if you sell to someone else.

Remember: 1000 shares sounds like a lot, but if you create 1,000,000,000 shares, you aren’t really giving much away.

Step 2: Create Company 2

Step 2: Create an entirely new company “Company 2”.

“Company 2” will automate Task Y.

This will be created with the seed money to invest in the tool.

This might also qualify you for start up incentives or R&D funding.

Step 3: Bankrupt Company 1

Step 3: Give all Task Y work to Company 2 and cease trading as Company 1.

Because your company is bankrupt, you have no legal obligation to your employees. You have no money to pay them. Effectively they are no longer your problem.

And yes, you are very sad that the new business opportunity didn’t work out. But, hey, that’s business.

The ‘people’ immediately become ‘debtors’ rather than ’employees’. And ‘debtors’ are legal entities, rather than ‘people’. Which makes them much more like ‘resources’.

People Problems Summary

And now you know.

People problems are easily solvable by removing the people.

The only problem is removing the people. But our 3 step people removal process can quickly remove the people, and the problem, if you can properly follow our process for removing the people, which, at the same time, removes the problem.


100% Automation isn’t possible

Tool vendors often try to convince you that 100% automation is feasible.

Clearly it isn’t.

100% Automation isn’t possible

Common sense tells us it isn’t, so we should listen to common sense.

Aim instead for 99% automation. Or any other number that isn’t 100%.

People only get concerned when you say 100% automation.

For example: if we were to automate 100% of Task Y, that would clearly be ridiculous since some of those steps are manual and involve pressing keys on the keyboard and moving the mouse. When we automate it will all be done by software. We won’t have a robot pressing the keys, or moving the mouse. We will no longer be pressing the keys and moving the mouse.

Let’s assume that 2% of the work is moving the mouse and 3% is typing on keys. Then if we automate through software we are really only automating 95% of the work.

True, we wanted to remove 100% of the people who originally performed Task Y, but we haven’t tried to automate 100% of Task Y.

Hint: Don’t automate an existing manual process, change it to something appropriate. Automate what is appropriate to automate.

We know 100% Automation isn’t possible

We know that 100% Automation isn’t possible.

Tool vendors can’t fool us that easily.

Aim for “less than 100%” and you’ll be fine.

This Ain’t No Summary

“Ultimately, there’s one investment that supersedes all others: Invest in yourself,” Warren Buffett

Tools are shiny and new.

Tools can help us.

Tools can help us automate things.

Everyone automates things.

We invest in tools.

We invest in things that help us.

Did this post help you?

Consider investing in one of our shiny things:

Or, you could invest in yourself on an ongoing basis via our Patreon program patreon.com/eviltester


Disclaimer: this post was not prepared by looking at any tool vendor’s marketing material. It is not intended to represent any real world entity. Any similarity to any company, material, person or writing, either living or dead or in/out of copyright, in the real world should not be inferred and if you do infer it then it is entirely co-incidental and only exists in your imagination via inference. No animals were harmed during the writing of this, I didn’t even drink any water.