Thursday, 29 September 2016

Adhoc testing of incrementally developed HTML and CSS

TLDR; Even simple changes can use step by step risk based development to allow multiple releases, and will require multiple levels of testing.

My website is in a constant state of flux - that’s the benefit of writing it all by hand.

Today I wanted to add nifty social media icons on the page because I hear tales that there is gold to be made from the contacts that come from such things. And never one to turn my back on boundless riches, I wanted to add some social media and contact icons.

And why tell you this?

  • the testing process around this has relevance
  • I learned something about CSS and I assume other people might like to learn that too

Monday, 26 September 2016

How to Use Chrome Developer Tools JavaScript Source Snippets

TLDR; With Chrome Code Snippets we can ‘run’ small chunks of code without using the console. And using a gist we can import and export them to a file.

We already know that we can run custom JavaScript code from the console. I explained that in this blog post.
And we can use Google Chrome Code snippets to re-use that code over time.
By default we can’t import and export, but people have written helpful code to let us do that as well.

Friday, 23 September 2016

How to use the JavaScript Console for Technical Web Testing

TLDR; learn how to use the dev console and basic JavaScript commands (for, do…while, if…then, variables) to automate under the GUI and augment your interactive technical web testing.

I used to only use the JavaScript console in the Chrome Dev tools was for viewing JavaScript errors.
But I read the documentation for the JavaScript console
And realised that I could issue commands and view objects. And I slowly came to realise that I could use this to help me test better.

Thursday, 22 September 2016

How to Treat your Application as an API - "App as API"

TLDR; Look at what your GUI does. Automate that. And you’ve implemented “APP as API”

A phrase I’ve found myself using in various training sessions is “App as API”.

I think I’ve mentioned that in some previous blog posts but want to expand the concept here.
  • Some applications provide an API
  • Most applications provide a GUI
When the application provides an API, I like to take advantage of that. But…
  • Sometimes the API doesn’t do everything I want.
  • Sometimes the functionality I want to use is only available from the GUI
Do I let that stop me?


So what do I do?

Wednesday, 21 September 2016

How to write a simple random test data sentence generator

TLDR; I wrote a random test data generator, not a slogan generator

On the Importance of Test Data

We all know that test data is really really important.

If you don’t have the right data, you can’t test the functionality. You can’t check that an email is sent out on the customer’s 65th Birthday unless you have a customer who has a date of birth that will trigger that functionality.

Some data is path invariant and has to be specific to control the path.

We know this.

But we don’t always randomise enough data and our test data becomes stale and etc. etc.

Monday, 19 September 2016

Difference between hacking, cheating and automating?

TLDR; Hack for information. Cheating breaks the rules and introduces more risk. Automate the interfaces with less risk.

Previously on "Evil Tester goes Hacking JavaScript Games" I demonstrated a ‘bot’ that could play ZType

My bot cheated and automated shooting, I hacked

The bot has been through a few iterations:
  • 0 - every few seconds it uses an electro magnetic pulse and then gives itself a new emp, therefore having infinite emp devices
  • 1- every 100 milliseconds it iterates through the letters available and shoots that letter watch bot 1 beat a human
  • 2- fixes a bug in (1) so that it only shoots letters which are used on the level
  • 3- once it knows what word is currently being shot at, it uses the letters in that word, but when it doesn’t know the word it shoots all the letters on the level
  • 4 - only shoots letters on screen, every 10 milliseconds, either the start of a word then continues on the word
  • 5 - only shoots letters on screen, which are the start of a word, then focusses on that word to shoot it to bits (doesn’t wait 10 milliseconds to finish the word) - 100% efficiency
  • 6 - essentially bot 5, but only waits 2 milliseconds watch bot 6 in action
The different versions represent ‘cheating’ or ‘automating’.
I had to ‘hack’ to get the information I needed to ‘cheat’ with bot zero.

Friday, 16 September 2016

Lessons learned from Automating - Instantiated

TLDR; take small steps when automating, keep your code working at all times, automate at an appropriate interface

Working on a ZType bot reinforced a few lessons learned from automating that are important enough to draw attention to (again).
  • Your Code can gradually iterate to improve
  • You are allowed to stop improving the code
  • Start simple to add immediate value
  • Debug in small chunks
  • Readable code is understandable
  • Automate at the appropriate interface

Wednesday, 14 September 2016

Hacking JavaScript Games to improve your testing

TLDR; Learning to hack JavaScript will develop the skill of automating using the browser dev tools console.

For the last 6 months or so, I have, on and off, been learning to code JavaScript. That process has mainly involved:
  • writing some simple games
  • hacking other games to see how they were written
I’m going back to the good old days when we could ‘break’ the ZX Spectrum game and view the source, or disassemble the games on the Atari ST and hook/hack them through debuggers and monitors.
To do all of that now, JavaScript and the Browser Dev tools are perfect companions.

Tuesday, 13 September 2016

Is there a difference between "Responsive Web Testing" and "Cross Browser Testing"?

TLDR; Testing responsive web does not mean test it on lots of devices and browsers. Look at the risk associated with your technical implementation and test those. You might still have to use lots of devices and browsers.

When you test your web application, do you differentiate between “Responsive Web Testing” and “Cross Browser Testing”?

What is Responsive Web Design?

I think people still argue about “Responsive Web Design”.
I use the term to mean how the website responds to different rendering environments.
You can read more about responsive web design:
For my purposes here, I’m going to say that Responsive Web Design is responding to the size of the screen. This might be done via CSS or JavaScript or server-side based on browser headers.
So, how do we test it?

Introducing Pious Sanctimonious Standard Compliance Boy

TLDR; Browsers Lie To Us
What is worse - a sanctimonious ex-somethingorotherer or a zealous recent convert?
I guess it doesn’t really matter, I didn’t choose one role above t’other.
After my HTML Validation experience
I adopted the persona of “Pious Sanctimonious Boy” and went to work.

Friday, 9 September 2016

Batch validation of HTML as part of your web testing with Total Validator

TLDR; Total Validator Pro will spider a site and check its HTML as well as links. Free version works 1 page at a time. 30 GBP for the pro version.

I started using Total Validator Pro as well as the w3 validator (the API which powers the ‘validate’ function in Charles).
I wanted to add a ‘batch’ checking into my HTML validation process.
And Total Validator Pro seemed pretty simple, cheap and cross platform, so I tried it out.

Thursday, 8 September 2016

How to add HTML validation into your test process without even trying

TLDR; Charles proxy can use and report results in the proxy GUI

After looking at my default test process for web and realising it didn't include HTML validation by default, I decided to find the easiest way I could to add it in to my process.

I generally find that the easier I can slot something into my process, the more likely I am to use it.

We all use Dev Tools by default now

Dev tools are bundled with every browser now, so we all use them. Right?

Previously we had to download plugins and add ons. But now that it is a right click away by default, we have no excuse.

But HTML validation doesn't appear to be there. I can 'audit' the performance of the page in the Chrome dev tools, and the CSS usage checking is quite useful. But no HTML validation.

Which is the higher technical risk to your products functionality? CSS? Speed? HTML?

What could possibly go wrong? And what to do about it?

TLDR; We test based on risk. If we don’t identify risk we don’t test for it. Automated tools can reveal risk that our technical knowledge can not.

True story:

A timeline of a true story of software development:
  • 15:00 I get an idea for an ‘app’ so start making notes
  • 15:30 decide it will be faster if I just start coding it
  • 17:30 basic version of "The Evil Tester Sloganizer" done
    • it is unstyled, but generates slogans
  • 21:30 add some styling and a ‘tweet this’ slogan button, and a hacked up navigation bar
  • 10:30 what the heck, I’ll just release it
  • 10:45 what the heck, I’ll just announce it as beta, because then it doesn’t really matter if it works, right?
Whoa there!
Did you even test it?

Tuesday, 6 September 2016

Dear Evil Tester: "I think that if I 'learn automation' I can get a new job FAST"

TLDR; If you want a new job, then build a portfolio while you 'learn'

Following on from the ’How do I learn ‘automation’?" question. We had to ask questions to see what they would do with this newly learned ability to automate. If it was to find a job, then how long did they want to wait? And did they have a specific job in mind?
If there was no job in mind, and they wanted it fast then that is not a good combination for a ‘how to’ answer to the question they asked.

If you have no job in mind

Chances are, I probably won’t recommend ‘learn automation’ to you.

Dear Evil Tester: "I think that if I 'learn automation' I can get this specific type of job"

TLDR; Learn with an end goal in mind. That will keep you focussed.

Following on from the ’How do I learn ‘automation’?" question. We had to ask questions to see what they would do with this newly learned ability to automate. If it was to find a job, and they had a specific type of job in mind then I think we could probably answer the question.

If you have a better job in mind

For the marketplace, if you want to ‘learn automation’ because:
  • you want to get a new job in a specific organisation,
  • you want to work with a specific technology
You have a better idea of the job you want in mind. You aren’t just competing for a general ‘job’.

Dear Evil Tester "I want to keep my testing skills up to date otherwise I might not get another job"

Following on from the ’How do I learn ‘automation’?" question. We had to ask questions to see what they would do with this newly learned ability to automate. And because they want to perform ongoing work and this is part of their learning plan we can answer.

I want to keep my testing skills up to date

This is great. You have a job, use it.

Monday, 5 September 2016

A Model of Automating

TLDR; I automate tasks by using tools and writing code. Understand what they abstract to learn more.

A Model of Automating

What do you automate?

I automate tasks.

Sometimes I use a tool to automate tasks. Sometimes I write code.
But of course my model that underlies that statement doesn’t have as clear a separation as the words suggest.
My ‘tool’ and ‘code’ models do not have a clear separation. They overlap. I use tools when I code. I can write code to extend some tools. I can wrap tools in code.
I can model ‘Tools’ and ‘Code’ as a ‘spectrum’.
  • Tool <-> Code
    • “Easy Entry Point” <-> “Longer Learning Curve”
    • “Limited to Tool Features” <-> “Ultimate Flexibility”
    • etc.

Friday, 2 September 2016

Dear Evil Tester: “How can I learn Automation?”

TLDR; Don't just 'learn automation'. Learn it for a reason.

When I attend conferences and meetups, people often ask me how they can learn to automate stuff.
I then have to ask some follow up questions. This is not my normal approach for a "Dear Evil Tester" answer.
A "Dear Evil Tester" answer makes assumptions and takes the question and the words in the question at face value. Because that is funnier, and allows me to address generic points. But doesn’t always target a specific person’s need.
When dealing with questions from people, I try to spend a little time understanding what underpins the question, to build a better model of their world, to try and provide an answer that helps them.
I might ask a few questions before making any statements. My first statements might be to check my model of their world rather than to start answering the question, and based on their response, I might ask more questions or start to answer their question.
Follow up questions to “How can I learn Automation?” might be:
  • what do you want to automate?
  • are you trying to automate something specific at work?
  • what programming languages can you write code in?
  • how will learning automation help you?