Tuesday, 13 September 2016

Is there a difference between "Responsive Web Testing" and "Cross Browser Testing"?

TLDR; Testing responsive web does not mean test it on lots of devices and browsers. Look at the risk associated with your technical implementation and test those. You might still have to use lots of devices and browsers.

When you test your web application, do you differentiate between “Responsive Web Testing” and “Cross Browser Testing”?

What is Responsive Web Design?

I think people still argue about “Responsive Web Design”.
I use the term to mean how the website responds to different rendering environments.
You can read more about responsive web design:
For my purposes here, I’m going to say that Responsive Web Design is responding to the size of the screen. This might be done via CSS or JavaScript or server-side based on browser headers.
So, how do we test it?

How we we test depends on how it is implemented

Very often I find people test on multiple devices because their organization has chosen to ‘support’ those devices.
And that’s it.
Different implementations expose us to different risks. And therefore… different test approaches.

“Cross Browser” Testing

Often the risk decision making criteria comes down to “we need to test it on this device because we want to say we support this device”.
The decision about ‘what’ to test on the device wasn’t based on the risk that the device presents. It was based on the risk that the marketing might not be accurate.
Issues with this strategy?
  • It will take too long, we have to test ‘everything’ on each of the devices
  • it still leaves us exposed to technical risk on devices that don’t come into scope.
If we take a technical decision about risk mitigation then we might stand a chance of reducing testing scope and therefore the time it takes to test.
We should look at the implementation approach to understand the risks around it.

Risk of Media Queries

Media Queries have been around for a long time.
Can I Use reports a pretty high browser support percentage for media queries. (How accurate is this? I don’t know!)
All of this, I could take to mean that CSS media queries represent a pretty low risk ‘responsive’ strategy.
But clearly it depends on what parts of media queries we use:
And if the media query itself is ‘essential’ to the site or application.
If it is essential that the functionality implemented using “hover” works then you probably want to make sure that Firefox and IE are part of your test browser environment, if you use “Can I Use” as your risk assessment tool, since “Can I Use” reports Firefox and IE 11 as having issues with CSS Hover. Your use case might actually work, even if the browsers are not viewed as completely compatible. (Certainly I use ‘hover’ on my main web site and I haven’t seen any issues with the hover on Firefox and IE.)
Cross-browser becomes important and viable when we have an identified risk with that browser platform.
If we ‘architect’ our applications to reduce risk then that provides an even better risk mitigation strategy than ‘testing’ since we can choose implementation solutions that are low risk.
Regardless… technical decisions around implementation can guide our testing.
Other than ‘platform’ implementation what else do we take into account?

Media Queries create screensize breakpoints

@media only screen and (max-width : 1290px) and (min-width : 1051px) {
   /* Medium desktop: 5 tiles */
   .thumbCell {
      width: 20%;
      padding-bottom: 20%;
   }
}
I took the above media query from the CSS used on “everybody’s favourite test consultancy” web site. And yes, that is my web site (thanks for noticing the shameless self promotion that I engage in).
The above query gives us a couple of breakpoints to work with and use those in our testing.
How could we test those?
  • choose devices that cover those sizes
  • resize the browser
  • use a responsive test tool with iframes set to those sizes
  • use the browser dev tools capabilities e.g. Chrome’s Device Toolbar
Are there risks in choosing one strategy above another?
  • What if we test on device but it doesn’t report the size we think it should?
  • What if we don’t have devices of the right size?
  • What if there are bugs in the tools we use?
  • what if the browser has bugs?
So many risks.
But really.
  • if the device implements the standard for media query screen sizing
  • if the browser sizes properly
  • and assuming that the CSS validates against the standard
  • and that the CSS validator validates correctly
Do we really need to check the media query on a device?
If you think so, then what additional risk did you identify that you will focus on with your testing?

Responsive Web Testing vs Cross-Browser Testing Summary

  • Use Cross-browser testing to target specifically identified platform risks.
  • Use responsive testing tools and browser features to target specific responsive web design standard features.
  • Validate your CSS
  • Find multiple trusted knowledge sources to help you assess risk

No comments:

Post a Comment