Saturday, 8 March 2008

A little abstraction when testing software with Selenium-RC and Java

image The testing tool Selenium goes from strength to strength. As evidenced at the recent Selenium Users Meeting, the future plans for Selenium look really interesting and Google seem to want to continue to invest heavily in Selenium which can only mean good things for the rest of us.

Testers have learned to use abstraction over the years. Abstracting the GUI changes away from the tests with a GUI Map, abstracting the testing language away from the scripting language through 'Action Word Testing'.

The development world came up with similar levels of abstraction when they started doing testing - hence the Fit and FitNesse frameworks.

Selenium-rc presents us with an API. Around which people tend to build abstraction layers. Many of the people using Selenium-rc use abstraction automatically because they come from a development background. So the simple use of refactoring leads to a degree of abstraction.



Take for example a simple Selenium test, built to automate a simple path through a default install of bugzilla.

The Selenium-IDE automatically 'captured' and generated the following 'Selenese' test script for me.

selenium_ide_test_login_logout_of_bugzilla
open /
type Bugzilla_login admin
type Bugzilla_password admin
clickAndWait log_in
clickAndWait link=Reports
clickAndWait link=My Requests
clickAndWait filter
clickAndWait link=My Votes
clickAndWait link=Preferences
clickAndWait link=Log out
clickAndWait link=Home

This type of script looks like an old 'action word' script. Where we have abstracted the implementation of the commands out to Selenium.

FitNesse users would typically want to see a higher level of abstraction because FitNesse targets 'acceptance testing' and 'acceptance tests' typically read as very specific worked examples which act as a minimal acceptance set. Some general criteria for automated acceptance tests may include: readable by 'user', robust in the face of application changes, use user 'language'. The script above covers none of these attributes.

So if we have a quick look at the Java code generated from this by the IDE.

I can see a similar level of abstraction here in that I now use the Selenium-rc API directly to automate the test. Still not quite the readable test that we can achieve through abstraction.

package com.example.tests;

import com.thoughtworks.selenium.*;
import java.util.regex.Pattern;

public class NewTest extends SeleneseTestCase {
public void setUp() throws Exception {
setUp("http://xxx.xxx.xxx.xxx/", "*chrome");
}
public void testNew() throws Exception {
selenium.open("/");
selenium.type("Bugzilla_login", "admin");
selenium.type("Bugzilla_password", "admin");
selenium.click("log_in");
selenium.waitForPageToLoad("30000");
selenium.click("link=Reports");
selenium.waitForPageToLoad("30000");
selenium.click("link=My Requests");
selenium.waitForPageToLoad("30000");
selenium.click("filter");
selenium.waitForPageToLoad("30000");
selenium.click("link=My Votes");
selenium.waitForPageToLoad("30000");
selenium.click("link=Preferences");
selenium.waitForPageToLoad("30000");
selenium.click("link=Log out");
selenium.waitForPageToLoad("30000");
selenium.click("link=Home");
selenium.waitForPageToLoad("30000");
}
}

A fairly typical process which follows on from the above script would involve refactoring some of the Selenium code into new methods. Which immediately creates a level of abstraction.


So I may create a new method called login...


 public void login() {
selenium.type("Bugzilla_login", "admin");
selenium.type("Bugzilla_password", "admin");
selenium.click("log_in");
selenium.waitForPageToLoad("30000");
}

...so that my test can start looking like...


        public void testNew() throws Exception {
selenium.open("/");
login();
...

...and so I carry on until I have a test that looks like this...


        public void testLoginAndClickOnTheMenuItems() throws Exception {   
gotoHomePage();
login();
gotoReportsPage();
gotoRequestsPage();
clickFilter();
gotoMyVotesPage();
gotoPreferencesPage();
logout();
gotoHomePage();
}

Now pretty much anyone can read and understand the test.


I would probably do some follow on refactoring:



  • move these private methods from the test class into a class of their own - possibly 'BugzillaAbstraction'
  • rename login() to loginAsAdmin() since I hardcoded the userID and password.

But, regardless, the above script represents one level of abstraction that I apply when writing Selenium tests.


I can apply further abstractions as and when required by my automation needs and the communication needs associated with the testing.


 


Addendum: Since Antony Marcano has written a post dealing with abstraction and acceptance testing in Selenium, I feel duty bound to bring the date of this posting forward, even though the abstraction above does not go as far as he describes in his blog post.

7 comments:

  1. Nice post Alan! I also use this approach of abstracting behind methods - however, I've found my refactorings taking me in the direction of using the Command pattern instead of the methods (for various reasons).

    Referencing the "literate API" example in the comment of my post - command classes provide the same aggregation of activities as methods but allow you to minimise change... it also makes it easier to write the syntactic sugar illustrated in the literate API example I tagged on the foot of the post you've referenced.

    I'll put together an example that I can share openly (without restriction of NDAs) and post to my blog in the next week or so (hopefully).

    Thanks for the comment Antony. I look forward to seeing the blog post you write. I like to see 'real' code examples on the internet for at least 2 reasons: so that I can learn, and so that I have more resources to point people at to help their learning.

    ReplyDelete
  2. [...] ajudar nestes casos, o ideal seria abstrair o código Selenium e modularizá-lo em funções ou operações de negócio, para que o código final possa ser lido e [...]

    ReplyDelete
  3. Hi. I liked your post. And admittantly, I found it through a link from Anthony Marcano's posting, which I liked too.

    This brings to mind a technique I've been finding myself using on occasion when TDD/BDD'ing code. I'd end up with a fixture looking like this:

    [Test]
    public void DrawingOnScreen()
    {
    GivenACanvasOfSize(100,100);
    WhenDrawingABlackPixelAtPosition(99,99);
    ThenTheResultingPictureShoulBe(expectedBitmap);
    }

    [Test]
    public void DrawingOffScreen()
    {
    GivenACanvasOfSize(100,100);
    WhenDrawingABlackPixelAtPosition(101,99);
    ThenTheResultingPictureShouldBe(expectedBitmap);
    }
    ...

    The tests are extremely readable and tracable back to stories/requirements, but I must admit having so much refactored untested test-code behind the scenes does make me feel uneasy.

    I've recently been coaching team members on writing acceptance tests with Selenium.
    We havn't tried any of such techniques (yet), and I too find that the tests focus too much on the task-level (type this and that, click that, assert this textbox etc.) instead of on the activity level (perform an invalid login, assert that we get a "login denied" message etc).

    I'm guessing too that stuff like UI-Map can get us closer to an activity-testing DSL, although it has a high learning curve for non-developers to use it. Same with Selenium RC (as opposed to HTML).

    - Avi

    ReplyDelete
  4. Aha! We use these techniques in our functional testing framework.

    It's written in Java and drives Selenium to test an ExtJS UI. Although the framework is in-house and source is not public, I wrote an article on it that you might find useful

    http://www.neocoders.com/portal/articles/selenium-and-extjs

    cheers,
    Lindsay

    Thanks Lindsay, an interesting read - everyone go check it out. :)

    ReplyDelete
  5. This is a good article. However I really having a tough time implementing Selenium in one of my application wherein I am using Ext JS. Folowing is a query, please see if someone could solve it and let me know same.

    I have a simple login page written in extension JS in following way:
    Ext.onReady(function()
    {
    Ext.QuickTips.init();
    Ext.form.Field.prototype.msgTarget = 'side';
    new Ext.Button({
    renderTo:'login',
    text: 'Login',
    width: 800
    }).on('click',function()
    {
    document.forms[0].submit();

    });
    });

    Now when I use selenium testing via browser I get a randomly generated id which when hard coded can very well be used in following way:
    selenium.click("ext-gen26");

    My concern is how to get reference of this generated id, so my selenium test case in JUnit is not dependent on randomly generated ids from Extension JS?

    Your suggestions appreciated.

    Regards,
    Prashant

    Prashant, I haven't worked with Ext so I don't know exactly how it works, and I don't know what html it uses to model the button.

    I assume that it must be more complicated than an xpath that checks for a button with the text login:
    //button[text()='Login']

    But if you are using renderTo, doesn't that mean that you already have an element on the form with the id 'login'? If so then don't you already know the top level element to use as a basis for an xpath for your click locator?

    You might find this article useful http://www.xeolabs.com/portal/node/34

    ReplyDelete
  6. This is a good article. I am a recent convert to Selenium from QTP. Looking to see some code examples of identifying webpage objects.

    Thanks for sharing...

    -Sakthi.

    ReplyDelete
  7. Sakthi, you can find many examples of Selenium code if you search in Google.

    ReplyDelete