Drupal Acceptance/QA Testing with Selenium - Screencast

For the past 6 months I've been lucky to be part of the development team for the newly launched, a Drupal-powered social networking site which "provides one of the safest online destinations for youth ages 9 through 18 to interact*". To say that Yoursphere is the most customized Drupal site I've worked on would be quite an understatement. One example of that, and subject of this article, the user registration system went from the standard single page - to one which uses 4 unique user creation forms which are integrated within several possible 'registration flows'. The most complex of these involves two of the user creation forms, 8 total screens and third-party identity verification.

Besides being an opportunity to get to know hook_user real well, at the end of creating this system we were left with a larger-than-normal nightmare of, "Wow, I wonder if my new small change just exploded the entire registration system for the site. Hmmmmmm."

Deciding on acceptance testing / Selenium
Faced with how to automate testing of these screens and use case, combined with limited time to implement the testing framework, a method of acceptance testing (using Selenium) was opted for instead of unit testing (e.g., using Simpletest) for several reasons (though perhaps in the future we'll have both unit and acceptance testing implemented):
  • Facilitate testing of the registration system by non-developers
  • There are registration screens which involve third-party interaction/functionality which cannot be unit-tested by us
  • Selenium opens the possibility of testing the registration system across multiple browsers
  • Seeing a browser step through all the registration steps just like a real person would do (except more quickly) is just so cool and offers a unique piece of mind about the integrity of the registration system
Selenium in action
When I last wrote about Selenium, I was trying to find an easy way to get Selenium RC to work *easily*, meaning without having to install a PEAR extension and/or PEAR itself. After (much) more research I can report conclusively that there is no *easy* way to get Selenium RC working, which in all honestly puts it outside my scope of interest, as I'm sure it would for most of the Drupal community.

6 October, 2008

Researching Drupal and acceptance testing. Or where Simpletest admits it falls short. (so what'd we gonna do about it?!)

(Since writing this article I've posted a newer article about Selenium here)

Based upon my recent research, there seems to be a sentiment in the Drupal-community-at-large (an undefinable thing to be sure) of, 'there's no need to look any further than simpletest for any testing needs' (unit, integration, acceptance, etc). This is counter to simpletest's own documentation (bottom of page) which explicitly suggests to look for other alternatives for acceptance testing.

After Googling to death 'selenium rc drupal', 'selenium rc php', etc I realized that almost no one seems to be implementing Selenium RC within their Drupal workflows.

Scratching my head at this I decided to figure out 'why' by doing an exhaustive search of all things Simpletest and found that it does indeed have support for making acceptance tests (see bottom of article) - BUT - and it's a big but - there's no javascript support.

So I started tracking down how the Drupal community plans to deal with the lack of JS support in Simpletest, and stumbled upon this effort to get some kind of JS testing framework into Drupal 7 head, which if I read things correctly appears to be more about unit testing than UI/acceptance testing. So the current Drupal roadmap for the latter doesn't seem to exist. (anyone have ideas/links to things related UI/acceptance testing for Drupal that I might have overlooked? UPDATE: Just found this link, but also seems to be more on the unit testing side and it's currently seems to be inactive)

At the moment I'm left wishing that getting Selenium RC to work wasn't so painful (it requires installing PHPUnit, which in-turn requires installing a PEAR extension, which for many requires installing PEAR itself).....or to put it another way....currently I'm feeling stuck between tackling something which does what I want but which isn't embraced (Selenium or acceptance testing) by the larger Drupal community, or else tackling something (Simpletest) which is supported by the larger Drupal community but doesn't offer the level of functionality I'm looking for.

Wondering how I can help.

Related articles:
Selenium and Drupal
Unit VS UI Testing
Develop an automated javascript testing framework

21 August, 2008
Subscribe to RSS - selenium