Drupal Acceptance/QA Testing with Selenium - Screencast

Background
For the past 6 months I've been lucky to be part of the development team for the newly launched Yoursphere.com, a Drupal-powered social networking site which "provides one of the safest online destinations for youth ages 9 through 18 to interact*". To say that Yoursphere is the most customized Drupal site I've worked on would be quite an understatement. One example of that, and subject of this article, the user registration system went from the standard single page - to one which uses 4 unique user creation forms which are integrated within several possible 'registration flows'. The most complex of these involves two of the user creation forms, 8 total screens and third-party identity verification.

Besides being an opportunity to get to know hook_user real well, at the end of creating this system we were left with a larger-than-normal nightmare of, "Wow, I wonder if my new small change just exploded the entire registration system for the site. Hmmmmmm."

Deciding on acceptance testing / Selenium
Faced with how to automate testing of these screens and use case, combined with limited time to implement the testing framework, a method of acceptance testing (using Selenium) was opted for instead of unit testing (e.g., using Simpletest) for several reasons (though perhaps in the future we'll have both unit and acceptance testing implemented):
  • Facilitate testing of the registration system by non-developers
  • There are registration screens which involve third-party interaction/functionality which cannot be unit-tested by us
  • Selenium opens the possibility of testing the registration system across multiple browsers
  • Seeing a browser step through all the registration steps just like a real person would do (except more quickly) is just so cool and offers a unique piece of mind about the integrity of the registration system
Selenium in action
When I last wrote about Selenium, I was trying to find an easy way to get Selenium RC to work *easily*, meaning without having to install a PEAR extension and/or PEAR itself. After (much) more research I can report conclusively that there is no *easy* way to get Selenium RC working, which in all honestly puts it outside my scope of interest, as I'm sure it would for most of the Drupal community.

The good news is that for my purposes, and I believe most people's purposes it doesn't really matter because during the course of this same research I discovered that Selenium IDE and/or Selenium Core offers most of the functionality of Selenium RC without all that extra PEAR-overhead. :-)

So enough talking, below is a screencast which shows a Selenium test suite which tests 6 different registration screens in a matter of seconds:


Find out more about Selenium
6 October, 2008

Comments

When I used the selenium IDE, the code it generated I considered "fragile" because if there were any changes to the site, the script would get lost. Form inputs, the most important part, seemed ok, it was more on verifying text on the page.

Do you have any tips for drupal developers to make tests more robust to site changes. If your recommendation is to simply add more id attributes to the resulting HTML, are there techniques particular to drupal to help with this?

Thanks for the writeup.

Great question - I think you're on to the right idea, basically. To carry that even further this link lists the several possible methods (css, dom/js-expression, xpath, and couple others) one can use for identifying page elements, including text.

I haven't come up with any particularly Drupal specific techniques for things (e.g., creating tests, automating test, or identifying elements yet), but I'm sure that possibility is there.

I'm working on creating an automated test which will eventually test my entire platform. For the moment, I am trying to automate the user registration process, which requires some external data to be called from my Postgresql table, {ldapprov}. At the moment getting the code value from the table using a SELECT query and a db_fetch_object either does not get the value, or does not seem to register and enter the value on time.
Has anyone found a way to include SQL information in their testing?