In this blog entry, I wanted to explain more about the FOSS automation tools we are using to do browser automation testing here at Wikimedia. These include Cucumber, Ruby, and Selenium with Watir web driver. We also use Page Object Design Pattern, which is a testing design pattern used to improve the maintenance of automation tests. I am excited to learn these tools not only because are they easy to learn and use, but also because they are up-and-coming browser automation testings tools starting to be seen more and more in the software testing world. Another plus is that when you mention you are working with Cucumber, you are sure to get a good reaction from others. Usually the reaction is something like this: "Huh? What is that? Never heard of it". In my daughter's case, her reaction was to insist to me that Cucumber is a vegetable and cannot be anything else but something you can eat!
One of Cucumber's strengths is that it is a communication tool as well as an automated testing tool.
Cucumber lets software development teams describe how software should behave in plain text. The text is written in a business-readable domain-specific language and thus can be used for written specifications, automated tests and as a development-aid - all rolled into one. Cucumber works with Ruby, Java, .NET, Flex or web applications written in any language. In our case, we are running it with Ruby.
Writing a test case in Cucumber is really as simple as writing a test case in plain english, except for the fact that it must be put into Cucumber's Given, When, and Then syntax. Here is an explanation and example of Cucumber taken directly from our Quality Assurance/browser testing document:
Cucumber demands specifications in a particular form, Given/When/Then:
- A Given statement should only ever mention starting conditions.
- A When statement should always be some sort of verb, or action.
- A Then statement always has the words "should" or "should not", or equivalent.
Feature: My Feature Scenario: Testing some aspect of My Feature Given <some initial condition> When <I take some action> Then <I should observe some result in the browser>
Any Given/When/Then statement may also have an arbitrary number of "And" clauses:
Scenario: Testing some complicated aspect of My Feature Given <some initial condition> And <some other condition> When <I take some action> And <I take some other action> And <I take yet another action> Then <I should observe some result in the browser> And <I should not observe some other result>
In order to write a cucumber test for the feature Search which should work in any given wiki, you should first write down how it is expected to behave in "plain english". It would like like this:
On any random page in any wiki, type "main" into the search box. A list of results should appear, and "Main Page" should be the top result. On another random page in any wiki, type "ma" into the search box and click the Search icon. I should then be on a page with a set of search results, and "Main Page" should be one of the results.
Now, here it is after being cucumberized:
Feature: Search Scenario: Search suggestions Given I am at random page When I search for: main Then a list of suggested pages should appear And Main Page should be the first result Scenario: Fill in search term and click search Given I am at random page When I search for: ma And I click the Search button Then I should land on Search Results page
See how simple that is? Easy Peasy :)
The next step would be to run this cucumber script from the command line, where it will detail the steps you need to take to put the test case into Ruby. More on this in my next entry.