Create your own Business Readable Domain Specific Language for Testing

HTML test automation tool in 60 minutes for someone without solid programming skills.

I had a project recently where I had to come up with a cheap (free), reliable and modular solution for User Acceptance Testing of a Web application. I have spent approximately two weeks validating tools and solutions, but none of them fulfilled our exact needs and requirements. Some felt short on controlling the application itself while others fell short on reporting. Some were not suitable due to requiring significant programming skills to operate. Apart from scripting in Python and JS, I do not have solid coding experience and yet the stack I have tick every boxes on our list. Read on if you are interested in the details!

Our goals were relatively simple given the circumstances

• Our website/application is constantly changing with a relatively high frequence means multiple times a week: Test cases should be easy to modify.
• Some of the sites in the web application were written using legacy technologies. The tool should be able to overcome the challenges of finding and controlling any element on the screen.
• Remote execution and reporting is a must, since the solution will be deployed worldwide
• We are in the middle of adopting proper TDD / ATDD: The tool should support rapid prototyping
• We would like to offer a relatively easy testing interface to enhance the support capabilities of our customers hence test cases should be very close to being human readable.

After experimenting with many things the following tool stack proved to be working fine for us:

• Python 2.7.9 or 3.4 (because of the out-of-box unit test framework)
• Selenium Webdriver (highly customizable, available for many languages, including python)
• Python bindings for Selenium webdriver
• Selenium Wrapper (saves a lot of work, because it provides ready-to-use object finder functions with retry and timeout functionalities and error messages)
• HTMLTestRunner for getting a more understandable output with no effort.

The entire stack is open source and setup is also relatively trivial.

These individual items do not make a good automation tool by themselves, but they are providing a solid base to build sophisticated Behavior Driven test cases on. The setup lends itself to this very nicely, following the principles similar to some of Cucumber’s.

• Each and every test case can be executed individually
• The top level test scripts are readable by the Business
• Already existing test cases are easy to convert into automated tests.

I was using Python’s built-in, already powerful unit test module to execute the tests. Fortunately it’s already providing an option for SetUp() and TearDown() methods which will run before and after each test cases, respectively. Browser types, profile settings, and any other configuration can be defined in the SetUp method. Cleaning-up the whole system, making it ready for the new test case, is easily done in TearDown(). More about this on the official python documentation on unit tests here. This setup completely fulfills our first point of requirements.

. In order to make the top layer readable I would recommend two expressions to keep in mind: simple user interaction and orchestration/middle layer or “glue code”. By simple user interaction I mean that I was forcing myself to implement the smallest things a user can do on the page, and give them a unique name. Think about a click and drag, opening a drop down, etc. I was paying attention to make it customizable by parameters if it was possible. By the “glue code” I have moved all of this definition into another .py file importing it into my unit test. This is important, because if you build you complex interaction by small definitions it will become very easy to maintain or modify the tests, while it takes minimal effort to follow any kind of UI/UX change

I have started to build complex actions in the following way:

(please note: this is just a example code, both available for download, with proper formatting):

In gluecode.py I have defined:

- Smallest user interaction like:

def open_the_sub_menu(browser, self):
try:
button = browser.href("dub_menu_ID")
button.click()
except NoSuchElementException:
self.fail("We could not find the sub menu, it might be not visible")
except:
raise

- Advanced user interaction built by the smallest user interaction like>

def go_to_submenu(browser, self):
open_the_main_menu(browser, self)
open_the_sub_menu(browser, self)

In the unittest.py file I was using only the advanced user interactions.

def test_case_01_open_the_submenu(self, browser):
Your_webpage.go_to_submenu(browser, self)
Your_webpage.validate_element_in_submenu_is_displayed(browser, self)

As you can see that the test_case_01 remained reasonably readable. It contains enough information to completely identify its purpose while the implementation in the glue layer makes it robust and still flexible. Having the smallest user interaction defined in only one places clearly increases the project’s maintainability as well.

For the reporting I was using HTMLTestrunner. It is very easy to use, free and works like a charm. Also, if you are familiar with python and PHP it is customizable as well (displaying screenshots taken at fails/errors, etc)

Usage of the reporter is very easy ,it will just sit on the top of the unit test. All you have to do is replace

unittest.main(verbosity=2)

With:

HTMLTestRunner.main()

At the end of the test file. I highly recommend to direct the output into a file with while using this method. Easiest way is to call the test file from command line with the following command:

python your_test_file.py > result.html

The results will be saved in the .html file next to your test. By reaching this state, you will have a simple Test automation tool to start with. You can customize it, build a UI around it, make it a standalone executable… it is all up to you. Possibilities are endless.

I hope this small tutorial can help you to make the first steps towards a proper test automation tool. On budget.

Best regards,

Laszlo Gombosi

Test Automation enthusiastic

If you have any question, or require assistance, please feel free to contact me at: gombosi.laszlo@gmail.com