Tribune DataViz

Matters of interest, from the data reporters and developers across Tribune Publishing

Posts Tagged ‘test runner

Experiments in Testing

leave a comment »

We’ve been flirting with automated testing for our Community Submissions app for some time. It’s an app that gets a lot of use, has some pretty dedicated members, and it talks to some outside APIs. By the end of last year, we were pretty happy with the tests that cover the backend and the basics of our views, but still wanted more robust coverage for the interface.

And then just before the holiday break, one of the three Ryans on our team (whose initials just happen to be RTM) posted a link in our group chat: “PhantomJS, Selenium, and Django: Headless Browser Testing for the Rest of Us.”

I decided to give it a try.

Getting it running wasn’t too bad, although the post uses a sudo command to install PhantomJS and that meant my app didn’t have permission to run it. I uninstalled and reinstalled without sudo and from then on was fine.

I didn’t have much experience with how Django manages tests, or with Selenium. It was interesting to read and think for the first time about different language bindings for the same API. I looked at how Python, Java and Ruby all accomplish the same thing and wondered what that took on the Selenium side. It was challenging and interesting to think about test servers, test databases, what kinds of configuration options are available to you for them. Not having a clear understanding of all of these parts meant that I made things work at first in ways that were a little more complicated than they needed to be.

In the process however, I learned how database aliases work, what parts of Django rely on them, wrote a test runner to use a custom setting in a custom alias, and wrote a custom settings file to manage the whole thing. The best part of this experience was going back and forth between the code that was needed to test the app, and the Ops changes I needed to make it happen.

I got great help from my team all along the way. One teammate suggested I put all the tests in a separate app in the project, to be able to hook into Django’s test system and allow for a one-line command to run them. Another helped me think about exactly what kinds of things would be useful to test. Another helped me realize I didn’t need the custom database alias or test runner to do what I wanted.

At first the tests I wrote went through each of the multi-step processes users do to accomplish specific goals, and contained assertions for each step. Later I broke most of them down into single tests for single goals, with one assertion for each. I wrote a few wrapper functions to be able to write clear assertions, assertSomethingContainsText, for example. I learned to use Selenium methods to wait until specific elements were clickable or present, rather than just have the webdriver wait a certain amount of time and hope they were – no better than guessing. Eventually I used one class to handle the setup and teardown and subclasses of it to run different kinds of tests, which is clearer and also helps reduce the load on our network.

Now we have a nice suite of browser tests that complement our functional tests, by doing all the common things users will do in our app. And I had a great time while learning many useful things. Team DevOps.


Written by jenlindner

January 21, 2014 at 10:20 am

Posted in Uncategorized

Tagged with ,