What Is A Promise?
Promises have a
then method which can be used to get the eventual return value of the operation. In this manner,
it’s common to see something along the lines of:
The widget is only updated with the data once it’s been returned, and in addition, the call to get the data from the external service does not block.
In the realm of browser tests this could lead to a long chains of
then functions, but WebdriverJS provides a Promise
Manager to get around that issue, thus ensuring that calls to the browser are run in sequence, and we only need to worry
about dealing with the Promises when we wish to do something with data from the page.
In practice, this means that we can ignore the returned promises while interacting with the page, until such time as we want to pass page data to something other then WebDriverJS. So, rather than having code which looks like this:
We can have code that looks like this:
The slightly tricky part is when we went to extract values from the page in question, at which point we have to explicitly handle the Promises ourselves:
There’s a good deal more information about this on the WebdriverJS page, but as you might expect, it’s still not overly straightforward.
Getting Set Up
The First Test
The intention with this test is to open a Chrome window, navigate to the Scott Logic Blog, and then assert that the
title is “Scott Logic Blog”. I’ve read the WebDriverJS documentation on bridging between the Promise Manager and
assertions, so I know to use a
then when getting the title of the page.
If you run this test it will print a collection of nice, green ticks to the console so we must have done everything right first time:
Celebratory coffee time! But wait… it ran in 31ms? Come to think of it, I never saw a browser window open and as awesome as my Scott Logic desktop machine is, I don’t think it’s that fast.
Connecting Mocha and WebdriverJS
Let’s look at the assertion that we have:
Although we’re clearly asserting on the title of the blog, that assertion is only executed once WebdriverJS has retrieved the title from the browser window, and we observed that the browser never appeared. The issue is that our testrunner has no idea that we want to wait for a browser to appear, as the interactions with the browser are non-blocking, remember?
So how do we tell the testrunner to wait for a result? In this case, mocha is Promise-aware so we can simple return that Promise from our test and rerun the test:
Well, well! Our passing test was doubly wrong! After a bit of correcting, we have this passing test, which does wait for a browser to appear and asserts correctly on the title:
It’s still not quite right, however, as the browser is left open after the test has completed despite the call to
quit in the
after block. By this point it probably won’t surprise you to learn that it’s because we’re not
returning the Promise to mocha so it doesn’t work as expected. In fact, we’re also not returning the Promise from
get either, and if you were to run the test you would see that mocha reports that
it block as passed before the
browser has appeared.
These false positives are a problem I have with using WebDriverJS. It is very easy to miss out a return and leave yourself with a bug-in-waiting, or a mysteriously passing test. Indeed, the final test is not very different to how we started out; just three return keywords required.
The example test above uses a pattern we’ve established to help with broken Promises, which is to separate out each browser
interaction into an
it block to ensure the spec output appears in sync with what’s happening on screen. We’ve also
found that it helps with those pesky intermittent failures that plague browser testing suites, if only to help narrow
down which interaction failed.
There is also a chai-as-promised library which could improve readability
of our tests, and reduce the need to explicitly drop in and out of
Something Of A Conclusion
I can understand why WebdriverJS has gone with Promises; browser interaction is largely asynchronous, especially with modern webapps using AJAX calls as opposed to testing older applications where you could just wait for a browser refresh. However, hiding those Promises away, in my opinion, confuses matters, making it more difficult to get into a flow when writing the tests.