In this part, we will look at end-to-end (E2E) testing: we will test the entire application, and we will do it from the user’s point of view, per se, automating all his actions.
In our case, the application consists only of the front-end – there is simply no back-end, so E2E testing will consist of opening the application in a real browser, performing a set of calculations, and checking the validity of the value on the screen.
Do we need to test all permutations like we did in unit tests? No, it’s already been tested! In E2E tests, we check the performance of not individual units, but the entire system all together.
How many E2E tests are required?
The first reason why there should not be many such tests is that well-written integration and unit tests should suffice. E2E tests should check that all elements are correctly related to each other.
The second reason is they are slow. If there are a hundred of them, like unit tests and integration tests, then testing will take a very long time.
The third reason is the unpredictable behavior of E2E tests. There is a post about such a phenomenon in the Google blog dedicated to testing. Unit tests don’t show this erratic behavior. They can either pass or fall – and without visible changes, solely because of I/O. Is it possible to remove unpredictability? No, but you can minimize it.
To get rid of unpredictability, do as few E2E tests as possible. Write one E2E test for ten others, and only when they are really needed.
Writing E2E tests
Let’s move on to writing E2E tests. We need two things: a browser and a server for our front-end code.
For E2E testing, as well as for unit testing, we will use Mocha. We will set up the browser and web server using the before function, and reset the settings using the after function. These functions run before and after all tests and set up the environment that the test functions can use. You can learn how they work in the Mocha documentation.
First, let’s take a look at setting up a web server.
Setting up a web-server in Mocha
A web server in Node? Immediately, express comes to mind, so without further ado, let’s see the code in our test:
let server
before((done) => {
const app = express()
app.use('/', express.static(path.resolve(__dirname, '../../dist')))
server = app.listen(8080, done)
})
after(() => {
server.close()
})In the before function, we create an express application, specify the dist folder for it, and set it to listen on port 8080. In the after function, we “kill” the server.
The dist folder is where we store our JS scripts and where we copy our HTML and CSS files. You can see we are doing this in the npm build script in package.json:
{
"name": "frontend-testing",
"scripts": {
"build": "webpack && cp public/* dist",
"test": "mocha 'test/**/test-*.js' && eslint test lib",
...
},This means that for E2E tests, you must first run npm run build and then npm test. Yes, it’s inconvenient. In the case of unit tests, this is not necessary, since they run under Node and do not require transpiling and bundling.
For the sake of completeness, let’s take a look at webpack.config.js, which describes how Webpack should build files:
module.exports = {
entry: './lib/app.js',
output: {
filename: 'bundle.js',
path: path.resolve(__dirname, 'dist')
},
...
}Webpack will read our app.js and bundle all the necessary files into bundle.js in the dist folder.
The dist folder is used both in the user environment and in E2E tests. This is important – E2E tests run in an environment that is as similar to production as possible.
Setting up a browser in Mocha
Our application is installed on the server – all that remains is to launch a browser for it. What library will we use for automation? I usually use the popular selenium-webdriver.
First, let’s see how we use it before we get started with the settings:
const {prepareDriver, cleanupDriver} = require('../utils/browser-automation')
//...
describe('calculator app', function () {
let driver
...
before(async () => {
driver = await prepareDriver()
})
after(() => cleanupDriver(driver))
it('should work', async function () {
await driver.get('http://localhost:8080')
//...
})
})In the before function, we prepare the driver, and in the after function, we clean it up. Preparing the driver will launch the browser, and clearing it will close it. Note that driver setup happens asynchronously and we can use async/await to make the code look nicer.
In the test function, we open the address http://localhost:8080, again using await, given that driver.get is an asynchronous function.
So how do those prepareDriver and cleanupDriver look like?
const webdriver = require('selenium-webdriver')
const chromeDriver = require('chromedriver')
const path = require('path')
const chromeDriverPathAddition = `:${path.dirname(chromeDriver.path)}`
exports.prepareDriver = async () => {
process.on('beforeExit', () => this.browser && this.browser.quit())
process.env.PATH += chromeDriverPathAddition
return await new webdriver.Builder()
.disableEnvironmentOverrides()
.forBrowser('chrome')
.setLoggingPrefs({browser: 'ALL', driver: 'ALL'})
.build()
}
exports.cleanupDriver = async (driver) => {
if (driver) {
driver.quit()
}
process.env.PATH = process.env.PATH.replace(chromeDriverPathAddition, '')
}This is a complex thing. And I have to admit something: this code was written in blood (oh, and it only works in Unix systems). It was written with the help of Google, Stack Overflow and the webdriver documentation and heavily modified by a rule of thumb. But it works!
In theory, you can just copy-paste the code into your tests without understanding it, but let’s take a look at it for a second.
The first two lines connect webdriver – a driver for the browser. The way Selenium Webdriver works is that it has an API (in the selenium-webdriver module that we import on line 1) that works with any browser, and it relies on browser drivers to… control different browsers. The driver I used is chromedriver imported in line 2.
The Chrome driver doesn’t need a browser on the machine: it actually installs its own Chrome executable when you do npm install. Unfortunately, for some reason that I can’t figure out, it can’t find it and the chromedriver directory has to be added to PATH (this addition to the path is what doesn’t work in Windows). We do this on line 9. We also remove it from PATH in the cleanup step, in line 22.
So, we have configured the browser driver. Now it’s time to set up (and return) the web-driver, which we do in lines 11-15. And since the build function is asynchronous and returns a promise, we wait for it with await.
Why are we doing this in lines 11–15? The reasons are hidden by the fog of experience. Feel free to copy-paste – no guarantees are included, but I’ve been using this code for a while with no issues.
Let’s get to the tests!
We’ve finished the setup – it’s time to take a look at the code that webdriver uses to control the browser and test our code.
Let’s break down the code piece by piece:
// ...
const retry = require('promise-retry')
// ...
it('should work', async function () {
await driver.get('http://localhost:8080')
await retry(async () => {
const title = await driver.getTitle()
expect(title).to.equal('Calculator')
})
//...Let’s skip the setup we saw earlier and move on to the test function itself.
The code navigates to the application and checks that its title is “Calculator”. We have already seen the first line – we open our application using the driver. And do not forget to wait for the end of the process.
Let’s move on to line 9. Here we ask the browser to return the title to us (we use await to respond because it’s asynchronous), and on line 10 we check that the title title has the correct value.
So why are we repeating this using the promise-retry module? The reason is very important, we will see that in the rest of the test the browser, when we ask it to do something, like go to a URL, will do it, but asynchronously. Don’t let await fool you! We are waiting for the browser to say “OK, I did it” rather than the end of the operation.
Looking at Elements
Moving on to the next part of the test!
const {By} = require('selenium-webdriver')
it('should work', async function () {
await driver.get('http://localhost:8080')
//...
await retry(async () => {
const displayElement = await driver.findElement(By.css('.display'))
const displayText = await displayElement.getText()
expect(displayText).to.equal('0')
})
//...Now we check that display is initially 0. To do this, we find the element that contains display – in our case, this is the display class. We do this on line 7 with the findelement function of the webdriver class object. We can search for elements using the By.id, By.css or other methods. I usually use By.css – it accepts a selector and is very versatile to use, although By.javascript is probably the most versatile of them all.
As you can see By is imported from selenium-webdriver.
On line 10, we use the getText() method to get the content of the element and check it. Remember to (a)wait for all methods to complete!
User interface
It’s time to test our application – click on the numbers and operators and check the result of the operations:
const digit4Element = await driver.findElement(By.css('.digit-4'))
const digit2Element = await driver.findElement(By.css('.digit-2'))
const operatorMultiply = await driver.findElement(By.css('.operator-multiply'))
const operatorEquals = await driver.findElement(By.css('.operator-equals'))
await digit4Element.click()
await digit2Element.click()
await operatorMultiply.click()
await digit2Element.click()
await operatorEquals.click()
await retry(async () => {
const displayElement = await driver.findElement(By.css('.display'))
const displayText = await displayElement.getText()
expect(displayText).to.equal('84')
})We first find the items we want to click on in lines 2-4. Then click on them in lines 6-7. In our test, we got the expression “42*2=”. We then repeat the process until we get the correct result, “84”.
Running the tests this far
So, we have E2E tests and unit tests, let’s run them all with npm test:
We’re green!
Some Words on the use of await
If you look at a lot of samples out there on the web, you will see they do not use async/await, or even wait on the result using promises. Nope, they write code that is synchronous. How does this work? Honestly, I don’t know, but it looks like some weird shenanigans going on inside the webdriver. And, as even selenium says, this was an interim solution, until Node gets async/await support.
Well, guess what happened?
Some Words on the Documentation
Selenium’s documentation is, well, Java-ish. And that was not a compliment. On the other hand, the information is there. So bear with it, after a few tests, you will get it.
