menu

// Behaviour-driven Development in automated tests

Is it worth investing in Behaviour-driven Development (BDD) when preparing automated tests? What are the pros and cons of such a solution? I will examine this issue and analyse the tools dedicated to it.

BDD in a nutshell

I will not elaborate on what BDD is – there are many reviews of this kind. However, it is worth remembering a few basic facts:

  • BDD is focused on how the given functionality should operate
  • We write test scenarios as stories using the Gherkin language – as a result, they look as follows:

Feature: Adding a trip to shop cart
Scenario: As a user I am able to add a trip to shop cart
Given user is on main page
When he sets Paris as destination in the search form,
And submits his request
And goes to the first found result record
And clicks on “add to cart button”
Then a new item is added to the cart
And the item’s description contains proper data
And overall price is counted properly

  • FEATURE – a description of a functionality
  • SCENARIO – a description of what the test is to check/perform
  • GIVEN – the initial condition
  • WHEN – execution of action
  • THEN – verification of the expected result/outcome
  • AND – the keyword, informing that the above step will be continued

Apart from the above keywords and annotations, the following ones may also appear in our story: BACKGROUND, EXAMPLES, SCENARIO OUTLINE, BUT or TAG.

Why use BDD?
The greatest advantage of this methodology is the clarity of the stories for all users – from the developer to the end customer. A well-written scenario provides immediate information on which functionality is being tested and what the purpose of each step is. Therefore, we know what is being tested without the need to analyse the code, etc.

If we add the report system (e.g., Allure) to this, then we will obtain a clear summary of the performed tests. In the event of a failure, we will know, in which step there has occurred a problem. Meanwhile, the business practice shows that a client often considers the purpose of including test automation in the project. It is due to his uncertainty about what is actually being verified. Such doubts can be cleared by tests written in BDD. It is worth emphasizing that they are ready to maintain, and once developed steps may be used again in many other stories.

Is it worth doing?

Definitely yes! However, a more detailed answer to this question requires referring to a concrete project situation. Let us consider then a few cases:

  • THE PROJECT STARTS FROM THE SCRATCH
    If our project is about to have its kick-off, then it is definitely worth choosing BDD. The arguments in favour are: clarity, easy maintenance and a user-friendly preview of tests for non-technical recipients.
    At this point, it is worth bearing in mind that even if we don’t need it now, things may change in the future.
  • THE PROJECT IS AT AN ADVANCED STAGE
    If we have the automated tests ready and they operate well, not requiring any special workload apart from their maintenance, then there is rather no point in converting to BDD.
    However, if we have already decided to apply this methodology, then it will be best to start from writing new scenarios, leaving the current ones unchanged. If time allows and there is such a need, then we can always come back to them and modify them at a further stage of the project.

Weak sides
When you decide to use the Behaviour-driven Development, bear in mind that at the same time you implement an additional level of abstraction – we have to code subsequent steps of our story.

Apart from that, you have to take into account the time needed to develop such a scenario itself. Is this a disadvantage? The answer depends on your needs and requirements of the project. However, it is worth remembering, that traditional tests which are ‘only coded’, may – on the other hand – require more time to understand, what they actually verify.

BDD? I’m in… So what’s next?
We may apply many tools supporting BDD, if we use the Java language for test automation, in particular: JBehave, Cucumber and Serenity.

  • CUCUMBER VERSUS JBEHAVE
    They are frameworks most often used for test automation in Java. Everything started with JBehave created by the father of BDD – Dan North, but nowadays Cucumber is the definite number one. Its popularity is best illustrated by the following review:

At first glance, both frameworks look very similar. However, the devil’s in the detail, as they say.

1. We save test scenarios as files with .story extension for JBehave and with .feature extension for Cucumber

2. Both frameworks support the Gherkin language standards. However, bear in minds that Cucumber was the precursor in that case

3. Another difference refers to defining the test step scenario – in JBehave an annotation is described with an ordinary text, and the parameter is preceded with the dollar sign (e.g., ‘$parameter’). Meanwhile, in Cucumber regular expressions are used for that

JBehave fans consider the so-called composite steps to be a huge advantage of this tool. In short, it is the possibility to create one step which consists of a few others, which have been already defined. In theory, it saves time, prevents code duplication and makes it easier to write scenarios. In practice, however, it may turn out that using this functionality without proper skills and consideration, may result in damage, rather than bring benefits. Due to this, the Gherkin standards do not contain the composite steps, and it is not recommended to apply them. Therefore, Cucumber does not support this solution.

You can find a more detailed description of the differences between the aforementioned frameworks here.

Based on my own experience, I would like to add that Cucumber seems to be quicker and easier to use during testing. It is also visible that Cucumber is being dynamically developed – the community around this tool is growing all the time, and its clear documentation and technical support (the official one and this at the Internet fora) makes solving problems easy and quick. On the contrary, JBehave is still supported, but the dynamics of its development and its community is several times smaller.

Interestingly, when I was writing this text, the official Cucumber website released news that the team responsible for its development was taken over by Smartbear – a company known for such products, as SoapUl, TestComplete, Zephyr, SwaggerHub or HipTest. What will happen next and what will its consequences be? Time will show.

  • SERENITY
    This tool makes our approach to creating tests in BDD slightly different. In what way? There is no story writing in Gherkin. The following steps are:
  1. we prepare classes containing definitions (in the form of annotations and methods) of subsequent steps
  2. we initiate appropriate objects and their methods in the class we will be performing our tests in
  3. after completing a test, a report is generated, which contains i.a. the created BDD story.

Apart from that, Serenity enables the integration of both JBehave and Cucumber. Then, of course, we reach again for Gherkin syntax.

BDD – curiosities

  • HIPTEST

It is a test management platform in the cloud. We can write our own scenarios, run tests, use continuous integration tools from the level of a web browser. And all this in a distributed environment. If you like UI tools, you can use a free trial version.

  • JGIVEN

This tool allows you to write tests in the following code form:
given().method().when().method().then()method

After the test, we receive a detailed report. One of its useful features is an automatic change of our programmatic writing into a story developed using Gherkin language.

  • KARATE

It is a framework that is used for REST API and SOAP tests in the form of stories with the use of Gherkin resources and BDD methodology, where:

  1. All logic is enclosed in test scenarios
  2. Steps and special commands are predefined
  3. We can add the missing elements on our own using Java or JavaScript

According to it’s author, Karate is supposed to be an alternative to the popular Rest Assured. However, while the conclusions from simple examples are encouraging, the maintenance of several dozen tests becomes troublesome, illegible and simply uncomfortable.

 

As you can see, BDD can be a useful solution. It has many advantages – provided that its use is justified. From my own experience I can also confirm that the Behaviour-driven Development can be one of the strongest arguments for introducing automatic tests in a project.

// Partners