Filed under: BDD, Cucumber, Test automation, — Tags: Behaviour-Driven Development, Testing iceberg — Thomas Sundberg — 2016-07-25
No tool is perfect for all jobs. When you start out automating Behvaiour-Driven Development, BDD, using Cucumber you tend to use it for much more than you need. This will work, but it is not a good idea for a few reasons:
The origin of Cucumber is collaboration. It has been described as "The world's most misunderstood collaboration tool" by Aslak Hellesøy, the creator of Cucumber.
The purpose with BDD is to collaborate and understand requirements for a system. Collaboration can be done in different ways. Performing an Example Mapping session is one way to understand what is needed.
The result of the Example Mapping can be captured using
Gherkin.
Gherkin is sometimes referred to as the Given/When/Then.
The result of the Example Mapping expressed in Gherkin can be automated using Cucumber. The result is an executable specification for a part of the system. It will drive the implementation as well as act as acceptance tests when the implementation is done. This is where Behaviour-Driven comes from.
A common question is at what layer we should use Cucumber. Should we use Cucumber to automate exclusively through the user interface? After all, our users will use the application through its user interface.
The answer is no. Not everything should be tested through the user interface. Testing can be done in any seam in the application as Michael Feathers describes in Working Effectively with Legacy Code.
UI testing is slow and brittle, there are other seams that are much faster and much more stable. You want to use these others seams as much as possible.
Matt Wynne and Seb Rose came up with the metaphor of an iceberg.
The majority of an Iceberg is under the surface. You can usually only see about 10% of an Iceberg. 90% of the iceberg is hidden below the water surface. Similar proportions can be used when deciding how much you should automate using Cucumber and how many of your tests that should be implemented using unit tests.
The parts that will interest the business, i.e. business-readable examples, should be automated using Cucumber. The rest, the technical testing, should be implemented using more developer friendly tooling such as JUnit.
Using the iceberg metaphor indicates that some of your tests should be done through the user interface, some of the tests should test an integrated part of the system without using the user interface and some tests should verify small parts. It emphasises that different seams should be used at different times.
Use the right tool for the job. When you start out using a new shiny tool, you try to use it for everything. Using the same tool for every problem is most likely to use it for the wrong problem in some cases.
Use Cucumber in the cases where the implementation of the test cases need to be business-readable. Use JUnit when it is a technical test that the business don't have to read.
I would like to thank Malin Ekholm for proof reading.
RSS