One of the many amazing sessions given at the DrupalCamp Leuven 2015 was about tests applied to small / medium sized projects. I was a bit sceptical, we all agree that testing should be part of any development process, but it can run you quickly out of budget on small projects. @pfrenssen has just proven the opposite.
Automated testing has a lot of benefits: it reduces maintenance costs, increases software quality and prevents regressions.In the long run having a test suite saves time and money for any project. However in practice most small to medium size client projects are delivered without any tests. Why? Because in the past tests have the reputation to be very expensive and difficult to write. Luckily with Behat this is no longer the case: tests can be written in plain English in a few minutes.
I will try to summarize this session.
Automated and human testing
Let's begin by an overview of all the tests that we can run
Automatable testing
- Functional testing : regression testing (does a bug reproduced after being fixed?), smoke testing / sanity testing (fast test, runned before the others), ...
- Unit testing (dedicate to a portion of the software)
- A/B testing (variation control)
- Stress / load / performance testing (hardware and software resistance)
Non automatable testing
- acceptance testing
- alpha/beta testing
- usability testing
Drupal specific testing
- Drupal 7 : SimpleTest (quite slow to run)
- Drupal 8.0 : WebTestBase, UnitTestBase (fast in Drupal 8), BrowserTestBase (see Mink, PhantomJS)
- Drupal 8.1 (?) JavascriptTestBase
Third parties testing tools
How to apply to small projects ?
In this case, testing should be
- cheap
- easy to write and give quick results
- automatic and part of the process
- restricted to important features and regressions (every public bug report should be part of future test).
If we look around for testing solutions
- SimpleTest (Drupal 7, 8) : functional, slow to write and to run
- PHPUnit (Drupal 8) : unit, very fast, very hard to write
- Behat : functional, fast, easy
Behat is a BDD framework available for PHP and it has a Drupal extension (version 3 works for Drupal 7 and 8), so, let's go for it : )
BDD stands for behavior-driven development. The idea is to apply test via human-readable sentences, so it also improves naturally the documentation and communication between client and developers. A user story is translated into Feature and Scenario.
Exemple :
#features/example.feature
Feature: Search
Search is the most important part of a site!@search
Scenario: Search with no test content produces no results
Given I am on "search/node"
And the response status code should be 200
And I fill in "keys" with "node"
When I press "edit-submit"
Then I should see "Your search yielded no results"@search
Scenario: Search with test content produces result
Given a node of type "article" with the title "My first node" exists
And cron has run
And I am on "search/node"
And the response status code should be 200
And I fill in "keys" with "node"
When I press "edit-submit"
Then I should not see "Your search yielded no results"
And I should see "My first node"
More information : https://www.drupal.org/project/behat_testing
A Scenario can also be qualified with tags : @api stands for the Drupal API, @javascript for a headless browser testing (Selenium).
Built in step definitions are organized in contexts
- MinkContext
- DrupalContext
- DrushContext
- MessageContext
- FeatureContext (for custom code, own custom steps)
Exemple of DrushContext :
Given I run drush <command>
The drush output should contain <text>
How to implement in my project?
You just need 3 components
- Behat
- A git provider (GitHub or BitBucket)
- A CI service like Travis CI or continuousphp
The setup of the Behat Drupal extension can be done with Composer (require), then, the only thing to edit is the drupal_root in the behat.yml file. Afterwards, you can begin to write tests and save them in the features directory.
Travis CI and continuousphp are both free for open source projects. You setup your project via your git repo. Travis is more flexible but harder to implement than continuousphp that is also cheaper for private projects. Once configured, the reporting of the tests are also available on GitHub.
Test hapilly !
Resources
- Phéna Proxima (July 2018) : I’ve been using Behat wrong this whole time
- Jeff Geerling (September 2018) : Logging in as an existing user in a Behat test with the Drupal Extension
- Pieter Frenssen @ Drupal Europe 2018 : Testing small and medium size client projects with Behat [PDF]
- Pieter Frenssen @ Mountain Camp 2017 : Getting up to speed with automated testing on Drupal 8 [PDF]
- Pieter Frenssen @ DrupalCon 2017 : Testing small to medium size client projects with Behat [PDF]
- Examples of Behat tests : Joinup website tests
- Lullabot : Running Behat Tests in a Vagrant Box
- Drupal.org : Behat Drupal extension
- Nuvole GitHub : Behat Drupal Extension
- Compare Behat scenario tables to see if they match expected values
- Matt Glaman : Setup Drupal Commerce for CI and Behat testing
Add new comment