Testing
Vortex supports running Unit, Kernel, Functional, and BDD tests.
For local development, the tests can be run directly or using handy Ahoy commands:
- Direct
- Ahoy
vendor/bin/phpunit # Run Unit, Kernel and Functional tests
vendor/bin/phpunit --testsuite=unit # Run Unit tests
vendor/bin/phpunit --testsuite=kernel # Run Kernel tests
vendor/bin/phpunit --testsuite=functional # Run Functional tests
vendor/bin/behat # Run BDD tests
ahoy test # Run Unit, Kernel and Functional tests
ahoy test-unit # Run Unit tests
ahoy test-kernel # Run Kernel tests
ahoy test-functional # Run Functional tests
ahoy test-bdd # Run BDD tests
In CI, tests are run by calling the test binaries directly.
Unit testing
Vortex uses PHPUnit as a framework for Unit testing.
It is configured to use a copy of Drupal core's core/phpunit.xml.dist
configuration file to allow customizing the test suite per project.
Reporting
Test reports are stored in .logs/phpunit
directory
separated into multiple files and named after the suite name.
These reports are usually used in CI to track tests performance and stability.
Boilerplate
Vortex provides Unit, Kernel and Functional tests boilerplate for custom modules, themes and scripts.
These boilerplate tests run in CI when you install Vortex and can be used as a starting point for writing your own.
Drupal settings tests
Vortex provides Drupal settings tests to check that Drupal settings are correct based on the environment type the site is running: with the number of custom modules multiplied by the number of environment types, it is easy to miss certain settings which may lead to unexpected issues when deploying a project to a different environment.
It is intended to be used in your site and kept up-to-date with the
changes made to the settings.php
file.
CI configuration tests
Vortex provides a CI configuration tests to check that the CI configuration is correct. It is intended to be used in your site and kept up-to-date with the CI configurations.
For example, there are tests for regular expressions used to filter the branches and tags before they are deployed to the hosting environment.
BDD testing
Vortex uses Behat for Behavior-Driven Development (BDD) testing. Behat allows to write human-readable stories that describe the behavior of the application. Behat tests primarily focus on critical user journeys, serving as comprehensive end-to-end validations.
Vortex provides full Behat support, including configuration in behat.yml
and a browser container to run tests interactively in a real browser with
a VNC viewer.
Additional features include:
- Behat Drupal Extension - an extension to work with Drupal.
- Behat steps - a library of re-usable Behat steps.
- Behat Screenshot - extension to capture screenshots on-demand and on failure.
- Behat Progress formatter - extension to show progress as TAP and failures inline.
- Parallel profiles - configuration to allow running tests in parallel.
FeatureContext
The FeatureContext.php
file comes with
included steps from Behat steps package.
You can add your custom steps into this file.
Profiles
Behat's default
profile configured with sensible defaults to allow running it
with provided extensions.
In CI, the profile can be overridden using $VORTEX_CI_BEHAT_PROFILE
environment variable.
Parallel runs
In CI, Behat tests can run within multiple runners to increase the speed of the
test suite. To achieve this, Behat tags are used to mark features and scenarios
with @p*
tags.
Out of the box, Vortex provides support for unlimited parallel
runners, but only 2 parallel profiles p0
and p1
: a feature can be tagged by
either @p0
or @p1
to run in a dedicated runner, or with both tags to run in
both runners.
Note that you can easily add more p*
profiles in your behat.yml
by copying
existing p1
profile and changing several lines of configuration.
Features without @p*
tags will always run in the first CI runner, so even
if you forget to tag the feature, it will still be allocated to a runner.
If CI has only one runner - a default
profile will be used and all tests
(except for those that tagged with @skipped
) will be run.
Skipping tests
Add @skipped
tag to a feature or scenario to exclude it from the test run.
Screenshots
Test screenshots are stored into .logs/screenshots
location by default,
which can be overwritten using $BEHAT_SCREENSHOT_DIR
variable (courtesy of
Behat Screenshot package).
In CI, screenshots are stored as build artifacts. In GitHub Actions, they can be
downloaded from the Summary
tab. In CircleCI they are accessible in
the Artifacts
tab.
Format
Out of the box, Vortex comes with Behat Progress formatter output formatter to show progress as TAP and failures inline. This allows to continue test runs after a failure while maintaining a minimal output.
Reporting
Test reports are stored in .logs/behat
directory.
CI usually uses them to to track test performance and stability.
Boilerplate
Vortex provides BDD tests boilerplate for homepage and login user journeys.
These boilerplate tests run in CI when you install Vortex and can be used as a starting point for writing your own.