April 22, 2025

Detect Upstream Changes: Weekly Module Testing with GitLab CI

by Jürgen Haas

Leveraging GitLab Templates for Proactive Module Testing

Keeping your Drupal modules aligned with the ever-evolving Drupal ecosystem is crucial for stability and maintainability. But manually testing module compatibility against various versions of Drupal core and other dependencies can be a time-consuming and error-prone process. Fortunately, Drupal offers a powerful solution: the GitLab CI Templates. These templates provide a pre-configured environment for running automated tests, ensuring your modules are ready for any upgrade or change.

The GitLab CI Templates – Your Foundation for Automated Testing

The GitLab CI Templates are a collection of pre-built configuration files designed to streamline the testing process for Drupal modules. They provide a robust framework for running various tests, all within a consistent and reliable environment.

What Can the Templates Do?

These templates are designed to cover a comprehensive range of testing scenarios, including:

Testing Against Multiple Drupal Versions: This is the core benefit. The templates are configured to run tests against the current version of Drupal core, as well as previous and future minor and major versions of Drupal core. This allows you to proactively identify compatibility issues before they impact your project.
- Testing Against Future Versions: Ensures your code works with the upcoming features and is prepared for deprecations.
- Testing Against Past Versions: Helps with backward compatibility if you need to maintain support for older Drupal versions.
Comprehensive Test Types: The templates include support for:
- Linting: Static analysis to identify code style violations.
- Static Code Analysis: More in-depth analysis for potential bugs and vulnerabilities.
- Unit Tests: Ensure individual components of your module function correctly.
- And More: You can expand the testing suite to include integration tests, functional tests, and other tests as needed.

Simple Setup – Ready for MRs, Commits, and Releases

The best part? These templates are incredibly easy to set up. They’re configured to automatically run whenever:

Merge Requests are created or updated: Catch potential issues before merging code.
Commits are made to your development branches: Ensure your changes don’t break existing functionality.
New releases of your module are published: Validate that the new version integrates seamlessly with Drupal core and other modules.

To make this work for your module, you only have to create a file called .gitlab-ci.yml in the root directory of the module’s code. It should contain these lines:

include:
  - project: $_GITLAB_TEMPLATES_REPO
    ref: $_GITLAB_TEMPLATES_REF
    file:
      - '/includes/include.drupalci.main.yml'
      - '/includes/include.drupalci.variables.yml'
      - '/includes/include.drupalci.workflows.yml'

That’s all you need to adopt all the defaults from the templates that are maintained by the community. In some cases, you may have to adjust the settings towards your own needs, and all the available options can be found in the GitLab CI Templates Documentation.

Recommended config changes

Over the years, we’ve learned that automated tests are only really valuable with the following adjustments:

Do not allow failure for PHPStan, PHPCS, CSpell
Increase the PHPStan level to 6

Disallowing test failures, which means that you get alerted if anything fails, can be achieved by altering the respective tasks in your .gitlab-ci.yml file:

cspell:
  allow_failure: false
phpcs:
  allow_failure: false
phpstan:
  allow_failure: false
phpstan (next minor):
  allow_failure: false
phpstan (next major):
  allow_failure: false

To increase the level for PHPStan, you need a file called phpstan.neon in the root of your module with the following instructions:

parameters:
  level: 6

More instructions may be following in that file, depending on the nature of your module and what types of warnings you may see and will want to ignore, probably.

Stable Modules and Unnoticed Component Updates

Maintaining compatibility of your Drupal modules is a continuous effort. Modules that haven’t seen updates for a significant period – we’ll refer to them as stable or mature – can quickly fall behind the current Drupal ecosystem. Without regular testing, subtle changes in Drupal core or other modules can lead to unnoticed component updates – issues that aren’t immediately apparent but can cause problems down the line. This chapter explores why these modules need extra attention and how to proactively identify and address these potential issues.

Why Stable Modules Need Extra Attention

Modules that haven’t received recent updates often rely on older versions of Drupal core or other modules. This creates a dependency chain, and even small changes in any part of the chain can create compatibility issues. The longer a module remains untouched, the more likely it is that its dependencies have evolved, and the greater the risk of unnoticed component updates.

Unnoticed Component Updates – The Risks

Without regular testing, these updates can manifest as:

Behavioural Changes: A module’s functionality might change subtly due to a newer version of Drupal core.
Deprecated Functionality: Function calls or APIs that were previously supported might be removed in newer versions.
Security Vulnerabilities: Older modules may not be patched against newly discovered security vulnerabilities.

Proactive Testing is Key

To mitigate these risks, it’s crucial to regularly test stable module releases against the current and future versions of Drupal core. This is where the GitLab CI Templates come in again.

An extra bonus from this approach comes in when there is a new merge in the issue queue that fixes something in your module, and the pipelines suddenly show failures that are totally unrelated to that fix. Had the module been tested and new failure been fixed proactively, that wouldn't have happened.

Weekly Scheduled Tests

Testing your Drupal modules shouldn’t be a one-off event triggered by a new commit. To truly ensure long-term stability and prevent unnoticed component updates, you need a consistent, automated testing process. This chapter outlines how to set up weekly scheduled tests using the GitLab CI templates.

Setting Up Weekly Scheduled Tests via the GitLab UI

The good news is that setting up these weekly tests is incredibly simple. Here’s how:

Navigate to your module’s GitLab page, e.g. https://git.drupalcode.org/project/eca (replace eca with the short name of your own module)
Find Pipeline Schedules in the “Build” section of the left toolbar
Configure the Schedule: Click the “New schedule” button to create a new scheduled pipeline. Here, you can define the frequency, time of day, and branch that should be tested.

GitLab will perform the same tests as already described above by using the already available configuration in the .gitlab-ci.yml file of the configured branch.

Applying the Approach to Private Custom Modules

The automated testing process we’ve outlined so far is primarily designed for modules that have been released to Drupal. However, the same principles – consistent testing and proactive detection of compatibility issues – apply equally to private custom modules. This chapter demonstrates how to extend this approach to your own private module development. It only requires your own instance of GitLab, where you can keep custom modules private, while still using the same infrastructure components for pipelines.

To make this work, add a .gitlab-ci.yml file to your custom module on your own GitLab instance and copy the following content to it:

include:
  - remote: 'https://gitlab.lakedrops.com/gitlab-ci-cd/drupal/-/raw/main/private-modules.yml?ref_type=heads'

This is using the template from our Drupal CI project. You can either use that directly or clone that project to your own GitLab instance and maintain it there yourself.

In addition to the CI configuration, you also have to provide your own GitLab runners, as we can’t use the infrastructure of the Drupal Association to run our private tests. To set up a runner for your project, go to the CI/CD settings of that project on your GitLab instance and follow the instructions in the “Runners” section. Simply use Docker as the runner executer and any PHP image as a starting point. The CI templates will bring their own docker images that are required to run all the tests.

This setup will test your private custom modules the same way as regular Drupal modules on drupal.org, and if you also configure weekly scheduled pipelines, you’ll benefit from those regular tests in the same way.

The Benefits of a Unified Approach

Using the same template for both public and private modules ensures a consistent and robust testing workflow, regardless of the module’s scope and status.

Conclusion

A Holistic Approach to Drupal Module Development

Throughout this guide, we’ve focused on establishing a robust testing framework for Drupal modules – both public and private. By implementing automated weekly tests and leveraging the GitLab CI/CD templates, you’ve equipped yourself with a powerful system for ensuring code quality, compatibility, and stability.

Looking Ahead – Real Drupal Site Dependency Testing

However, testing your modules is just one piece of the puzzle. Truly proactive development requires a real-time understanding of your module’s health in a full-blown environment. That’s where the concept of reference sites comes in.

To learn more about setting up and managing reference sites for real-time monitoring and dependency testing, please refer to our companion article: Real Drupal Site Dependency Testing – A Strategic Approach

Resources & Further Reading

Thank you for completing this guide! We hope you’ve gained valuable insights and practical techniques for improving your Drupal module development workflow.

Disclosure: I've used the Gemma3 LLM in Ollama to build the structure and create some content suggestions based on my own understanding of the problem space and the solutions that we've implemented. I've also used Deepl and LanguageTools for spelling and grammar improvements. Writing the article itself I consider my own work, and AI has simply been a supporting tool.