The ultimate subscription and notification solution for Drupal

Current situation

Almost all websites have one requirement in common: getting notified when something specific is happening. Whether that's a new blog post, a new comment, a modified version of a node, an exception in the logs or a deleted user account, either the site admin or website visitors would like to know about some or all of these.

While this requirement is pretty common, each and every individual has very different views on how they want to get notified. Without getting into any discussion on effectiveness of emails, it seems enough to just think about all these other available channels like Slack, mobile push, ticket systems or even SMS.

Building websites with Drupal makes you wonder, why the solution stack to those requirements is that fragmented as it has been for over a decade. For Drupal 8 and 9 alone you'll find over 100 modules for the "notify" search term. At least 40 of them should be considered serious solutions as they have stable releases and security advisory coverage. But each of them only ever addresses one single aspect, i.e. only nodes or only comments or workflow states or exceptions or what ever. And then, most of them only offer email notifications for each of the event types they've been built for.

Our solution

As this is not appropriate for an enterprise grade web application framework like Drupal, it's been time to sit down and address this space in a generic, scalable, configurable and maintainable fashion such that the community can build upon it and be ready for current and future requirements - because the subscription and notification requirements are under permanent pressure to adjust and improve, because the world around us is changing all the time as well.

The team at bitegra Solutions in Germany spent almost 2 years in problem analysis, solution design, software architecture and development to make available the ultimate solution for Drupal. We've decided to publish all available components on drupal.org and let the community participate in the benefits but also future maintenance and development of existing and additional features and notification channels for free.

We take you on a journey with us

This is part 1 of a blog post series which introduces the idea and implementation and how you can use them yourselves on Drupal projects of any size:

• Part 1 (you are here): Introduction and system architecture
• Part 2: Getting started with subscriptions and notifications
• Part 3: How to add new notification channels
• Part 4: Covering more events for being notified
• Part 5: Customizing the notification output

System architecture

Let's get started with a list of requirements:

• Support everything notable:
  • Content, i.e. entities:
    • Create, update, delete and pusblish/unpublish
    • Every entity type and bundle
    • Each bundle with individual and detailed settings
  • Configuration: track all changes to the site's configuration
  • Logger: track all log entries above a configured threashold
  • External events: allow other services in the enterprise network to notify the same audience
  • Allow developers to provide new plugins to create notifications
• Be flexible with recipients
  • Authenticated users should be able to subscribe/unsubscribe to/from any available event configured on the site and made available for subscription
  • Site configuration should allow pre-selection of recipients for each available event type
  • Content editors with appropriate permission should be able to overwrite notification defaults per entity action (create, update, delete, publish, unpublish)
  • Allow developers to apply custom rules on recipient selection
• Support multiple notification channels
  • Allow site builders to activate any number of different notification channels at once
  • Allow users to define their own preferences on channel priority
  • Make sure that each user only gets notified once for each event
  • Ensure and trace notification delivery
  • Provide good defaults for notification rendering per channel
  • Allow site builders to modify the notification output
  • Allow developers to add new channels

While collecting and structuring the requirements, it became obvious that there are 3 isolated phases that need to be addressed separately:

1. Event capturing
2. Recipient selection and notification creation
3. Notification delivery

Yes, phase 2 contains two steps: for each notable event all relevant recipients need to be determined, and for each of them a notification entity gets created in order to further handle that notification for each recipient individually. 

Bild

This led to another major decision in the solution design: capturing events and selecting recipients for notifications is one major building block, and delivering those notifications to users over a number of channels is another one. Both building blocks can operate independently and hence we broke this down into 2 separate modules:

DANSE: Notification Framework

The DANSE module (DANSE: Drupal - Audit - Notification - Subscription - Event) addresses the first two phases: event capturing and recipient selection. Both phases are implemented with their own plugin managers and hence can easily be enhanced to support additional requirements.

Out of the box, DANSE supports event capturing for content entities, configuration, logging and webhooks (external events). Each of those event types are implemented in separate submodules so that site builders can enable just those actually needed.

The recipient selection plugin manager allows to implement your own business logic but brings 2 plugins already with it: "all admin users" and "all active users". For the content entity event capturing plugin above, site builders can also configure on a per-bundle basis, which user roles are allowed to subscribe to supported event types - available subscriptions can be recognized as a recipient selection plugin in its own right.

DANSE comes with 3 entity types to handle its duties:

• Events (danse_event): each notable event gets stored with its creation timestamp and further details depending on the event type
• Notification (danse_notification): for each recipient of each event a single notification gets stored
• Action (danse_notification_action): to track the activity around that notification including an API to store delivery attempts and the receipt flag

As a result, this module allows you to audit the activity on your website by reviewing all these entities for events, notifications and actions so that you can easily tell what happened when and why. Of course, it also stores who got notified how and when. So, that made the final piece of the module name, just in cased you asked yourself: audit.

An extra benefit is the notification widget that comes with DANSE. Without even delivering notifications out to the recipients, there is a huge value in telling your users what's new and changed on the site. The stored notifications allow you to do that easily. DANSE comes with a view for unseen notifications and you can place that view as a block on the site for authenticated users.

With this in mind, DANSE can be used on its own for any existing Drupal 8 and Drupal 9 site.

Push Framework

In order to also send notifications out to their recipients, the Push Framework module comes in handy. It hooks into the API provided by DANSE but can also be used together with any other source of information that should be delivered. To make this possible, the push framework comes with a source plugin manager and DANSE is just the first module which implemented a plugin for this. But of course, other sources can be provided easily like e.g. a newsletter module.

The push framework module handles all the logic of collecting the source content which is due for delivery, determining the appropriate channel for delivery which depends on available channels and user preferences, and then delivers the notification and reports back to the source on the delivery success. If delivery fails, each channel can support any number of retries and continue with another channel, if all retries failed.

The output channels are made available through the channel plugin manager which allows developing additional channels very easily. Currently supported (or planned) channels are:

• Email
• OneSignal
• Alerta
• Slack (planned)
• Mattermost (planned)
• Twilio (planned)

Diagram of the overall solution

Both of the above modules make intensive use of queues, i.e. all the heavy lifting is operated asynchronously. That helps to ensure reliability of both event capturing and recipient selection as well as delivery - without any impact on the websites performance.

Bild

Both DANSE and Push Framework have been carefully designed to provide a rock solid solution for all Drupal sites of any size. It comes with a common feature set for (a) event types which create notifications and (b) notification channels. But we are well aware that lots of sites have their own business logic for most of this solution space - even our own first customer project, where we used this, had such a specific requirement - which can still be satisfied by developing custom plugins or public modules, if the extra feature were of general interest.

The subsequent parts of this blog post series will give you all the insight into what's required to use these modules and how to enhance them towards your own requirements.

Update 8/11/2020: Due to an issue with the old module name, we've changed the name and updated this blog post accordingly.