On simplicity & maintainability: CDN module for Drupal 8

The first release of the CDN module for Drupal was 9.5 years ago yesterday: cdn 5.x-1.0-beta1 was released on January 8, 2008!

Excitement

On January 27, 2008, the first RC followed, with boatloads of new features. Over the years, it was ported to Drupal 6¹, 7 and 8 and gained more features (I effectively added every single feature that was requested — I loved empowering the site builder). I did the same with my Hierarchical Select module.

I was a Computer Science student for the first half of those 9.5 years, and it was super exciting to see people actually use my code on hundreds, thousands and even tens of thousands of sites! In stark contrast with the assignments at university, where the results were graded, then discarded.

Frustration

Unfortunately this approach resulted in feature-rich modules, with complex UIs to configure them, and many, many bug reports and support requests, because they were so brittle and confusing. Rather than making the 80% case simple, I supported 99% of needed features, and made things confusing and complex for 100% of the users.

Learning

In my job in Acquia’s Office of the CTO, my job is effectively “make Drupal better & faster”.

In 2012–2013, it was improving the authoring experience by adding in-place editing and tightly integrating CKEditor. Then it shifted in 2014 and 2015 to “make Drupal 8 shippable”, first by working on the cache system, then on the render pipeline and finally on the intersection of both: Dynamic Page Cache and BigPipe. After Drupal 8 shipped at the end of 2015, the next thing became “improve Drupal 8’s REST APIs”, which grew into the API-First Initiative.

All this time (5 years already!), I’ve been helping to build Drupal itself (the system, the APIs, the infrastructure, the overarching architecture), and have seen the long-term consequences from both up close and afar: the concepts required to understand how it all works, the APIs to extend, override and plug in to. In that half decade, I’ve often cursed past commits, including my own!

That’s what led to:

• my insistence that the dynamic_page_cache and big_pipe modules in Drupal 8 core do not have a UI, nor any configuration, and rely entirely on existing APIs and metadata to do their thing (with only a handful of bug reports in >18 months!)
• my “Backwards Compatibility: Burden & Benefit” talk a few months ago
• and of course this rewrite of the CDN module

CDN module in Drupal 8: radically simpler

I started porting the CDN module to Drupal 8 in March 2016 — a few months after the release of Drupal 8. It is much simpler to use (just look at the UI). It has less overhead (the UI is in a separate module, the altering of file URLs has far simpler logic). It has lower technical complexity (File Conveyor support was dropped, it no longer needs to detect HTTP vs HTTPS: it always uses protocol-relative URLs, less unnecessary configurability, the farfuture functionality no longer tries to generate file and no longer has extremely detailed configurability).

In other words: the CDN module in Drupal 8 is much simpler. And has much better test coverage too. (You can see this in the tarball size too: it’s about half of the Drupal 7 version of the module, despite significantly more test coverage!)

all the fundamentals

• the ability to use simple CDN mappings, including conditional ones depending on file extensions, auto-balancing, and complex combinations of all of the above
• preconnecting (and DNS prefetching for older browsers)
• a simple UI to set it up — in fact, much simpler than before!

changed/improved

1. the CDN module now always uses protocol-relative URLs, which means there’s no more need to distinguish between HTTP and HTTPS, which simplifies a lot
2. the UI is now a separate module
3. the UI is optional: for power users there is a sensible configuration structure with strict config schema validation
4. complete unit test coverage of the heart of the CDN module, thanks to D8’s improved architecture
5. preconnecting (and DNS prefetching) using headers rather than tags in , which allows a much simpler/cleaner Symfony response subscriber
6. tours instead of advanced help, which very often was ignored
7. there is nothing to configure for the SEO (duplicate content prevention) feature anymore
8. nor is there anything to configure for the Forever cacheable files feature anymore (named Far Future expiration in Drupal 7), and it’s a lot more robust

removed

1. File Conveyor support
2. separate HTTPS mapping (also mentioned above)
3. all the exceptions (blacklist, whitelist, based on Drupal path, file path…) — all of them are a maintenance/debugging/cacheability nightmare
4. configurability of SEO feature
5. configurability of unique file identifiers for the Forever cacheable files feature
6. testing mode

For very complex mappings, you must manipulate cdn.settings.yml — there’s inline documentation with examples there. Those who need the complex setups don’t mind reading three commented examples in a YAML file. This used to be configurable through the UI, but it also was possible to configure it “incorrectly”, resulting in broken sites — that’s no longer possible.

There’s comprehensive test coverage for everything in the critical path, and basic integration test coverage. Together, they ensure peace of mind, and uncover bugs in the next minor Drupal 8 release: BC breaks are detected early and automatically.

The results after 8 months: contributed module maintainer bliss

The first stable release of the CDN module for Drupal 8 was published on December 2, 2016. Today, I released the first patch release: cdn 8.x-3.1. The change log is tiny: a PHP notice fixed, two minor automated testing infrastructure problems fixed, and two new minor features added.

We can now compare the Drupal 7 and 8 versions of the CDN module:

• 149 support requests for the Drupal 7 version, with 14 in the last 12 months (the module is stable now after all these years of course) and 83 bug reports over 6.5 years (78 months), with ~6000 sites using it.
• 7 support requests for the Drupal 8 version in the last 8 months and 1 bug report (a bug in a test). With ~500 sites using it.

In other words: maintaining this contributed module now requires pretty much zero effort!

Conclusion

For your own Drupal 8 modules, no matter if they’re contributed or custom, I recommend a few key rules:

• Selective feature set.
• Comprehensive unit test coverage for critical code paths (UnitTestCase)² + basic integration test coverage (BrowserTestBase) maximizes confidence while minimizing time spent.
• Don’t provide/build APIs (that includes hooks) unless you see a strong use case for it. Prefer coarse over granular APIs unless you’re absolutely certain.
• Avoid configurability if possible. Otherwise, use config schemas to your advantage, provide a simple UI for the 80% use case. Leave the rest to contrib/custom modules.

This is more empowering for the Drupal site builder persona, because they can’t shoot themselves in the foot anymore. It’s no longer necessary to learn the complex edge cases in each contributed module’s domain, because they’re no longer exposed in the UI. In other words: domain complexities no longer leak into the UI.

At the same time, it hugely decreases the risk of burnout in module maintainers!

And of course: use the CDN module, it’s rock solid! :)

Related reading

Finally, read Amitai Burstein’s “OG8 Development Mindset”! He makes very similar observations, albeit about a much bigger contributed module (Organic Groups). Some of my favorite quotes:

1. About edge cases & complexity:

   Edge cases are no longer my concern. I mean, I’m making sure that edge cases can be done and the API will cater to it, but I won’t go too far and implement them. […] we’ve somewhat reduced the flexibility in order to reduce the complexity; but while doing so, made sure edge cases can still hook into the process.

2. About tests:

   I think there is another hidden merit in tests. By taking the time to carefully go over your own code - and using it - you give yourself some pause to think about the necessity of your recently added code. Do you really need it? If you are not afraid of writing code and then throwing it out the window, and you are true to yourself, you can create a better, less complex, and polished module.

3. About feature set & UI:

   One of the mistakes that I feel made in OG7 was exposing a lot of the advanced functionality in the UI. […] But these are all advanced use cases. When thinking about how to port them to OG8, I think found the perfect solution: we did’t port it.