renderviz: tool visualizing Drupal 8's render tree

published on April 7, 2015

I’m working on making Drupal 8 faster as part of my job at Acquia. The focus has been on render caching12, which implies that cacheability metadata is of vital importance in Drupal 8.

To be able to render cache all things that can possibly be render cached, Drupal 8 code must:

  • set the right cache max-age — to ensure only the cacheable parts of the page are cached
  • set the right cache contexts — to ensure content is varied as expected (per language, per role, per timezone, per user …)
  • set the right cache tags — to ensure rendered content is invalidated when the data it depends on is modified

Before Drupal 8, approximately zero attention was given to cacheability of the rendered content: everything seen on a Drupal 7 page is rendered dynamically, with only the occasional exception.

By flipping that around, we make developers more conscious about the output they’re generating, and how much time it takes to generate that output. This in turn allows Drupal 8 to automatically apply powerful performance optimizations, such as:

  1. enabling Drupal’s internal page cache (for anonymous users) by default: d.o/node/606840, which requires cache tags to be correct
  2. smartly caching partial pages for all users (including authenticated users): d.o/node/2429617, which requires cache contexts to be correct
  3. sending the dynamic, uncacheable parts of the page via a BigPipe-like mechanism: d.o/node/2429287

(The first of those three will likely happen this week. We’re working hard to make the last two a reality.)

Visualization

Caching means better performance, but it also means that without the correct cacheability metadata, the wrong content may be served to end users: without the right cache contexts, the wrong variation may be sent to a user; without the right cache tags, stale content may be sent to a user. Therefore we should make it as easy as possible to analyze the cacheability of a rendered block, entity (node/user/taxonomy term/…), view, region, menu, and so on.

It should work not only for cacheability metadata, but for all bubbleable metadata3: it’d be very valuable to be able to see which part of the page caused an expensive cache context or tag 4, but it’d be at least equally valuable to see which part of the page attached a certain asset5.

Since bubbling happens across a tree, it’s important to visualize the hierarchy. The best hierarchy visualization I know in the web developer world is the Firefox Developer Tools 3D view.

Firefox 3D view of wimleers.com

I think a tool for visualizing, analyzing and understanding the bubbleable metadata (cache contexts, cache tags, cache max-age, assets) should work in a similar way. The developer should be able to:

  • look at the document in 3D in different layers and/or queries (assets, cacheability as a whole, but also only cache contexts, only cache tags, or only max-age)
  • zoom in a specific element, and look at all of its bubbleable metadata
  • reposition the 3D view, to look from different angles — humans are very proficient at processing visual data

Prototype

So, over the past weekend, I worked on a prototype. I read the CSS Transforms spec6 and read a CSS 3D transforms introduction. As somebody with little CSS knowledge and not having touched CSS nor 3D programming in years, it was fun to play with this :) The result:

renderviz prototype, visualizing the 'timezone' cache context.

And finally, a short screencast demonstrating it in action:

Give it a try yourself by applying the attached patch to Drupal 8 at commit daf9e2c509149441d4d9a4d1964895179a84a12c and installing the renderviz module.

Want to help?

There are many rough edges — actually there are only rough edges. Everything needs work: CSS, JavaScript, UI (there isn’t any yet!), even the name!

But it’s a lot of fun to work on, and it’s very different from what most of us tend to work on every day. If you’d like to be able to build sites in Drupal 8 with a developer tool like this, please contact me, or leave a comment :)

  1. Avoiding to render exactly the same chunks of HTML endlessly on every request. See d.o/developing/api/8/render/arrays/cacheability. ↩︎

  2. For more about that, see the Render caching in Drupal 7 and 8 talk I did with Fabian Franz & Marco Molinari at DrupalCon Amsterdam. ↩︎

  3. Drupal is all about reusable content and reusable components. That’s why starting in Drupal 8, we don’t attach assets at the page-level (i.e. global), but we attach them to the places where we actually need them (e.g. when rendering a taxonomy term, we attach the assets to style the taxonomy term to the taxonomy term’s render array). They then “bubble” the render tree, just like JavaScript events bubble the DOM tree. The assets bubble all the way to the response level, i.e. to a HTML response’s <head> element. Similarly, cache tags and contexts bubble to a response’s X-Drupal-Cache-Contexts and X-Drupal-Cache-Tags headers. ↩︎

  4. A cache tag is expensive if it’s invalidated relatively frequently (which causes all render cache items that have that tag to be invalidated). A cache context is expensive if it causes many variations (for example: per-user caching requires a variation of the render array to be created for every single authenticated user). ↩︎

  5. In Drupal 8, all assets are defined in asset libraries, which can contain any number of CSS or JS assets, and which can depend on other asset libraries. See d.o/theme-guide/8/assets. ↩︎

  6. Firefox’ 3D view is built using WebGL, not CSS Transforms. We might eventually need that too, but not just yet. Oh, and for the origins of Firefox’ 3D view, see https://github.com/victorporof/Tilt. ↩︎

Comments

Wim Leers's picture
Wim Leers

Hah, I forgot to mention it in the blog post, but I was definitely thinking about devel_themer! I even looked at its code to see how it was gathering its metadata. This can’t use the same approach, but fortunately, at the same time, the metadata we need is much simpler to gather.

So: yes, definitely, this should most definitely learn from devel_themer!

Wim Leers's picture
Wim Leers

It’s only available as a patch, attached to this very blog post :)

Looking for people that want to help with this before I publish it as a module … and for a better name :)

Clemens Tolboom's picture
Clemens Tolboom

LOL. I missed the modules/renderviz directory as my .gitignore has modules in it.

git apply ~/Downloads/renderviz-prototype.patch git diff

did not show the module. (doh)

Ivan Tsekhmistro's picture
Ivan Tsekhmistro

Here is an updated patch-attached, compatible with current latest Drupal 8.1.2

Note, there cache debug info is rendered with one exclusion: // W3C documentation: HTML Comments are markup, // thus it's not allowed to use HTML Comments inside HTML element attributes. // Example: >a link is as wrong as >a link. // Following condition is applied to avoid breaking HTML structure. // In result elements that are used as string value of the parent element attribute are skipped. // (Example, form_token value).

jedihe's picture
jedihe

Just wanted to say thanks for sharing this; it allowed me to find why logged-in users were getting uncacheable responses from Dynamic Page Cache, but anonymous users were working fine; it ended up being the token in a form, which I managed to resolve by using a simple #lazy_builder instead of a direct call to form builder service. Again, thanks!

Ivan's picture
Ivan

This is great idea to add cache info to the page render! You’ve safe my day. Thanks a lot!