Backwards Compatibility

Burden & Benefit

Wim Leers

Principal Software Engineer, OCTO, Acquia logo

Things I learned so I know slightly better what I do not know

I'm responsible for the πŸ‘ & πŸ‘Ž.

My analysis in a nutshell

can be used for anything!


All code must have an API to be generic & overridable
but not enough time to carefully design every API


… yet underengineered!


BC = nightmare. Let's get better.

BC = promise of updating without problems


Q: Why does Drupal have so many APIs?
A: Optimized for targeted overrides
    β‡’ granular APIs
    β‡’ many, many APIs

  • forest ~ Drupal
  • tree ~ Drupal component
  • branch ~ Drupal component feature
  • leaf ~ Drupal component feature method

Drupal allows you to replace a particular leaf.
Others require replacing a branch or even tree.

Drupal has 3 types of APIs

Explicit APIs
hooks, plugins, tagged services
Implicit APIs
markup structure, render array structure, call order (weights, priorities)
Accidental APIs
many (most?) interfaces (and even classes!)

d.o/core/d8-bc-policy: @api vs @internal

But in Drupal core…

0 occurrences
53 occurrences

99% "undocumented"
β‡’ 99% considered API

APIs elsewhere?

Screenshot listing WordPress APIs
Screenshot listing Django APIs

The API assumption

  • Drupal: X is API
  • Others: X is NOT an API


Part 1: Accidental API

Example of an accidental API

  1. D8: OOPify everything
  2. Every class must have an interface
  3. Interfaces coupled to the sole implementation
  4. BC broken by bugfixes & new implementations

Issue #2266809: Make QuickEditEntityFieldAccessCheck::access() use the $account that's passed in

    * @param string $field_name
    *   The field name.
+   * @param \Drupal\Core\Session\AccountInterface $account
+   *   The user for which to check access.
    * @return \Drupal\Core\Access\AccessResultInterface
    *   The access result.
-  public fn accessEditEntityField($entity, $field_name);
+  public fn accessEditEntityField($entity, $field_name, AccountInterface $account);


(Introduced by yours truly in #1824500: In-place editing for Fields on Dec 21, 2012.
Suffering the consequences >4 years later.)

Poorly designed APIs make BC very difficult

API support cost

Prefer duplication over the wrong abstraction

API when:

  1. data to prove soundness of API design
  2. sufficient demand

API discoverability & complexity

Little work + high complexity (granular APIs)


More work + low complexity (duplication)


Part 2: orthogonality

[…] how a relatively small number of components can be combined in a relatively small number of ways to get the desired results. It is associated with simplicity; the more orthogonal the design, the fewer exceptions. This makes it easier to learn […]

API dependencies

  • Automated Cron: 5 (Form + Config + EventSub + Cron + State)
  • BigPipe: 5 (Render + AJAX + Cache + EventSub + Req/Resp)
  • Entity API uses >10 APIs
β‡’ using Entity API === using >10 APIs!

API cascades

NodeInterface $node

1. NodeInterface extends ContentEntityInterface, EntityChangedInterface, EntityOwnerInterface, RevisionLogInterface, EntityPublishedInterface
2. ContentEntityInterface extends \Traversable, FieldableEntityInterface, RevisionableInterface, TranslatableInterface
3. FieldableEntityInterface extends EntityInterface
4. EntityInterface extends AccessibleInterface, CacheableDependencyInterface, RefinableCacheableDependencyInterface

FieldableEntityInterface::get() aka $node->get($field_name)

1. FieldItemListInterface extends ListInterface, AccessibleInterface
2. ListInterface extends TraversableTypedDataInterface, \ArrayAccess, \Countable
3. TraversableTypedDataInterface extends TypedDataInterface, \Traversable
4. TypedDataInterface

AKA a rabbit hole

Rabbit hole example

  • REST module: Taxonomy term REST responses don't list parent term, because:
  • Taxonomy module does not define parent the proper way in Entity API, but cannot be fixed because:
  • Views module breaks for multi-value base Fields on Entities

AKA a deep rabbit hole

Massive composition & long inheritance chains require massive knowledge


Part 3: assumptions

  1. Drupal core does X
  2. Module/API Foo assumes X
  3. Install contrib module Bar: X β‡’ Y
  4. 😭
Zero config, no UI!

Breaks when advagg is installed 😭

     $request = $this->requestStack->getCurrentRequest();
     $link_headers = $request->attributes->get('http2_server_push_link_headers', []);
     foreach ($elements as &$element) {
+      if (!static::isLinkRelStylesheet($element)) {
+        continue;
+      }
       // Locally served CSS files that are sent to all browsers can be pushed.
-      if ($element['#tag'] === 'link' && $element['#browsers']['!IE'] === TRUE && $element['#browsers']['IE'] === TRUE && $element['#attributes']['href'][0] === '/' && $element['#attributes']['href'][1] !== '/') {
+      if (isset($element['#attributes']['href']) && static::hasRootRelativeUrl($element, 'href') && static::isUnconditional($element)) {
         $link_header_value = '<' . $element['#attributes']['href'] . '>; rel=preload; as=style';
         $link_headers[] = $link_header_value;
Make assumptions explicit

even better:

Test assumptions

Extreme one end

REST + Serialization

REST in Drupal 8.0.0

  • Only thin happy path test coverage
  • Slightest mistake β‡’ incomprehensible errors 😩😨

REST module: what + API?

  1. Server side: define more REST resource plugins (PHP), deploy some (config)
  2. Client side: HTTP API, so: URL + request/response headers/body

Wim in 2016

  • bugfixingβ€¦πŸ˜β€¦πŸ˜©β€¦πŸ˜‘
  • comprehensive test coverage testing every mistake an end user might make 😡 β‡’ uncovered dozens more bugs…
  • bugfixing++ πŸ™‚

No more incomprehensible errors, nor BC breaks!

😡 "serialization gaps" 😡

  • #2751325: All serialized values are strings, should be integers/booleans when appropriate
  • #2543726: Expose $term->parent in serialized taxonomy terms
  • … 6 more in #2852860.
Response === API β‡’ extreme care
When API surface is great, test coverage must be greater


Clearly define what isΒ an API (and hence BC guaranteed)

Extreme other end

BigPipe + Dynamic Page Cache

Dynamic Page Cache: what + API?

  • Reverse proxy (transparently, much like cache)
  • Inspects metadata (on Responses) β‡’ act or not
  • X-Drupal-Dynamic-Cache response header
  • No API!
1 task + 3 minor (clean-up) bugs + 1 support request.
5 issues total for millions of responses accelerated!

BigPipe: what + API?

  • Alternative HTML rendering+delivery technique
  • Inspects metadata (on Responses) β‡’ act or not
  • Surrogate-Control response header
  • No API!
  • Critical functionality β‡’ vast test coverage!
Explicitly functionality, not API. @internal all the things!
Handful of bugs over the course of a year. 0 bugs last 6 months.
Try hard to not provide an API

(Then BC is kept as long as functionality works!)

Contrib: organic feature growth

Hierarchical Select 7.x-3.0

CDN 7.x-2.9

CDN 8.x-3.0

Functionality first, API later
or @internal first, @api later
Prefer duplication over the wrong abstraction
Test 1) critical path, 2) edge cases, 3) assumptions
(especially for APIs)