Media embedding in Drupal 8.8

published on September 3, 2019

2007 is the year of my first DrupalCon, and the year the #1 most wanted end-user feature was Better media handling. 2019 is the year that Drupal will finally have it. Doing things right takes time!

Back then I never would’ve believed I would someday play a small role in making it happen :)

Without further ado, and without using a mouse:

Reusing and embedding media, using only the keyboard.

The text editor assisted in producing this HTML:

<p>Let's talk about llamas!</p>

<drupal-media alt="A beautiful llama!" data-align="center" data-entity-type="media" data-entity-uuid="84911dc4-c086-4781-afc3-eb49b7380ff5"></drupal-media>

<p>(I like llamas, okay?)</p>

If you’re wondering why something seemingly so simple could have taken such a long time, read on for a little bit of Drupal history! (By no means a complete history.)

2007 and Drupal five

Twelve years ago, in Dries’ State of Drupal talk 1, Better media handling was deemed super important. I attended a session about it — this is the (verbatim) session description:

  • Drupal’s core features for file management and media handling
  • common problems and requirements (restrictions, performance issues, multi-lingual content, dependencies between nodes and files)
  • first approaches: own node types for managing, improved filemananger.module (example: Bloomstreet,European Resistance Archive, Director’s Cut Commercials)
  • next step: generic media management module with pluggable media types, mutli server infrastructure, different protocols, file systems, file encoding/transcoding

It’s surprisingly relevant today.

By the way, you can still look at the session’s slides or even watch it!

2007–2013 (?)

The era of the venerable Media module, from which many lessons were learned, but which never quite reached the required level of usability for inclusion in Drupal core.

2013 (?) – 2019

The Media initiative started around 2013 (I think?), with the Media entity module as the first area of focus. After years of monumental work by many hundreds of Drupal contributors (yes, really!), only one missing puzzle piece was left: WYSIWYG embedding of media. The first thing I worked on after joining Acquia was shipping a WYSIWYG editor with Drupal 8.0, so I was asked to take this on.

To help you understand the massive scale of the Media Initiative: this last puzzle piece represents only the last few percent of work!

Drupal has always focused on content modeling and structured content. WYSIWYG embedding of media should not result in blobs of HTML being embedded. So we’re using domain-specific markup (<drupal-media>) to continue to respect structured content principles. The result is document transclusion combined with an assistive “WYSIWYG” editing UX — which we wished for in 2013.

A little less than two months ago, we added the MediaEmbed text filter to Drupal 8.8 (domain-specific markup), then we made those have previews using CKEditor Widgets for assistive “WYSIWYG” editing, followed by media library integration and per-embed metadata overriding (for example overriding alt, as shown in the screencast).

I was responsible for coming up with an architecture that addressed all needs. To maximally learn from past lessons, I first worked on stabilizing the contributed Entity Embed module. So this builds on many years worth of work on the Entity Embed module from Dave Reid, cs_shadow and slashrsm! Thanks to the historical backlog of reported issues there, we are able to avoid those same problems in Drupal core. The architecture of the CKEditor integration was designed in close collaboration with Krzysztof Krzton of the CKEditor team (thanks CKSource!), and was also informed by the problems reported against that module. So this stabilization work benefited both Drupal core as well as tens of thousands of existing sites using Entity Embed.

I then was able to “just” lift Drupal core patches out of the Entity Embed module (omitting some features that do not make sense in Drupal core). But it’s phenaproxima (thanks Acquia!), oknate and rainbreaw who got this actually committed to Drupal core!

Complete media management shipped in increments

Fortunately, for many (most?) Drupal 8 sites, this will not require significant rework, only gradual change. Drupal 8.8 will ship with complete media management, but it’ll be the fifth Drupal core release in a little over two years that adds layers of functionality in order to arrive at that complete solution:

  • Drupal 8.4 added foundational Media API support, but still required contributed modules to be able to use it
  • Drupal 8.5 made Media work out-of-the-box
  • Drupal 8.6 added oEmbed support (enabling YouTube videos for example) and added an experimental Media Library
  • Drupal 8.7 made the Media Library UI-complete: bulk uploads, massively improved UX
  • Drupal 8.8 will contain the key thing that was blocking Media Library from being marked stable (non-experimental): WYSIWYG integration

Today is the perfect moment to start looking into adopting it shortly after Drupal 8.8 ships in December!

Thanks to phenaproxima for feedback on this post!


  1. See slide 40! ↩︎

Comments

Gopi Ramanujam's picture
Gopi Ramanujam

Exciting news! How about migration from Drupal 7 file to <drupal-media> tags?

drico's picture
drico

To add some details to this history, from 2007 to 2013 two solutions were available for media handling, the famous Media module and the Scald suite which already enabled reusability of media content and drag and drop of Scald atoms into WYSIWYG-enabled fields. The fusion of the ideas led to the media entity into Drupal 8.

slybud's picture

Such a great leap in Drupal’s history, I’ve been waiting for this moment for years with the Scald team! Great work Wim, thank you for it and this feels like the landing of Apollo 11, what a great timing!

Wim Leers's picture
Wim Leers

I wouldn’t quite compare it to that, but I’m glad your so excited! :D

frank kelly's picture

With Drupal 8, I use the contrib Juicebox module to create photo galleries. These consist of thousands of jpg files (and thumbnail versions of jpg files) spread across dozens of Galleries. I upload the files to a directory structure that I manage using FTP within sites/default/files. If I want to re-use these jpg files within an article content type I use the contrib IMCE module. It lets me browse through my directory structure and insert the image into the article using CKEditor. It works fine.

So, now what happens? Will media library “discover” my existing files and directory structure? Do I leave the existing files where they are and start adding new files to Media Library? Is there a conversion or migration step. My current Galleries depend on a pre-specified set of files in a pre-specified directory structure. If this changes my galleries go ker-plunck.

Wim Leers's picture
Wim Leers

It’s going to be up to https://www.drupal.org/project/juicebox to provide an update path. There actually already is an issue for adding Media support to the Juicebox module, and it’s been open for more than 2.5 years: https://www.drupal.org/project/juicebox/issues/2840494. It also seems that the Juicebox module is not actively maintained: the latest release is a beta, which is more than 2.5 years old too…

Then again, this module seems highly optimized for one particular use case that you could also choose to continue to use this.

There is no simple answer here: because it depends.

The high-level meta issue for migrating into Media is https://www.drupal.org/project/drupal/issues/2930514.

Michelle Cox's picture
Michelle Cox

I use Juicebox and views to make image galleries from core media entities and it works fine. It sounds like that issue was maybe for the contrib version of Media? Works fine with core.

Mike's picture
Mike

Why only media? Same goes for the media gallery module which provides the browser to populate media reference fields. That’s like making moderation only available for nodes.

Wim Leers's picture
Wim Leers

There’s so many reasons to only do embedding of media and not arbitrary entities! This was discussed in detail at https://www.drupal.org/project/drupal/issues/2801307.

Here are some of the high-level reasons:

  • To offer a great embedding UX, you also need a great selection UX. For Media, there’s the Media Library. For all other entity types, there’s only title-based search. Which is fine for some, but definitely not all entity types.
  • There needs to be a sensible embeddable representation of the embedded entity. This is evident for media. For many other entity types, this is not evident. Even for node entities, it’s non-trivial since for example the “full” view mode includes the full comment list and the “teaser” view mode is usually used in listings, not in embeddings (when embedded, things like author and published date are less important).
  • We didn’t want to block bringing embedding of media entities on having embedding of arbitrary entities. But it’s been designed in a forward-compatible way — the generated & stored HTML is highly structured, making it easy for functionality to be expanded in the future.
  • For those who want arbitrary entity embedding, there’s still https://www.drupal.org/project/entity_embed — which has to do some very complex trickery and requires some rather confusing configuration to make arbitrary entity types embeddable.
Joseph Olstad's picture
Joseph Olstad

Nice work Wim, media embedding is finally about to make it into “core”. I haven’t yet tried out 8.8.x however I did recently try the varbase distribution and was very impressed with it’s media handling, very very nice.

https://www.drupal.org/project/varbase

Wim Leers's picture
Wim Leers

I’m glad to see you’re excited about this too! I see that https://www.drupal.org/project/varbase has pioneered integrating the many prior contributed Media modules into a coherent whole. Do you plan to transition it from Entity Browser to Media Library?

Victor's picture
Victor

Congrats !!! , Media is finally here , a few years after Wordpress did it , but importantly we did a million times better. Thanks to everybody in the media team for all the work. The demonstration video with the keyboard is super awesome and a nice touch.

Wim Leers's picture
Wim Leers

Glad you like it so much! :D

Paweł Philipczyk's picture
Paweł Philipczyk

I’m not following Drupal so closely these days but this was one of main things I looked for few years, and finally it’s here! Reading this story brought some good memories :) Congrats and thank you for all the work you and others put into it!

Wim Leers's picture
Wim Leers

Thanks for the kind words and sharing the good memories — and hopefully see you back in the Drupal community soon? :)

Iain Houston's picture
Iain Houston

I have a rather confusing hotch-potch of modules that don’t quite do a good job. Which ones of the following will I no longer need in Drupal 8.8?

  • Embed
  • Entity Embed
  • DropzoneJS entity browser widget
  • Entity Browser
  • Entity Browser IEF
Wim Leers's picture
Wim Leers

Great question!

You won’t need any of those modules anymore :)

  • The Media Library module supersedes Entity Browser + Entity Browser IEF + DropzoneJS entity browser widget. It offers a better UX, but only handles media entities. The 3 modules you listed could handle arbitrary entity types, but because of that could not optimize the UX as much.
  • The Media module’s “Embed media” text filter plus accompanying “Insert from Media Library” CKEditor button provide everything Embed + Entity Embed provided, but again restricted to only Media entities, again to provide a better UX.

The Media and Media Library modules ship with Drupal core. So it’s only a matter of enabling those, adding that filter to your text format and adding that button to your text editor. That’s it :)

phenaproxima's picture
phenaproxima

I wouldn’t say that Media Library supersedes Entity Browser or any of its related modules; I think this has been confusing for people (and disappointing for some). But, the fact of the matter is that Media Library was built to be simple and fairly limited, on purpose. Entity Browser is a LOT more flexible and you can select anything with it. It’s got superpowers. Media Library should work for most standard use cases with a little flexibility, but if you need to really take control and build a very customized UI, or support things that aren’t media, Entity Browser is the way to go. Same thing goes for Entity Embed – Media Library provides the most important media-specific functionality, but Entity Embed is where it’s at for more complicated uses.

Wim Leers's picture
Wim Leers

You’re of course right! Note I did say

but only handles media entities

However, that is indeed still a too simplistic answer from me. If you’re doing “typical” media stuff, then my answer is correct. As soon as you have complex needs, Entity Browser + Entity Embed are indeed still the way to go.

Iain Houston's picture

Thanks Wim. At last I am doing the deed: I have ditched the modules I listed above and mostly I have what I need except for an issue I raised in Drupal Answers about aligning embedded Media (Images)

Wim Leers's picture
Wim Leers

Excellent — that’s exactly what the intent and hope was! It’s nice to see that being confirmed!

frank kelly's picture

I guess I’ll just have to install 8.8 dev locally and see what happens. But, quickly … if I have an article content type with an image “embedded” (maybe I should say “inserted”?) with codes like:

<p><img alt="billsuegayfrank.jpg" height="703" src="/drupal8/sites/default/files/styles/large/public/inline- images/billsuegayfrank.jpg"width="938" /></p>
  1. Will the article still display properly?
  2. Will the image files stay where they are in default/files … ?
  3. If we choose to include them in the media library will it make an extra “copy” somewhere and then we incrementally go through our content transitioning to the new media library and embedding as time permits ?

For a module such as Juicebox which uses an XML file to set up the layout and options for a photo gallery, contains an images and thumbnails directory for the images and uses a Javascript program to display the Gallery, forcing a transition to media library would seem unnecessary and disruptive.

Wim Leers's picture
Wim Leers
  1. Yes, that will still display properly. Existing data is not modified.
  2. Yes, the image files will stay in that same location. Existing data is not modified.
  3. There is no automatic update path just yet, so for now none of the existing uploaded images will appear in the Media Library. For that, see https://www.drupal.org/project/drupal/issues/2930514. I already explained that in my response to your previous comment at https://wimleers.com/comment/20428#comment-20428.

There is no forced transition to Media Library. Juicebox and any other contributed Drupal module will continue to work as-is. Every site can choose for itself to migrate to Media + Media Library. By default, nothing happens. Precisely for the reason you cite: to avoid disruption!

Balu Ertl's picture

Congratulations to the whole Media Initiative team, and especially thank you, Wim, for taking the time to communicate these milestone achievements publicly.

Serge Melis's picture
Serge Melis

Wauw! I was in that session in 2007 as well.

We’ve come a long way, and it sure is great to see this in Core!

A hat tip to all involved, but also a nod of recognition to all to forged their own path.

For instance, we used the venerable Scald Module (https://www.drupal.org/project/scald) integrated with EnterMedia for asset management (https://entermediadb.org/) to power https://news.un.org/ !

I look forward to seeing the same kind of power and usability be available in core.

Wim Leers's picture
Wim Leers

Wow! What an amazing coincidence! :D

Nadia's picture
Nadia

Hi, thank you so much for your amazing work and this blog post! Keyboard accessibility is really so important but often overlooked. I also would like to ask what keys you are using. According to the CKEditor docs, it’s ALT+F10 to enter the toolbar, tab + arrows to navigate between options, space or enter to select/insert … and it all works, but I cannot reproduce the final Edit image part. Would be very greatful if you could give a hint :)

Lisa's picture
Lisa

Problems with oEmbed content regarding adding remote videos with Media Module. Complicated solutions to a big potential security risk make me question its use, especially since it is a known issue—it ALWAYS makes it a risk without clear instructions as to how to resolve. Why should we have to resolve it in the first place on a core module? It’s unusable with that kind of risk.

Wim Leers's picture
Wim Leers

Which concrete risks are you referring to?