Config validation

Drupal's next leap?

Wim Leers
Senior Principal Software Engineer
Acquia logo 's Drupal Acceleration Team

What's this about really? 😅


icon_base_uri: 'public://media-icons/generic'
iframe_domain: ''
oembed_providers_url: 'https://oembed.com/providers.json'
standalone_url: false

media.settings.yml

What's this about really? 😅


_core:
  default_config_hash: PlWtVQXY5oKYZqCMPXh6SPamXagn5BoZqgAI8EY9WsY
icon_base_uri: 'public://media-icons/generic'
iframe_domain: ''
oembed_providers_url: 'https://oembed.com/providers.json'
standalone_url: false

media.settings.yml when installed

A brief history of config in Drupal
era	storage	blobbiness
2004	`variable` table	extremely	🙈
2012	YAML files	diffable	🤓𝍔
2013	[…] + schema	introspectable	🤔💡
2014	`config` table + […]	unchanged	🏎️
2017	[…] + […] + constraints	validatable	✅

Trend: more structure

introspectable (👋 variable module)
translatable (👋 i18n module)
exportable (👋 ctools exportables)

type: uuid

👆

the only validatable config type

😱

Config schema 101

📚

type: uuid use

config_entity:
  type: mapping
  mapping:
    uuid:
      type: uuid
      label: 'UUID'
  …

type: uuid definition

# A UUID.
uuid:
  type: string
  label: 'UUID'
  constraints:
    Uuid: {}

core/config/schema/core.data_types.schema.yml

"Uuid" 👉 a @Constraint plugin ID

namespace Drupal\Core\Validation\Plugin\Validation\Constraint;

use Symfony\Component\Validator\Constraints\Uuid;
use Symfony\Component\Validator\Constraints\UuidValidator;

/**
 * Validates a UUID.
 *
 * @Constraint(
 *   id = "Uuid",
 *   label = @Translation("Universally Unique Identifier", context = "Validation"),
 * )
 */
class UuidConstraint extends Uuid {

  /**
   * {@inheritdoc}
   */
  public function validatedBy() {
    return UuidValidator::class;
  }

  public $message = 'This is not a valid UUID.';

}

namespace Symfony\Component\Validator\Constraints;

class UuidValidator extends ConstraintValidator {

    public function validate(mixed $value, Constraint $constraint) {
        if (!$constraint instanceof Uuid) {
            throw new UnexpectedTypeException($constraint, Uuid::class);
        }
        if (!\is_scalar($value) && !$value instanceof \Stringable) {
            throw new UnexpectedValueException($value, 'string');
        }
        …
    }

}

🧑‍🎓🥳

Before `type: uuid`

     uuid:
-      type: string
-      label: 'UUID'
+      type: uuid

Any string 😳

🤔

🤯

media.settings

Data	Types
	Config	Storage	Perceived
`'f962b8c7- 4c74-4100- b6de-08e6a65ff43d'`	`uuid`	`string`	UUID	👍
`'https://oembed.com/ providers.json'`	`uri`	`string`	URL	👍
`''`	uri	`string`	🤪	🙈
	intent	shape	semantics

Validators needed! 🙏

It gets worse.

media.settings

Data	Types
	Config	Storage	Perceived
`'f962b8c7- 4c74-4100- b6de-08e6a65ff43d'`	`uuid`	`string`	UUID	👍
`'https://oembed.com/ providers.json'`	uri	`string`	URL	👍
`''`	uri	`string`	🤪	🙈
`'first-uuid'`	`uuid`	`string`	🤪	😱

Validators are not executed today.

The new blobbiness

🫠

Many validators missing.
Validators not executed.

Why?

🤷‍♀️

Writing configuration via JSON:API
Admin UI improvements 🪦 JS Admin UI
Reliable config_split, config_filter …
Recipes Initiative
Automatic Updates Initiative
Ambitious site builder experiences

There are many capabilities blocked on this. It's not just that this blocks writing configuration via JSON:API, which in turn blocked the JS Admin UI initiative which has since stopped. It's also that it is impossible to reuse the same validation logic for the JS admin UI: it literally needs to be re-implemented, which is a big security risk. If we move the current logic out of forms, we benefit multiple times: custom admin UIs (on the server or client) become possible, UIs become simpler to maintain, additional validation (from a contrib or custom module) becomes easy to add, et cetera. Talking to Fabian Bircher, maintainer of https://www.drupal.org/project/config_split and https://www.drupal.org/project/config_filter, he said a key problem is to know if a key is required or not. Sometimes when a value is null or the array empty we can omit the key and sometimes not. These are problems that are easily solvable with validation.

Drupal Core

Ugly

>95% of config types not validatable

#3324984 👉 test to track this %

Bad

Every test checks schema …

… but does not validate!

#3361534 👉 validate()

Actually, the 'first-uuid' as a UUID example comes from that issue: it's a real example from Drupal core! 😅 It was fixed a few days ago! 🥳

Good

More config validatable!

CKEditor 5 configuration + every plugin!

type: config_dependencies

block.block.*:plugin

#2920678 👉 type: machine_name

#3341682 👉 type: label

So what else besides type: uuid is supported?

Good

More validation constraints to reuse!

ValidKeys: <infer>
PluginExists: plugin.manager.block
ExtensionName: {}
ExtensionExists: module
ConfigExists: {}
…

The most powerful and impactful one was written by phenaproxima and committed to core almost exactly six months ago: the ValidKeys constraint. This constraint accepts a list of … valid keys, unsurprisingly! But what truly makes it powerful is the ability to specify the <infer>

Interested?

Issue tags:

Configuration Inspector

d.o/project/config_inspector

Since 2.1.1

Drupal Contrib

You can start using this today!

CDN module: since 2019

Copy ValidatableConfigFormBase which extends \Drupal\Core\Form\ConfigFormBase
Subclass ValidatableConfigFormBase instead of ConfigFormBase
💰🌈

Copy ValidatableConfigFormBase which extends \Drupal\Core\Form\ConfigFormBase
Subclass ValidatableConfigFormBase instead of ConfigFormBase
💰🌈
Optional: ConfigEvents::SAVE subscriber to validate your config
🥳🥇

Examples

Search for constraints
in *.schema.yml files

How to adopt?

🪩

Analyze your entire site

drush config:inspect --format=csv > ~/Desktop/core.csv


$ drush config:inspect --filter-keys=user.settings --detail --list-constraints
 Legend for Data:
  ✅❓  → Correct primitive type, detailed validation impossible.
  ✅✅  → Correct primitive type, passed all validation constraints.
 ------------------------------------------ --------- ------------- ------ ---------------------------------
  Key                                        Status    Validatable   Data   Validation constraints
 ------------------------------------------ --------- ------------- ------ ---------------------------------
  user.settings                              Correct   74%           ✅❓
   user.settings:                            Correct   NOT           ✅❓
   user.settings:_core                       Correct   NOT           ✅❓
   user.settings:_core.default_config_hash   Correct   NOT           ✅❓
   user.settings:anonymous                   Correct   NOT           ✅❓
   user.settings:cancel_method               Correct   NOT           ✅❓
   user.settings:langcode                    Correct   NOT           ✅❓
   user.settings:notify                      Correct   NOT           ✅❓
   user.settings:notify.cancel_confirm       Correct   Validatable   ✅✅   ↣ PrimitiveType: {  }
   user.settings:notify.password_reset       Correct   Validatable   ✅✅   ↣ PrimitiveType: {  }
…
   user.settings:notify.status_activated     Correct   Validatable   ✅✅   ↣ PrimitiveType: {  }
   user.settings:notify.status_blocked       Correct   Validatable   ✅✅   ↣ PrimitiveType: {  }
   user.settings:notify.status_canceled      Correct   Validatable   ✅✅   ↣ PrimitiveType: {  }
   user.settings:password_reset_timeout      Correct   NOT           ✅❓
   user.settings:password_strength           Correct   Validatable   ✅✅   ↣ PrimitiveType: {  }
   user.settings:register                    Correct   NOT           ✅❓
   user.settings:verify_mail                 Correct   Validatable   ✅✅   ↣ PrimitiveType: {  }
 ------------------------------------------ --------- ------------- ------ ------------------------

Using the drush command to list the constraints of all property paths inside user.settings gives us a nice overview of the property paths still missing validation. Some already have validation, but only by accident: because they're type: boolean. Let's start with the first: user.settings:anonymous.


  user.settings:
    type: config_object
    label: 'User settings'
    mapping:
      anonymous:
        type: label
        label: 'Name'
+       constraints:
+         NotBlank: []
+         Regex:
+           pattern: '/[[:alnum:]]+/'
+           message: 'Using only emojis is not allowed because it may not be supported in all contexts.'

user.schema.yml

Because there is no validation logic in the form (which implies that pretty much any value was allowed until now, or at least any value that the HTML form input would have allowed to pass through), we have to use common sense to determine what are acceptable values. In this case, we know that pretty much anything goes, except for no input at all (which an maps to the empty string). So we use NotBlank. But is that enough? Another reasonable requirement is to ensure that the name for the anonymous user works in all contexts. Not all contexts allow for the use of emojis. So, let's require the bare minimum: at least some of the characters must not be emojis, so that even after transliteration, there's something meaningful left.

Next, let's find out which UI actually modifies this config. A quick search for user.settings reveals a LOT of matches. But … we know ConfigFormBase has a getEditableConfigNames() method. Let's use that to our advantage, and maybe this is a cool PHPStorm trick you didn't know yet! 🤓


-class AccountSettingsForm extends ConfigFormBase {
+class AccountSettingsForm extends ValidatableConfigFormBase {


   public function submitForm(array &$form, FormStateInterface $form_state) {
     parent::submitForm($form, $form_state);

-    $this->config('user.settings')
-      ->set('anonymous', $form_state->getValue('anonymous'))
-      ->set('register', $form_state->getValue('user_register'))
-      ->set('password_strength', $form_state->getValue('user_password_strength'))
-      ->set('verify_mail', $form_state->getValue('user_email_verification'))
-      ->set('cancel_method', $form_state->getValue('user_cancel_method'))
-      ->set('notify.status_activated', $form_state->getValue('user_mail_status_activated_notify'))
-      ->set('notify.status_blocked', $form_state->getValue('user_mail_status_blocked_notify'))
-      ->set('notify.status_canceled', $form_state->getValue('user_mail_status_canceled_notify'))
-      ->save();


+  /**
+   * {@inheritdoc}
+   */
+  protected static function mapFormValuesToConfig(FormStateInterface $form_state, Config $config): Config {
+    switch ($config->getName()) {
+      case 'user.settings':
+        $config->set('anonymous', $form_state->getValue('anonymous'))
+          ->set('register', $form_state->getValue('user_register'))
+          ->set('password_strength', $form_state->getValue('user_password_strength'))
+          ->set('verify_mail', $form_state->getValue('user_email_verification'))
+          ->set('cancel_method', $form_state->getValue('user_cancel_method'))
+          ->set('notify.status_activated', $form_state->getValue('user_mail_status_activated_notify'))
+          ->set('notify.status_blocked', $form_state->getValue('user_mail_status_blocked_notify'))
+          ->set('notify.status_canceled', $form_state->getValue('user_mail_status_canceled_notify'));
+    }
+    return $config;
+  }

🕺

#3364506

🙏

Joris Vercammen aka @borisson_ — Calibrate
Adam Hoenich aka @phenaproxima — Acquia
Ben Mullins aka @bnjmnm — Acquia
Fabian Bircher aka @bircher — Nuvole
Daniel Wehner aka @dawehner ^(2018!)
Alex Pott aka @alexpott ^{(2018–now!)}

🚀

Challenge: partially validatable
Functionality: JS admin UI, JSON:API, recipes
Operational: reliability (CI!), module maintainability

🙇🏻‍♂️

META	#2164373 👀 ⤵
Required values	#3364109
Required keys	#3364108
Validate in tests	#3361534
Metadata for rich UX	#2949888

Which config changes have caused outages for YOU? 🫵

Config validation

Drupal's next leap?

Config schema 101

🧑‍🎓🥳

Before type: uuid

🤔

🤯

The new blobbiness

Why?

Drupal Core

Ugly

Bad

Good

Good

Interested?

Configuration Inspector

Drupal Contrib

CDN module: since 2019

Examples

How to adopt?

🪩

🕺

🙏

🚀

🙇🏻‍♂️

Before `type: uuid`