Configuration Management in Drupal 8

In the first article of this serie s we talked about the Drupal 8 Core approach to configuration management, how it helps us, and how it can present certain challenges in our site support and development work. In this article, we will focus on contributed solutions for the two main challenges presented in Part 1, handling environment-specific config and config changes made on live sites.

Handling Environment-Specific Config

Drupal 8 Core config management does not include the option to specify different config for specific site instances. One reason we might want this is to be able to have development helper modules like Devel enabled on a local dev site, without these modules also getting enabled on the live site when we do a live import of config files from development. Let’s look at some of the solutions the Drupal community has come up with for handling environment-specific config.

Drush config-export --skip-modules

As early as 2015, there was a command option for the drush config-export command, --skip-modules, that was intended to let you skip exporting the config for a list of modules while still exporting all the other config. While it seems like that would be useful for keeping config for development modules out of a live import, there were reports by some that it didn’t work well, and it was eventually removed with the release of drush 8.1.11 on April 21, 2017. This was not such a bad thing, because by that time there was already a more robust option available, the Config Split module.

Config Split Module - Part 1

The Configuration Split module came along a little later in the D8 timeline. The first commit was on Aug 18, 2016 by bircher. Config Split offers new, flexible and customizable options for our config management workflows. When I first tried Config Split, I followed the instructions in an article by Jonathan Minder. I was able to create a custom split for the Devel module, resulting in the config files for Devel being written to a separate directory on export (drush cex), so when I pushed the exported files in the default sync directory to the live site and did the import (drush cim) the Devel config files were not part of that. Also the core.extension.yml file in the default sync directory, which tells the system which modules are enabled, did not have an entry for Devel. Even though Devel was still enabled on my local site, it did not get enabled on the live site. You can do the same with any number of modules you’d like to use only on particular site instances.

Config Split groups Drupal functionality and its related config files by module. When I created the split for the Devel module, Config Split identified all config depending on the Devel module and “blacklisted” that from being written to the default sync directory on export. The blacklisting only happens on the site instance where you have that particular split activated, which is the local dev site in this case. Based on this, we can use Config Split for more than just having a different set of modules enabled on particular site instances. We can also specify a different configuration for a module that is enabled on all instances. This might come in handy for modules like Search API Solr which require different configuration for each site instance. Where we had to specify site-specific config for Solr in a local settings.php file before, now we can use Config Split to accomplish the same thing, and the bonus is that in the admin interface we will see the config that is actually being used on that site instance.

Handling Config Changes on the Live Site

What can we do about config changes that are made on a live Drupal 8 site while other config changes are being made in our development workflow? The expectation of Config management core developers was that config changes would be made in development, exported, moved to the live site and imported there. So if users make config changes on the live site, they will be surprised (and not particularly happy) when their changes disappear after the next release when we import our development config on live. Luckily, we have some options available that will help us avoid the “dark night of config confusion” that our releases can turn into when we try to reconcile simultaneous live and dev config changes manually.

Configuration Read-only Mode

The Config Read-only Mode module was created early in the Drupal 8 timeline. The first commit was Feb 13, 2014 by scor. With this module, you can lock the live environment such that no configuration changes can be made there at all. Period. That’s how the Drupal 8 core developers intended config management to work, after all. It’s a simple solution for users who can agree to let developers handle all config changes within a development and deployment workflow. For users who plan to be more hands-on with the administration of their Drupal website, a different solution is required.

Drush CMI Tools

CMI stands for Configuration Management Initiative, which is what they called config management in the early days of Drupal 8. The initial commit for Drush CMI Tools was by larowlan on Aug 23, 2016.

Drush CMI Tools adds some new commands to Drush that extend what you can do with config management. One of these commands, cexy, is like the regular drush config-export (drush cex) command except that it comes with a --ignore-list option for excluding some config from the export. The excluded config is specified by config filename patterns contained in a text file.

The use case for “cexy” is when you are working on a local dev site and you know that some config changes are regularly made on the live site to certain kinds of things like contact forms and webforms - according to the documentation on GitHub. After loading a copy of the live database locally and importing your development config into that, you make some config changes as part of a new feature you are working on. When it’s time to export your changed config and commit it to the repository, you would use the drush cexy command along with the --ignore-list option to point to a text file listing patterns for config you know is often changed on the live site. This exports the active config, then uses the ignore list to “remove unwanted configuration”.

This ensures that your config changes will not include any files matching the pattern in your ignore list. But since the ignored files are deleted from the dev config, how do we prevent their config from being deleted on the live site when we do a config import there? I ran into this question when I tested drush cexy for the first time and wound up deleting my live contact form with the config import. In subsequent tests, I found that I could use the --partial option with my live config import (drush cim --partial) which prevents it from deleting active config for missing config files.

But what about when you actually want some active config to be deleted as part of your development config changes? The CMI Tools “cimy” command can handle that. Use drush cimy with the --delete-list option to point it to a yml file listing config you want to delete. Cimy runs like cim --partial in that it won’t delete active config that is missing from the config import, but it will delete any active config listed in that “delete list” file. Read more about Drush CMI Tools, cexy and cimy here.

Config Ignore

The Config Ignore module, first commit by tlyngej on Nov. 28, 2016, allows you to specify some config to ignore when doing a config import. When testing this module I wanted to ignore the contact form config when importing, since I expect that to be changed on the live site occasionally. Config Ignore provides a user interface to add config and config patterns to ignore. So I entered contact.form.* in the UI and saved. Then I changed the contact from on the live site and also on my dev site. When I exported config on dev I saw that the contact form config change was exported. Then I deployed the changed config from dev to live and did a config import on live. The contact form config that had been exported on dev was not imported, so the form on the live site remained unchanged. I like this solution for its simplicity and ease of use, which seems good enough unless you want the additional flexibility that the Config Split module provides for a similar use case.

Config Split Module - Part 2

The Config Split module is multi talented! Not only can it help with environment specific config differences but it can also exclude config that is routinely changed on the live site. In my tests I confirmed this use case and found that the “Graylist” option provides some additional flexibility with exclusions which other solutions don’t provide.

The Graylist option can be hard to understand at first. There was some discussion about that in a drupal.org issue which resulted in changing the Graylist field description. The new description still seemed confusing to me, so I ran some tests to find out how “Graylist” really works. The testing scenario: contact form and Webform config is changed on the live site while other config changes happen in development. To prevent the live config changes from being overwritten on config import, I created an “excluded” config split setting in the module’s user interface and added an “excluded” config directory to my site’s file structure. By the way, there are many blog articles that go into the specifics of using Config Split. My favorite is this article by Jonathan Minder.

My “excluded” split setting listed contact form and Webform config in the Graylist field like so:
webform.webform.*
contact.form.*
I activated the “excluded” setting on the live site, by adding a line to the localized settings.php file on live:
$config['config_split.config_split.excluded']['status'] = TRUE;

But when I exported config in dev and deployed that on the live site, the live forms were overwritten by the dev config. Why? It turned out that I had missed an important step in my deployment process. Before importing the config from dev on the live site you have to first export the excluded config on the live site. To export only the excluded config into the “excluded” config directory on live, I ran this drush command:
drush config-split-export excluded
(or with the alias: drush csex excluded)

I should mention that my “excluded” sync directory is not tracked in my Git repository (it has an entry in my .gitignore file). I don’t want file changes from a config export to register with my live Git repo because uncommitted changes on a live server could cause the next Git deployment there to fail (this may vary depending on the deployment process used on the live server).

After exporting the live, excluded config, I imported my development config (drush cim) on live, and the live contact and webform config was not overwritten. Now I have a better idea of what Graylist does. If the config files in my Graylisted split (my “excluded” directory) are different from the same files in my default sync directory then that config is treated the same as blacklisted config. In other words, it won’t get overwritten by what is in the default sync directory when I do a config import. So why not just use the Blacklist field instead of Graylist for excluded config? Because the Graylist option does a bit more than that.

What if we get a request to create a new feature for the site that will require a new Webform? Since we’ve excluded Webform config, does that mean we have to create the new form on the live site? Not if we’ve excluded it via the Graylist option. That’s the cool thing about Graylist. Any changes we make in development to config files that already exist in the live Graylist-excluded directory won’t get imported. But if we add new config for modules that are Graylisted, like Webform, in development, that config will get imported on live. I ran the following test to verify this.

On my development site I created a new Webform and exported the config for that. I deployed my config changes to the live site and imported config there (drush cim). The new Webform was imported, and the preexisting contact form and other Webforms on the live site remained unchanged. This time I didn’t export the excluded config prior to the import. That was OK because there were no new changes to excluded forms made on the live site. But normally before doing a live import we’d always export the excluded config first. If there have been any changes made to excluded contact forms or Webforms on the live site, and we don’t export the excluded config, the ‘drush cim’ command will also import config files present in the “excluded” split directory, and those live forms will revert to the config of the last “excluded” export.

The next time we export the excluded config on live, will config for the new Webform, which we just added, be exported to the “excluded” directory? No. Why not? I think the reason is that we have not changed any config for the new Webform on live yet. At this point in my testing, I could still make changes to the new form in development, and import those changes successfully on live. So the form was not excluded yet. But after I made a change to the new form on the live site and exported the config there (using ‘drush csex excluded’), the config for this form was exported into the “excluded” directory. From that point on, the form was “blacklisted” and new config imports from dev did not affect it.

Based on these tests I would describe the Graylist option this way:

When the active, Graylisted config is different from the config files in the default sync directory:
- If the active, Graylisted config has been changed on the live site:
  - A drush config-split-export to the excluded directory will export the changed live config to the excluded directory.
  - If the changed config has been exported to the split directory:
    - This config is now blacklisted - it will not be overwritten on config import by config that is different in the default sync directory.
  - If the changed config has NOT been exported to the split directory:
    - This config will be overwritten on config import by config that is different in the default sync directory.
- If the active, Graylisted config has never been changed on the live site:
  - A drush config-split-export to the excluded directory will NOT export the Graylisted live config to the excluded directory.
  - This config will be overwritten on config import by config that is different in the default sync directory.
- New config for Graylisted modules that does not exist in the excluded directory will be imported from the default sync directory.

Excluded Config is Content

If most of the config changes made on the live site are to things like contact forms and webforms, for instance, then we can predict that these are our potential trouble areas when doing a live import of development config. As we’ve seen, some of the solutions to this are to exclude troublesome config from our config management workflow altogether. What this means is that we begin to treat the excluded config as content. Like all other content, it will be under the control of authorized users to be updated on the live site, and will be backed up in database archives, not as config files in a Git repository.

What’s Next?

We have many options available for tailoring our config management strategy to meet the needs of both end users and developers. But no one strategy will be suitable for every project, or even for different phases of the same project. A new site build may call for a different strategy than an ongoing support arrangement. After learning what our options are for dealing with different config scenarios, I think the next step is to discuss these options with all stakeholders and work out an agreement about how configuration changes will be handled on the site going forward. After that, we will have a better idea of which contributed tools we need (and which we don’t need), and will have a chance to implement a strategy that will help us achieve our goals while avoiding potential setbacks.

Additional Resources
Configuration Management in Drupal 8 - Part 1 | Blog
Building REST Endpoints with Drupal 8 | Blog
Resource Guide to Drupal 8 | Blog