Since the release of Drupal 8 in November 2015, developers and site maintainers have been coming to terms with the configuration management that is now a big part of how we support Drupal 8 sites. I took some time to learn the strategies different developers have adopted to make configuration management work for them. In reviewing the material found online, I noticed a definite progression in these strategies from the release of Drupal 8 to the time of writing this nearly two years later. It’s more information than can easily fit into one article, so we’ll cover it in a series. In this first article, we’ll take a look at the Drupal core approach to config management, a few limitations of that approach, and the most basic options we have to introduce some flexibility or customization into the process. Subsequent articles will get into more advanced strategies for configuration management.
Config Management Part 1 - Drupal 8 Core, 2015
The release of Drupal 8 in 2015 marks the first time that a configuration management system has been included in core. This is important because configuration management allows Drupal developers to include configuration changes with source code management (usually git), allowing us to make configuration changes on a development site that are portable to other test instances of the same site and to the live site. Without this kind of workflow, we’d have to repeat the same configuration changes on each instance of the site, which would be more error-prone and time consuming.
Drupal 8 Core Approach
With Drupal 8 core, all of the site’s configuration is exported from the database and stored as YAML files in a single directory. When you do a config import, all the exported YAML files are imported into the database in one operation. There is an option to export or import one piece of configuration separately (the equivalent of one YAML file), but no option to do partial exports or imports that would group some specified configuration together and only act on that.
The key to understanding the new, Drupal 8 approach to configuration management and the development workflow it implies can be summed up in this statement quoted from the article on drupal.org titled, “Managing your site’s configuration”.
“Making configuration changes on a live site is not recommended. The system is designed to make it easy to take the live configuration, test changes locally, export them to files, and deploy to production. Your site's configuration can be stored as part of your codebase and integrated with version control.”
- Managing Your Site's Configuration
This approach implies the following assumptions:
- In a multi-tier environment, with live, local, and often more site instances, all instances of the same site will operate with the same configuration.
- Configuration changes will not be made directly on the live site, but will first be made in a development instance of the site and later be deployed to the live site via configuration management.
Handling Environment-Specific Config
But we don’t always want to operate all instances of the same site with the same configuration. In local development, we will want to enable the Devel module and other modules that aid our development. We don’t want those modules enabled on the testing or live sites. If we use Solr search, we don’t want a test site to point to the same Solr host as the live site. And there are other configuration items that we may want to customize per site instance.
Settings File Config Overrides
Just like in Drupal 7, in Drupal 8 we can override some configuration in settings files. We can deploy different settings.local.php files, excluded from git control, to each site instance. In those files we can specify some custom configuration for each instance of the site - in the same way that we specify different database information for each site. Configuration specified this way overrides the active configuration. There are a couple of drawbacks to this. One is that you cannot specify whether a module is enabled or disabled this way. Another is that when you override config in settings files the admin interface will still show the non-overridden values from the imported YAML files (as of this writing), which has often been a source of confusion and false alarms.
Handling Config Changes on the Live Site
Although Drupal 8 recommends that we don’t make config changes on the live site, we don’t always have control over this. Clients often expect to be able to make live changes to things like contact forms and Webforms. After all, they could do this in Drupal 7 without any problem. And although we may ask users not to do it, we often find that when we compare the live config export to the development export there are differences beyond what we’ve changed in development. This means that if we import the config from development on the live site, we will roll back any changes that users have made there during our development cycle.
Excluding Config Files with git
If there is some configuration that we know is more likely to be changed directly on the live site, and we are using git to move config changes from one environment to another, we can tell git to ignore the related config YAML files by including the list of “excluded” files in a .gitignore file. This way any changes to the excluded files made by exporting config on a development instance will not ultimately overwrite changes made to the same config on the live site. This solution is error-prone, because you have to first identify each YAML file that is related to the configuration you wish to exclude, and manually add those file names to your .gitignore file. And you will have to keep doing this as new config files appear in exports that are related to the config that you wish to exclude.
We’ve done all that but still...
- After importing config from development into a live site:
- We’ve lost some recent config changes that were made on the live site.
- We have to disable some modules that are only used on testing or development sites.
- We have to enable some modules that are only used on the live site.
- After importing config from development into a development or testing site:
- We have to disable some modules that are only used on the live site.
- We have to enable some modules that are only used on the testing or development site.
- In development, after making changes to configuration and exporting the changed config:
- In the sync directory we see that there are some config changes that should not be included in a live import because they were only made to operate the site in a development mode, or help with problem solving. Then, by some manual process, we discover and only include specific config files in the set that will eventually be part of a live import.
- When preparing to do a release of development changes to the live site, we have to pause because we don’t know if by importing our development config on the live site we will wipe out some important config changes that have been made there during our development cycle. Then we have to do something like the following.
- Load the latest live database into our development site which has our up-to-date development config files in the sync directory.
- Export the live config on the development site, overwriting our development config files in the sync directory.
- Use our source code management tool (probably git) to see what files changed in the sync directory after the live export.
- Manually figure out which changes in config files came from our development process and which came from a change to the live site.
- Manually merge changes in config files that were changed both in development and on the live site and add those to the set that will be imported on live.
- Manually add config files that were changed either in development or on live to the set that will be imported on live.
- If we make a mistake or have some uncertainty at the end of this process we have to do it all over again.
- Hope that in the process of doing all this, nobody has made any further config changes on the live site.
- Because of config management, our releases take longer with Drupal 8 than they did with Drupal 7.
What’s Next
Fortunately, we don’t have to wrestle with these problems all by ourselves. If you’ve been involved in the Drupal community for awhile, you know that if you are having a problem, chances are that someone else has had the same problem and may have already figured out a solution. Google is your friend, and when you go looking for solutions to the issues you encounter with config management in Drupal 8, you will find that there is a lot of information out there and it is growing. In 2017 there are much better strategies available for managing configuration than there were in 2015 and 2016. But these strategies only help if we take the time to learn about them and integrate them into our Drupal 8 projects.
In the next article of this series, I will go into more advanced config management strategies and how they can help overcome some of the issues we encounter.