Config Review

Multiple Configurations. It’s not uncommon for one application to have multiple configuration files. Clients also prefer very configurable applications to minimize code changes after project completion. And don’t forget that fact that there are many configuration formats out there, depending on the programmer, library or language.

Multiple Environments. Larger setups have an additional Training environment. QA and Prod will be load balanced, so they belong in a cluster with multiple instances. Some Prod setups are shared. Have you ever tried adding a new configuration to all these environments?

Configuration in Artifact. What is easier than a configuration file? That will be constant. Some clients prefer a very configurable application, like SQL in configuration files. So it’s a balancing act between simplicity and configurability. When things do go into configuration files, they should not be part of the build artifact. If not, how can you make them vary per environment?

Version Control. Configuration files are version controlled like source code. In reality, it’s not that simple. Local configuration is not typically source controlled. The more environments and instances, the more likely only the base copy is source controlled. There is a one to many relationship between code and configuration, so storing and retrieving configuration in version control is not as straightforward as doing it for source code. Configuration should be separate from source code, and highly configurable applications should have separate code and configuration version numbers. In case you store configuration files with source code, how is that working out with your continuous integration and deployment? What about cases where configuration files are edited directly on the server and are now out of sync with source control?

Too Technical. Configuration files are meant to be easy to change, but in reality, it’s too technical for people who are not programmers or system administrators. Some clients don’t give Prod access, which means technicians have to work closely with system administrators to make this configuration change for locked down environments. Some companies use automation tools like Ansible, but it is still too technical.

Change Review. With a big team working on multiple applications, it’s sometimes hard to keep track of all the changes done to configuration files. Businesses need to keep track of changes to make sure their local environment is up to date, but more importantly, to make sure they have the right configuration for Prod deployment or rollback. In locked down environments, system administrators need to review or make the changes themselves.

Runtime Errors. Have you ever made configuration changes that ruined Prod? It could have been a simple typo in vi or Notepad. One way of testing configuration changes is running some read-only unit test against the configuration, assuming you have the necessary permissions. Another way is triple checking changes or asking a peer to review. Would be nice to have a configuration compiler or lint, to validate configuration and detect errors early.

Security. Configuration files can contain sensitive information like passwords and keys. It’s important that sensitive information is encrypted in some way, especially since this is data at rest. Some people don’t bother, and many are among the guilty. It adds a layer of complexity since the data needs to be encrypted in the configuration and decrypted in the code. Manually encrypting data for configuration files is a process in itself. Decrypting them for troubleshooting purposes doesn’t make life easier as well.

Documentation. Like source code, configuration needs to be documented. Like applications, configuration needs technical documentation. Easier said than done. Not a lot of thought is normally given to configuration documentation. If they are, they are either in a separate document that can easily get out of sync or can be part of the configuration file itself, where multiple copies become a maintenance problem. Some configuration files don’t allow comments. Many popular open source applications have very detailed configuration documentation as part of the configuration file. This is great to start but can get overwhelming. What about configuration file locations? And what about searching for configuration across all environments, or doing a diff between environments?

Configuration Server. Some, not all, pain points can be solved by using a configuration server. ZooKeeper quickly comes to mind, but many are fond of Redis. The main problem is that it is no longer an application concern but a company concern. This approach needs buy-in, system administration, and code changes. It’s not just one server as they don’t want a single point of failure. There must be a better way.