twitter
Content is available under Creative Commons Attribution.


Articles

Configuration Management

Introduction

Configuration management is both one of the most import and most overlooked aspects of the software development life cycle. In brief, the idea behind configuration management is to unify how configurations are handled across environments. Our goals are to:

Proper configuration management is the foundation upon which all your deployments are built upon; getting it right is vital. Let’s see how Leroy helps us out here.

Properties

Leroy has a hierarchy of “entities” which can each have properties. The entities are organized in a tree structure:

Global properties are overridden by more local properties. So, if a global property called ‘myProperty’ is defined in controller.xml, it will be overridden by a property with the same name in an environment or in an agent. In this way, you can have a set of global properties that define a general set of properties and then refine and tailor the properties by environments or by agents as needed.

How it Relates

An application typically has some set of configurations; paths, databases, ports, and other such things. Further, the application will typically be deployed to some set of environments (development, test, stage, production, etc.). The application probably uses monolithic configuration files, but it is rare that these configurations are completely distinct across environments. Usually, at least some settings are conceptually shared. And in practically all cases, each environment needs a complete set of configuration settings in order to run correctly.

In any given application, there will be some set of configurations that are similar for all environments (perhaps a port number, for instance, may be the same across all environments), some that are environment specific (host names, databases) and some that may even be specific to a particular node/host in an environment (cluster settings, installation paths).

This structure is the exact same structure Leroy uses to manage properties in a deployment; global, per-environment, and per-node (agent). This is where Leroy’s configuration management component comes in.

Using Leroy for Configuration Management

Leroy has the ability to take a set of template files, called resources, and basically search-and-replace tokens in those templates with Leroy properties. This is similar to how Maven’s resources plugin works, except it is more targeted to the configuration management task and therefore easier to set up and use.

What Leroy will do, is take a folder with template files and copy them with the property tokens in the files replaced with Leroy properties into a structured filesystem. It’s easier to show an example than to try and explain it, so let’s say we have this directory structure:

Let’s say you have a production environment named ‘prod’ with three agents in it named: web01, web02, and db01

Leroy’s configuration management feature will generate a structure like this:

Basically, a set of files in a directory name for the environment and agent will be generated for each agent in the environment. Each file will have it’s replacements done “from the perspective” of the given agent-environment pair. This structure will then be zipped up and put into your artifacts directory, ready to be deployed to your host and then copied out however you like.

Each file will contain tokens like this:

Where ${propertyName} will be replaced by Leroy property values. What you can then do, is “boil down” your configuration files into a set of Leroy properties and then use the configuration management feature to “compile” your configurations.

This is done with the <configuration> workflow task in the Leroy workflow:

The configurations will automatically be built for whatever environment you are running the workflow against. Each time you build configurations, certain integrity checks will be run. See below.

How to get from A to B

No doubt you already have a set of configurations that were not constructed with ease of deployment in mind; figuring out how to deploy software is rarely the first priority when developing a product, so it’s very common to wind up with some set of configuration files that will need to be re-organized in order to make them environment-portable.

The first step is to make your basic structures and your initial files for both your Leroy property files and your configuration templates. Let’s start with the Leroy property files.

As noted above, our goal is to turn our flat configuration files into a hierarchy of properties. So, in your Leroy controller directory, create a structure like this :

Where “EnvironmentName.xml” is named based on your environment as defined in environments.xml (dev.xml, test.xml, etc.) and “AgentName.xml” is based on your agents as defined in agents.xml (linux-web-node1.xml or whatever). All of these files are optional; your should always have a global.xml, with the other files being necessary based on the complexities of your individual application. It’s possible, for instance, that you do not need node-level (agent) specific configuration parameters.

Now, let’s start creating some configuration templates. Find all the configuration files that your system uses, and keep track of where they are installed (a spreadsheet is a great tool for this to map configuration files to application names and directory locations). Gather together samples of all your configuration files and put them in directories in some logical grouping.

For instance, if you have a system with a web front end and a Java back-end server, a structure like this might make sense:

This becomes the foundation of your configuration templates, and these files should wind up in your source control so that your configurations can be revisioned along with everything else.

Now comes the tedious work! The good news is, once this has been completed, changes become a lot easier in the future. But it’s usually a lot of careful work the first time around.

What you want to do, is go through your configuration templates, and convert all your configuration parameters to Leroy properties. The smartest way to do this is to open your template files up in one window, and open your global.xml in another file. As you replace things in the configuration templates, you should populate global.xml with reasonable values. These values will be overriden per environment in the next pass, but for now, we just want to make sure that every possible value is defined in global.xml — even if a field is blank.

The reason is we want global.xml to act almost like a glossary of what configuration parameters are available; this makes adding a new environment a lot easier, and serves as living documentation of your configuration.

Let’s say you have 3 configuration files that look like this:

Something to note right off the bat is that the java server’s port (8888) is a common point that is shared between components in the environment. If that port changes, it will need to change in both places; therefore, it will be advantageous to make sure that property is shared. Here’s what our global.xml should look like:

Now, our template files would look like this:

 

Now, finally, we make our files for other environments.

Now … to define a new environment, you just need to copy production.xml to, say, test.xml and change just the values you need to change. You still have the power to override a base-level configuration that’s in global.xml (like a port), but you have reasonable defaults set so that if you do not need to override it, you don’t have to.

This makes shifts in your configuration easier to deal with in the future. Corporate decides all the servers are to have a new naming convention? No sweat! It’s all in one place now.

In your deployment workflow, you would have a block like this:

A cool “side effect” of putting all your configuration properties in Leroy, is they are available to your workflow. Want to refer to ${web.base_path} in your workflow? Go right ahead — it works just like you would expect.

Note: When expressing properties in resource files literally that you do not want parsed use double-backslashes like so: \\${prop}

Leave a Reply



by: Epic Force