Tag: puppet

How to change a file using Puppet

One of the first things I was curious about Puppet (or configuration management in general) was “how can I change a file?” (besides the pretty obvious alternatives of copying a static one and im-sure-they-do-support templates).

So, leaving aside solutions that apply to very specific types of files and more esoteric options like those involving manually exec’ing sed and things like that, there are five ways to do it:

File resource, take one

To the simplest problem, the simplest solution. If you want the file to have a static predefined content, you just copy one using the File resource:

file { '/tmp/file01':
  ensure => file,
  owner  => 'root',
  group  => 'root',
  mode   => '0644',
  source => 'puppet:///modules/mymodule/file01',

You also use the file resource to create symlinks and directories, and you can copy dirs and its files recursively too. Check the reference for more info. In the example above, the source attribute references a file inside a puppet module, which is the most common way of doing it. That works out-of-the box when you use modules. But puppet has a internal file server that you can tweak with for more advanced use cases.

File resource, take two

You can also use File to define the file’s content instead of the source:

file { '/tmp/file02':
  ensure  => file,
  content => 'Yeah, I am file02, so what?',

But you usually use that with templates:

$say_hello_to = 'dude'
$myname = 'file03'

file { "/tmp/$myname":
  ensure  => file,
  content => template('mymodule/polite-file.erb'),

That template, stored inside the module mymodule, looks like this:

$ cat mymodule/templates/polite-file.erb 
<% if @say_hello_to -%>
Hello <%= @say_hello_to %>,
<% end -%>

I'm <%= @myname %>, on a <%= @operatingsystem %> system, nice to meet you.

After applying that manifest, we have a /tmp/file03 with this content:

Hello dude,

I'm file03, on a Ubuntu system, nice to meet you.

Now, where that @operatingsystem variable (replaced here by “Ubuntu”) came from? Well, besides variables defined within your puppet manifest, templates can use values provided by Facter.

File_line resource

If you wanna change just one simple line on an existent file, File_line is your friend:

file_line { 'someline':
  path  => '/etc/hosts',
  line  => ' somehost',
  match => '^192\.168\.0\.1.*',

It adds only that line if it’s not already there. If you supply the optional match parameter, it will replace the line that matches the regex, instead of adding a new one. File_line is not in puppet core tough, but in the widely used puppetlabs/stdlib module.


Ok, let’s start to complicate things a little bit. The ripienaar/concat module does … well … file concatenation:

concat { '/tmp/file04':
  mode  => '0644',
  owner => 'root',
  group => 'root',

concat::fragment { 'file04_fa':
  target  => '/tmp/file04',
  content => "another piece a\n",

concat::fragment { 'file04_fb':
  target  => '/tmp/file04',
  content => "another piece b\n",

concat::fragment { 'file04_f0':
  target  => '/tmp/file04',
  content => "this is the first piece\n\n",
  order   => 01,

The Concat resource is pretty much like a File resource. The other fragments link to it by their target parameter and may also define the order. How is that complicated? Well, it isn’t in fact. Not the code. But puppet’s report log, oh boy, goes wild:

Notice: /Stage[main]/Concat::Setup/File[/var/lib/puppet/concat]/ensure: created
Notice: /Stage[main]/Concat::Setup/File[/var/lib/puppet/concat/bin]/ensure: created
Notice: /Stage[main]/Concat::Setup/File[/var/lib/puppet/concat/bin/concatfragments.sh]/ensure: defined content as '{md5}256169ee61115a6b717b2844d2ea3128'
Notice: /Stage[main]//Concat[/tmp/file04]/File[/var/lib/puppet/concat/_tmp_file04]/ensure: created
Notice: /Stage[main]//Concat[/tmp/file04]/File[/var/lib/puppet/concat/_tmp_file04/fragments]/ensure: created
Notice: /Stage[main]//Concat[/tmp/file04]/File[/var/lib/puppet/concat/_tmp_file04/fragments.concat]/ensure: created
Notice: /Stage[main]//Concat::Fragment[file04_fa]/File[/var/lib/puppet/concat/_tmp_file04/fragments/10_file04_fa]/ensure: created
Notice: /Stage[main]//Concat[/tmp/file04]/File[/var/lib/puppet/concat/_tmp_file04/fragments.concat.out]/ensure: created
Notice: /Stage[main]//Concat::Fragment[file04_fb]/File[/var/lib/puppet/concat/_tmp_file04/fragments/10_file04_fb]/ensure: created
Notice: /Stage[main]//Concat::Fragment[file04_f0]/File[/var/lib/puppet/concat/_tmp_file04/fragments/01_file04_f0]/ensure: created
Notice: /Stage[main]//Concat[/tmp/file04]/Exec[concat_/tmp/file04]/returns: executed successfully
Notice: /Stage[main]//Concat[/tmp/file04]/Exec[concat_/tmp/file04]: Triggered 'refresh' from 5 events
Notice: /Stage[main]//Concat[/tmp/file04]/File[/tmp/file04]/ensure: defined content as '{md5}7586b5db4d26995af5e7d6eb23f7eb7e'
Notice: Finished catalog run in 0.78 seconds

Concat is great, very handy. But it is also very verbose, makes the report log harder to read. Besides that, it needs to generate the file sections locally on the destination machine in runtime. Doing a –noop run will now throw errors at you because those file sections do not get created when running noop, and everything that depends on the final resulting file will fail as consequence.

When you have concat in the recipe, troubleshooting and testing it can get quite annoying. I use it frequently, but only when I need it.

Bring the bazooka: Augeas

Augeas is a configuration editing tool. It parses configuration files in their native formats and transforms them into a tree. Configuration changes are made by manipulating this tree and saving it back into native config files. – augeas.net

Augeas is way cool, and it’s not exclusive to puppet, not at all. There is a cli interface (augtool) you can use from anywhere. There is also a puppet resource type for it. Here’s an example of using it to define a setting of your fstab:

augeas { 'setfsoptions':
  context => "/files/etc/fstab",
  changes => [
    "set *[spec = '/dev/sda5']/opt[2] noauto",

UPDATE: Aron Roberts pointed out in a comment a new puppet+augeas documentation available here. Nice!

I’ve just defined that the filesystem in /dev/sda5 shouldn’t be automatically mounted. It will do just that. The rest of the fstab line for that file system is unaffected, keeping it unmanaged by puppet. You can’t do that with File_line or anything else show above.

Install augtool and run augtool print /files/etc/fstab to understand how augeas parses that file. Augeas understands the syntax of files by what it calls lenses, and it comes with several of them ready for dealing with a range of common unix configuration files. If you are brave (and need it) enough, you can create the lens you miss by yourself.

But getting the augeas commands right might be tricky, particularly if you have to deal with numbered entries.  I usually test the commands using augtool before getting them into puppet, so I can have a faster debugging cycle.

The sixth way: use ready-for-use modules and don’t care

As I finished this I realized there is a sixth way that was worth mentioning: grab a ready for use module from the Forge that takes your parameters and does the job for you, changing the file you need to change. No need to know how it does that!

Need to change an ini-style file? See puppetlabs/inifile.  Like augeas power? domcleal/augeasproviders uses it to implement several higher level puppet resource types. Windows work? puppetlabs/registry may help.

And much more. At the Forge you find modules to help manage all kinds of stuff: apache, mysql, tomcat, sudo, dhcp, resolv.conf, apt, … and the list goes on and on. Check it out.

service blog start

Well, it’s about time to kick this blog alive.

So this is a blog about automation in the context of system administration. Automation is such a broad term. For example, monitoring is for me a little bit automation. It automates away the need to regularly check things by hand, which is paramount, by the way.

Shell scripting, ssh multiplexers, libraries and tools for automating non-interactive deployments or both text and visual interactive applications are all great tools to have under any sysadmin’s belt, and I’ve been using then for many years. They are still very useful, but recently I added the ultimate automation tool to my belt: a configuration management software. And that’s a role new game, called Infrastructure as Code.

Infrastructure as code is not just about making you type less, tough. It’s about consistency, repeatability, executable documentation, versioning, peer reviewing, testing. Automation is cool, but realizing the benefits from all these things together is what really excites me.

There are quite a few config management tools out there, and I decided to go with Puppet. So far I’m very happy with it, and it will certainly be a frequent topic here.

Update: my friend foscarini pointed out another quite interesting config management alternative.