Puppet future parser — what to expect that you’ll have to update in your manifests…

The Puppet Future Parser is the new implementation of the manifest parser which will become the default in 4.0, so I thought I’d take a look to see what I’d need to update.

Also, there are some fancy new features like iteration and that you can use [1,2] array notation or {a=>b} hash notation anywhere that you’d previously used a variable containing an array or hash.

The iteration and lambda features are intended to replace create_resources calls, as they are more flexible and can loop round repeatedly to create individual definitions.

For example, here’s a dumb “sudo” profile which uses the each construct to iterate over an array:

class profiles::sudo {
  # This is a particularly dumb version of use of sudo, to allow any commands:
  $admin_users = hiera_array('admin_users')
  # Additional users with special sudo rights, but no ssh access (e.g. root):
  $sudo_users  = hiera_array('sudo_users')

  class { ::sudo: }

  $all_sudo_users = concat($sudo_users, $admin_users)

  # Create a resource for each entry in the array:
  each($all_sudo_users) |$u| {
    sudo::entry { $u:
      comment  => "Allow ${u} to run anything as any user",
      username => $u,
      host     => 'ALL',
      as_user  => 'ALL',
      as_group => 'ALL',
      nopasswd => false,
      cmd      => 'ALL',
    }
  }
}

Making this work with create_resources and trying to splice in the the username for each user in the list into a hash looked like it would be messy, requiring at least an additional layer of define — this method is much neater.

This makes it much easier to create data abstractions over existing modules — you can programmatically massage the data you read from your hiera files and call definitions using that data in a much more flexible way than when passing hashes to create_resources. This “glue” can be separated into your roles and profiles (which could be the subject of another post but are described well in this blog post), creating a layer which separates the use of the module from the data which drives that use nicely.

So this all sounds pretty great, but there are a few changes you’ll possibly encounter when switching to the future parser:

  • Similar to the switch from puppet master to puppet server, the future parser is somewhat more strict about data formats. e.g. I found that my hiera data definitely needed to be properly quoted when I started using puppet server, so entries like mode : 644 in a file hash wouldn’t give the number you were expecting… (needs mode : 0644 or mode : '644' to avoid conversion from octal to decimal…). The future parser extends this to being more strict in your manifests, so a similarly-incorrect file { ... mode => 644 } declaration needs quoting or a leading zero. If you use puppet-lint you’ll catch this anyway — so use it! 🙂
  • It’s necessary to use {} instead of undef when setting default values for hiera_hash (and likewise [] instead of undef for hiera_array), to allow conditional expressions of the form if $var { ... } to work as intended. It seems that in terms of falseness for arrays and hashes that undef is in fact true… (could be a bug, as this page in the docs says: “When used as a boolean, undef is false”)
  • Dynamically-scoped variables (which are pretty mad and difficult to follow anyway, which is why most languages avoid them like the plague…) don’t pass between a class and any sub-classes which it creates. This is in the docs here, but it’s such a common pattern that it could well have made it through from your old (pre-Puppet 2.7) manifests and still have been working OK until the switch to the future parser. e.g.:
    class foo {
      $var = "x"
    }
    
    class bar {
      include foo
      # $var isn't defined here, as dynamic scope rules don't allow it in Puppet >2.7
    }
    

    Instead you need to explicitly qualify your variables to pull them out of the correct scope — $foo::var in this case. In your erb templates, as a common place where the dynamically-scoped variables might have ended up getting used, you can now use scope['::foo::var'] as a shorthand for the previously-longer scope.lookupvar('::foo::var') to explicitly qualify the lookup of variables. The actual scope rules for Puppet < 2.7 are somewhat more complicated and often led to confusing situations if you unintentionally used dynamic scoping, especially when combined with overriding variables from the parent scope…

  • I’m not sure that expressions of the form if "foo" in $arrayvar { ... } work how they should, but I’ve not had a chance to investigate this properly yet.

Most of these are technically the parser more strictly adhering to the specifications, but it’s easy to have accidentally had them creep into your manifests if you’re not being good and using puppet-lint and other tools to check them.

In conclusion : Start using the Future Parser soon! It adds excellent features for iteration which make abstracting data a whole lot easier than using the non-future (past?) parser allows. Suddenly the combination of roles, profiles and the iteration facilities in the future parser mean that abstraction using Puppet and hiera makes an awful lot more sense!

Packaging mod_auth_cas for CentOS 7

Mark wrote a useful post about building mod_auth_cas for CentOS 7. It works, but I prefer to build RPM packages on a build server and deploy them to production servers, rather than building on production servers.

Basically I took the spec file from the mod_auth_cas source package for CentOS 6 from the EPEL 6 repository, tweaked it, and replaced the source tarball with the forked copy Mark recommended. This built cleanly for CentOS 7.

I’ve sent a pull request back to the upstream project with the Red Hat build files and documentation and for those who are keen, here are the EL7 RPM and source RPM:

Update

There’s an even better way of building this for CentOS 7. Fedora 21 includes Apache 2.4 and mod_auth_cas and it is really easy to backport this source package.

First grab the latest version of the source package from the Fedora mirror onto your CentOS 7 build server. Always make sure there isn’t a newer version available in the updates repo.

Rebuild is as simple as issuing:

rpmbuild --rebuild mod_auth_cas-1.0.8.1-11.fc22.src.rpm

It will spit out an RPM file suitable for deployment on CentOS 7. Add --sign if you routinely sign your packages with an RPM-GPG key.

mod_auth_cas on CentOS7 / Apache 2.4

For CentOS, this is now available in the EPEL repo

mod_auth_cas (https://github.com/Jasig/mod_auth_cas) is an Apache module that plugs into the Apache mod_auth framework, to provide authentication against a Jasig CAS SSO server.  Unfortunately development on it seems to have stalled, and it currently doesn’t support Apache 2.4 – https://github.com/Jasig/mod_auth_cas/issues/49.

There is a fork that does support Apache 2.4, so this willl cover how to build and install it.

# Clone the forked github repo
git clone https://github.com/klausdieterkrannich/mod_auth_cas.git .

# install development libraries
yum install gcc httpd-devel openssl-devel libcurl-devel automake

# Run configure
./configure

# Run Make
make

# If you get errors like:
/opt/mod_auth_cas/missing: line 81: aclocal-1.12: command not found
WARNING: 'aclocal-1.12' is missing on your system.
# then symlink the binaries as follows:
ln -s /usr/bin/aclocal /usr/bin/aclocal-1.12
ln -s /usr/bin/automake /usr/bin/automake-1.12
# and run make again

# Assuming it built OK, then install it:
make install

# On CentOS this will put the binaries into:
#/usr/lib64/httpd/modules
# and you can then copy them from here to your production systems.

 

Using Puppet to deploy code from Git

I’ve revisited the way that we at ResNet deploy our web applications to web servers. We decided to store the application code in a Git repository. As part of our release process, we create a tag in Gitlab.

Rather than check the code out manually, we are using a Forge module called puppetlabs/vcsrepo to clone a tagged release and deploy it. Our app repos do not permit anonymous cloning so the Puppet deployment mechanism must be able to authenticate. I found the documentation for puppetlabs/vcsrepo to be a bit lacking and had spend a while figuring out what to do to make it work properly.

I recommend you generate a separate SSH key for each app you want to deploy. I generated my key with ssh-keygen and added it to Gitlab as a deploy key which has read-only access to the repo – no need to make a phantom user.

Here’s a worked example with some extra detail about how to deploy an app from git:

# Define docroot
$docroot = '/var/www/app'

# Deploy SSH key to authenticate git
file { '/etc/pki/id_rsa':
  source => 'puppet:///modules/app/id_rsa',
  owner  => 'root',
  group  => 'root',
  mode   => '0600',
}
file { '/etc/pki/id_rsa.pub':
  source => 'puppet:///modules/app/id_rsa.pub',
  owner  => 'root',
  group  => 'root',
  mode   => '0644',
}

# Clone the app from git
vcsrepo { 'app':
  ensure   => present,
  path     => $docparent,
  provider => git,
  source   => 'git@gitlab.resnet.bris.ac.uk:resnet/app.git',
  identity => '/etc/pki/git_id_rsa',
  revision => '14.0.01',
  owner    => 'apache', # User the local clone will be created as
  group    => 'apache',
  require  => File['/etc/pki/id_rsa', '/etc/pki/id_rsa.pub'],
}

# Configure Apache vhost
apache::vhost { 'app'
  servername    => 'app.resnet.bris.ac.uk',
  docroot       => $docroot,
  require       => Vcsrepo['app'],
  docroot_owner => 'apache',  # Must be the same as 'owner' above
  docroot_group => 'apache',
  ...
}

To deploy a new version of the app, you just need to create a new tagged release of the app in Git and update the revision parameter in your Puppet code. This also gives you easy rollback if you deploy a broken version of your app. But you’d never do that, right? 😉