Prerequisites

• A working puppet master and some clients to test with
• Hiera in use, with some “common” section which is applied to all nodes. This isn’t strictly required, but it makes the set-up a lot simpler.

What you’ll end up with

• SSL’d connections between all involved daemons
• An admin user who can run “mco ping” and get responses from all the mcollective-enabled nodes

Basic setup (without SSL)

First you’ll want to get mcollective up and running *without* SSL to be sure that the module is working, before leaping on to use SSL and certificates.

Since this is a very basic setup all the examples with use the puppet master as the “broker”, where activemq queues all the requests and replies centrally, rather than a separate machine. Replace the puppet.example.org and node[12].example.org references with your host names as appropriate 🙂

• Install the puppetlabs mcollective module and its dependencies with:
  puppet module install puppetlabs-mcollective
  This installs many other modules to help it along.
• Add the mcollective class to your included class list for all nodes. e.g.:

  classes:
    ... (probably plenty of others)
    - mcollective
  

• Add these variables to your hiera configs for all hosts:

  mcollective::middleware_hosts: puppet.example.org
  

• Add these variables to apply only to your puppet master:

  mcollective::middleware: true
  mcollective::server    : true
  mcollective::client    : true
  

• If you have firewalling on your test puppet master (and you should!), open port 61613. For me this required adding another hiera entry:

   site::firewall:
     '040 accept all stomp/activemq connections':
       state : ['NEW']
       proto : 'tcp'
       dport : 61613
       action: accept
  

• Now run puppet (or wait for a run to occur automatically, if you have that set up) on the puppet master and a couple of nodes, and you should get the mcollective agent installed and configured on all nodes, plus the activemq daemon set up and running on the broker.

At this point you should be able to run mco ping and get some ping time results back from the nodes which are running the mcollective agent. You can also look at the output of netstat on the activemq server and see a number of ESTABLISHED connections to the 61613 port from the mcollective nodes.

Adding SSL support

Now you have a basic set-up working the steps required to get SSL working are :

• Allow firewall connections to port 61614 on the broker (same host as the puppet master in this simple case)
• Generate a strong random password for the mcollective user — I use pwgen -s -y 20 to generate good long random passwords
• Generate a certificate (public/private) pair for the new admin user, using puppet cert generate admin-user-name on the puppet master
• Generate a certificate (public/private) pair for the mcollective and activemq daemons to use, with puppet cert generate mcollective-servers
• Create a site-local mcollective certificates set of directories under your module directories. For example, we have:

    /etc/puppet/modules/project_zoned/files/mcollective/
    /etc/puppet/modules/project_zoned/files/mcollective/client_certs
    /etc/puppet/modules/project_zoned/files/mcollective/certs
    /etc/puppet/modules/project_zoned/files/mcollective/private_keys
  

• Copy the certificates to the appropriate places:

    cp /var/lib/puppet/ssl/certs/ca.pem /etc/puppet/modules/project_zoned/files/mcollective/certs/
    cp /var/lib/puppet/ssl/certs/mcollective-servers.pem /etc/puppet/modules/project_zoned/files/mcollective/certs/
    cp /var/lib/puppet/ssl/private_keys/mcollective-servers.pem /etc/puppet/modules/project_zoned/files/mcollective/private_keys/
    cp /var/lib/puppet/ssl/certs/admin-user-name.pem /etc/puppet/modules/project_zoned/files/mcollective/client_certs/
    cp /var/lib/puppet/ssl/private_keys/admin-user-name.pem /etc/puppet/modules/project_zoned/files/mcollective/private_keys/
    cp /var/lib/puppet/ssl/certs/admin-user-name.pem /etc/puppet/modules/project_zoned/files/mcollective/client_certs/
  

• Add a declaration of an mcollective::user. We have various site-local classes which get auto-expanded into multiple calls to puppet defines, so we add a list like this:

  site::mcollective::users:
    - 'admin-user-name':
      group : 'admin-user-group'
      certificate: 'puppet:///modules/project_zoned/mcollective/client_certs/admin-user-name.pem'
      private_key: 'puppet:///modules/project_zoned/mcollective/private_keys/admin-user-name.pem'
  

Re-run puppet on the puppet master and you’ll have an install of mcollective which is set up to use SSL, and an admin user who has a ~/.mcollective config file, along with keys in their ~/.mcollective.d directory. This user should be able to run mco ping and get results from any nodes which have had the mcollective setup installed on them too (which is any that you run puppet on after adding the mcollective class and settings into a common yaml file for all nodes).

Now you should have a basic SSL-encrypted setup which works, and you can start adding mcollective plugins to do useful stuff like manage services and the puppet agent runs etc.

Adding more admin users just requires generating certificates for the user, copying them to the mcollective files repository in your “project” module, and adding them to the site::mcollective::users hash. Then they get the config added to their homedir automatically.

I can see this being rather useful as the number of machines we are managing continues to grow 😀

Docs which were useful / further reading

• The standard mcollective deployment guide covers this process in much more depth
• The component overview for mcollective
• The github docs for the puppetlabs-mcollective module which describe the various classes and defines.