How to manage public and private Puppet modules with Vagrant

If you use Puppet for provisioning Vagrant VMs, the chances are that you want to use some standard public Puppet modules which could be found at Puppet Forge and the question becomes how to get them installed as part of provisioning process? You see, you can’t just download them as part of your Puppet provisioning as Puppet catalog will not compile because the modules will be missing at the time of Puppet run. It is a chicken and egg situation.

You can, of course, manually download the modules from Forge and store them in the repository of your project in puppet/modules directory alongside with your own Puppet code. This is rather messy, you have to manually maintain the dependencies and it makes your repository bigger than necessary. There has to be a better way of doing this.

I did some reading and found few ideas how to deal with this issue. One possibility is to make use of Puppet module Librarian-puppet. Librarian-puppet, once installed, allows you to specify a list of Puppet modules that your infrastructure depends on in Puppetfile and it will manages the installation of these modules and their dependencies for you.

The trick is you have to install Librarian and make it install modules from Forge *before* you can run your Puppet provisioner, otherwise Puppet will complain about missing modules while compiling the catalog. So, we need to use shell provisioner, get Librarian-puppet installed and then use it to install modules from Forge. After that you can safely use Puppet provisioner to run your own Puppet code knowing that all the dependencies that your code needs have been already installed.

Here are the steps:

In your Vagrantfile add the following line:

  # install librarian-puppet and run it to install puppet common modules.
  # This has to be done before puppet provisioning so that modules are available
  # when puppet tries to parse its manifests
  config.vm.provision :shell, :path => "provision/shell/main.sh"
 

It should be added *before* any puppet provisioners.

This will run main.sh which for lives in /provision/shell directory and looks like this:
(I borrowed this script from this project , most of the ideas in this post come from it)

#!/usr/bin/env bash

# Directory in which librarian-puppet should manage its modules directory
PUPPET_DIR=/vagrant/provision/puppet

# NB: librarian-puppet might need git installed. If it is not already installed
# in your basebox, this will manually install it at this point using apt or yum

$(which git > /dev/null 2>&1)
FOUND_GIT=$?
if [ "$FOUND_GIT" -ne '0' ]; then
  echo 'Attempting to install git.'
  $(which apt-get > /dev/null 2>&1)
  FOUND_APT=$?
  $(which yum > /dev/null 2>&1)
  FOUND_YUM=$?

  if [ "${FOUND_YUM}" -eq '0' ]; then
    yum -q -y makecache
    yum -q -y install git
    echo 'git installed.'
  elif [ "${FOUND_APT}" -eq '0' ]; then
    apt-get -q -y update
    apt-get -q -y install git
    echo 'git installed.'
  else
    echo 'No package installer available. You may need to install git manually.'
  fi
else
  echo 'git found.'
fi

if [ "$(gem search -i librarian-puppet)" = "false" ]; then
  gem install librarian-puppet
  cd $PUPPET_DIR && librarian-puppet install --path modules-contrib
else
  cd $PUPPET_DIR && librarian-puppet update
fi

Main.sh cd-es into your provision/puppet directory which will be mounted to Vagrant VM as /vagrant/provision/puppet and installs Librarian. It then reads Puppetfile and installs the Puppet modules defined in Puppetfile into provision/puppet/modules-contrib directory on your host machine. Librarian-puppet installs modules to the directories relative to location of Puppetfile and you can specify the name of the root directory by passing –path param to librarian-puppet when it is run first time:

librarian-puppet install --path modules-contrib

Puppetfile just gives a location of forge and lists modules that you want to pull off forge (the format is flexible enough to be able to pull code repos from github as well) :

# Puppetfile
# Configuration for librarian-puppet. For example:
# 
forge "http://forge.puppetlabs.com"
mod "garethr/docker"
mod "camptocamp/archive"
mod "puppetlabs/vcsrepo"
mod "maestrodev/wget"
mod "puppetlabs/git"

And finally, in your Vagrantfile for Pupper provisioner configuration you need to define “module_path” and list the locations of your custom puppet modules and the public ones which you installed with Librarian:

  # Provide basic configuration, install git
  config.vm.provision "puppet" do |d|
    d.manifests_path = 'provision/puppet/manifests'
    d.manifest_file = 'site.pp'
    d.module_path = [ 'provision/puppet/modules-contrib', 'provision/puppet/modules' ]
    #d.options = "--verbose --debug"
  end

make sure that empty “modules-contrib” folder exists in your project, it needs to be there at the time when you do ‘vagrant up’ otherwise Vagrant won’t be able to mount this folder to your VM.

git doesn’t allow to keep empty directories but you can get around this by adding the following .gitignore into modules-contrib folder.

# Ignore everything in this directory
*
# Except this file
!.gitignore
Advertisements
This entry was posted in Puppet, Vagrant and tagged , . Bookmark the permalink.

2 Responses to How to manage public and private Puppet modules with Vagrant

  1. Thanks for your awesome post. I was really stuck on finding a solution for public and private modules using librarian-puppet. Nice work!

  2. Martin Meyer says:

    Thanks for the post. I was also struggling on how to best use librarian-puppet with Vagrant. The Vagrant plugin wasn’t really an option as I’m on a Windows machine.

    The script provided by you didn’t work out for me. But the referenced original script on https://github.com/purple52/librarian-puppet-vagrant worked on a box based on ubuntu/trusty64.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s