How to make Vagrant and Puppet to clone private github repo

I have been playing with Vagrant and Puppet lately to automate my home development environment and got to the point when I needed to clone / build code from several private github repos. I ended up spending two days to make this work as it turned out to be more difficult then I expected.

1. To start with you need to have working SSH private / public keys on your host computer. Public key needs to be added to your github private repo. Follow this instructions to setup and test.

2. Now you have a private key on your machine and public key sitting on github side. You also want to enable SSH agent forwarding to save you time of typing in the passphrase associated with your private key every time you login to github. Here are the instructions how to enable ssh forwarding.

3. At this point you probably want to get you Vagrant provisioning process to be able to login to your private github repo as well. There are two ways of doing this:

  1. you can setup a Vagrant task which will find your private and public keys and copy them over to Vagrant VM (into /root/.ssh because Vagrant runs provisioning using root account). This is possible but not very convenient.
  2. much better way is to enable Vagrant VM to use already existing ssh keys from your host machine. This will allow you to share your Vagrantfile with other developers who might have access to your private repo and they will be able to use their SSH keys

Add the following to your Vangrantfile:

 config.ssh.private_key_path = [ '~/.vagrant.d/insecure_private_key', '~/.ssh/id_rsa' ]
 config.ssh.forward_agent = true

You also need to add “github.com” hostname into a list of ssh known hosts in your Vagrant VM.

The problem is even if you enable ssh forwarding from your Vagrant VM, when cloning job makes a first time connection to github it will get the following message and fail

RSA key fingerprint is 16:27:ac:a5:7c:28:2d:36:63:2b:56:4d:eb:df:a6:48.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added 'github.com,205.232.175.93' (RSA) to the list of known hosts.

One way around this is to set option StrictHostKeyChecking=no for ssh forwarding agent but this opens you up to certain security risk. Better way is to add github.com to /root/.ssh/known_hosts as part of provisioning. You can do it with the following rule in Vagrantfile which needs to be executed before you make ssh connection to github:

  # add github to the list of known_hosts
  config.vm.provision :shell do |shell|
    shell.inline = "mkdir $1 && touch $2 && ssh-keyscan -H $3 >> $2 && chmod 600 $2"
    shell.args = %q{/root/.ssh /root/.ssh/known_hosts "github.com"}
  end

or ( better! ) use the following Puppet module:

 # -*- mode: ruby -*-
# vi: set ft=ruby :

class known_hosts( $username = 'root' ) {
    $group = $username
    $server_list = [ 'github.com' ]

    file{ '/root/.ssh' :
      ensure => directory,
      group => $group,
      owner => $username,
      mode => 0600,
    }

    file{ '/root/.ssh/known_hosts' :
      ensure => file,
      group => $group,
      owner => $username,
      mode => 0600,
      require => File[ '/root/.ssh' ],
    }

    file{ '/tmp/known_hosts.sh' :
      ensure => present,
      source => 'puppet:///modules/known_hosts/known_hosts.sh',
    }

    exec{ 'add_known_hosts' :
      command => "/tmp/known_hosts.sh",
      path => "/sbin:/usr/bin:/usr/local/bin/:/bin/",
      provider => shell,
      user => 'root',
      require => File[ '/root/.ssh/known_hosts', '/tmp/known_hosts.sh' ]
    }
}

this rule will create /root/.ssh directory and execute the following known_hosts.sh

#!/bin/bash
array=( 'github.com' )
for h in "${array[@]}"
do
    #echo $h
    ip=$(dig +short $h)
    ssh-keygen -R $h
    ssh-keygen -R $ip
    ssh-keyscan -H $ip >> /root/.ssh/known_hosts
    ssh-keyscan -H $h >> /root/.ssh/known_hosts
 done

you need to add this bash script into ‘files’ directory of your module, e.g. for me it lives in “puppet/modules/known_hosts/files/known_hosts.sh”

All you need now is the rule to clone the github repo, I am using Puppet module vcrepo and it looks like this:

    vcsrepo { "/opt/code/${repo}":
      ensure => latest,
      owner => $username,
      group => $username,
      provider => git,
      require => [ Package[ 'git' ] ],
      source => "git@github.com:<your account name>/<your project name>.git",
      revision => 'master',
    }

Here we clone from github into local Vagrant directory “/opt/code/repo” and this directory will be owned by username that you can define.

You can also add a rule to build your code:

    exec{ 'make': 
      command => "make",
      environment => "HOME=/home/${username}",
      cwd => "/opt/code/${repo}",
      path => "/sbin:/usr/bin:/usr/local/bin/:/bin/",
      require => [ Vcsrepo[ "/opt/code/${repo}" ], Package[ 'erlang' ] ],
    }

Hopefully these instructions will save you some time.

Advertisements
This entry was posted in Vagrant and tagged , , , . Bookmark the permalink.

3 Responses to How to make Vagrant and Puppet to clone private github repo

  1. Josh N says:

    You may want to add the following edit:
    mode => ‘0755’,
    for the shell script, otherwise it won’t execute

  2. scamartist26 says:

    The `config.vm.provision` is just what I needed, thank you! I am having a problem now however where there is no check to see if the file exists should I run `vagrant provision` again. I am not familiar with Ruby. How can I run the `config.vm.provision` shell only on condition that the .ssh/known_hosts file does not exist?

    Thanks!

  3. Piotr Pasich says:

    This line – shell.inline = “mkdir $1 && touch $2 && ssh-keyscan -H $3 >> $2 && chmod 600 $2” will work better when mkdir will have -p flag

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s