We at Isos recently had a need to test working with multiple minions in Salt Stack. The example provided by renoirb on GitHub is a good example on how to use one Vagrantfile to spin up a master and a minion with pre-seeded keys. I found there are some critical steps that are not discussed. I had to spend quite a bit of time digging to figure out what those missing pieces are. This blog should help anyone with basic knowledge of Vagrant and Salt Stack create a multi-box environment for Enterprise Application Development within minutes.
enterprise application development

PROJECT DIRECTORIES SETUP

  1. Start by creating a new project directory to serve as your workspace, and create the following directories in your workspace (the root of your project directory):
    • multiples-example/salt/key/

      This directory will contain pem and pub files for the master and the minion.  See How keys can be obtained for more info.

    • multiples-example/salt/roots/

      This directory will hold salt states, pillar files and top.sls file.

    • multiples-example/salt/master:

      Salt master config file.

    • multiples-example/salt/minion:

      Salt minion config file.

  2. Open a terminal and change directory to your new workspace (i.e. multiples-example).
  3. Type “vagrant init” to generate a new default Vagrantfile and then open it in your favorite text editor.
  4. Copy and paste over the default contents, using the example Vagrantfile code from the link below:
  5. Then create the master and minion config files based on the general template examples found in the links below and read the next section for the minimal settings:
  6. Test some Salt Stack state files.

MINIMUM SETTINGS IN THE MASTER AND MINION CONFIG FILES

The default master and minion config files basically have every option you could need, except everything is commented out. Maybe I didn’t dig deep enough, but I couldn’t find anyone explaining what the minimal requirements are for a multiple machine environment to work. I’m going to show you the minimum editing necessary to make a master-minion environment work. Other config options are available to use.
In the master file, un-comment and make edits to the following lines:

publish_port: 4505
ret_port: 4506
pki_dir: /etc/salt/pki/master
 ## Edit to be
 pki_dir: /etc/salt/keys
open_mode: False
 ## Edit to be
 open_mode: True
file_roots:
 base:
 - /srv/salt
 ## Edit to be
 - /srv/

In the minion file, un-comment and make edits to the following lines:

master: localhost
## Edit to be
master: 10.10.10.2
master_port: 4506
pki_dir: /etc/salt/pki
##Edit to be
pki_dir: /etc/salt/keys

HOW KEYS CAN BE OBTAINED

If you have Salt Stack already installed locally, you can generate the keys locally. When I was setting this up, I didn’t have that and needed a machine from which to generate the keys.
So now that we have the directories and config files created, we can spin up the master VM to create the keys necessary to pre-seed the master and minions so that they can all communicate when they are started.

    1. First, in your Vagrantfile, temporarily comment out the following (this is so just the master can be brought up with salt stack installed):
# config.minion_config = "salt/minion"
# config.master_config = "salt/master"
# config.minion_key = "salt/key/minion.pem"
# config.minion_pub = "salt/key/minion.pub"
# config.master_key = "salt/key/master.pem"
# config.master_pub = "salt/key/master.pub"
# config.seed_master = {salt: "salt/key/master.pub", app0: "salt/key/minion.pub"}
    1. Start the master VM. This will start only the master VM which is defined as “salt” in the Vagrantfile:
multiples-example$ vagrant up salt
    1. When the machine completes the build process, type:
multiples-example$ vagrant ssh salt
    1. Then type the following to generate the keys for the master and the minion:
vagrant@master:~$ sudo salt-key --gen-keys=master
vagrant@master:~$ sudo salt-key --gen-keys=minion
    1. Now you should have the following new files
master.pem
master.pub
minion.pem
minion.pub
    1. Copy these key files to the /etc/salt/keys directory which is shared with the host OS in “multiples-example/salt/key”
vagrant@master:~$ sudo cp master.pem /etc/salt/keys/
vagrant@master:~$ sudo cp master.pub /etc/salt/keys/
vagrant@master:~$ sudo cp minion.pem /etc/salt/keys/
vagrant@master:~$ sudo cp minion.pub /etc/salt/keys/
    1. Now you can exit the ssh session, and destroy the VM by typing
multiples-example$ vagrant destroy salt
  1. Next open the Vagrantfile and un-comment the lines from step 1, save the file, and type “vagrant up” in the terminal to create the two VM’s with the keys automatically accepted.

The master (salt) might come up and be unable to accept minion keys.
Part of the process is to send the master the minion key to be accepted, and the master should auto-accept.

Waiting for minion key...
Salt Master did not receive minion key.

If you see this, the process of bringing up both VM’s has stopped. So, you have to type “vagrant up” again.
It will recognize that salt is already created and running and will only bring up the minion (app0).

TEST OUT SOME SALT STATES

    1. In your roots directory, add a top.sls file and paste the following…
base:
  '*':
  - git
  - java7
    1. For this example we will install Oracle Java 7 and Git. So, set up the directories like below:
enterprise application development
    1. In each of the app directories, create an init.sls file.

For Git, paste into the ../roots/git/init.sls

git:
  pkg.installed:
    - name: git

For Java, paste into the ../roots/java7/init.sls

java7_ppa:
  pkgrepo.managed:
    - ppa: webupd8team/java
    - require_in:
    - pkg: jdk7
jdk7-accept-license:
  cmd.run:
    - name: "echo oracle-java7-installer shared/accepted-oracle-license-v1-1 select true | /usr/bin/debconf-set-selections"
    - unless: "debconf-get-selections | grep -q shared/accepted-oracle-license-v1-1"
    - user: root
jdk7:
  pkg.installed:
    - name: oracle-java7-installer
    - requires:
      - cmd: jdk7-accept-license
    - pkgrepo: java7_ppa
    1. Type “vagrant up” and then “vagrant ssh salt” to login to the master VM.
multiples-example$ vagrant up
Bringing machine 'salt' up with 'virtualbox' provider...
Bringing machine 'app0' up with 'virtualbox' provider...
[salt] Importing base box 'precise64'...
...
multiples-example$ vagrant ssh salt
    1. Type this to verify the keys are accepted (you should see “app0” and “master”):
vagrant@master:~$ sudo salt-key -L
Accepted Keys:
app0
master
Unaccepted Keys:
Rejected Keys:
    1. Running this will install java and git on both the master and the minion.
vagrant@master:~$ sudo salt '*' state.highstate
      Type the minion’s name in place of ‘*’ to only install on the minion.
vagrant@master:~$ sudo salt 'app0' state.highstate