Efficient Vagrantfiles – Part 3 – Code and data separation

Emerson Prado - 14/01/2020

In the second part of this series, we got rid of lots of repeated code (and open doors for typing mistakes) by using arrays, hashes, loops and conditionals. In this part, we'll take a break from Vagrantfiles and check out a file format that will be quite useful once we get back to them.

TL;DR
Problem - messing with code to each environment change
Solution - each thing where it belongs
Yaml
Next step

TL;DR

If you already know about YAML files, you might skip this part of the articles, and get going with what I've summarized below.

Code and data mixed up in the Vagrantfile makes maintenance and versioning messy. We can make it easier and more organized by putting data in YAML files and load them into the Vagrantfile.

YAML files can store objects like strings, numbers, arrays and hashes. These last two can be nested. Some exaples to refresh your mind (read each one as a file):

  ---
  # Start with a line with 3 dashes, telling the OS it's an YAML file
  # It MUST be the FIRST line and THREE dashes. Exactly.
  # Hash signs indicate comments. Standalone or inline.
  # This file has a number
  3.14

  --- # Another file
  This file has this string

  --- # Array (items classes can vary)
  - One
  - -3.14
  - Blowin' in the Wind
  # Items MUST be prepended by ONE dash and ONE space. Exactly.

  --- # Hash (keys and values can vary)
  One: 1
  2: Two
  Multi-word key: Multi-word value
  "Key with colon: just quote it": Value
  # Keys MUST be prepended by NOTHING and appended by ONE colon. Exactly.
  # Vaues MUST be separated from keys by spaces.

  --- # Nesting from an array
  - # Outer array item 1 is middle array below
    - item_1_1
    - item_1_2
    - # More nesting - Middle array item 3 is inner array below
      - item_1_3_1
      - item_1_3_2
    - # Nesting hashes into arrays
      key_1_4_1: value_1_4_1
      key_1_4_2: value_1_4_2
  - # Outer array item 2
    ...

  --- # Nesting from a hash
  key_1: # Outer hash element 1 value is middle hash below
    key_1_1: value_1_1
    key_1_2: value_1_1
    key_1_3: # More nesting - Middle hash element 3 value is inner hash below
      key_1_3_1: value_1_3_1
      key_1_3_2: value_1_3_2
    key_1_4: # Nesting arrays into hashes
      - item_1_4_1
      - item_1_4_2
  key_2: # Outer hash element 2
    ...

For the Vagrantfile integration part, go to part 4 (coming soon).

Problem - messing with code for each environment change

Your environment will change. Period. You might have to test an application's new version, or debug something in it, or try it in more OS options, or update boxes, etc. That's one of the main reasons you're using Vagrant in first place. In the common Vagrant use, every change means editing your Vagrantfile. This means trouble because:

You end up sometimes having a big Vagrantfile with everything in it. Changing things becomes increasingly more difficult.
When you have to edit code, you have data around. And vice-versa. It's harder to focus in just one of them, and easier to mess with the other.
Reusing Vagrantfiles becomes tedious - you have to copy and paste Vagrantfiles then manually edit changes.
Version tracking becomes a mess - no matter what had to change, your versioning system will always show the same change: Vagrantfile.

Solution - each thing where it belongs

If the problem is code and data mixed up, the solution is obvious: let's take them apart.

Since the Vagrantfile is intended to have code describing your environment, it's already the right place for code. So, the data should be somewhere else.

A good way to store data is, you bet, in files which format is intended to store data. Lucky we, there's one such format which is a close friend to Ruby.

Yaml

YAML is a structured file format which can store a Ruby object, which can be a single value, an array, a hash, or nested arrays/hashes, without much hassle (given you get familiar to the picky format).

So, placing data in YAML files makes it easy to load data into the Vagrantfile. Then, the Vagrantfile will be only code, and the YAML files, only data. Cheers!

Basics

The YAML file contents format depends primarily on the type of object it contains:

Start with 3 dashes in the first line. This tells the OS it's an YAML file. Not quite required, but highly recommended.
For a single value, just put it plainly in the additional line. It could be next to the dashes in the same line, but would look weird.
For an array, put each item in a line, prepended by one dash and one space. I do mean one. Remember: YAML is picky!
For a hash, put each key in the beginning of a line, followed by a colon, then space(s), and the value. I do mean the beginning, and also space(s) between the colon and the value. Remember: YAML is picky!
Comments are indicated by a hash sign.

Examples (each one is a file)

  ---
  # A number
  3.14

  ---
  # A string
  Hello, world!

  ---
  # An array (items classes can vary)
  - One
  - -3.14
  - Blowin' in the Wind

  ---
  # A hash (keys and values classes can vary)
  One: 1
  2: Two
  Multi-word key: Multi-word value
  "Key with colon: just quote it": Value

Nesting

Just as arrays and hashes can be nested in Ruby objects, so can they in YAML files. Nesting is indicated by indentation, plus some special attention in "outer" objects:

Start "outer" objects as normal, but omitting the contents - the array item or the hash value. That is:
1. For an "outer" array, just put the dash
2. For an "outer" hash, just put the key and the colon
Indent "inner" objects with two spaces. I do mean two. Remember: YAML is picky!
For additional nesting, do exactly the same, but indented with two additional spaces per depth level. I do mean two. Remember: YAML is picky!
Never indent with tabs. Ever! Don't know if I already warned you, but YAML is picky!

Examples (each one is a file)

  ---
  # Nesting from an array
  - # Outer array item 1 is middle array below
    - item_1_1
    - item_1_2
    - # More nesting - Middle array item 3 is inner array below
      - item_3_1
      - item_3_2
    - # Nesting hashes into arrays
      key_1_4_1: value_1_4_1
      key_1_4_2: value_1_4_2
  - # Outer array item 2
    ...

  ---
  # Nesting from a hash
  key_1: # Outer hash element 1 value is middle hash below
    key_1_1: value_1_1
    key_1_2: value_1_1
    key_1_3: # More nesting - Middle hash element 3 value is inner hash below
      key_1_3_1: value_1_3_1
      key_1_3_2: value_1_3_2
    key_1_4: # Nesting arrays into hashes
      - item_1_4_1
      - item_1_4_2
  key_2: # Outer hash element 2
    ...

  ---
  # Is this trip really necessary?
  key_1:        # This element value is the array below
    -               # This item value is the hash below
      key_1_1_1:        # This element value is the array below
        - item_1_1_1_1
        - item_1_1_1_2
      key_1_1_2:
        - item_1_1_2_1
        - item_1_1_2_2
    -
      key_1_2_1: ...
  key_2:
    ...

References

For real YAML learning, check out:

This nice overview from Ansible docs. It is quite a thorough overview, but still a simple and light-reading one.
A full tutorial from Tutorials Point. Walks you through everything (I guess).
The official manual. Official manuals should always be at hand.

Next step

Now, all we have to do is to take the huge hash from part 2, move it to an YAML file, then load the object in the Vagrantfile. On to part 4 (coming soon)!