[author:alex]
Perhaps the title should be “Automatically creating Vagrant boxes with Jidoteki”, but as some of you might already know, Jidoteki is Japanese for: automatically.
One of the features we’ve had at the back of our minds since the beginning of Jidoteki, was having the ability to create a Vagrant box from scratch. I won’t write specifically about Vagrant because there’s lots of information out there on how to use it.
Rather, I want to explain what we’ve done to make the creation (export) possible, and how we automated it.
Starting with Jidoteki
Your first task when using Jidoteki, is to choose an OS (operating system) to be installed. You can customize the installation if you want, but we opted to use our standard minimal pre-installed OSes (to save time).
The second step is to provision the OS. We love Ansible, and we love Bash, so we created 2 things:
We provision the OS by providing 3 URLs. The first URL is the RAW debian bootstrapping script hosted on GitHub. The second URL is the contents of the entire repo containing our vagrant/provisioning files. The third URL is the tarball of Ansible version 1.3.0.
Jidoteki will upload those 3 files to the OS, and then execute the first script (the bootstrapping script written in Bash).
A few minutes later, we had a fully-provisioned Vagrant box ready to be shared with the world.
You can find our Vagrant / Ansible Playbooks and scripts here.
To avoid duplication, we wrote our Ansible playbook to run on every platform we support (CentOS, Debian, FreeBSD).
That’s essentially how Jidoteki is designed to provision an OS. You can provision using anything you want: chef-solo, puppet apply, ./magic_install_script.
So how does it actually work?
The boostrapping script expects the 2 other files to be present in the /root directory.
You can start by looking at the bootstrap script for Debian. It installs some dependencies, extracts the ansible tarball and extracts the playbooks, runs the Ansible playbook called vagrant.yml, and then cleans up after itself.
You always want to make your bootstrap/provisioning scripts exit with 0 if it succedes, or exit with non-zero if it fails. Jidoteki catches that exit code, stdout, and stderr at the end of the provisioning.
You’ll notice something interesting. We run Ansible in local mode on the OS itself. This is why we install the dependencies first. We thought it would be a pretty big security risk to run untrusted Ansible playbooks over SSH, so we just dump Ansible and all the files on the OS, and run it locally. This is a bit of a headache, but it gives you ELEVENTY BILLION levels of flexibility in regards to what you want to do. We don’t force you to write your scripts/recipes/playbooks/manifests in any specific way, and we don’t try to abstract things for every possible configuration management system. In other words, we support everything :)
The vagrant playbook simply creates the vagrant user, sets the passwords, downloads the insecure public SSH key and disables DNS as a small SSHd fix.
Once this is complete, Jidoteki handles creating the actual Vagrant .box file by inserting the required metadata.json (virtualbox provider) and a READTHIS.md file which contains some basic information about the box, where it came from, and a picture of a cat.
Nice feature
We added a small feature that’s kind of fun:
Just like an OS installation, every minute Jidoteki will take a screenshot of the console. Here’s the fun part, when we launch your bootstrap script, we append “ | tee /dev/console” which makes all output go to the console as well.
This way you can even see the progress of the provisioning directly in the screenshots, without having to wait for it to complete in order to know if it’s working correctly. Have a look:
That’s about it. At the moment our Vagrant boxes are not yet public. We’re still trying to figure out how to distribute them without blowing up our bandwidth costs. I’m thinking of using torrents, but Vagrant doesn’t handle torrent file downloads.. perhaps I should add that feature? ;) ;)
In any case, we’ll make sure to announce when the Vagrant boxes are publicly available for download.