Ansible secrets and vaults

One chicken-and-egg-problem with setting up servers is taking care of secrets. Secrets are hard, and they’re especially hard to keep secret. When working with systems management, you really want to keep your secrets secret, but you also want to share them with your peers. In essence, this is a conflict of interests and a hard problem to solve in a truly usable manner. But it’s not impossible.

Lately, i’ve been re-introducing myself to Ansible because it’s a way to efficiently communicate intent between peers. I still like the philosophy of Puppet more – a topic for a completely different discussion – but as my peers prefer Ansible to Puppet, then in the interest of mutual intelligibility, efficiency and common dispenility¹, i’m fine with Ansible.

Since last time, Ansible has introduced Collections, essentially a way to package a bunch of roles under the same umbrella and started referring to stuff with a Fully Qualified Collection Names (absolute “dot notation” like fully qualified domain names) in favour of “shortnames”. I guess i can handle it. There’s probably more too, but a key part of learning is discovering stuff you don’t know (and then shamelessly blogging about it).

Ansible also has a way of handling secrets, namely the Vault.² In short, it’s just a way of encrypting data “at rest” (variables and their values when stored on disk)³ in a way that it can be used fluently in your Ansible code. Vaults are protected by passwords, which can be given at runtime or stored in a file. As a new thing for Ansible, you can have several vault passwords for storing different secrets. I’m not sure i understand why just yet, but once i get it, i might post it. Or update this.

One way to go about things would be along these lines: Define a Vault Identity, essentially a password with a name; you can have several vault identities and you tell one from the other using its name (or more accurately, identifier). Then tell Ansible that said password can be found in a file. Here i’d call a password “pass” (yeah, not very imaginative) and it is stored, unencrypted, in the file .vault-pass in the project root.

# ansible.cfg snippet
---
[defaults]
...
vault_password_file=.vault-pass
vault_identity_list=pass@.vault-pass
...

(Here i should note that i’ve had varying degrees of success; on one machine, i didn’t need to specify the vault_password_file as it was already listed in the vault_identity_list whereas on a second one, both config lines were needed. The problem is most certainly elsewhere, but i haven’t quite caught it yet. YMMV.)

You should chmod 600 .vault-pass to protect it from possible evildoers and make sure the password doesn’t end up in your git repository:

# .gitignore snippet
.vault-pass
...

The vault-pass itself should be stored in your password management system (eg. gopass) in a manner that it can be securely shared in your group or with yourself if you’re working on multiple computers. And you are, because you’re a sysadmin, right? :)⁴

How to vault

On to the secrets then! To create a fully encrypted file, use the incantation ansible-vault create secrets.yml. Here, secrets.yml is the name of the encrypted file. Typically, you’d have it within one of your variables folders, either as something you would have automagically included in your run (by use of groups, hostnames or default variable files) or explicitly referred to by having something like this at the top of your playbooks:

include_vars: vars/secrets.yml

To edit your secrets, the command to use is ansible-vault edit secrets.yml, though as a productive developer/sysadmin/devop, you would probably install a plugin to your favvo editor to manipulate these secrets in-flow.

A variable file can also contain a mix of plaintext and encrypted. This is not as bad as it sounds if you have a web developer background, but the execution is a bit clunky. Send the string to encrypt to the command ansible-vault encrypt_string either interactively or by redirection, then copy the encrypted data blob and paste it into your regular varible yaml file. Note that the ride will be smoother if you did set up the vault identity from ansible.cfg.

ansible-vault encrypt_string --vault-password-file .vault_pass --name hello World

If you don’t want to leave the string to encrypt in the command line history, use the following incantation. Note that the flag name is replaced by stdin-name (which i think is a sign of lazy programming, but what do i know about these things).

ansible-vault encrypt_string --vault-password-file .vault_pass --stdin-name hello

You can then either enter the string to encrypt in interactive mode or use your favoured way to paste and pipe the secret to ansible-vault (eg pbpaste | ansible-vault encrypt_string ...)

With the secrets encrypted, it’s now possible to store them with at least a bit of confidence on a version control system such as git.

This also has a nice side effect. If you find yourself storing secrets on a publicly accessible git repo, it should serve you as a reminder to ask yourself whether you’re doing the right thing. The secrets may be unhackable or they may not. But if they’re secrets, maybe that’s an indicator that you should at least access-control who can get to them, as those secrets probably contain a hint that they shouldn’t be for everyone anyway?