Automate Let's Encrypt Certificate Install on VMware vSphere ESXi

Overview

We’ll use Ansible playbooks that will do the following:

• Create a private key and CSR on local machine.
• Generate a Let’s Encrypt Certificate with a common name and multiple SANs for each host.
• Use DNS for authentication (Cloudflare Ansible Module)
• Copy the private key and certificate to multiple hosts and restart the hostd service.

The following steps have been tested on three VMware vSphere ESXi 6.7 virtual machines running on a MacBook Pro.

Prerequisites

You’ll need to enable SSH on your ESXi hosts and prepare them to run Ansible by following the steps in this tutorial: How to Connect to an ESXi host with Ansible.

Once you’ve enabled SSH on each host, we’ll begin by creating the inventory file.

Step 1: Create Inventory File

First of all create a folder on your local machine to store the following files in. For example, mkdir ~/esxi.

Now create the inventory.yml file by running.

vim inventory.yml

Add the following contents to the file (replacing the hosts with yours).

---
esxi:
  hosts:
    esxi01.graspingtech.com:
    esxi02.graspingtech.com:
    esxi03.graspingtech.com:
  vars:
    ansible_python_interpreter: /usr/bin/python3
    ansible_connection: ssh
    ansible_user: root
    ansible_ssh_private_key_file: ~/.ssh/id_rsa

Remember to make sure the hosts in the file above are resolvable, either by DNS or editing the /etc/hosts file.

Step 2: Create Vars File

Next we’ll create the file containing our variables.

Now create the vars.yml file by running.

vim vars.yml

Add the following contents.

---
certs_path: ~/lets-encrypt
crt_common_name: vc01.graspingtech.com
crt_subject_alt_name:
  - esxi01.graspingtech.com
  - esxi02.graspingtech.com
  - esxi03.graspingtech.com
cf_zone: graspingtech.com
cf_account_email: cloudflarelogin@example.com
cf_account_api_token: YOUR_CF_API_KEY

The crt_common_name variable contains the common name of the certificate. You can use your primary domain name or subdomain, the example above uses my vCenter hostname as the common name.

crt_subject_alt_name contains a list of all the fully qualified domain names of your ESXi hosts. Let’s Encrypt will allow 100 domains to be used here. If you have more, you’ll need to split it into multiple certificates.

The cf_zone, cf_account_email and cf_account_api_token are used by the Ansible cloudflare_dns module to create TXT records that Let’s Encrypt can use to validate you own the domain name. If you don’t use Cloudflare for your DNS, there’s a module for Amazon Route 53 or you can modify the SSL playbook to use HTTP authentication instead.

Step 3: Create SSL Playbook

The SSL playbook will run on your local machine. It will use the Ansible acme_certificate module to automatically generate and validate a Let’s Encrypt certificate that will work on multiple ESXi hosts in your cluster.

Create the ssl.yml file by running.

vim ssl.yml

Add the following contents.

---
- hosts: localhost
  gather_facts: no
  vars_files:
    - vars.yml
  tasks:
  - name: create directory to store certs
    file:
      path: "{{ certs_path }}"
      state: directory

  - name: generate account key
    openssl_privatekey:
      path: "{{ certs_path }}/account-key.pem"
      size: 2048

  - name: generate signing key
    openssl_privatekey:
      path: "{{ certs_path }}/{{ crt_common_name }}.pem"
      size: 2048

  - name: generate csr
    openssl_csr:
      path: "{{ certs_path }}/{{ crt_common_name }}.csr"
      privatekey_path: "{{ certs_path }}/{{ crt_common_name }}.pem"
      common_name: "{{ crt_common_name }}"
      subject_alt_name: "DNS:{{ crt_subject_alt_name | join(',DNS:') }}"

  - name: create acme challenge
    acme_certificate:
      acme_version: 2
      terms_agreed: yes
      account_key_src: "{{ certs_path }}/account-key.pem"
      src: "{{ certs_path }}/{{ crt_common_name }}.csr"
      cert: "{{ certs_path }}/{{ crt_common_name }}.crt"
      challenge: dns-01
      acme_directory: https://acme-v02.api.letsencrypt.org/directory
      remaining_days: 60
    register: challenge

  - name: create cloudflare TXT records
    cloudflare_dns:
      domain: "{{ cf_zone }}"
      record: "{{ challenge.challenge_data[item]['dns-01'].record }}"
      type: TXT
      value: "{{ challenge.challenge_data[item]['dns-01'].resource_value }}"
      solo: true
      account_email: "{{ cf_account_email }}"
      account_api_token: "{{ cf_account_api_token }}"
      state: present
    with_items: "{{ [crt_common_name] + crt_subject_alt_name }}"
    when: challenge is changed
  
  - name: validate acme challenge
    acme_certificate:
      acme_version: 2 
      account_key_src: "{{ certs_path }}/account-key.pem"
      src: "{{ certs_path }}/{{ crt_common_name }}.csr"
      cert: "{{ certs_path }}/{{ crt_common_name }}.crt"
      fullchain: "{{ certs_path }}/{{ crt_common_name }}-fullchain.crt"
      chain: "{{ certs_path }}/{{ crt_common_name }}-intermediate.crt"
      challenge: dns-01
      acme_directory: https://acme-v02.api.letsencrypt.org/directory
      remaining_days: 60
      data: "{{ challenge }}"
    when: challenge is changed

  - name: delete cloudflare TXT record
    cloudflare_dns:
      domain: "{{ cf_zone }}"
      record: "{{ challenge.challenge_data[item]['dns-01'].record }}"
      type: TXT
      account_email: "{{ cf_account_email }}"
      account_api_token: "{{ cf_account_api_token }}"
      state: absent
    with_items: "{{ [crt_common_name] + crt_subject_alt_name }}"
    when: challenge is changed

Step 4: Create Deploy Playbook

Next we’ll create a playbook that deploys the validated Let’s Encrypt certificate to all the hosts in the inventory.yml file.

Create the deploy.yml file by running.

vim deploy.yml

Add the following contents.

---
- hosts: esxi
  vars_files:
    - vars.yml
  tasks:
  - name: copy key to host
    copy:
      src: "{{ certs_path }}/{{ crt_common_name }}.pem"
      dest: "/etc/vmware/ssl/rui.key"

  - name: copy cert to host
    copy:
      src: "{{ certs_path }}/{{ crt_common_name }}-fullchain.crt"
      dest: "/etc/vmware/ssl/rui.crt"
    register: cert

  - name: restart hostd
    command: /etc/init.d/hostd restart
    when: cert is changed

Step 5: Create Final Playbook

The final playbook is for convenience so that both playbooks will run one after the other.

Create the play.yml file by running.

vim play.yml

Add the following contents.

---
- import_playbook: ssl.yml
- import_playbook: deploy.yml

Step 6: Run the playbook

Run the playbook with the following command:

ansible-playbook -i inventory.yml play.yml

After a successful run, you should notice the ~/lets-encrypt directory contains the private key and certificates and they should have also been copied and installed to each host specified in the inventory.yml file.

When you visit the login screen of the ESXi host, you’ll see the Let’s Encrypt certificate has been applied.

You can also see it in the Certificates section of the management screen.

Conclusion

In this tutorial we used Ansible to generate a Let’s Encrypt certificate from a local machine and have it installed to multiple ESXi hosts in a cluster. By extending the inventory file we’re able to deploy it to thousands of hosts by running one simple command.

The authentication method we used was DNS based with the Cloudflate API. If you’re not using Cloudflare, there’s also an Ansible module for Amazon Route 53, or if you don’t want to use DNS, you can modify the SSL playbook to use HTTP authentication instead.