Using Enough

Requirements

Technical

  • The openrc.sh credentials for a Public cloud project at OVH. No other OpenStack provider is supported.
  • The Public cloud project has a Private network.

Knowledge

Enough is based on Ansible, OpenStack and Debian GNU/Linux. Debian GNU/Linux based virtual machines are created on an OpenStack tenant and provisioned with Ansible playbooks.

Each service adds more components such as Let’s encrypt, Docker or Snap. And they also implement concepts such as Virtual Private Network, Reverse Proxy or Certificate Authority.

If all goes well, Enough can be used even if the user knows nothing more than what is in this guide. When something goes wrong, fixing the problem or understanding the cause will require intimate knowledge about all of the above. If that happens, it is best to start a discussion in the forum and ask for help.

Installation

  • Install Docker.
  • Copy openrc.sh in ~/.enough/example.com/openrc.sh and edit to replace $OS_PASSWORD_INPUT with the actual password.
  • Add the enough CLI to ~/.bashrc:
eval "$(docker run --rm enoughcommunity/enough:latest install)"
  • Verify that it works:
enough --version

Create the DNS name server

Assuming the domain name (example.com for example) is registered, it must be delegated to a dedicated name server before any service can be created by Enough:

enough --domain example.com service create bind

Note

Other hosts are created such as postfix-host or icinga-host because they are part of the core services that Enough needs. They however do not require manual intervention.

Upon successfull completion, a machine named bind-host exists and its public IP must be used as a GLUE record.

$ enough --domain example.com openstack server list
+---------------+--------------+--------+-----------------------+-----------+--------+
| ID            | Name         | Status | Networks              | Image     | Flavor |
+---------------+--------------+--------+-----------------------+-----------+--------+
| e4f50405-f58b | icinga-host  | ACTIVE | Ext-Net=51.178.60.119 | Debian 10 | s1-2   |
| 75610dd4-eba7 | postfix-host | ACTIVE | Ext-Net=51.178.60.120 | Debian 10 | s1-2   |
| 2b9a1bda-c2c0 | bind-host    | ACTIVE | Ext-Net=51.178.60.121 | Debian 10 | s1-2   |
+---------------+--------------+--------+-----------------------+-----------+--------+

To verify the DNS works as expected:

dig @ip-of-the-bind-host bind.example.com

It can then be used to instruct the registrar of example.com to delegate all domain name resolutions to this IP. The exact method for performing this DNS delegation depends on the registrar (Gandi is different from OVH, etc.). But it needs to be done only once.

Note

It will take time for the delegation to be effective. Delays from 24h to 72h are not uncommon.

To verify the delegation is effective:

getent hosts bind.example.com

Create or update a service

The following services are available:

As an example, the cloud service can be created as follows:

enough --domain example.com service create cloud

Note

If the command fails, because of a network failure or any other reason, it is safe to run it again. It is idempotent.

When it completes successfully, it is possible to login https://cloud.example.com with user admin and password mynextcloud.

Restore a service

Stateless services such as bind do not need backup: they can be rebuilt from scratch if the machine hosting them fails. For instance, if bind-host is lost:

$ enough --domain example.com host create bind-host
$ enough --domain example.com playbook

However, most services such as file sharing and translations rely on persistent information that are located in a encrypted volume attached to the machine. A daily backup is made in case a file is inadvertendly lost.

Restore a service from a backup

To restore the volume attached to a service from a designated backup:

$ enough --domain example.com openstack volume snapshot list
...
| 6b75f34e | 2020-04-12-cloud-volume | None | available | 100 |
...
$ enough --domain example.com backup restore 2020-04-12-cloud-volume

In this example, the restoration is done as follows:

  • The cloud service is created, if it does not already exist.
  • The machine (cloud-host) attached to the volume (cloud-volume) is stopped. The volume is detached and deleted.
  • A new volume cloud-volume is created from the 2020-04-12-cloud-volume backup and attached to cloud-host.
  • The machine (cloud-host) is restarted.

Create a clone of a service from a backup

It is convenient to create a clone of an existing service based on a backup for:

  • testing and experimenting without disrupting production
  • verify an upgrade won’t loose any data
  • teaching
  • etc.

Assuming the credentials for the OpenStack region on which the clone must be placed are in the ~/.enough/test-clouds.yml file:

$ enough --domain example.com openstack volume snapshot list
...
| 6b75f34e | 2020-04-12-cloud-volume | None | available | 100 |
...
$ enough --domain example.com backup restore \
         --target-domain test.d.enough.community \
         --target-clouds ~/.enough/test-clouds.yml 2020-04-12-cloud-volume

Once the service is cloned, it will be available at https://cloud.test.d.enough.community. In this example, the cloning is done as follows:

  • A volume is created from the 2020-04-12-cloud-volume, in the ~/.enough/test-clouds.yml OpenStack region.
  • The cloud service is created, as well as all the services it depends on, if they do not already exist. Including the DNS server.
  • The test.d.enough.community domain is delegated to the DNS server located in the ~/.enough/test-clouds.yml OpenStack region so that https://cloud.test.d.enough.community resolves to the newly created cloud service.

Infrastructure services and access

Networks

By default all hosts are connected to two networks: one with a public IP and the other with a private IP. This can be changed by setting the openstack_network variable in ~/.enough/example.com/inventory/group_vars/all/network.yml, using this example.

The default can also be changed for a given host (for instance weblate-host) by setting the desired value in the ~/.enough/example.com/inventory/host_vars/weblate-host/network.yml file.

VPN

A VPN can optionally be installed for clients to access hosts that do not have public IPs.

enough --domain example.com service create openvpn

The certificates for clients to connect to the VPN will be created from the list in the openvpn_active_clients variable in ~/.enough/example.com/inventory/group_vars/all/openvpn.yml, using this example.

For each name in the openvpn_active_clients list, a .tar.gz file will be created in the ~/.enough/example.com/openvpn/ directory. For instance, for

---
openvpn_active_clients:
 - loic

The file ~/.enough/example.com/openvpn/loic.tar.gz will be created and contains OpenVPN credentials. The specific instructions to use them depends on the client.

Certificates

By default certificates are obtained from Let’s Encrypt. But if a host is not publicly accessible, it can be configured to obtain a certificate from a certificate authority dedicated to the Enough instance. The default for certificate_authority should be set in ~/.enough/example.com/inventory/group_vars/all/certificate.yml, using this example.

The default can also be changed for a given host (for instance weblate-host) by setting the desired value in the ~/.enough/example.com/inventory/host_vars/weblate-host/network.yml file.

Attached volumes

Provisioning

A volume can be created and attached to the host. It can be resized at a later time, when more space is needed. For instance, before creating weblate-host, the desired volume size and name can be set in the ~/.enough/example.com/inventory/host_vars/weblate-host/network.yml file like so:

---
openstack_volumes:
  - name: weblate-volume
    size: 10

Encrypting and Mounting

The volume can then be encrypted, formatted and mounted by specifying the mount point in the encrypted_device_mount_point variable like so:

---
openstack_volumes:
  - name: weblate-volume
    size: 10
encrypted_device_mount_point: /srv

By default Docker or Snap will be set to reside in the encrypted_device_mount_point directory so that the data it contains is encrypted. It can be disabled with the encrypted_volume_for_docker and encrypted_volume_for_snap like so:

---
openstack_volumes:
  - name: weblate-volume
    size: 10
encrypted_device_mount_point: /srv
encrypted_volume_for_docker: false
encrypted_volume_for_snap: false

Services

The following services are always available:

Access

The SSH public keys found in files matching authorized_keys_globs are installed on every machine.

---
authorized_keys_globs:
  - ssh_keys/dachary.pub
  - ssh_keys/glen.pub

Low level commands

The following are not useful if only relying on the service command above. They can however be helpful to run Ansible or OpenStack manually.

Managing hosts

The hosts (OpenStack virtual machines) are created automatically when a service is provided. It is however possible to create a new host or destroy an existing one.

The first step is to edit ~/.enough/example.com/inventory/all.yml and add the name of the new host:

---
all-hosts:
 hosts:
  my-host:
  bind-host:
  forum-host:
  ...

Creating a new host:

enough --domain example.com host create my-host

SSH to a host:

enough --domain example.com ssh my-host

Deleting a host:

enough --domain example.com host delete my-host

Running openstack

The openstack CLI can be used as follows:

$ enough --domain example.com openstack -- help

Which is exactly equivalent to:

$ OS_CLIENT_CONFIG_FILE=~/.enough/example.com/inventory/group_vars/all/clouds.yml \
  openstack --os-cloud ovh help

Running ansible-playbook

The ansible-playbook CLI can be used as follows:

$ enough --domain example.com playbook -- --limit localhost,icinga-host \
  --private-key ~/.enough/example.com/infrastructure_key \
  ~/.enough/example.com/enough-playbook.yml

It implicitly uses the following inventories (via multiple –inventory options), in order (the last inventory listed has precedence):