# 3.3.2 Provision environment

{% hint style="success" %}
In OpenCRVS v1.3.1 hotfix scheduled for November 6th 2023, we will be releaseing a Github Action to automate this process. :-)
{% endhint %}

[Ansible](https://docs.ansible.com/) is required to be installed on your local development machine in order to provision the server with required software. Installation instructions are [here](https://docs.ansible.com/ansible/latest/installation_guide/intro_installation.html). Ensure that you have ssh access using the root user to all the servers that you are trying to configure.

{% hint style="info" %}
Ansible is an IT automation tool that you **install locally and run from your local computer**. **It uses SSH to connect to your servers automatically** and installs all the supporting software and configures Ubuntu for you to be able to deploy OpenCRVS. The Ansible script we provide will install all the dependencies onto your server nodes, configure a secure firewall, open required ports and provision the optional automated backup of OpenCRVS for use in production. You may need to customise our Ansible playbooks depending on your hosting provider and the access you have.
{% endhint %}

{% hint style="info" %}
You will need a docker container registry account on [Dockerhub](https://hub.docker.com/) to build and push your country configuration image, in order to use our Ansible script. This is because the server needs access to your containrer registry account to pull images from it. [Dockerhub](https://hub.docker.com/) is a free containerisation repository. You can customise our Ansible script if you wish to use a different registry.
{% endhint %}

{% hint style="success" %}
In OpenCRVS v1.3, all server setup configuration and deployment files exist in the country configuration repository - e.g. opencrvs-countryconfig. This allows you to completely customise them for your needs without needing to fork opencrvs-core
{% endhint %}

1. Create an account on [Dockerhub](https://hub.docker.com/) as Ansible and Github Actions will require your Dockerhub username and password in order to login.\\
2. Referring to the country configuration repository e.g. opencrvs-countryconfig, duplicate the **example-X.ini** inventory\_file of choice where X is relative to the number of servers. These can be found in the [infrastructure/server-setup](https://github.com/opencrvs/opencrvs-countryconfig/tree/master/infrastructure/server-setup) directory, depending upon whether or not you are deploying to 1, 3 or 5 servers.\
   \
   For example: If you are only deploying to 1 server, you need to make a copy of the **inventory\_file**: [example-1.ini](https://github.com/opencrvs/opencrvs-countryconfig/blob/master/infrastructure/server-setup/example-1.ini) file to run with the Ansible **playbook\_file:** [playbook-1.yml ](https://github.com/opencrvs/opencrvs-countryconfig/blob/master/infrastructure/server-setup/playbook-1.yml)explained below.\
   \
   If you are deploying to a standard production deployment of 3 servers, you need to make a copy of the **inventory\_file:** [example-3.ini ](https://github.com/opencrvs/opencrvs-countryconfig/blob/master/infrastructure/server-setup/example-3.ini)file to run with the Ansible **playbook\_file:** [playbook-3.yml](https://github.com/opencrvs/opencrvs-countryconfig/blob/master/infrastructure/server-setup/playbook-3.yml) explained below.\
   \
   If you are deploying to 5 servers, you need to make a copy of the **inventory\_file:** [example-5.ini ](https://github.com/opencrvs/opencrvs-countryconfig/blob/master/infrastructure/server-setup/example-5.ini)file to run with the Ansible **playbook\_file:** [playbook-5.yml](https://github.com/opencrvs/opencrvs-countryconfig/blob/master/infrastructure/server-setup/playbook-5.yml) explained below.

{% hint style="danger" %}
Deploying to only 1 server is not recommended and is **CONSIDERABLY RISKY** for a production installation. **YOU COULD LOSE CITIZEN DATA IF THE SERVER CRASHES.** Docker Swarm's power is that it can load balance between servers in a cluster. If one of the servers goes down, the others will still be operational.
{% endhint %}

You will be required to uncomment some lines to enter the IP addresses and hostnames, e.g.:\\

```
;manager1 ansible_host="ENTER YOUR MANAGER HOST IP"
```

\
becomes:

```
manager1 ansible_host="159.223.11.243"
```

\
... and:

```
;data1_hostname=ENTER_HOSTNAME_1
```

becomes:

```
data1_hostname=<your server hostname>
```

4\. Using a strong password generator, such as [1Password](https://1password.com/) you should create and **safely store** the following parameters.:

{% hint style="warning" %}
We suggest that your passwords are unguessable [RFC 4122](https://www.rfc-editor.org/rfc/rfc4122) alphanumeric version 1 or 4 [UUIDs](https://en.wikipedia.org/wiki/Universally_unique_identifier) . If you use special characters in the passwords, they will need to be escaped.
{% endhint %}

{% hint style="danger" %}
The passwords and secret keys that you make in this section are critically important to be saved and stored securely. **They are the master keys to access your citizen data!** Consider [ISO270001](https://iso-docs.com/blogs/iso-27001-standard-1/iso-27001-annex-a-9-access-control) Password Management Policies!! Tools such as [1Password](https://1password.com/) can help you here. You can customise our playbook to store the keys in a [hardware security module](https://en.wikipedia.org/wiki/Hardware_security_module).
{% endhint %}

*mongodb\_admin\_username*

*mongodb\_admin\_password*

*disk\_encryption\_key (The* [*LUKS*](https://en.wikipedia.org/wiki/Linux_Unified_Key_Setup) *Disk encryption secret, encrypting your citizen data at rest.)*

*elasticsearch\_superuser\_password*

{% hint style="danger" %}
You must store these keys safely for future use, as you will need the details in the [deploy](https://documentation.opencrvs.org/v1.3/setup/3.-installation/3.3-set-up-a-server-hosted-environment/3.3.6-deploy-automated-and-manual) process.
{% endhint %}

5\. You are now ready to call the Ansible command passing these **required** parameters and additionally some optional parameters.

**Required parameters:**

*dockerhub\_username*

*dockerhub\_password*

*mongodb\_admin\_username*

*mongodb\_admin\_password*

*elasticsearch\_superuser\_password*

*disk\_encryption\_key*

*encrypted\_disk\_size*

{% hint style="info" %}
Depending on the size of your server, *encrypted\_disk\_size* should be equal to about 80% of your available disk. E.G. for a 320GB disk, encrypt 256GB to store OpenCRVS data. You need to leave enough unencrypted space for your operating system and software. The encrypted\_disk\_size prop for 256GB is: `256g`
{% endhint %}

{% hint style="warning" %}
**PRODUCTION NOTE:** In production, we advise that you provision a [Hardware Security Module](https://en.wikipedia.org/wiki/Hardware_security_module) and amend the country configuration [playbook.yml](https://github.com/opencrvs/opencrvs-countryconfig/blob/9502c638fab53e2e6fb8ee3af572c86aa2c3b6e9/infrastructure/server-setup/playbook-1.yml#L429), [decrypt.sh](https://github.com/opencrvs/opencrvs-countryconfig/blob/9502c638fab53e2e6fb8ee3af572c86aa2c3b6e9/infrastructure/decrypt.sh#L35), [emergency-backup-metadata.sh](https://github.com/opencrvs/opencrvs-countryconfig/blob/9502c638fab53e2e6fb8ee3af572c86aa2c3b6e9/infrastructure/emergency-backup-metadata.sh#L104) and [emergency-restore-metadata.sh](https://github.com/opencrvs/opencrvs-countryconfig/blob/9502c638fab53e2e6fb8ee3af572c86aa2c3b6e9/infrastructure/emergency-restore-metadata.sh#L68) scripts at the linked locations in order to change the approach to storing and accessing the ***disk\_encryption\_key*** and **MongoDB and Elasticsearch passwords**. **Secure secret storage is currently outside the scope of OpenCRVS.**

*MOSIP's documentation on the requirements of a* [*Hardware Security Module*](https://docs.mosip.io/1.1.5/build-and-deploy/hardware-security-module-hsm-specifications) *is useful reading.*
{% endhint %}

**Optional parameters:**

For the optional automated daily external data backup to another server, these parameters must be prepared:

*external\_backup\_server\_ip*

*external\_backup\_server\_user*

*external\_backup\_server\_ssh\_port*

*external\_backup\_server\_remote\_directory*

{% hint style="warning" %}
You must ensure that you are in your local computer and that both your local directories opencrvs-core and opencrvs-\<your country> are on the same release version **v\*** or **master** branch before running Ansible. Ansible is run from your local machine and it connects to your server using SSH and automatically runs commands on it.
{% endhint %}

Ansible playbooks are run like this **from your local machine**:

5. If you are on the root directory of the country configuration repository - e.g. opencrvs-countryconfig, navigate to the [server-setup](https://github.com/opencrvs/opencrvs-countryconfig/tree/master/infrastructure/server-setup) folder:

```
cd infrastructure/server-setup
```

Now you can run the playbook like this, substituting the parameters as required:

```
ansible-playbook -i <inventory_file> <playbook_file> -e " \
dockerhub_username=<your dockerhub username> \
dockerhub_password=<your dockerhub password> \
mongodb_admin_username=<mongo username> \
mongodb_admin_password=<mongo password you generated> \
elasticsearch_superuser_password=<elastic password you generated> \
disk_encryption_key=<a strong disk encryption password> \
encrypted_disk_size=<your available disk size for encryption, e.g. 256g>"
```

Or with all the possible **optional props**:

```
ansible-playbook -i <inventory_file> <playbook_file> -e " \
dockerhub_username=<your dockerhub username> \
dockerhub_password=<your dockerhub password> \
mongodb_admin_username=<mongo username> \
mongodb_admin_password=<mongo password you generated> \
elasticsearch_superuser_password=<elastic password you generated> \
disk_encryption_key=<a strong disk encryption password> \
encrypted_disk_size=<your available disk size for encryption, e.g. 256g> \
external_backup_server_ip=<your_external_backup_server_ip> \
external_backup_server_user=<your_external_backup_server_user> \
external_backup_server_ssh_port=<your_external_backup_server_ssh_port> \
manager_production_server_ip=<your_manager_production_server_ip> \
external_backup_server_remote_directory=<your_external_backup_server_remote_directory>"
```

Once this command is finished the servers are prepared for an OpenCRVS deployment. You can read more about how the external backups work in the [Automated & manual backup...](https://documentation.opencrvs.org/v1.3/setup/3.-installation/3.3-set-up-a-server-hosted-environment/3.3.8-automated-and-manual-backup-and-manual-restore) section