# 3.3.6 Deploy (Automated & Manual) ### Automated continuous deployment The easiest way to deploy OpenCRVS to your servers is to use [automated continuous deployment](https://www.atlassian.com/continuous-delivery/principles/continuous-integration-vs-delivery-vs-deployment). Refer to our supplied [**"Publish image to Dockerhub"**](https://github.com/opencrvs/opencrvs-countryconfig/blob/master/.github/workflows/publish-to-dockerhub.yml) Github Action and our supplied [**"Deploy"**](https://github.com/opencrvs/opencrvs-countryconfig/blob/develop/.github/workflows/deploy.yml) Github Action in the country configuration repo to set up appropriate environments for your use case or customise your own delivery pipeline. You could rewrite our Actions for use in [Gitlab](https://about.gitlab.com/), [Jenkins](https://www.jenkins.io/), [Travis](https://www.travis-ci.com/), [CircleCI](https://circleci.com/) or [Azure Pipelines](https://azure.microsoft.com/en-gb/products/devops/pipelines/) for example. In our example, we have 3 environments provisioned in Github. These environments allow you to provision different subdomains, secrets and optional deployment properties depending on your chosen environment when running the action. All of our workflows can be manually run from a shell script, so you can deploy from your local machine too if you like. Manual steps are explained below each automated step. To use our automated Github Action workflow, first you need to ensure that you have set up at least one, or optionally all, of the following Git [environments](https://docs.github.com/en/actions/deployment/targeting-different-environments/using-environments-for-deployment): a) **Staging** - A useful environment for testing, where the environment variable NODE\_ENV is set to **development** and you can create test user accounts with a 6 zero "000000" 2FA code during login. This allows us to see a debug experience. b) **QA** - A quality assurance/pseudo production environment for software testers, where the environment variable NODE\_ENV is set to **production** and a secondary exception variable QA\_ENV is set to **true.** This allows us to see a production like experience, but with the capability of still creating test user accounts with a 6 zero "000000" 2FA code during login. c) **Production** - A live environment, where NODE\_ENV is set to **production** & QA\_ENV is set to **false** and random 2FA is enabled so a comms Gateway must be active as in [step 3.3.3.](https://documentation.opencrvs.org/v1.3/setup/3.-installation/3.3-set-up-a-server-hosted-environment/3.3.3-provision-a-comms-gateway) {% hint style="warning" %} Our supplied Github Actions are examples that cannot be edited from a [fork](https://github.blog/2020-08-03-github-actions-improvements-for-fork-and-pull-request-workflows/). You should duplicate these Github Actions files if you want to make changes for your infrastructure and update the branches that dispatch them ([here for example](https://github.com/opencrvs/opencrvs-countryconfig/blob/master/.github/workflows/publish-to-dockerhub.yml#L5)) from master, develop to master-\, develop-\ as we described in the fork [section 3.2.1](https://documentation.opencrvs.org/v1.3/setup/3.-installation/3.2-set-up-your-own-country-configuration/3.2.1-fork-your-own-country-configuration-repository). {% endhint %} ### Configure docker-compose files There are a few environment variables you need to manually set in docker compose files for your installation. These are: [docker-compose.countryconfig.staging-deploy.yml](https://github.com/opencrvs/opencrvs-countryconfig/blob/develop/infrastructure/docker-compose.countryconfig.staging-deploy.yml) [docker-compose.countryconfig.qa-deploy.yml](https://github.com/opencrvs/opencrvs-countryconfig/blob/develop/infrastructure/docker-compose.countryconfig.qa-deploy.yml) [docker-compose.countryconfig.prod-deploy.yml](https://github.com/opencrvs/opencrvs-countryconfig/blob/develop/infrastructure/docker-compose.countryconfig.prod-deploy.yml) a) Replace any instance of the comma separated available language strings for your localisation needs. E.G.: ``` LANGUAGES=en,fr ``` b) Replace any instance of the COUNTRY code. E.G.: ``` COUNTRY=FAR ``` ### Publishing your country configuration Before you can deploy, you need to make sure that your country configuration Docker image has compiled and has been pushed to your container registry (E.G. Dockerhub). **Option 1: Using our automated action to publish your country configuration** We supply an **automated action** to do this for you. Our [**"Publish image to Dockerhub"**](https://github.com/opencrvs/opencrvs-countryconfig/blob/master/.github/workflows/publish-to-dockerhub.yml) Github Action is set to **automatically run** on your country configuration repository **whenever code is pushed to a branch named master, main or develop**. This action will build and push your Docker image to Dockerhub. You must edit this action if you wish to use a different container registry. {% hint style="info" %} The image will be tagged with the short Git commit hash. This hash is important to refer to and use in the deploy step. {% endhint %} **Global repository secrets** {% hint style="success" %} In OpenCRVS v1.3.1 hotfix scheduled for November 6th 2023, we will be releaseing a Github Action to automate the creation of environments and secrets :-) {% endhint %} These secrets below must be set as **global repository secrets** in Github as they apply to all environments and are used by the [**"Publish image to Dockerhub"**](https://github.com/opencrvs/opencrvs-countryconfig/blob/master/.github/workflows/publish-to-dockerhub.yml) Github Action:thumbsup:
ParameterDescription
DOCKER_USERNAMEYour Dockerhub username to access the container registry. If you are using a different container registry, you will need to manually edit the deploy.yml appropriately.
DOCKER_PASSWORDYour Dockerhub password.
DOCKERHUB_ACCOUNTThe name of your Dockerhub account or organisation that forms the URL to your country config docker image on Dockerhub before the slash. e.g: opencrvs
DOCKERHUB_REPOThe name of your Dockerhub repository that forms the URL to your country config docker image on Dockerhub after the slash.. e.g. ocrvs-farajaland
In Github, navigate to "Actions" and click "Publish image to Dockerhub" to see the output of the action that automates whenever code merges to "develop", "main", "master" branches. Copy the git commit hash in the Action logs to see how the Docker image was tagged. You will use it in the deploy step.
**Option 2: Manually publishing your country configuration** You can navigate to the country configuration repository to build your image from the command line if you wish: ``` cd opencrvs-countryconfig ``` Export all of the required secrets as environment variables: ``` export DOCKER_PASSWORD= \ export DOCKER_USERNAME= \ export DOCKERHUB_ACCOUNT= \ export DOCKERHUB_REPO= ``` Run the following script: ``` yarn compose:push:version ``` Take note of the githash at the end of the process to use in the next step ...
### Deploying Next, you need to deploy to your server environments. 1. **First create appropriate secrets and environments in Github** Create the following [Github secrets](https://docs.github.com/en/codespaces/managing-codespaces-for-your-organization/managing-encrypted-secrets-for-your-repository-and-organization-for-codespaces) for the usernames and passwords you created earlier when provisioning the servers using Ansible in [step 3.3.2](https://documentation.opencrvs.org/v1.3/setup/3.-installation/3.3-set-up-a-server-hosted-environment/3.3.2-install-dependencies), along with other secrets Github will use to SSH into your servers, set the Traefik SSL hostname and connect to your container registry (E.G. Dockerhub) etc. {% hint style="info" %} Note: Using a strong password service such as [1Password](https://1password.com/) you should store the usernames and passwords you create in this section as you will need them regularly. {% endhint %} **Global repository secrets** These secrets below must be set as **global repository secrets** in Github as they apply to all environments and are used by the Github Action: | Secret | Description | | ---------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | SUPER\_USER\_PASSWORD | This is an OpenCRVS superuser National System Admnistrator password used when seeding the databases in deployed environments. | | ELASTICSEARCH\_SUPERUSER\_PASSWORD | [*Created in 3.3.2.*](https://documentation.opencrvs.org/v1.3/setup/3.-installation/3.3-set-up-a-server-hosted-environment/3.3.2-install-dependencies) The Elasticsearch superuser password. You can also use this to login to Kibana with the username "**elastic**" and you have superuser Elastic privileges. Kibana URL: \ | | KIBANA\_USERNAME | A username for a regular Kibana user to login and monitor OpenCRVS stack health. Useful for developers as this user will have no superuser privileges. | | KIBANA\_PASSWORD | A password for a regular Kibana user to login and monitor OpenCRVS stack health | | MONGODB\_ADMIN\_USER | [*Created in 3.3.2.*](https://documentation.opencrvs.org/v1.3/setup/3.-installation/3.3-set-up-a-server-hosted-environment/3.3.2-install-dependencies) The MongoDB superuser admin username. A powerful account that has all rights to OpenCRVS data | | MONGODB\_ADMIN\_PASSWORD | [*Created in 3.3.2.*](https://documentation.opencrvs.org/v1.3/setup/3.-installation/3.3-set-up-a-server-hosted-environment/3.3.2-install-dependencies) The MongoDB superuser admin password. | | MINIO\_ROOT\_USER | A username for a Minio superuser admin to login to the Minio console to view supporting document attachments submitted during registrations. \ | | MINIO\_ROOT\_PASSWORD | A password for a Minio superuser admin | | DOCKER\_USERNAME | Your [Dockerhub](https://hub.docker.com/) username to access the container registry. If you are using a different container registry, you will need to manually edit the deploy.yml appropriately. | | DOCKER\_PASSWORD | Your [Dockerhub](https://hub.docker.com/) password. *(Docker doesnt like special characters in its passwords when we tested this recently. So choose a long, obfuscated alphanumeric password of 13 chars or more.)* | | DOCKERHUB\_ACCOUNT | The name of your Dockerhub account or organisation that forms the URL to your country config docker image on Dockerhub ***before*** the slash. e.g: **opencrvs** | | DOCKERHUB\_REPO | The name of your Dockerhub repository that forms the URL to your country config docker image on Dockerhub ***after*** the slash.. e.g. **ocrvs-farajaland** | | SMTP\_HOST | Described in [step 3.3.4](https://documentation.opencrvs.org/v1.3/setup/3.-installation/3.3-set-up-a-server-hosted-environment/3.3.4-set-up-an-smtp-server-for-opencrvs-monitoring-alerts) | | SMTP\_PORT | Described in [step 3.3.4](https://documentation.opencrvs.org/v1.3/setup/3.-installation/3.3-set-up-a-server-hosted-environment/3.3.4-set-up-an-smtp-server-for-opencrvs-monitoring-alerts) | | SMTP\_USERNAME | Described in [step 3.3.4](https://documentation.opencrvs.org/v1.3/setup/3.-installation/3.3-set-up-a-server-hosted-environment/3.3.4-set-up-an-smtp-server-for-opencrvs-monitoring-alerts) | | SMTP\_PASSWORD | Described in [step 3.3.4](https://documentation.opencrvs.org/v1.3/setup/3.-installation/3.3-set-up-a-server-hosted-environment/3.3.4-set-up-an-smtp-server-for-opencrvs-monitoring-alerts) | | ALERT\_EMAIL | Described in [step 3.3.4](https://documentation.opencrvs.org/v1.3/setup/3.-installation/3.3-set-up-a-server-hosted-environment/3.3.4-set-up-an-smtp-server-for-opencrvs-monitoring-alerts) | **Copy your id\_rsa.pub key into the authorized\_keys file** {% hint style="danger" %} The deploy Github Action uses [this library](https://github.com/shimataro/ssh-key-action) to SSH into your environments. This library depends on an PEM(RSA), PKCS8, and RFC4716(OpenSSH) SSH key. The id\_rsa.pub associated with the same key must be saved into the [authorized\_keys ](https://github.com/opencrvs/opencrvs-countryconfig/blob/develop/infrastructure/authorized_keys)file here along with the id\_rsa.pub for any other developer who requires SSH access to your server. **Be aware that these developers have root level access to your system and must be trusted. If they leave your organisation, it is your responsibility to remove their id\_rsa.pub from the authorized\_keys file in Git and on the servers.** {% endhint %} **Environment secrets** The following secrets are likely to be unique for each environment so they should be duplicated as **environment secrets** in Github and are used by the Github Action: | Secret | Description | Example | | ------------ | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------- | | KNOWN\_HOSTS |

You will need the lines in the .ssh/known\_hosts file relevant to the environments that the SSH Key uses. This can be generated using a test SSH connection using your key. From your local computer, SSH in to your manager node using your domain name. E.G.:

ssh root@\

This will save an entry into your \~/.ssh/known\_hosts. You will need a copy of the line in your \~/.ssh/known\_hosts relevant to the host domain name for your environment in this secret so that Github can access your server using your SSH\_KEY below.

| | | SSH\_KEY | This is a copy of the **id\_rsa** file for the SSH Key, not the id\_rsa.pub! Described [here](https://docs.github.com/en/authentication/connecting-to-github-with-ssh/managing-deploy-keys#setup-2). | | **Environment variables** The following variables are likely to be unique for each environment so they should be duplicated as **environment variables** in Github and are used by the Github Action: | Variable | Description | Example | | ----------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | --------------------------- | | REPLICAS | The number of replicas: **1, 3 or 5** depending on how many servers are in the environment cluster as explained in [step 3.3.1](https://documentation.opencrvs.org/v1.3/setup/3.-installation/3.3-set-up-a-server-hosted-environment/3.3.1-provision-your-server-nodes-with-ssh-access) | | | DOMAIN | The host **domain name** (without www!) for your environment. **You must make sure that you can ping this domain and that the ping resolves to your manager server's IP address.** If this does not resolve, there must be a problem with your A record configuration explained in the previous [step 3.3.5](https://documentation.opencrvs.org/v1.3/setup/3.-installation/3.3-set-up-a-server-hosted-environment/3.3.5-setup-dns-a-records). | | | CONTENT\_SECURITY\_POLICY\_WILDCARD | This string is supplied to the clients and nginx config and ensures that the format of your domain above can be configurable for CORS purposes. | \*.\ or \*\ | **Customisable, optional environment secrets** The following secrets depend on your choice of communications provider. The Github [**deploy**](https://github.com/opencrvs/opencrvs-countryconfig/blob/9502c638fab53e2e6fb8ee3af572c86aa2c3b6e9/.github/workflows/deploy.yml#L110) action and the [**deploy.sh**](https://github.com/opencrvs/opencrvs-countryconfig/blob/develop/infrastructure/deploy.sh) script that the action calls will likely need to be edited according to your needs in order to customise these values. | Parameter | Description | | -------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | | EMAIL\_API\_KEY | If you are using Email for communications, then you will likely require an API key. We use this secret for the [SendGrid](https://sendgrid.com/) API. Its likely you will have your own custom requirements depending on your chosen provider. Amend the [handler](https://github.com/opencrvs/opencrvs-countryconfig/tree/develop/src/api/notification) appropriately to suit your needs. | | SENDER\_EMAIL\_ADDRESS | If you are using Email for communications, then the sender email address that appears in all emails will need to be configured. Its likely you will have your own custom requirements depending on your chosen provider. Amend the [handler](https://github.com/opencrvs/opencrvs-countryconfig/tree/develop/src/api/notification) appropriately to suit your needs. | | SENTRY\_DSN | OpenCRVS can report application errors to [Sentry](https://sentry.io/) in order to help you debug any issues in production. | | INFOBIP\_API\_KEY | If you are using SMS for communications, then you will likely require an API key. We use this secret for the [Infobip](https://www.infobip.com/) API. Its likely you will have your own custom requirements depending on your chosen provider. Amend the [handler](https://github.com/opencrvs/opencrvs-countryconfig/tree/develop/src/api/notification) appropriately to suit your needs. | | INFOBIP\_SENDER\_ID | If you are using SMS for communications, then the sender id that appears in all SMS messages will need to be configured. Its likely you will have your own custom requirements depending on your chosen provider. Amend the [handler](https://github.com/opencrvs/opencrvs-countryconfig/tree/develop/src/api/notification) appropriately to suit your needs. | | INFOBIP\_GATEWAY\_ENDPOINT | We use this prop to store the endpoint URL for the Infobip API. Its likely you will have your own custom requirements depending on your chosen provider. Amend the [handler](https://github.com/opencrvs/opencrvs-countryconfig/tree/develop/src/api/notification) appropriately to suit your needs. | 2. **Run the Deploy action** You can deploy to your server using the automated [**"Deploy"**](https://github.com/opencrvs/opencrvs-countryconfig/blob/develop/.github/workflows/deploy.yml) Github Action. Navigate to "Actions", and click "Deploy."
a) You will be required to select the environment that you wish to deploy to. b) You will be required to enter the OpenCRVS Core Dockerhub image tag for any tagged build on Dockerhub) to refer to the OpenCRVS Core release of choice. Usually this will be an official release if you have performed no customisations to core. E.G. **v1.3.0** c) You will be required to enter the OpenCRVS Country Configuration version (or short Git hash tag for any tagged custom country configuration build on Dockerhub) to refer to your country configuration image and githash created by the previous "Publish image to Dockerhub" action. E.G. **4e39a2a** d) Click Run workflow, and watch the output to make sure that the deployment was successful.
Once the deployment is complete, you should be able to navigate to your OpenCRVS URLs. To login to OpenCRVS: \ To login to OpenHIM: \ To login to Kibana: \ To login to Minio: \ **Option 2: Manual deploy** You can navigate to the country configuration repository to deploy from the command line if you wish: ``` cd opencrvs-countryconfig ``` Export all of the required secrets as environment variables, and repeat lines for any customisable secrets you require: ``` export DOCKER_PASSWORD= \ export DOCKER_USERNAME= \ export DOCKERHUB_ACCOUNT= \ export DOCKERHUB_REPO= \ export SUPER_USER_PASSWORD= export ELASTICSEARCH_ADMIN_USER=elastic \ export ELASTICSEARCH_SUPERUSER_PASSWORD= \ export MONGODB_ADMIN_USER= \ export MONGODB_ADMIN_PASSWORD= \ export MINIO_ROOT_USER= \ export MINIO_ROOT_PASSWORD= \ export KIBANA_USERNAME= \ export KIBANA_PASSWORD= \ export SMTP_HOST= \ export ALERT_EMAIL= \ export SMTP_USERNAME= \ export SMTP_PASSWORD= \ export SMTP_PORT= ``` Run the following script substituting the parameters as explained in the table below: ``` yarn deploy --environment= --host= --version= --country_config_version= --replicas= --clear_data=no ``` | Parameter | Description | Example | | ------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------- | | ENVIRONMENT | Can be set to "staging", "qa" or "production" with the impact on 2FA and comms delivery as previously explained in [step 3.3.3](https://documentation.opencrvs.org/v1.3/setup/3.-installation/3.3-set-up-a-server-hosted-environment/3.3.3-provision-a-comms-gateway). **"staging"** = a development environment where NODE\_ENV is equal to "development". **"qa"** = a quality assurance environment where NODE\_ENV is equal to "production", but all users will be test users and comms will not be sent - 2FA is disabled. **"production"** = a production environment where NODE\_ENV is equal to "production" and comms will be sent - 2FA is enabled. | staging | | DOMAIN | The host **domain name** (without www!) for your environment. **You must make sure that you can ping this domain and that the ping resolves to your manager server's IP address.** If this does not resolve, there must be a problem with your A record configuration explained in the previous [step 3.3.5](https://documentation.opencrvs.org/v1.3/setup/3.-installation/3.3-set-up-a-server-hosted-environment/3.3.5-setup-dns-a-records). | | | CORE\_VERSION | The OpenCRVS Core Dockerhub image tag for any tagged build on Dockerhub) to refer to the OpenCRVS Core release of choice. Usually this will be an official release if you have performed no customisations to core. E.G. **v1.3.0** | v1.3.0 | | COUNTRY\_CONFIG\_VERSION | The OpenCRVS Country Configuration version (or short Git hash tag for any tagged custom country configuration build on Dockerhub) to refer to your country configuration image and githash created by the previous "Publishing your country configuration" step. | 4e39a2a | | REPLICAS | The number of replicas: **1, 3 or 5** depending on how many servers are in the environment cluster as explained in [step 3.3.1](https://documentation.opencrvs.org/v1.3/setup/3.-installation/3.3-set-up-a-server-hosted-environment/3.3.1-provision-your-server-nodes-with-ssh-access) | 1 | | --clear\_data=no | **THIS IS A DESTRUCTIVE ACTION!!!** - This parameter should be set to "**no**". Unless you wish to entirely clear all data on your server. That may be useful each time you deploy to a development environment. To **PERMANENTLY DELETE ALL DATA** on a server after deployments, set this to "**yes**" | no |