7.1 Pre-Deployment Checklist

Pre-Deployment Checklist

We provide you with the pre-deployment checklist Excel file that you must complete before going live in your country. Some points are explained in this video and table:

Step Explanation

Step	Explanation
Provisioning & Deployment pipelines for Backup, QA, staging & production working	Your Github Actions should all be running without any errors in logs. No red crosses!
SMTP service tested for emails and alerts	Not only should you confirm that emails are sending for staff, but you should receive emailed alerts too. If you have enabled Slack as the ALERT_EMAIL you would see an SSH login alert in Slack like this:
Setup DNS & TLS (HTTPS)	Your SSL certificate should be valid.
Verify production backups restore on staging (pre-prod/mirror)	Ensure that the backup & restore test is completed as documented.
Verify logs roll over	Ensure that the logs are rotating so as not to use up disk-space. Logfiles are configured to rotate automatically. Refer to logrotate.conf in your country config package.
Deploy latest release	Ensure that OpenCRVS is as up to date as it can be. Refer to recent releases.
Alert sent of SSH login	Ensure the SSH login alert is firing as mentioned above in the "SMTP service tested" description.
Verify a warning is received when disk space is at 50%, alert at 70%	To test this you need to create a temporary large file on your server. Kibana will broadcast an alert. This example terminal command creates a file of 200G: `sudo dd if=/dev/zero of=/large-file-to-trigger-alerts bs=200G count=100.`
Send Sentry errors to Slack from QA and Prod	A test Sentry error in Slack looks something like this:
Are National System Admin passwords updated, minimum 12 chars long and stored in a Password Manager.	The National System Admin must immediately change their password from the password that exists in prod-employees.csv . As they are a super user, they must confirm that the new password is saved in a password manager tool.
Github account is owned by country and OpenCRVS Core team are removed as admins	If the OpenCRVS team have assisted you to set up your Github organisation, they must no longer have "Admin" rights in your countryconfig repository otherwise they can potentially access citizen data.
Disable password SSH access, and enable 2FA SSH access. Verify users are required to set up 2FA and that the code is asked on every login after that	Our Provision Action should have enforced this but it is always a good idea to make sure that this is working successfully and no bugs have been introduced.
Root login is disabled	As above.
Verify firewall hides all ports from the public internet	As above.
Clear all data from production instances created during testing	Reset the production environment so that it is clear of any test registrations.
Delete all test backups on backup server	SSH into the backup server and delete any test backups. Especially if you have configured the backup server to store more than the default 7 days of backups.
Delete all terminal history on all production, staging and backup servers	You might have exported secrets in Terminal to use when resetting environments or debugging. Clear the terminal history like this: `history -c history -w`
Remove OpenCRVS from SSH access to production, staging & backup servers	If the OpenCRVS team have assisted you to provision your production, staging and backup servers, their SSH access must be removed otherwise they can still access citizen data. Follow the steps as documented here.

Provisioning & Deployment pipelines for Backup, QA, staging & production working

Your Github Actions should all be running without any errors in logs. No red crosses!

SMTP service tested for emails and alerts

Not only should you confirm that emails are sending for staff, but you should receive emailed alerts too. If you have enabled Slack as the ALERT_EMAIL you would see an SSH login alert in Slack like this:

Setup DNS & TLS (HTTPS)

Your SSL certificate should be valid.

Verify production backups restore on staging (pre-prod/mirror)

Ensure that the backup & restore test is completed as documented.

Verify logs roll over

Ensure that the logs are rotating so as not to use up disk-space. Logfiles are configured to rotate automatically. Refer to logrotate.conf in your country config package.

Deploy latest release

Ensure that OpenCRVS is as up to date as it can be. Refer to recent releases.

Alert sent of SSH login

Ensure the SSH login alert is firing as mentioned above in the "SMTP service tested" description.

Verify a warning is received when disk space is at 50%, alert at 70%

To test this you need to create a temporary large file on your server. Kibana will broadcast an alert. This example terminal command creates a file of 200G:

sudo dd if=/dev/zero of=/large-file-to-trigger-alerts bs=200G count=100.

Send Sentry errors to Slack from QA and Prod

A test Sentry error in Slack looks something like this:

Are National System Admin passwords updated, minimum 12 chars long and stored in a Password Manager.

The National System Admin must immediately change their password from the password that exists in prod-employees.csv . As they are a super user, they must confirm that the new password is saved in a password manager tool.

Github account is owned by country and OpenCRVS Core team are removed as admins

If the OpenCRVS team have assisted you to set up your Github organisation, they must no longer have "Admin" rights in your countryconfig repository otherwise they can potentially access citizen data.

Disable password SSH access, and enable 2FA SSH access. Verify users are required to set up 2FA and that the code is asked on every login after that

Our Provision Action should have enforced this but it is always a good idea to make sure that this is working successfully and no bugs have been introduced.

Root login is disabled

As above.

Verify firewall hides all ports from the public internet

As above.

Clear all data from production instances created during testing

Reset the production environment so that it is clear of any test registrations.

Delete all test backups on backup server

SSH into the backup server and delete any test backups. Especially if you have configured the backup server to store more than the default 7 days of backups.

Delete all terminal history on all production, staging and backup servers

You might have exported secrets in Terminal to use when resetting environments or debugging. Clear the terminal history like this:

history -c
history -w

Remove OpenCRVS from SSH access to production, staging & backup servers

If the OpenCRVS team have assisted you to provision your production, staging and backup servers, their SSH access must be removed otherwise they can still access citizen data. Follow the steps as documented here.

Pre-Deployment Checklist Excel

Download our Excel checklist in the "Technical" zip in the OpenCRVS Requirements Templates

Ongoing-Costs

As you have probably gathered from the server configuration section, some additional tooling is required to be paid for. These costs are negligble and support your installation with a secure code repository, bug tracking systems, alerting and forensic analysis.

The Ongoing-Costs Excel sheet outlines the ownerhsip, status and costs and helps you project manage the provisioning of these tools. They are explained in this video.

Ongoing-Costs Excel

Download the Excel sheet in the "Technical" zip in the OpenCRVS Requirements Templates

Data Security Framework

The purpose of this document is to provide organisations with:

An understanding of data security and privacy risks.
An understanding of the technical steps taken in OpenCRVS to mitigate against these risks.
A guidance framework for the development of context-specific data security policies and procedures that should be designed and introduced by a government that has chosen to install OpenCRVS and digitise their civil registration system.
Security guidance for project managers and all staff involved on a temporary or continual basis in the following stages of an OpenCRVS project: a) design & implementation b) monitoring & maintenance and c) day-to-day usage of OpenCRVS.

Make sure that this document is shared with key stakeholders.

Data Security Framework

Download the document from the "Technical" zip in the OpenCRVS Requirements Templates

Previous7. Go-live Next8. Operational Support

Last updated 12 days ago