`kobo-install` on a fresh Ubuntu 18.04 LTS VPS

#1

This is an annotated transcript of installing https://github.com/kobotoolbox/kobo-install on a fresh “Linode 2GB” instance, with 1 CPU core, 2 GB of RAM (surprise), and 50 GB of SSD storage. It costs $10/month. In the Linode Manager, I used “Create a new Disk” to create 8 GB of swap prior to “Deploy an image,” since the latter only offers swap sizes up to 512 MB. I deployed the Ubuntu 18.04 LTS image and set a root password.

:information_source: Anywhere that <snip> appears indicates that I’ve removed some output for brevity and readability.

First, before doing anything on the new server, I verified that the DNS for my three domains was configured properly. I’ve replaced my VPS’ real IP address with 9.8.7.6 throughout this transcript:

john@world$ dig +noall +answer kf.self-hosted-ssl-test.kbtdev.org kc.self-hosted-ssl-test.kbtdev.org ee.self-hosted-ssl-test.kbtdev.org
kf.self-hosted-ssl-test.kbtdev.org. 3 IN A	9.8.7.6
kc.self-hosted-ssl-test.kbtdev.org. 6 IN A	9.8.7.6
ee.self-hosted-ssl-test.kbtdev.org. 60 IN A	9.8.7.6

With the DNS working properly and the server booted up, let’s begin by opening a SSH session with the new server:

john@world$ ssh root@kf.self-hosted-ssl-test.kbtdev.org
<snip>
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added 'kf.self-hosted-ssl-test.kbtdev.org,9.8.7.6' (ECDSA) to the list of known hosts.
root@kf.self-hosted-ssl-test.kbtdev.org's password: 
Welcome to Ubuntu 18.04.2 LTS (GNU/Linux 4.15.0-45-generic x86_64)
<snip>

Installing the Docker repository’s GPG key:

root@localhost:~# curl -fsSL https://download.docker.com/linux/ubuntu/gpg | apt-key add -
OK

Adding the Docker repository:

root@localhost:~# apt-add-repository "deb [arch=amd64] https://download.docker.com/linux/ubuntu bionic stable"
<snip>
Fetched 5,118 kB in 2s (3,103 kB/s)                          
Reading package lists... Done

Installing the usual package updates, the latest Docker, and pip, which we’ll use to install Docker Compose:

root@localhost:~# apt upgrade && apt install docker-ce python-pip
Reading package lists... Done
Building dependency tree       
Reading state information... Done
Calculating upgrade... Done
<snip>
105 upgraded, 5 newly installed, 0 to remove and 0 not upgraded.
Need to get 175 MB of archives.
After this operation, 357 MB of additional disk space will be used.
Do you want to continue? [Y/n] 
<snip>
0 upgraded, 82 newly installed, 0 to remove and 0 not upgraded.
Need to get 125 MB of archives.
After this operation, 478 MB of additional disk space will be used.
Do you want to continue? [Y/n] 
<snip>

Installing the latest Docker Compose using pip, one of many methods:

root@localhost:~# pip install docker-compose
Collecting docker-compose
<snip>
Successfully installed PyYAML-3.13 backports.ssl-match-hostname-3.7.0.1 bcrypt-3.1.6 cached-property-1.5.1 certifi-2019.3.9 cffi-1.12.3 chardet-3.0.4 docker-3.7.2 docker-compose-1.24.0 docker-pycreds-0.4.0 dockerpty-0.4.1 docopt-0.6.2 functools32-3.2.3.post2 jsonschema-2.6.0 paramiko-2.4.2 pyasn1-0.4.5 pycparser-2.19 pynacl-1.3.0 requests-2.20.1 texttable-0.9.1 urllib3-1.24.3 websocket-client-0.56.0

Adding a new non-root user for KoBo, which is nice for tidiness if not security (see the next step):

root@localhost:~# adduser kobo
Adding user `kobo' ...
Adding new group `kobo' (1000) ...
Adding new user `kobo' (1000) with group `kobo' ...
Creating home directory `/home/kobo' ...
Copying files from `/etc/skel' ...
Enter new UNIX password: 
Retype new UNIX password: 
passwd: password updated successfully
Changing the user information for kobo
Enter the new value, or press ENTER for the default
	Full Name []: 
	Room Number []: 
	Work Phone []: 
	Home Phone []: 
	Other []: 
Is the information correct? [Y/n] 

Warning: adding a user to the docker group effectively grants root privileges. See https://docs.docker.com/install/linux/linux-postinstall/.

root@localhost:~# usermod -aG docker kobo

Switching to the new user and making sure it’s a member of the docker group:

root@localhost:~# su kobo
kobo@localhost:/root$ groups
kobo docker

Changing to the kobo user’s home directory and cloning the kobo-install repository:

kobo@localhost:/root$ cd 
kobo@localhost:~$ git clone https://github.com/kobotoolbox/kobo-install
Cloning into 'kobo-install'...
<snip>

Starting the included run.py script and answering its questions; notice that I accepted almost all of the defaults:

kobo@localhost:~$ cd kobo-install/
kobo@localhost:~/kobo-install$ ./run.py 
╔═══════════════════════════════════════════════════════════════╗
║ Welcome to KoBoInstall!                                       ║
║                                                               ║
║ You are going to be asked some questions that will            ║
║ determine how to build the configuration of `KoBoToolBox`.    ║
║                                                               ║
║ Some questions already have default values (within brackets). ║
║ Just press `enter` to accept the default value or enter `-`   ║
║ to remove previously entered value.                           ║
║ Otherwise choose between choices or type your answer.         ║
╚═══════════════════════════════════════════════════════════════╝
Where do you want to install?
[/home/kobo/kobo-docker]: 
Please confirm path [/home/kobo/kobo-docker]
	1) Yes
	2) No
[1]: 
Do you want to see advanced options?
	1) Yes
	2) No
[2]: 
What kind of installation do you need?
	1) On your workstation
	2) On a server
[2]: 
Public domain name [kobo.local]: self-hosted-ssl-test.kbtdev.org
KPI sub domain [kf]: 
KoBoCat sub domain [kc]: 
Enketo Express sub domain name [ee]: 
Do you want to use HTTPS?
	1) Yes
	2) No
[1]: 
╔════════════════════════════════════════════════════════════════════╗
║ Please note that certificates must be installed on a reverse-proxy ║
║ or a load balancer.                                                ║
║ KoBoInstall can install one, if needed.                            ║
╚════════════════════════════════════════════════════════════════════╝
Auto-install HTTPS certificates with Let's Encrypt?
	1) Yes
	2) No - Use my own reserve-proxy/load-balancer
[1]: 
╔════════════════════════════════════════════════╗
║ Domain names must be publicly accessible.      ║
║ Otherwise Let's Encrypt won't be able to valid ║
║ your certificates.                             ║
╚════════════════════════════════════════════════╝
Email address for Let's Encrypt: mymail@gmail.com
Please confirm [mymail@gmail.com]
	1) Yes
	2) No
[1]: 
Cloning `nginx-certbot` repository to `/home/kobo/nginx-certbot` 
Cloning into '/home/kobo/nginx-certbot'...
remote: Enumerating objects: 128, done.
remote: Total 128 (delta 0), reused 0 (delta 0), pack-reused 128
Receiving objects: 100% (128/128), 20.78 KiB | 2.97 MiB/s, done.
Resolving deltas: 100% (66/66), done.

To have KoBo send email through Gmail’s SMTP server, you must first generate an “App Password” within your Google account, and then use that as your SMTP password here:

SMTP server: smtp.gmail.com
SMTP port [25]: 567
SMTP user: mymail@gmail.com
SMTP password: my-generated-APP-PASSWORD
Use TLS?
	1) True
	2) False
[1]: 2
From email address [support@self-hosted-ssl-test.kbtdev.org]: mymail@gmail.com
Super user's username [super_admin]: 
Super user's password [random!string]: super_test
Do you want to activate backups?
	1) Yes
	2) No
[2]: 

You may skim past the output below, but I’ve left it in its entirety to illustrate that some warning messages are normal.

Cloning into '/home/kobo/kobo-docker'...
remote: Enumerating objects: 67, done.
remote: Counting objects: 100% (67/67), done.
remote: Compressing objects: 100% (51/51), done.
remote: Total 2820 (delta 32), reused 37 (delta 16), pack-reused 2753
Receiving objects: 100% (2820/2820), 2.08 MiB | 27.31 MiB/s, done.
Resolving deltas: 100% (1724/1724), done.
Already on 'master'
From https://github.com/kobotoolbox/kobo-docker
 * branch            master     -> FETCH_HEAD
Creating network "nginx-certbot_default" with the default driver
Pulling certbot (certbot/certbot:)...
Pulling nginx (nginx:1.15-alpine)...
Creating nginx-certbot_nginx_1 ... done
Removing network kobo-docker_default
WARNING: Network kobo-docker_default not found.
Removing network kobo-docker_kobo-fe-network
WARNING: Network kobo-docker_kobo-fe-network not found.
Stopping nginx-certbot_nginx_1 ... done
Removing nginx-certbot_nginx_1 ... done
Removing network nginx-certbot_default
Launching environment
Creating network "kobo-docker_default" with the default driver
Pulling redis_main (redis:3.2)...
Pulling mongo (mongo:3.4)...
Pulling postgres (mdillon/postgis:9.5)...
Creating kobo-docker_mongo_1       ... done
Creating kobo-docker_redis_main_1  ... done
Creating kobo-docker_postgres_1    ... done
Creating kobo-docker_redis_cache_1 ... done
Creating network "kobo-docker_kobo-fe-network" with driver "bridge"
WARNING: Found orphan containers (kobo-docker_redis_cache_1, kobo-docker_redis_main_1, kobo-docker_postgres_1, kobo-docker_mongo_1) for this project. If you removed or renamed this service in your compose file, you can run this command with the --remove-orphans flag to clean it up.
Pulling nginx (kobotoolbox/nginx:latest)...
Pulling kobocat (kobotoolbox/kobocat:2.019.16a)...
Pulling enketo_express (kobotoolbox/enketo-express-extra-widgets:1.76.2)...
Pulling kpi (kobotoolbox/kpi:2.019.16)...
Creating kobo-docker_nginx_1          ... done
Creating kobo-docker_kobocat_1        ... done
Creating kobo-docker_enketo_express_1 ... done
Creating kobo-docker_kpi_1            ... done
Creating network "nginx-certbot_default" with the default driver
Creating nginx-certbot_nginx_1   ... done
Creating nginx-certbot_certbot_1 ... done
Waiting for environment to be ready. It can take a few minutes.
........................................
╔══════════════════════════════════════════════════╗
║ Ready                                            ║
║ URL: https://kf.self-hosted-ssl-test.kbtdev.org/ ║
║ User: super_admin                                ║
║ Password: super_test                             ║
╚══════════════════════════════════════════════════╝

Success! https://kf.self-hosted-ssl-test.kbtdev.org/, https://kc.self-hosted-ssl-test.kbtdev.org/, and https://ee.self-hosted-ssl-test.kbtdev.org/ all worked at this point. I registered a new account, received an activation link via email, logged in, deployed a form, made a submission with Enketo, and viewed the collected data.

2 Likes
Installing Wildcard SSL Certificates on Kobo-Docker
Unable to initiate kobotoolbox in an EC2 instance
#2

Hello everyone,

I’m looking for your help here is the screenshot

#3

Could you share some information about your environment? OS version, amount of RAM, swap size, and CPU would be helpful. Also, did you follow the steps above, or did you choose a different configuration?

There should be another step after this where the front-end containers are restarted. I’m guessing that fails as well, but please confirm.

Try running ./run.py --logs, either after the services fail to start or, in another terminal, during Waiting for environment to be ready. You may notice a useful error.

#4

Well noted, I will resume step by step in accordance with your instructions. I’ll come back to you later.

#7

hi, i need your help


But i can’t start docker-compose service…