`kobo-install` on a fresh Ubuntu 18.04 LTS VPS

This is an annotated transcript of installing https://github.com/kobotoolbox/kobo-install on a fresh “Linode 2GB” instance, with 1 CPU core, 2 GB of RAM (surprise), and 50 GB of SSD storage. It costs $10/month. In the Linode Manager, I used “Create a new Disk” to create 8 GB of swap prior to “Deploy an image,” since the latter only offers swap sizes up to 512 MB. I deployed the Ubuntu 18.04 LTS image and set a root password.

:information_source: Anywhere that <snip> appears indicates that I’ve removed some output for brevity and readability.

First, before doing anything on the new server, I verified that the DNS for my three domains was configured properly. I’ve replaced my VPS’ real IP address with 9.8.7.6 throughout this transcript:

john@world$ dig +noall +answer kf.self-hosted-ssl-test.kbtdev.org kc.self-hosted-ssl-test.kbtdev.org ee.self-hosted-ssl-test.kbtdev.org
kf.self-hosted-ssl-test.kbtdev.org. 3 IN A	9.8.7.6
kc.self-hosted-ssl-test.kbtdev.org. 6 IN A	9.8.7.6
ee.self-hosted-ssl-test.kbtdev.org. 60 IN A	9.8.7.6

With the DNS working properly and the server booted up, let’s begin by opening a SSH session with the new server:

john@world$ ssh root@kf.self-hosted-ssl-test.kbtdev.org
<snip>
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added 'kf.self-hosted-ssl-test.kbtdev.org,9.8.7.6' (ECDSA) to the list of known hosts.
root@kf.self-hosted-ssl-test.kbtdev.org's password: 
Welcome to Ubuntu 18.04.2 LTS (GNU/Linux 4.15.0-45-generic x86_64)
<snip>

Installing the Docker repository’s GPG key:

root@localhost:~# curl -fsSL https://download.docker.com/linux/ubuntu/gpg | apt-key add -
OK

Adding the Docker repository:

root@localhost:~# apt-add-repository "deb [arch=amd64] https://download.docker.com/linux/ubuntu bionic stable"
<snip>
Fetched 5,118 kB in 2s (3,103 kB/s)                          
Reading package lists... Done

Installing the usual package updates, the latest Docker, and pip, which we’ll use to install Docker Compose:

root@localhost:~# apt upgrade && apt install docker-ce python-pip
Reading package lists... Done
Building dependency tree       
Reading state information... Done
Calculating upgrade... Done
<snip>
105 upgraded, 5 newly installed, 0 to remove and 0 not upgraded.
Need to get 175 MB of archives.
After this operation, 357 MB of additional disk space will be used.
Do you want to continue? [Y/n] 
<snip>
0 upgraded, 82 newly installed, 0 to remove and 0 not upgraded.
Need to get 125 MB of archives.
After this operation, 478 MB of additional disk space will be used.
Do you want to continue? [Y/n] 
<snip>

Installing the latest Docker Compose using pip, one of many methods:

root@localhost:~# pip install docker-compose
Collecting docker-compose
<snip>
Successfully installed PyYAML-3.13 backports.ssl-match-hostname-3.7.0.1 bcrypt-3.1.6 cached-property-1.5.1 certifi-2019.3.9 cffi-1.12.3 chardet-3.0.4 docker-3.7.2 docker-compose-1.24.0 docker-pycreds-0.4.0 dockerpty-0.4.1 docopt-0.6.2 functools32-3.2.3.post2 jsonschema-2.6.0 paramiko-2.4.2 pyasn1-0.4.5 pycparser-2.19 pynacl-1.3.0 requests-2.20.1 texttable-0.9.1 urllib3-1.24.3 websocket-client-0.56.0

Adding a new non-root user for KoBo, which is nice for tidiness if not security (see the next step):

root@localhost:~# adduser kobo
Adding user `kobo' ...
Adding new group `kobo' (1000) ...
Adding new user `kobo' (1000) with group `kobo' ...
Creating home directory `/home/kobo' ...
Copying files from `/etc/skel' ...
Enter new UNIX password: 
Retype new UNIX password: 
passwd: password updated successfully
Changing the user information for kobo
Enter the new value, or press ENTER for the default
	Full Name []: 
	Room Number []: 
	Work Phone []: 
	Home Phone []: 
	Other []: 
Is the information correct? [Y/n] 

Warning: adding a user to the docker group effectively grants root privileges. See https://docs.docker.com/install/linux/linux-postinstall/.

root@localhost:~# usermod -aG docker kobo

Switching to the new user and making sure it’s a member of the docker group:

root@localhost:~# su kobo
kobo@localhost:/root$ groups
kobo docker

Changing to the kobo user’s home directory and cloning the kobo-install repository:

kobo@localhost:/root$ cd 
kobo@localhost:~$ git clone https://github.com/kobotoolbox/kobo-install
Cloning into 'kobo-install'...
<snip>

Starting the included run.py script and answering its questions; notice that I accepted almost all of the defaults:

kobo@localhost:~$ cd kobo-install/
kobo@localhost:~/kobo-install$ ./run.py 
╔═══════════════════════════════════════════════════════════════╗
║ Welcome to KoBoInstall!                                       ║
║                                                               ║
║ You are going to be asked some questions that will            ║
║ determine how to build the configuration of `KoBoToolBox`.    ║
║                                                               ║
║ Some questions already have default values (within brackets). ║
║ Just press `enter` to accept the default value or enter `-`   ║
║ to remove previously entered value.                           ║
║ Otherwise choose between choices or type your answer.         ║
╚═══════════════════════════════════════════════════════════════╝
Where do you want to install?
[/home/kobo/kobo-docker]: 
Please confirm path [/home/kobo/kobo-docker]
	1) Yes
	2) No
[1]: 
Do you want to see advanced options?
	1) Yes
	2) No
[2]: 
What kind of installation do you need?
	1) On your workstation
	2) On a server
[2]: 
Public domain name [kobo.local]: self-hosted-ssl-test.kbtdev.org
KPI sub domain [kf]: 
KoBoCat sub domain [kc]: 
Enketo Express sub domain name [ee]: 
Do you want to use HTTPS?
	1) Yes
	2) No
[1]: 
╔════════════════════════════════════════════════════════════════════╗
║ Please note that certificates must be installed on a reverse-proxy ║
║ or a load balancer.                                                ║
║ KoBoInstall can install one, if needed.                            ║
╚════════════════════════════════════════════════════════════════════╝
Auto-install HTTPS certificates with Let's Encrypt?
	1) Yes
	2) No - Use my own reserve-proxy/load-balancer
[1]: 
╔════════════════════════════════════════════════╗
║ Domain names must be publicly accessible.      ║
║ Otherwise Let's Encrypt won't be able to valid ║
║ your certificates.                             ║
╚════════════════════════════════════════════════╝
Email address for Let's Encrypt: mymail@gmail.com
Please confirm [mymail@gmail.com]
	1) Yes
	2) No
[1]: 
Cloning `nginx-certbot` repository to `/home/kobo/nginx-certbot` 
Cloning into '/home/kobo/nginx-certbot'...
remote: Enumerating objects: 128, done.
remote: Total 128 (delta 0), reused 0 (delta 0), pack-reused 128
Receiving objects: 100% (128/128), 20.78 KiB | 2.97 MiB/s, done.
Resolving deltas: 100% (66/66), done.

To have KoBo send email through Gmail’s SMTP server, you must first generate an “App Password” within your Google account, and then use that as your SMTP password here:

SMTP server: smtp.gmail.com
SMTP port [25]: 567
SMTP user: mymail@gmail.com
SMTP password: my-generated-APP-PASSWORD
Use TLS?
	1) True
	2) False
[1]: 2
From email address [support@self-hosted-ssl-test.kbtdev.org]: mymail@gmail.com
Super user's username [super_admin]: 
Super user's password [random!string]: super_test
Do you want to activate backups?
	1) Yes
	2) No
[2]: 

You may skim past the output below, but I’ve left it in its entirety to illustrate that some warning messages are normal.

Cloning into '/home/kobo/kobo-docker'...
remote: Enumerating objects: 67, done.
remote: Counting objects: 100% (67/67), done.
remote: Compressing objects: 100% (51/51), done.
remote: Total 2820 (delta 32), reused 37 (delta 16), pack-reused 2753
Receiving objects: 100% (2820/2820), 2.08 MiB | 27.31 MiB/s, done.
Resolving deltas: 100% (1724/1724), done.
Already on 'master'
From https://github.com/kobotoolbox/kobo-docker
 * branch            master     -> FETCH_HEAD
Creating network "nginx-certbot_default" with the default driver
Pulling certbot (certbot/certbot:)...
Pulling nginx (nginx:1.15-alpine)...
Creating nginx-certbot_nginx_1 ... done
Removing network kobo-docker_default
WARNING: Network kobo-docker_default not found.
Removing network kobo-docker_kobo-fe-network
WARNING: Network kobo-docker_kobo-fe-network not found.
Stopping nginx-certbot_nginx_1 ... done
Removing nginx-certbot_nginx_1 ... done
Removing network nginx-certbot_default
Launching environment
Creating network "kobo-docker_default" with the default driver
Pulling redis_main (redis:3.2)...
Pulling mongo (mongo:3.4)...
Pulling postgres (mdillon/postgis:9.5)...
Creating kobo-docker_mongo_1       ... done
Creating kobo-docker_redis_main_1  ... done
Creating kobo-docker_postgres_1    ... done
Creating kobo-docker_redis_cache_1 ... done
Creating network "kobo-docker_kobo-fe-network" with driver "bridge"
WARNING: Found orphan containers (kobo-docker_redis_cache_1, kobo-docker_redis_main_1, kobo-docker_postgres_1, kobo-docker_mongo_1) for this project. If you removed or renamed this service in your compose file, you can run this command with the --remove-orphans flag to clean it up.
Pulling nginx (kobotoolbox/nginx:latest)...
Pulling kobocat (kobotoolbox/kobocat:2.019.16a)...
Pulling enketo_express (kobotoolbox/enketo-express-extra-widgets:1.76.2)...
Pulling kpi (kobotoolbox/kpi:2.019.16)...
Creating kobo-docker_nginx_1          ... done
Creating kobo-docker_kobocat_1        ... done
Creating kobo-docker_enketo_express_1 ... done
Creating kobo-docker_kpi_1            ... done
Creating network "nginx-certbot_default" with the default driver
Creating nginx-certbot_nginx_1   ... done
Creating nginx-certbot_certbot_1 ... done
Waiting for environment to be ready. It can take a few minutes.
........................................
╔══════════════════════════════════════════════════╗
║ Ready                                            ║
║ URL: https://kf.self-hosted-ssl-test.kbtdev.org/ ║
║ User: super_admin                                ║
║ Password: super_test                             ║
╚══════════════════════════════════════════════════╝

Success! https://kf.self-hosted-ssl-test.kbtdev.org/, https://kc.self-hosted-ssl-test.kbtdev.org/, and https://ee.self-hosted-ssl-test.kbtdev.org/ all worked at this point. I registered a new account, received an activation link via email, logged in, deployed a form, made a submission with Enketo, and viewed the collected data.

3 Likes

Hello everyone,

I’m looking for your help here is the screenshot

Could you share some information about your environment? OS version, amount of RAM, swap size, and CPU would be helpful. Also, did you follow the steps above, or did you choose a different configuration?

There should be another step after this where the front-end containers are restarted. I’m guessing that fails as well, but please confirm.

Try running ./run.py --logs, either after the services fail to start or, in another terminal, during Waiting for environment to be ready. You may notice a useful error.

Well noted, I will resume step by step in accordance with your instructions. I’ll come back to you later.

hi, i need your help


But i can’t start docker-compose service…

I am also having trouble completing the kobo docker install. I’m able to walk through all the steps you’ve laid out above till I get to the kobo docker installation itself. when I run the setup, the install fails right after creating the nginx certbot. I’m installing it on a basic DigitalOcean droplet. All the other stuff with installing docker and docker compose went well. See below a screenshot of the error message and where it exits. Would appreciate any pointers as to what the problem could be.

I am trying to install days ago and it always gives me errors. You can help me?

Already on ‘master’
From https://github.com/kobotoolbox/kobo-docker

  • branch master -> FETCH_HEAD
    Creating network “nginx-certbot_default” with the default driver
    Pulling certbot (certbot/certbot:)…
    Pulling nginx (nginx:1.15-alpine)…
    Creating nginx-certbot_nginx_1 … done
    Error response from daemon: Container 5d3e20d604eb76503b09b8f3ca995263afadb02a099e331b13de6ab1e0571f11 is restarting, wait until the container is running
    An error has occurred
    kobo@kobo-salta:~/kobo-install$

hi @ddiazsiempro

There was an issue with some config files in the nginx-certbot repo.
Do you mind to delete the folder (nginx-certbot) and restart kobo-install installation (python run.py --setup)

╔════════════════════════════════════════════════╗
║ Domain names must be publicly accessible. ║
║ Otherwise Let’s Encrypt won’t be able to valid ║
║ your certificates. ║
╚════════════════════════════════════════════════╝
Email address for Let’s Encrypt [siempro.salta@gmail.com]:
Please confirm [siempro.salta@gmail.com]
1) Yes
2) No
[1]:
Cloning nginx-certbot repository to /root/nginx-certbot
fatal: destination path ‘/root/nginx-certbot’ already exists and is not an empty directory.
An error has occurred
root@kobo-salta:~/kobo-install#

As I said, can you delete /root/nginx-certbot (completely. i.e. rm -rf /root/nginx-certbot) first, then retry the installation.

I delete the directory, but it’s like it’s part of a docker container. Try also deleting the container and keep giving error

ok let’s try this:

$kobo-install> python run.py --stop
$kobo-install> rm -rf ../nginx-certbot
$kobo-install> ls -l ../nginx-certbot

It should display this

ls: cannot access '../nginx-certbot': No such file or directory

Then, again

$kobo-install> python run.py --setup
1 Like

Sorry about the inconvenience. I also faced the issue. I forgot to update kobo-install too.

Please try this:

$kobo-install> python run.py --stop
$kobo-install> docker ps  # no containers should be up, otherwise stop, them manually
$kobo-install> git pull origin master
$kobo-install> git log -n 1 --oneline
a61b14c (HEAD -> master, origin/master) Changed nginx-certbot init script template to fetch files for kobotoolbox repo instead certbot repo
$kobo-install> cd /root/nginx-certbot/
$nginx-certbox> git log -n 1 --oneline
6a033b9 (HEAD -> master, origin/master, origin/HEAD) Use local ssl files instead of certbot repo
$nginx-certbox> rm -rf data/certbot
$nginx-certbox> cd kobo-install
$kobo-install> python run.py --setup

It should solve your issues. Please give me updated.

2 Likes

I installed on a server from 0 everything with a new domain and it was installed perfectly with the last post it gave me. The only problem I have is that new users cannot register. And when I restart the server, I must manually start the kobo, the docker does not take me. Thank you for all your help.

@kenGIS,
Can you try the command I gave to ddiazsiempro?

I’ve noticed you install kobo-install as root.
You should install it as normal user and ensure that this user is allowed to start docker. https://docs.docker.com/install/linux/linux-postinstall/

To restart the containers on reboot, we use a crontab to achieve this.
Create the file /etc/cron.d/kobo-install and add this line into it

@reboot <username> cd <path.to.kobo-install> && /usr/bin/python run.py

Leave the empty line at the end of the file, otherwise cron won’t run it

Obviously, adapt it to your config.

2 Likes

Thanks, I was able to configure it to start automatically. The problem I have is that new users cannot register, I get error 500

It’s often due to bad SMTP settings. Ensure to enter correct ones when prompted by kobo-install.

1 Like