General questions: Data Migration / MongoDB recreate

support

#1

Greetings!

I’m a new KoboToolbox admin and have a KoboDocker server under Linux. I have learned the most part of the documents accessible through the internet and have a pair of questions.

  1. The Data migration during Updates
  • It is supposed that Kobo code is referring some special fields in its internal databases. Is there exist a possibility that a new code would be incompatible with the previous databases? What to do in this case?
  1. The document published at

https://www.kobowiki.org/index.php?title=Restore_procedures

say, that it is possible to recreate MongoDB based on the PostgreSQL data, however, the PostgreSQL dump done with the backup_postgres.bash shows no data on the running Kobo server that already contain many survey data. The dump specify just a public scheme and some general database configurations.


#2

For the second question it seems that I found the solution. Anyway it is interesting to listen to the opinion of the more experienced admins on how canonical is it.

Problem:
In the script backup_postgres.bash it is used a KOBO_POSTGRES_DB_NAME environment variable, however it is absent in the clean KoboDocker installations.

Solution:
Append into the files envfile.local.txt and envfile.server.txt the next line:
KOBO_POSTGRES_DB_NAME=kobotoolbox

After the docker-compose restart, the backup_postgres.bash will recover it’s work.


#3

Hello friends and Happy Christmas to all.

Finally I found an answer on my first question. Yes, the database scheme can change with updates, and it is exactly what was happened in the case of my server. To detect the database change I used the pgquarrel tool, and compared my production server database (obviously the old one) with the database of the up to date KoboDocker clean install. As long as my production server for the moment make a step from the pilot project to the official investigation instrument, the question opened in this thread is important for me personally.

To automate investigation of the KoboDocker migration procedure the https://github.com/YKolokoltsev/KoboAdminTools script pack was created. In the KoboAdminTools description I tried to present the clean update recipes, and invite you to exchange your experience about possible special techniques in KoboDocker administration. Of course the KoboAdmin tools are not finished yet and I actively update this project. But the main idea is now clear: automate administration routines to avoid possible bugs and critical errors.

The main motivation of making server updates is obvious: the mobile client can occasionally became incompatible with the old frozen server version. This would mean death for the server with a lots of data and a wide client base, because migrate frozen DB to the new one can be too error prone for those who is not a direct developer of the KoboToolpack.

So let’s track somewhere the discussion about how an admin shell react onto the recent server changes. If there happen any migration task – the correct recipe shell appear and shell be well tested. I suggest the KoboAdminTools project because I have not found yet any other place dedicated to this server administration.

Updates of the KoboDocker I’m planning to track automatically and promise you to react as soon as possible.