Greetings,
We are hosting our dedicated Kobotoolbox server, it worked all fine until today and the homepage returns this error: “502 Bad Gateway nginx”.
AFAIK, nobody touched the backend or config files, it just appeared to occur out of the blue on my end.
When digging into the log as follow:
docker ps shows that some containers keeps restarting (as per image below, see STATUS):
Then I dug into the logs of those containers but the one that striked me the most was from docker logs kobo-docker-kpi-1 --tail 100
I get this message constantly occurring:
KPI initializing… Waiting for container mongo. wait-for-it: waiting 40 seconds for mongo.collecte.private:27017 wait-for-it: timeout occurred after waiting 40 seconds for mongo.collecte.private:27017
The solutions I’m being given is to edit config files where anything related to MONGO_HOST or KPI_MONGO_HOST should be changed from mongo.collecte.private to mongo. But I am skeptical of that because edit history of my files shows that no file has been edited since the starting of the server so no change should have had happened related to config files.
Restarting the server did not help.
Please note that the whole site is down, nothing is accessible and we are being very careful approaching this issue as we do not want to accidentally delete all our collected data.
Thanks for any help.
