I’m hosting KoBo on my own server. I’ve succesfully built a rather large form (~2,4MB) and now I need to share the Enketo form URL with the people responsible of submitting data. The first time the Enketo URL is requested, it takes a while to display the form (about 30 mins). I’ve read that this is “normal” behavior since the HTML generation step seems to be compute intensive (CPU usage is stuck at 100% in a few cores during the 30 mins). Subsequent requests to Enketo are displayed much more quicker, since I suppose they are being served with the HTML cached in the Redis secondary instance. After a few hours, the Enketo URL suddenly stops working throwing a 404 Survey with this ID not found error. Indeed, the Enketo URL to the form changes in the KoBo dashboard and the cache is lost. My two questions are:
Should the Enketo URLs be permanent? If yes, do you know what might be causing the ID obsolescence I’m experiencing? If no, how do you handle providing a permanent URL to the form for people to submit data to it?
Thanks for your help. When the form is deployed to the KoBoToolbox online server the expiring Enketo URLs issue doesn’t occur. I think the problem with my own KoboToolbox instance is that for reasons that I currently ignore, the main and secondary Redis instances lose their data. Do you have any pointers regarding things or settings to look for in my own instance in order to solve the URL rotation problem?
Well, in this case would you mind providing the OS and other relevant details you have installed in your device so that i could flag it out to our developers.
I didn’t set up the firewall so unfortunately I won’t be able to give you the step-by-step directions. Also, instructions will change depending on your system specs and configuration. However, the problem was what @jnm said, i.e. that there were Redis ports open to the Internet allowing external unauthorized people to mess up with the database. Restricting access in those ports through a firewall fixed the issue.
After a while I solve this issue with two iptables rules, the process I follow was:
iptables-save > iptables-rules.orig
You can create a copy:
cp iptables-rules.orig iptables-rules.new
Edit the file and add this two rule in filter rules at first after “DOCKER-USER - [0:0]”:
-A FORWARD -i eth0 -p tcp -m tcp --dport 6379 -j DROP
-A FORWARD -i eth0 -p tcp -m tcp --dport 6380 -j DROP
Something like this:
# Generated by iptables-save v1.6.1 on Wed Apr 15 22:58:31 2020
*filter
:INPUT ACCEPT [777488:413876809]
:FORWARD DROP [0:0]
:OUTPUT ACCEPT [813511:419978487]
:DOCKER - [0:0]
:DOCKER-ISOLATION-STAGE-1 - [0:0]
:DOCKER-ISOLATION-STAGE-2 - [0:0]
:DOCKER-USER - [0:0]
-A FORWARD -i eth0 -p tcp -m tcp --dport 6379 -j DROP
-A FORWARD -i eth0 -p tcp -m tcp --dport 6380 -j DROP
...
To load the new rules use iptables-restore command:
iptables-restore < iptables-rules.new
And that’s all no more Enketo URL arbitrary changes, and it’s possible use them rules to secure other open ports like mongo (27017) and/or postgresql (5432).
