Could you help with data server errors that we have in a few Kobo projects? They are all on the global server https://kf.kobotoolbox.org/ :
Interestingly these are all based on the same template form, but other projects also based on the same template form work well. The data were collected on the CyberTracker application (we developed a specific application for this project that retrieves Kobo forms associated with it. Let me know if you need more details on this!
@dianedetoeuf, could you kindly provide more details on this? So you meant you did not use the Collect Android App or Enketo to collect your data for the project that you deployed through the KoboToolbox server. Instead, you used your custom-made CyberTracker application.
You are right, we did not use the KoboCollect or Enketo app, but an application called CyberTracker, that retrieves froms from KoboToolbox. We’ve used that application for a while and everything was working perfectly until now.
I thought I had send you a private message about the server 500 issue but it looks like the message was sent on the public thread (Data Server Error (500))
I had shared with you all the username details, were you able to check what is happening?
Hi @Kal_Lam we’re helping Diane look into this and there are 2 things that are strange…
When we log into the Kobo account via the web UI, we can see the form with name TEMPLATE Site Mapping and id a52oRefc52PVsTBGNjQFVm listed in Projects … which is good. But then…
When we click into the form > and then the Data tab, none of the data shows and there is a 500 Internal Server error displayed on the screen (see screenshot)
Then when we try to query the form’s data via the API, we also encounter this 500 error in response to our GET 'https://kf.kobotoolbox.org/api/v2/assets/a52oRefc52PVsTBGNjQFVm/data/?format=json'
When we try steps 1 and 2 with another form in this account (e.g., aSAjhptyRae7BcSvDBpthD), we’re able to query the data no problem. Was the data for the form above perhaps somehow corrupted?
Hello! Congratulations on developing your own application this way.
What’s going on here is that CyberTracker constructs submission XML in ways that KoboToolbox does not expect. The two main issues I see are:
Some submissions do not have a <meta> block with <instanceID> inside. Other CyberTracker submissions do have this, so perhaps its something you’ve already addressed. Either way, please make sure this is always included as required by the OpenRosa specification: “The only required element in the form submission metadata is <instanceID>, which must be a universally unique string identifying this specific submission.” We currently invent a value for <instanceID> (submission UUID) if the submission lacks it, but we may stop accepting those submissions in the future.
The cause of this particular 500 error: all CyberTracker submissions that I’ve seen lack the <formhub> block, which should have <uuid> inside: this is the UUID of the form, not the submission. There must be a recent change we’ve made in Kobo that relies on this being present, and I think we can work around it. Generally, though, you should make sure that CyberTracker respects what the XForm says ought to be included in the submission. For example, one of your XForms contains this:
<model odk:xforms-version="1.0.0">
<!-- other elements snipped -->
<instance>
<aNWAzVredactedredacted id="aNWAzVredactedredacted" version="3 (2024-05-30 08:51:02)">
<formhub>
<uuid/>
</formhub>
<!-- other elements snipped -->
</aNWAzVredactedredacted>
</instance>
<!-- other elements snipped -->
<bind nodeset="/aNWAzVredactedredacted/formhub/uuid" type="string" calculate="'50f7b62bredactedredactedredacted'"/>
</model>
That dictates that submissions for this form should include <formhub><uuid>50f7b62bredactedredactedredacted</uuid></formhub>. You should watch for things like this, or, perhaps more easily than grappling with the XForm specification, try creating submissions with Collect or Enketo. Then, compare the submission XML to what’s being generated by CyberTracker for the same form. Make sure you have a good rationale for anything missing from the CyberTracker submission that’s present in the submission made by Collect or Enketo.
We’ll post an update when we have a resolution for the missing <formhub><uuid> issue.
I have deployed a change to CyberTracker which includes the formhub uuid in all future submissions. We will still get some data coming through without it until everyone has updated.
