jdugh
June 11, 2025, 3:44pm
1
Hi everyone,
I wanted to share an issue I’ve encountered and analyzed on KoboToolbox related to editing submissions to add multiple images in a repeat group .
Summary of the problem
The problem is that the first batch is accepted, but the following batches fail — the user gets a 400 error saying:
“The server did not accept the data. Please contact the survey administrator (400)”
What I found
This issue does not happen when making a new submission (even large ones), only when editing an existing one with new attachment adding in the edition.
Enketo , KPI , or KoBoCat , but I’ve opened a GitHub issue, along with a temporary workaround we applied on our instance:
Link to the GitHub issue
I’d love to hear from the Kobo team or other community members:
Has anyone else experienced this?
Thanks in advance for your help!
Jonathan
1 Like
Welcome back to the community, @jdugh ! Have you tried this outside the repeat group?
jdugh
June 12, 2025, 5:54pm
3
Hi @Kal_Lam , thank you !
1 Like
Thank you @jdugh , seems like you already have created a GitHub issue for this:
opened 07:19AM - 12 Jun 25 UTC
Back end
[moved here from enketo/enketo#1406] [edits by @jnm]
**Describe the bug**
When … **editing** a submission to add multiple images within a begin repeat group, Enketo splits the data into multiple "batches" to stay under the 10MB payload limit. This triggers a 400 error on the second batch during the edit submission process.
**To Reproduce**
1. On KoboToolbox, create a form with:
A simple text field
A begin repeat group containing a single image field
2. Deploy the form and open it in Enketo.
3. Submit a response (with or without filling in the text field).
4. Edit the same submission.
5. In the repeat group, add enough images to exceed the 10MB limit (to trigger Enketo’s batching behavior).
6. Submit the edited form.
**Expected behavior**
The edited submission (even with multiple batches) should be accepted by the server without any errors.
**Additional context**
From investigation and debugging:
- Enketo splits large submissions into multiple HTTP POST requests ("batches").
- In **edit mode**, Enketo sends each batch to:
[`https://<kpi_url>/api/v2/asset_snapshots/<id>/submission`](https://github.com/enketo/enketo/blob/main/packages/enketo-express/app/controllers/submission-controller.js#L77)
- On the first batch:
- KPI receives the XML and attachments.
- ~~It reads the `instanceID` from the XML, finds the matching instance in MongoDB.~~
- [The XML includes a new `instanceID`, generated by Enketo, and a `deprecatedID`, which identifies the submission being edited]
- [KPI finds the matching `Instance` in PostgreSQL]
- ~~It creates a new version of the submission by setting `deprecatedID`, generating a new `instanceID`, and saving the updated record.~~
- [KPI overwrites the `Instance` found in PostgreSQL with the new XML, representing the edit, received from Enketo. THis new XML includes the new, Enketo-generated `instanceID` and the `deprecatedID` pointing to the `instanceID` of the previous revision of this submission.]
- On the **second batch**:
- KPI again tries to find the original instance via the same ~~`instanceID`~~ [`deprecatedID`] from the XML.
- But it no longer exists (it has already been updated in the first batch).
- This causes a `Instance.DoesNotExist` and a `400` error is returned.
- https://github.com/kobotoolbox/kpi/blob/main/kpi/deployment_backends/openrosa_backend.py#L400-L412
Note: In **new submissions** (not edits), even with multiple batches, Enketo sends data to:
`https://<kobocat_url>/<user_name>/submission`
This endpoint does not fail under the same conditions.
The URL used for submission [of edits] (`/api/v2/asset_snapshots/<id>/submission`) seems to come from Redis (`main` DB) and is specific to **edit mode** ? [Enketo likely does store it in Redis temporarily; it comes into Enketo via the ["Load an instance for editing" Enketo API endpoint](https://apidocs.enketo.org/v2/#/post-instance), which is called by KPI [here](https://github.com/kobotoolbox/kpi/blob/05ffd4cc62b1cb137f4d7c4ea03654124f672c95/kpi/views/v2/data.py#L675)]
---
### Notes
- On very old KoboToolbox versions (3+ years old), **even edit mode** used the `kobocat` endpoint, and the issue did not occur.
- In our own deployment, we implemented a **temporary workaround** in the `kobocat_backend` (KPI), inside `edit_submission()`:
- When a `Instance.DoesNotExist` is raised, we fall back to searching in MongoDB using [it's immaterial for this bug, but to keep the record correct, are you sure it's not [PostgreSQL](https://github.com/kobotoolbox/kpi/blob/05ffd4cc62b1cb137f4d7c4ea03654124f672c95/kpi/deployment_backends/openrosa_backend.py#L344-L348)?]:
- `meta/instanceID` [Yes, this is likely what needs to be added]
- `meta/deprecatedID` [That was already being done; if you got `Instance.DoesNotExist`, then there was no match]
- `meta/rootUuid` [This should not be done]
- We then retrieve the correct `_id` of the associated submission and re-fetch the object accordingly.
- This workaround is likely not ideal or robust I think, but it bypasses the issue temporarily.
---
### Request for Feedback
We're open to proposing a PR with a more stable fix, but are unsure what would be considered the **cleanest or most correct approach** to handle this kind of multi-batch edit scenario.
- Should Enketo use a different URL or include more metadata per batch? I think no, Enketo is behaving according to the OpenRosa spec (https://github.com/enketo/enketo/issues/1406#issuecomment-2963520245) [Agreed!]
- Should KPI handle fallback via `rootUuid` or similar logic? [Not `rootUuid`, but likely just `instanceId`]
Any guidance or suggestions from maintainers would be greatly appreciated.
---
Let me know if a minimal reproducible XML form is needed or if you want our complete **temporary workaround** code — happy to provide one.
Summary of the problem
What I found
I’m not sure if this is an issue with Enketo, KPI, or KoBoCat, but I’ve opened a GitHub issue, along with a temporary workaround we applied on our instance: