Complex Form Logic, Data Validation, and Submission Issues in KoBoToolbox for Large-Scale Surveys

Hi All,

I’m currently building a multi-section survey in KoBoToolbox for a large-scale field data collection project, and I’m running into several technical challenges that are affecting both the form design and data reliability. The form includes multiple groups, repeats, and conditional logic based on previous answers. While the basic structure works, once I started layering more complex skip logic and validation constraints, I began noticing inconsistent behavior especially when certain fields are left blank or when users navigate back to previous questions and modify responses.

One major issue I’m facing is with cascading select questions and dynamic filtering using choice_filter. In some cases, the filtered options don’t update correctly based on prior answers, particularly when the form is used on mobile devices in offline mode. This leads to users seeing irrelevant or outdated options, which compromises the data quality. I’ve double-checked the syntax in my XLSForm and ensured that the columns are properly defined, but the issue persists intermittently, making it difficult to debug.

Another challenge is related to validation constraints and required fields. I’ve added constraint expressions to enforce numeric ranges, regex patterns, and logical consistency between fields. However, I’ve noticed that some constraints are either not triggering at all or are bypassed when users quickly navigate through the form. Additionally, required fields inside repeat groups sometimes allow submission even when left empty, which defeats the purpose of enforcing completeness in critical sections of the survey.

I’m also dealing with performance issues as the form has grown significantly in size. It now includes over 300 questions, multiple repeat groups, and embedded calculations. On lower-end Android devices, the form takes a long time to load, and there is noticeable lag when navigating between questions. This is particularly problematic in field conditions where enumerators need to work efficiently without delays. I’ve tried optimizing by reducing unnecessary calculations and grouping questions, but the improvements have been minimal.

Another area of concern is data submission and syncing. Some users report that completed forms remain stuck in the “sending” queue within KoBoCollect, even when they have a stable internet connection. In a few cases, submissions appear to go through but are not visible on the server dashboard. This inconsistency is making it difficult to track data collection progress in real time and raises concerns about potential data loss.

I would appreciate any insights or best practices for managing complex form logic, ensuring reliable validation, and improving performance in large KoBoToolbox forms. Are there recommended design patterns for structuring large XLSForms, especially when using repeats and dynamic logic? Additionally, any tips on debugging submission issues or ensuring consistent syncing between KoBoCollect and the server would be extremely helpful.

Is there anyone who can guide?

Hi @juliepaul First, welcome to KoboToolbox.

There is a lot described above, so it is rather hard to know where to start. Suffice to say all these things fundamentally do work correctly in Kobo. But it is a matter of understanding perhaps where things are going wrong with your specific form(s).

I might suggest focusing on just one thing at a time; eg start with cascading selects. There is a lot of online documentation available for this, both in our Kobo docs as well as out there in the broader ODK community; XLSForm cascading selects are the same for everybody, whether you happen to be using KoboToolbox, KoboCollect, Enketo, ODK Central, ODK Collect, etc.

Then, once you have a solid grasp of that, perhaps focus on constraints and validation. Again, there are Kobo docs on this, and plenty of other online resources on how to do validation in XLSForms (again, they are the same for everybody).

If you do run into something in particular that isn’t working for you, try to reduce your form into a simpler example that focuses on just reproducing that, and post something to the Forum here and I am sure somebody will be able to help you figure it out.

If you havent already I might also suggest looking at our online course materials for KoboToolbox Essentials and slightly more advanced XLSForm Fundamentals. They’re both well worth it IMO and will give you a solid grounding in writing forms; I’m sure you will learn something new. :slightly_smiling_face:

1 Like

Hi @juliepaul

I understand you may be quite a ways along writing your xlsform and may not want to explore an alternative, but I wanted to introduce you to a tool I created that lets you write the survey in very simple text and it renders the logic flow immediately. It also will render to an interactive html view of the survey. And once you have your survey the way you like, you can render it to XForm for use in KoboToolbox.

Simple, Readable, Parseable surveys for humans and computers

Feel free to message me if you have any questions or follow-up. Cheers!