Problems with data in generated repeat groups in Enketo

Hi,

Thanks to the amazing advice and guidance on this forum & ODK we created a form where repeat groups are generated based on the choices in a previous select-multiple.

However, we encountered 2 main problems:
Problem 1:
a) user selects option 1, 2, 3, 4, 5 in the select multiple. 5 repeat groups are generated.
b) user inputs data for in the repeat groups generated for options 1, 2, and 5
c) user goes back and de-selects options 3, 4 from the select-multiple
d) the data originally input in the repeat group for option 5 disappears, and the user has to input it again. often the user forgets and we end up with missing data.

Problem 2:
a) user selects option 1, 2, 3, 4, 5 in the select multiple. 5 repeat groups are generated.
b) user inputs data for in the repeat groups generated for options 1, 2, and 5
c) user goes back and de-selects options 3, 4 from the select-multiple
d) user then goes back and once again selects options 3,4
e) the fields are not generated correctly, there are these grey fields (see screenshots)

Has anyone encountered similar issues? How did you fix it? Thank you.

XLS attached, testing version of the form here: https://ee.kob4.ifrc.org/single/::jvOwDK02

kobo_indicators_form2409.xls (242.5 KB)

Hi @daria_gordina
Thank you so much for your compliments. I am glad that you find the community forum useful in enhancing your experience on the platform.

On the above issue, could you kindly

  1. Highlight the specific questions that are affected so that we test.
  2. Point us to the exact execution of the component below. I cannot seem to see a relation.
  1. Use the actual questions to explain this problem i.e. based on the test form you sent as opposed to the hypothetical so that it is easier to understand the below caption
  1. Send screenshots depicting Problem 1 and Problem 2.

Stephane

1 Like

Hi @stephanealoo

Thanks so much for your quick response. Apologies, I realised I had originally uploaded & test deployed the wrong form. I have fixed it, it should make more sense now.

For problem 1:
For example, choose “COVID-19 testing”, “COVID-19 screening”, “Ambulance services”, and “Management of the dead” from the “Health indicators” select multiple. Input a number to “Management of the dead”. Scroll back up and de-select “Ambulance services”. You’ll see that the number you input into “Management of the dead” has disappeared.

For problem 2: see screenshot now attached in original post. This is the more pressing error.

Hi @daria_gordina
I have gone through your form on your installation instance, and also uploaded the form on our HHI servers and I noted the same issues with Problem 1 and Problem 2.

I then decided to run console as I filled the form for Problem 1. I noticed that everything was working fine until when I deselected ambulance as you had indicated.

Before I confirm whether this is a bug, I would like to potentially point out this as a potential form design issue based on the following logics.

PROBLEM 1

  1. Your repeat count is dynamically created based on the number of indicators selected (cell Y27)
  2. In the initial selection of the 4, burials, is taken to the 4th instance of repeat. When you deselect ambulance, you inadvertently cause the burial to be instance 3 i.e. instance 4 does not exist and hence the earlier filled response.
  3. Since the number being entered in burial is not burial specific but rather repeat iteration specific, I foresee that this changes based on the number of repeats.

Potential Conclusion: Behaviour is not a system issue but rather a form design logic issue, the repeat instance changes and as such the data position.

PROBLEM 2
Similar context of repeat instances being generated by the number of indicators selected would apply for this. A repeat instance would not be unique to an indicator but rather to the number of indicators selected. That is why option 5 entered values would be removed because the repeat instances are regenerated on de-selection and selection.

I also notice failures on server issue which I will be asking our developers to look at and inform us on what could be the issue

INDICATIVE PROPOSAL TO MOVE FORWARD,
I recommend a redesign of your form to make it an indicator specific group which is only appearing when indicator is selected as opposed to a begin repeat group which is dynamically generated based on number of selected indicators. Workload Wise I dont think it would be much considering that you have the different group items already presented. From a data management perspective, this allows you to lock a column to the specific indicator making the overall data management per indicator a much easier process.

I hope the above would help you get a way forward with your form.

Regards,
Stephane

1 Like

@daria_gordina
In addition to the issues I have stated above, I would like to point out that there a number of errors that are flagged when you test your form here https://getodk.org/xlsform/

SEE THE LIST

[row : 24] Question has no label: {‘type’: ‘select_multiple choose_indicators’, ‘control’: {‘appearance’: ‘w10’}, ‘bind’: {‘jr:constraintMsg’: {‘Arabic (ar)’: ‘عذرًا، لا يمكنكم اختيار “لا شيء ممّا سبق” مع المؤشّرات الأخرى.’, ‘French (fr)’: ‘Désolé, vous ne pouvez pas sélectionner “Aucune de ces activités” simultanément à d’autres indicateurs.’, ‘Spanish (es)’: ‘Lo siento, no puede elegir “Ninguna de las anteriores” con otros indicadores.’, ‘English (en)’: “Sorry, you cannot choose ‘None of the above’ with other indicators.”}, ‘required’: ‘true’, ‘jr:requiredMsg’: {‘Arabic (ar)’: ‘عذرًا، هذا السؤال ضروري. وفي حال لم تنطبق أيّ من هذه الأنشطة، يُرجى اختيار الخيار الأخير.’, ‘French (fr)’: “Désolé, vous devez répondre à cette question. Si aucune des activités ne s’applique à votre Société nationale, veuillez sélectionner la dernière option.”, ‘Spanish (es)’: ‘Lo siento, es necesario contestar a esta pregunta. Si no se aplica ninguna de esas actividades, por favor elija la última opción.’, ‘English (en)’: ‘Sorry, this question is required. If none of these activities apply, please choose the last option.’}, ‘constraint’: “if(selected(.,‘na’), count-selected(.)=1, count-selected(.)>=1)”}, ‘name’: ‘choose_indicators’, ‘hint’: {‘Arabic (ar)’: ‘ستُوفِّر الاستمارة تلقائيًا خاناتٍ لإدخال البيانات وأسئلة متعلّقة بالمؤشّرات التي اخترتموها. وفي حال عدم انطباق أيّ من المؤشّرات، يُرجى اختيار الخيار الأخير.’, ‘French (fr)’: “Le formulaire générera automatiquement des champs de saisie de données et des questions pour les indicateurs que vous avez choisis. Si aucun des indicateurs ne s’applique à votre Société nationale, veuillez sélectionner la dernière option.”, ‘Spanish (es)’: ‘El formulario generará automáticamente campos de introducción de datos y preguntas para los indicadores que haya elegido. Si ninguno de los indicadores es aplicable, por favor, elija la última opción.’, ‘English (en)’: ‘The form will automatically generate data entry fields and questions for the indicators you have chosen. If none of the indicators are applicable, please choose the last option.’}}

[row : 27] Question has no label: {‘type’: ‘begin_repeat’, ‘control’: {‘jr:count’: ‘${count_indicators}’, ‘appearance’: ‘w10’}, ‘bind’: {‘relevant’: “not(selected(${choose_indicators}, ‘na’))”}, ‘name’: ‘data_repeat’}

Repeat behavior has changed. Previously, some clients like ODK Collect prompted users to add the first repeat. Now, the user will only be prompted to add repeats after the first one. Representing 0 repetitions will require changing the form design. Read more at http://xlsform.org#representing-zero-repeats.

[row : 29] Question has no label: {‘type’: ‘begin_group’, ‘control’: {‘appearance’: ‘w10’}, ‘name’: ‘indicator_data’}

[row : 48] Question has no label: {‘type’: ‘begin_group’, ‘name’: ‘gender’}

[row : 53] Question has no label: {‘type’: ‘begin_group’, ‘control’: {‘appearance’: ‘w10’}, ‘bind’: {‘relevant’: “${agd}=‘age’”}, ‘name’: ‘age’}

[row : 77] Question has no label: {‘type’: ‘begin_group’, ‘bind’: {‘relevant’: “${agd}=‘genderage’”}, ‘name’: ‘genderage’}

[row : 155] Question has no label: {‘type’: ‘begin_group’, ‘bind’: {‘relevant’: “selected(${this_selection}, ‘washhf’) or selected(${this_selection}, ‘hf_covid’) or selected(${this_selection}, ‘mh_hf’) or selected(${this_selection}, ‘burial’)”}, ‘name’: ‘begin_group_number_data’}

[row : 163] Question has no label: {‘type’: ‘select_multiple choose_indicators_se’, ‘control’: {‘appearance’: ‘w10’}, ‘bind’: {‘jr:constraintMsg’: {‘Arabic (ar)’: ‘عذرًا، لا يمكنكم اختيار “لا شيء ممّا سبق” مع المؤشّرات الأخرى.’, ‘French (fr)’: ‘Désolé, vous ne pouvez pas sélectionner “Aucune de ces activités” simultanément à d’autres indicateurs.’, ‘Spanish (es)’: ‘Lo siento, no puede elegir “Ninguna de las anteriores” con otros indicadores.’, ‘English (en)’: “Sorry, you cannot choose ‘None of the above’ with other indicators.”}, ‘required’: ‘true’, ‘jr:requiredMsg’: {‘Arabic (ar)’: ‘عذرًا، هذا السؤال ضروري. وفي حال لم تنطبق أيّ من هذه الأنشطة، يُرجى اختيار الخيار الأخير.’, ‘French (fr)’: “Désolé, vous devez répondre à cette question. Si aucune des activités ne s’applique à votre Société nationale, veuillez sélectionner la dernière option.”, ‘Spanish (es)’: ‘Lo siento, es necesario contestar a esta pregunta. Si ninguna de estas actividades se aplica, por favor elija la última opción.’, ‘English (en)’: ‘Sorry, this question is required. If none of these activities apply, please choose the last option.’}, ‘constraint’: “if(selected(.,‘na’), count-selected(.)=1, count-selected(.)>=1)”}, ‘name’: ‘choose_indicators_se’, ‘hint’: {‘Arabic (ar)’: ‘ستُوفِّر الاستمارة تلقائيًا خاناتٍ لإدخال البيانات وأسئلة متعلّقة بالمؤشّرات التي اخترتموها. وفي حال عدم انطباق أيّ من المؤشّرات، يُرجى اختيار الخيار الأخير.’, ‘French (fr)’: “Le formulaire générera automatiquement des champs de saisie de données et des questions pour les indicateurs que vous avez choisis. Si aucun des indicateurs ne s’applique à votre Société nationale, veuillez sélectionner la dernière option.”, ‘Spanish (es)’: ‘El formulario generará automáticamente campos de introducción de datos y preguntas para los indicadores que haya elegido. Si ninguno de los indicadores es aplicable, por favor, elija la última opción.’, ‘English (en)’: ‘The form will automatically generate data entry fields and questions for the indicators you have chosen. If none of the indicators are applicable, please choose the last option.’}}

[row : 166] Question has no label: {‘type’: ‘begin_repeat’, ‘control’: {‘jr:count’: ‘${count_indicators_se}’, ‘appearance’: ‘w10’}, ‘bind’: {‘relevant’: “not(selected(${choose_indicators_se}, ‘na’))”}, ‘name’: ‘data_repeat_se’}

[row : 168] Question has no label: {‘type’: ‘begin_group’, ‘control’: {‘appearance’: ‘w10’}, ‘bind’: {‘relevant’: ‘${count_indicators_se}>0’}, ‘name’: ‘indicator_data_se’}

[row : 184] Question has no label: {‘type’: ‘begin_group’, ‘bind’: {‘relevant’: “${agd_se}=‘gender’”}, ‘name’: ‘gender_se’}

[row : 189] Question has no label: {‘type’: ‘begin_group’, ‘bind’: {‘relevant’: “${agd_se}=‘age’”}, ‘name’: ‘age_se’}

[row : 213] Question has no label: {‘type’: ‘begin_group’, ‘bind’: {‘relevant’: “${agd_se}=‘genderage’”}, ‘name’: ‘genderage_se’}

[row : 293] Question has no label: {‘type’: ‘begin_group’, ‘bind’: {‘relevant’: “selected(${this_selection_se}, ‘f_records’) or selected(${this_selection_se}, ‘f_reports’) or selected(${this_selection_se}, ‘mg_branch’)”}, ‘name’: ‘begin_group_no_se’}

[row : 301] Question has no label: {‘type’: ‘select_multiple choose_indicators_ns’, ‘control’: {‘appearance’: ‘w10’}, ‘bind’: {‘jr:constraintMsg’: {‘Arabic (ar)’: ‘عذرًا، لا يمكنكم اختيار “لا شيء ممّا سبق” مع المؤشّرات الأخرى.’, ‘French (fr)’: ‘Désolé, vous ne pouvez pas sélectionner “Aucune de ces activités” simultanément à d’autres indicateurs.’, ‘Spanish (es)’: ‘Lo siento, no puede elegir “Ninguna de las anteriores” con otros indicadores.’, ‘English (en)’: “Sorry, you cannot choose ‘None of the above’ with other indicators.”}, ‘required’: ‘true’, ‘jr:requiredMsg’: {‘Arabic (ar)’: ‘عذرًا، هذا السؤال ضروري. وفي حال لم تنطبق أيّ من هذه الأنشطة، يُرجى اختيار الخيار الأخير.’, ‘French (fr)’: “Désolé, vous devez répondre à cette question. Si aucune des activités ne s’applique à votre Société nationale, veuillez sélectionner la dernière option.”, ‘Spanish (es)’: ‘Lo siento, es necesario contestar a esta pregunta. Si ninguna de estas actividades se aplica, por favor elija la última opción.’, ‘English (en)’: ‘Sorry, this question is required. If none of these activities apply, please choose the last option.’}, ‘constraint’: “if(selected(.,‘na’), count-selected(.)=1, count-selected(.)>=1)”}, ‘name’: ‘choose_indicators_ns’, ‘hint’: {‘Arabic (ar)’: ‘ستُوفِّر الاستمارة تلقائيًا خاناتٍ لإدخال البيانات وأسئلة متعلّقة بالمؤشّرات التي اخترتموها. وفي حال عدم انطباق أيّ من المؤشّرات، يُرجى اختيار الخيار الأخير.’, ‘French (fr)’: “Le formulaire générera automatiquement des champs de saisie de données et des questions pour les indicateurs que vous avez choisis. Si aucun des indicateurs ne s’applique à votre Société nationale, veuillez sélectionner la dernière option.”, ‘Spanish (es)’: ‘El formulario generará automáticamente campos de introducción de datos y preguntas para los indicadores que haya elegido. Si ninguno de los indicadores es aplicable, por favor, elija la última opción.’, ‘English (en)’: ‘The form will automatically generate data entry fields and questions for the indicators you have chosen. If none of the indicators are applicable, please choose the last option.’}}

[row : 304] Question has no label: {‘type’: ‘begin_repeat’, ‘control’: {‘jr:count’: ‘${count_indicators_ns}’, ‘appearance’: ‘w10’}, ‘bind’: {‘relevant’: “not(selected(${choose_indicators_ns}, ‘na’))”}, ‘name’: ‘data_repeat_ns’}

[row : 306] Question has no label: {‘type’: ‘begin_group’, ‘control’: {‘appearance’: ‘w10’}, ‘bind’: {‘relevant’: ‘${count_indicators_ns}>0’}, ‘name’: ‘indicator_data_ns’}

[row : 320] Question has no label: {‘type’: ‘begin_group’, ‘bind’: {‘relevant’: “${agd_ns}=‘gender’”}, ‘name’: ‘gender_ns’}

[row : 325] Question has no label: {‘type’: ‘begin_group’, ‘bind’: {‘relevant’: “${agd_ns}=‘age’”}, ‘name’: ‘age_ns’}

[row : 349] Question has no label: {‘type’: ‘begin_group’, ‘bind’: {‘relevant’: “${agd_ns}=‘genderage’”}, ‘name’: ‘genderage_ns’}

[row : 426] Question has no label: {‘type’: ‘begin_group’, ‘bind’: {‘relevant’: “selected(${this_selection_ns}, ‘nss_is’) or selected(${this_selection_ns}, ‘nss_funding’)”}, ‘name’: ‘begin_group_no_ns’}

[row : 429] Question has no label: {‘type’: ‘begin_group’, ‘bind’: {‘relevant’: “selected(${this_selection_ns}, ‘nsr’) or selected(${this_selection_ns}, ‘nsar’) or selected(${this_selection_ns}, ‘nss_bcp’) or selected(${this_selection_ns}, ‘nss_fr’)”}, ‘name’: ‘begin_group_binary_ns’}

2 Likes

Hi @stephanealoo,
First of all thank you so much for your comprehensive review & help! This is amazing support.

Re: potential server issues
Thank you, I am very grateful.

Re: form design & proposal to move forward
Yes, I of course acknowledge that this is very much a form design issue, hence why I asked if anyone on the forum had encountered a similar problem before and how they had fixed it. Thank you very much for helping isolate the issue.

I thought that the calculate field “this_selection” (W28) was tying the repeat group to the multiple selection choice:

string (if(position(…)-1 < count-selected(${choose_indicators}),
selected-at(${choose_indicators}, position(…)-1), -1))

When pulling data from KoBo the tables for repeat groups have a column “this_selection”, which shows us what the data corresponds to (see data extract download attached).

However if I understand correctly this isn’t actually tying the repeat group to the multiple choice selection, as you said the repeat groups are generated by the count of selected options, not the selection itself. Do you have any suggestions on how I could tie a repeat group to a selection in a multiple choice? Before this version of the form I had initially created a super-long form with the same groups, but “relevant” for each possible option. This just refused to load as it was so large.

Re: other issues
Thank you for flagging this. All of the errors are “Question has no label” - does this actually matter, does it affect the performance of the form?

Thank you again for this help, I am so so so grateful. Apologies if some of the questions are elementary, this is so far the one & only form I have designed.

TEST form Indicators - latest version - False - 2020-09-25-07-13-27 (1).xlsx (16.7 KB)

Best,
Daria

Hi Daria,

Just to directly address your two questions as I think they are of interest:

  • Question has no label error severity depends on what type of question its referring to. For a calculation it doesnt really matter. For a select question its important.
  • For your question on tying the repeat group to the multiple select, is this question about analysing the exported data or about behaviour of the form during data collection. If its about analysis then the answer is by using the index and parent_index columns. Or by including more calculations inside the repeat groups

Best regards,
Noel

1 Like

Hi @NoelCartONG,

Thank you for your help.
Re: no label - clear, thanks very much.

Re: tying the repeat group - it’s about the behaviour of the form during data collection.
Specifically problems 1 & 2 I referenced in my post: data disappears when a choice is “de-selected”, and gets a bit confused when a choice “between” two other choices is selected after initial data entry.

Hi,

I remembered an old form from years ago and realised there is a way to connect the repeat group directly to the select_multiple and not just to the phase of the repeats. Please see attached modification. Its quite a small change compared to your form above but changes completely the structure of the repeat group. You can repeat the change for the other 2 sets of indicators as well. It seems to prevent all of the problems you mentioned and also gives a much more logical form of data exports afterwards. Do please let me know if you have any questions or hit further barriers implementing this change for indicators_se and indicators ns.

kobo_indicators_form_reworked.xls (246 KB)

Noel

2 Likes

Hi,
Looking at problem 2 this seems to be a bug which I will be documenting for our developers.

On problem 1, there are a number of clarifications I would like to make.

Yes your original design actually ties the repeat instance to the count of selected indicators as opposed to the specific indicators. In that essence, repeat iteration 3 will not necessarily be tied to the third (3rd) indicator. This is what @NoelCartONG has tried solving the form that they reworked

I however noticed an issue with @NoelCartONG approach which showed that the first note is repeated to all the subsequent repeat instances as can be seen on the screenshot below.

@daria_gordina, I would thus recommend you look at the proposal by @NoelCartONG and see what amendments you can make on your end.

Stephane

1 Like

Hi @stephanealoo and @NoelCartONG

Thank you both so much for all of your help & support, this is so beyond what I was expecting! The KoBo community is truly awesome.

I’ll work on amending @NoelCartONG 's proposal, and update on the forum on any solution we find.

Have a nice weekend,
Best,
Daria

1 Like

Many thanks for pointing this out stephanealoo, you are correct that the notes are not displaying correctly. Your analysis of Enketo functionality above is also very interesting and I hope can lead to a closer alignment between the webforms and mobile forms in future.

The error you mention in my attached file seems to be because I used the position(…) function inside a subgroup where it will always calculate as 1. Please see attached updated version where the position is captured by a calculation outside the group before being referred to indirectly in any relevancies. It seems to tidy up this detail.

kobo_indicators_form_reworked (2).xls (246.5 KB)

The change from the last version is highlighted in yellow. I hope that it will meet all the requirements for this form but if there is anything missing do feel free to let me know.

Best regards,

Noel

3 Likes

Thank you so much @NoelCartONG !!

1 Like