Editing data (with Enketo = default) not possible when using external csv?

Kal_Lam · April 27, 2021, 10:57am

Yes, this is known and we would advise to use you the select_one_from_file question type if you wish to make edits from the server.

york_rff · April 27, 2021, 11:48am

Thanks for the hint. Following it, I found: “At the moment, this feature is only available using XLSForms and the data can only be entered through Enketo and is not supported by ODK or KoBoCollect.”

Do I understand correctly, that if we are using KoboCollect to capture data offline, we cannot use the the select_one_from_file question type, and if we want to edit cata, we can use only Enketo? So, either one can collect data offline or one can edit them?

Kal_Lam · April 27, 2021, 2:14pm

@york_rff, kindly please be informed that select_one_from_file question type was initially not supported by Collect android app but it’s now supported by both Enketo as well as Collect android app. We will be updating our support article soon but you could use the same through both as stated above.

york_rff · April 27, 2021, 2:26pm

…oh, this sounds like good news, thanks a lot!

york_rff · April 27, 2021, 8:33pm

It seems that select_one_from_file is not the solution, as the mentioned need for cascading selections (choice list depending on previous choice, as typically used for geographical drill-downs) is not part of its concept, correct?

york_rff · April 28, 2021, 3:44pm

I continued to try various things and think that I might have found a (semi-comfortable) solution – almost.

I started out with the example questionnaire from Creating a form with cascading select, from a list in an external csv file - User Support / Form Building - KoBoToolbox Community Forum

First of all, this questionnaire created an error message on KoboCollect right for the first question: “File: …forms/EXAMPLE Selects with Choice Filter Search with from file_2-media/cbes.csv is missing.”

After deleting all entries (“search…”) form the appearance column, the example worked. Then I could enter an additional question, simulating a cascade select (as I need it).

Conclusion: select_one_from_file allows cascade select, too, just not as comfortably.

Before, my external csv contained several blocks of a number of columns, the first in each block always representing the name and the second the label, but not being called “name” and “label”, but for instance “district_name” and “district_label”. This allows to add data (e.g. schools) over time easily.

Now, all data with name and label function need to go in the same columns, called “name” and label" – and all additional columns for filtering and pulling data need to be more or less far from there to the right. Slightly more easy to make faulty edits, but it seemed to be working!

Seemed… While it was possible to see and change the choices in Enketo now, the system didn’t let me save it: “Submission Failure. Authentication required. Please authenticate here in a different browser tab and try again.” After following the procedure, it still complained. Interestingly, DATA/Reports (instead of DATA/Tables) showed error 500. Temporary problem or have I still not found a solution to my problem to use cascade select with an external file AND be able to edit data (with Enketo)?

Kal_Lam · April 28, 2021, 4:06pm

@york_rff, yes, kindly please be informed that the select_one_from_file has the following limitations (which will be addressed in in the future soon):

The Data>Table view is not viewable
Data download is not possible. But if you use the Legacy (format) you should be able to download your data.

york_rff · April 28, 2021, 6:21pm

…maybe I didn’t express myself clearly: not viewable (error 500) is the Data>Reports view. Data>Tables view is perfectly viewable, Edit opens the form and allows editing (showing all choices), but DOESN’T allow submission due to alleged login issues.

What is the problem here?

york_rff · April 29, 2021, 7:59am

This morning, the text of the error message changed (when trying to edit an uploaded form) – are there works going on = is this a temporary issue?

Would be great to understand, because I need to take an approach decision (kobo yes or no). Thanks for your support.

Error occured during the loading of this form. It is recommended not to edit this record until this is resolved.

Please contact support@kobotoolbox.org with the link to this page and the error message below:

Failed to load data from /media/get/https/kc.kobotoolbox.org/york_rff/xformsMedia/668118/1393627.csv

Kal_Lam · April 29, 2021, 8:03am

@york_rff, looking at your current error message i would say it should be an issue with your CSV file. Would you mind having a closer look at your CSV file?

york_rff · April 29, 2021, 8:40am

You mean, on Enketo there could be an issue with a csv file which works perfectly when using it for completing forms? What nature could this issue have? Where is the name of the csv file changed into a number?

Kal_Lam · April 29, 2021, 9:41am

What I mean here is that the CSV file could have issues. Would you mind sharing the same with the community? Maybe the community would be able to help you out.

york_rff · April 29, 2021, 2:18pm

OK, in order to provide you the file, I deleted most of the lines to not make it too confusing, but allow to quickly recognize the structure:
facilities_na-test.csv (2.9 KB) facilities_na-test.xlsx (12.6 KB) test_NA.xlsx (24.0 KB)

However, after creating a new project with these files and filling in a form (where the lists from the external file worked perfectly), I noticed that the error message has changed again: It is still coming now BEFORE the form opens in ENKETO (the first time it was when trying to save), but now the csv is not shown with a cryptic number, but with its real name – just with a space in front of it. It really gives the impression that some is working on your system right now…!?

Enketo Express for KoBo Toolbox

Loading Error. Error occured during the loading of this form. It is recommended not to edit this record until this is resolved.

Please contact support@kobotoolbox.org with the link to this page and the error message below:

Failed to load data from jr://file-csv/ facilities_na-test.csv

Kal_Lam · April 29, 2021, 2:35pm

@york_rff, will go through it and get back to you when resolved!

york_rff · April 29, 2021, 2:55pm

Thanks a lot. Just wondering whether it helps you to identify the problem, if a share my first attempt to test the from_external_file and its editing in Enketo (the one where I can make changes, but not save, because the system keeps on asking for credentials):
id_names.csv (9.0 KB) test.xlsx (42.0 KB)

york_rff · May 4, 2021, 6:25am

Dear Kal_Lam

Were you able to find out, why the editing function is not working (for our files)?

Kind regards
York

Kal_Lam · May 4, 2021, 7:46am

@york_rff, have you tried this in a new dummy survey project? If you try to use the search under the appearance column, you may have issues while editing the submissions but if you use other approaches as outlined above, you should not have issues while editing submission.

york_rff · May 5, 2021, 2:24pm

It is not so easy to extract the essence of this thread and make sure that there are no contradicting statements or circular considerations. Is the following correct?

There are quite some options to access (cascading) choice lists:

select_one
select_one_external
select_one_from_file xyz.csv
dynamic selects from pre-loaded data or even other approaches which have not been discussed here

Both select_one and select_one_external need to be followed by the term used on the choices or external_choices tab in the column “list_name”. The term in the column “choice_filter” (compared to the precondition variable) defines the column on the choices tab which determines further which of the choices will be listed. So it easy to produce cascading selects, but all data need to be in the xlsform (not possible to put long choice lists into external files).

Use of an external file is possible, if we we put one single line with generic terms in the choices tab into the columns “list_name/name/label” and add a search term in the column “appearance” to access an external file – e.g. minimal search(‘extfilename’,‘contains’, 'outputcolum, *, ‘conditioncolumn’, ${alreadyselectedcondition} – then the choice_filter term is not necessary (or must not be there?).

However, this approach doesn’t allow editing of the uploaded forms under Enketo (the default programme when clicking the edit icon under Data/Tables) – with varying issues ranging from not showing the choices to not allowing to save.

An alternative to this “search” approach to pick data from an external file is using select_one_from_file xyz.csv, which must contain at least the columns “list_name/name/label”, too, plus the columns to be pointed at by the term in the column “choice_filter”. The field name in the xlsform must be repeated on the choices tab in one line, followed by “name” and “label” in the columns name and label, as well as in the “list_name” column in the csv file.

Is this all correct?

If yes, the problem remains: If I press the pencil icon in Data/Table for an uploaded file, the error message is: Failed to load data from /media/get/https/kc.kobotoo…

In can’t find anything bad in the csv file – of what nature could the issue there be?

stephanealoo · May 5, 2021, 7:01pm

Hi @york_rff
I would like to join this conversation and picking up from the discussion trail, I would like to request that you just make some minor tweaks on your CSV naming so that we see if its resolves the issue:

SUPPORT ISSUE 1

Proposed change: Could you change the name of your CSV file from to facilities_na_test.csv
Rationale: This change tests the hypothesis that inclusion of - instead of _ may be causing an issue in the system during editing and not during data collection.

SUPPORT ISSUE 2

I would also like to revive one issue that you had probably set aside

I would really like to check your form and see what could have been the issue with how you wrote this. My experience with this search() parameter has shown that this approach actually works quite easily even with large data CSV data sets.

"Closing Remark"
Kindly start with Support Issue 1 and then update as you look at issue 2,

Stephane

york_rff · May 6, 2021, 4:38am

Dear Stephane

Thanks so much for joining the discussion.

I will work on suggestion #1 soonest possible, but can react to suggestion #2 immediately: I didn’t change the linked example, just opened a form in KoboCollect and received an error message when arriving at the select question. Indeed, with my original approach, search in appearance worked perfectly for collection, but then I discovered the issue of not being able to use the single edit function in Data/Tables – which is the still unsolved core issue.

Not sure whether related: I followed the instruction for select_one_from_file to use the same term for “name” in the survey sheet, for “list_name” in the choices field (which the example above doesn’t) and for “list_name” in the csv file – however, the latter has no influence on the selection (fully determined by the content in “choice-filter”. So, I wonder: Is the instruction wrong (like the example above seems to have a bug)? Can you point at a perfect description of how to use select from file properly (and thus allowing editing later)?

Kind regards
York

Gesendet von meinem BlackBerry 10-Smartphone.

Von: Stephane Aloo via KoBoToolbox Community Forum
Gesendet: Mittwoch, 5. Mai 2021 21:11
An: York Lunau
Antwort an: KoBoToolbox Community Forum
Betreff: [KoBoToolbox Community Forum] [User Support/Data Management] Editing data (with Enketo = default) not possible when using external csv?

stephanealoo Core Team
May 5