Editing data (with Enketo = default) not possible when using external csv?

Hello

For various fields , we store the possible choices (select_one) in an external csv file under settings/media and this works perfectly with KoboCollect – e.g. appearance = minimal search(‘facilities_na’,‘contains’, ‘org_label’, *, ‘org_empty’, ${empty}).

The problem comes in, if I want to edit/clean the data by accessing a form on the server (data/table/edit). The respective fields don’t show the selected value (although this value is showing in the table, before opening the respective form), but are empty and only show the field name (like: org_label). Because this field is mandatory, I cannot save the form again.

So, does the use of external data sources prevent any kind of editing?

Yes, this is known and we would advise to use you the select_one_from_file question type if you wish to make edits from the server.

Thanks for the hint. Following it, I found: “At the moment, this feature is only available using XLSForms and the data can only be entered through Enketo and is not supported by ODK or KoBoCollect.”

Do I understand correctly, that if we are using KoboCollect to capture data offline, we cannot use the the select_one_from_file question type, and if we want to edit cata, we can use only Enketo? So, either one can collect data offline or one can edit them?

@york_rff, kindly please be informed that select_one_from_file question type was initially not supported by Collect android app but it’s now supported by both Enketo as well as Collect android app. We will be updating our support article soon but you could use the same through both as stated above.

1 Like

…oh, this sounds like good news, thanks a lot!

1 Like

It seems that select_one_from_file is not the solution, as the mentioned need for cascading selections (choice list depending on previous choice, as typically used for geographical drill-downs) is not part of its concept, correct?

I continued to try various things and think that I might have found a (semi-comfortable) solution – almost.

I started out with the example questionnaire from Creating a form with cascading select, from a list in an external csv file - User Support / Form Building - KoBoToolbox Community Forum

First of all, this questionnaire created an error message on KoboCollect right for the first question: “File: …forms/EXAMPLE Selects with Choice Filter Search with from file_2-media/cbes.csv is missing.”

After deleting all entries (“search…”) form the appearance column, the example worked. Then I could enter an additional question, simulating a cascade select (as I need it).

Conclusion: select_one_from_file allows cascade select, too, just not as comfortably.

Before, my external csv contained several blocks of a number of columns, the first in each block always representing the name and the second the label, but not being called “name” and “label”, but for instance “district_name” and “district_label”. This allows to add data (e.g. schools) over time easily.

Now, all data with name and label function need to go in the same columns, called “name” and label" – and all additional columns for filtering and pulling data need to be more or less far from there to the right. Slightly more easy to make faulty edits, but it seemed to be working!

Seemed… While it was possible to see and change the choices in Enketo now, the system didn’t let me save it: “Submission Failure. Authentication required. Please authenticate here in a different browser tab and try again.” After following the procedure, it still complained. Interestingly, DATA/Reports (instead of DATA/Tables) showed error 500. Temporary problem or have I still not found a solution to my problem to use cascade select with an external file AND be able to edit data (with Enketo)?

@york_rff, yes, kindly please be informed that the select_one_from_file has the following limitations (which will be addressed in in the future soon):

  • The Data>Table view is not viewable
  • Data download is not possible. But if you use the Legacy (format) you should be able to download your data.

…maybe I didn’t express myself clearly: not viewable (error 500) is the Data>Reports view. Data>Tables view is perfectly viewable, Edit opens the form and allows editing (showing all choices), but DOESN’T allow submission due to alleged login issues.

What is the problem here?

This morning, the text of the error message changed (when trying to edit an uploaded form) – are there works going on = is this a temporary issue?

Would be great to understand, because I need to take an approach decision (kobo yes or no). Thanks for your support.

Error occured during the loading of this form. It is recommended not to edit this record until this is resolved.

Please contact support@kobotoolbox.org with the link to this page and the error message below:

  • Failed to load data from /media/get/https/kc.kobotoolbox.org/york_rff/xformsMedia/668118/1393627.csv

@york_rff, looking at your current error message i would say it should be an issue with your CSV file. Would you mind having a closer look at your CSV file?

You mean, on Enketo there could be an issue with a csv file which works perfectly when using it for completing forms? What nature could this issue have? Where is the name of the csv file changed into a number?

What I mean here is that the CSV file could have issues. Would you mind sharing the same with the community? Maybe the community would be able to help you out.

OK, in order to provide you the file, I deleted most of the lines to not make it too confusing, but allow to quickly recognize the structure:
facilities_na-test.csv (2.9 KB) facilities_na-test.xlsx (12.6 KB) test_NA.xlsx (24.0 KB)

However, after creating a new project with these files and filling in a form (where the lists from the external file worked perfectly), I noticed that the error message has changed again: It is still coming now BEFORE the form opens in ENKETO (the first time it was when trying to save), but now the csv is not shown with a cryptic number, but with its real name – just with a space in front of it. It really gives the impression that some is working on your system right now…!?

Enketo Express for KoBo Toolbox

Loading Error. Error occured during the loading of this form. It is recommended not to edit this record until this is resolved.

Please contact support@kobotoolbox.org with the link to this page and the error message below:

  • Failed to load data from jr://file-csv/ facilities_na-test.csv
1 Like

@york_rff, will go through it and get back to you when resolved!

Thanks a lot. Just wondering whether it helps you to identify the problem, if a share my first attempt to test the from_external_file and its editing in Enketo (the one where I can make changes, but not save, because the system keeps on asking for credentials):
id_names.csv (9.0 KB) test.xlsx (42.0 KB)

Dear Kal_Lam

Were you able to find out, why the editing function is not working (for our files)?

Kind regards
York

@york_rff, have you tried this in a new dummy survey project? If you try to use the search under the appearance column, you may have issues while editing the submissions but if you use other approaches as outlined above, you should not have issues while editing submission.

It is not so easy to extract the essence of this thread and make sure that there are no contradicting statements or circular considerations. Is the following correct?

There are quite some options to access (cascading) choice lists:

  • select_one
  • select_one_external
  • select_one_from_file xyz.csv
  • dynamic selects from pre-loaded data or even other approaches which have not been discussed here

Both select_one and select_one_external need to be followed by the term used on the choices or external_choices tab in the column “list_name”. The term in the column “choice_filter” (compared to the precondition variable) defines the column on the choices tab which determines further which of the choices will be listed. So it easy to produce cascading selects, but all data need to be in the xlsform (not possible to put long choice lists into external files).

Use of an external file is possible, if we we put one single line with generic terms in the choices tab into the columns “list_name/name/label” and add a search term in the column “appearance” to access an external file – e.g. minimal search(‘extfilename’,‘contains’, 'outputcolum, *, ‘conditioncolumn’, ${alreadyselectedcondition} – then the choice_filter term is not necessary (or must not be there?).

However, this approach doesn’t allow editing of the uploaded forms under Enketo (the default programme when clicking the edit icon under Data/Tables) – with varying issues ranging from not showing the choices to not allowing to save.

An alternative to this “search” approach to pick data from an external file is using select_one_from_file xyz.csv, which must contain at least the columns “list_name/name/label”, too, plus the columns to be pointed at by the term in the column “choice_filter”. The field name in the xlsform must be repeated on the choices tab in one line, followed by “name” and “label” in the columns name and label, as well as in the “list_name” column in the csv file.

Is this all correct?

If yes, the problem remains: If I press the pencil icon in Data/Table for an uploaded file, the error message is: Failed to load data from /media/get/https/kc.kobotoo…

In can’t find anything bad in the csv file – of what nature could the issue there be?

Hi @york_rff
I would like to join this conversation and picking up from the discussion trail, I would like to request that you just make some minor tweaks on your CSV naming so that we see if its resolves the issue:

SUPPORT ISSUE 1

  • Proposed change: Could you change the name of your CSV file from image to facilities_na_test.csv
  • Rationale: This change tests the hypothesis that inclusion of - instead of _ may be causing an issue in the system during editing and not during data collection.

SUPPORT ISSUE 2

I would also like to revive one issue that you had probably set aside

I would really like to check your form and see what could have been the issue with how you wrote this. My experience with this search() parameter has shown that this approach actually works quite easily even with large data CSV data sets.

Closing Remark
Kindly start with Support Issue 1 and then update as you look at issue 2,

Stephane

1 Like