Editing data (with Enketo = default) not possible when using external csv?

Thanks a lot. Just wondering whether it helps you to identify the problem, if a share my first attempt to test the from_external_file and its editing in Enketo (the one where I can make changes, but not save, because the system keeps on asking for credentials):
id_names.csv (9.0 KB) test.xlsx (42.0 KB)

Dear Kal_Lam

Were you able to find out, why the editing function is not working (for our files)?

Kind regards
York

@york_rff, have you tried this in a new dummy survey project? If you try to use the search under the appearance column, you may have issues while editing the submissions but if you use other approaches as outlined above, you should not have issues while editing submission.

It is not so easy to extract the essence of this thread and make sure that there are no contradicting statements or circular considerations. Is the following correct?

There are quite some options to access (cascading) choice lists:

  • select_one
  • select_one_external
  • select_one_from_file xyz.csv
  • dynamic selects from pre-loaded data or even other approaches which have not been discussed here

Both select_one and select_one_external need to be followed by the term used on the choices or external_choices tab in the column “list_name”. The term in the column “choice_filter” (compared to the precondition variable) defines the column on the choices tab which determines further which of the choices will be listed. So it easy to produce cascading selects, but all data need to be in the xlsform (not possible to put long choice lists into external files).

Use of an external file is possible, if we we put one single line with generic terms in the choices tab into the columns “list_name/name/label” and add a search term in the column “appearance” to access an external file – e.g. minimal search(‘extfilename’,‘contains’, 'outputcolum, *, ‘conditioncolumn’, ${alreadyselectedcondition} – then the choice_filter term is not necessary (or must not be there?).

However, this approach doesn’t allow editing of the uploaded forms under Enketo (the default programme when clicking the edit icon under Data/Tables) – with varying issues ranging from not showing the choices to not allowing to save.

An alternative to this “search” approach to pick data from an external file is using select_one_from_file xyz.csv, which must contain at least the columns “list_name/name/label”, too, plus the columns to be pointed at by the term in the column “choice_filter”. The field name in the xlsform must be repeated on the choices tab in one line, followed by “name” and “label” in the columns name and label, as well as in the “list_name” column in the csv file.

Is this all correct?

If yes, the problem remains: If I press the pencil icon in Data/Table for an uploaded file, the error message is: Failed to load data from /media/get/https/kc.kobotoo…

In can’t find anything bad in the csv file – of what nature could the issue there be?

Hi @york_rff
I would like to join this conversation and picking up from the discussion trail, I would like to request that you just make some minor tweaks on your CSV naming so that we see if its resolves the issue:

SUPPORT ISSUE 1

  • Proposed change: Could you change the name of your CSV file from image to facilities_na_test.csv
  • Rationale: This change tests the hypothesis that inclusion of - instead of _ may be causing an issue in the system during editing and not during data collection.

SUPPORT ISSUE 2

I would also like to revive one issue that you had probably set aside

I would really like to check your form and see what could have been the issue with how you wrote this. My experience with this search() parameter has shown that this approach actually works quite easily even with large data CSV data sets.

"Closing Remark"
Kindly start with Support Issue 1 and then update as you look at issue 2,

Stephane

1 Like

Dear Stephane

Thanks so much for joining the discussion.

I will work on suggestion #1 soonest possible, but can react to suggestion #2 immediately: I didn’t change the linked example, just opened a form in KoboCollect and received an error message when arriving at the select question. Indeed, with my original approach, search in appearance worked perfectly for collection, but then I discovered the issue of not being able to use the single edit function in Data/Tables – which is the still unsolved core issue.

Not sure whether related: I followed the instruction for select_one_from_file to use the same term for “name” in the survey sheet, for “list_name” in the choices field (which the example above doesn’t) and for “list_name” in the csv file – however, the latter has no influence on the selection (fully determined by the content in “choice-filter”. So, I wonder: Is the instruction wrong (like the example above seems to have a bug)? Can you point at a perfect description of how to use select from file properly (and thus allowing editing later)?

Kind regards
York

Gesendet von meinem BlackBerry 10-Smartphone.



Von: Stephane Aloo via KoBoToolbox Community Forum
Gesendet: Mittwoch, 5. Mai 2021 21:11
An: York Lunau
Antwort an: KoBoToolbox Community Forum
Betreff: [KoBoToolbox Community Forum] [User Support/Data Management] Editing data (with Enketo = default) not possible when using external csv?

stephanealoo Core Team
May 5

Hi @york_rff
I would like to join this conversation and picking up from the discussion trail, I would like to request that you just make some minor tweaks on your CSV naming so that we see if its resolves the issue:

SUPPORT ISSUE 1

  • Proposed change: Could you change the name of your CSV file from imageto facilities_na_test.csv
  • Rationale: This change tests the hypothesis that inclusion of - instead of _ may be causing an issue in the system during editing and not during data collection.

SUPPORT ISSUE 2

I would also like to revive one issue that you had probably set aside

york_rff:

I started out with the example questionnaire from Creating a form with cascading select, from a list in an external csv file - User Support / Form Building - KoBoToolbox Community Forum

First of all, this questionnaire created an error message on KoboCollect right for the first question: “File: …forms/EXAMPLE Selects with Choice Filter Search with from file_2-media/cbes.csv is missing.”

I would really like to check your form and see what could have been the issue with how you wrote this. My experience with this search() parameter has shown that this approach actually works quite easily even with large data CSV data sets.

"Closing Remark"
Kindly start with Support Issue 1 and then update as you look at issue 2,

Stephane

Hi Stephane

My other reply was somehow attached to MY last contribution – please see above.

On suggestion #1: I created a simplified xlsform (only left the cascade select questions in) and renamed the csv without using ANY special characters. Same result: using the form (collection) is no problem, but editing is not possible: " Error occured during the loading of this form. It is recommended not to edit this record until this is resolved.

Please contact support@kobotoolbox.org with the link to this page and the error message below:

  • Failed to load data from /media/get/https/kc.kobotoolbox.org/york_rff/xformsMedia/673180/1399606.csv"

Here are the files: facilitiestest.csv (140.3 KB) test_no_specialchar.xlsx (20.8 KB)

So, by switching to select FROM FILE I don’t gain any editing power and additionally lose the Data/Reports view… :frowning:

@york_rff, here is a good example of how you could use a cascading select question with select_one_from_file (this should solve your issue):

Taking a very quick view at the provided xls and csv files, I already saw things which might not be in line with the official description. Anyway, I just uploaded the two files to kobotoolbox, but when trying to get blank form on KoboCollect, an error was reported. So this can’t be an ideal example.

Interesting – the example I cited in the beginning of the thread ( id_names.csv (9.0 KB) test.xlsx (42.0 KB)) also didn’t work - it even had “search search” in one of the cells. So there really seems to be a lack of proper documentation on how to use the select_one_from_file (with all features working, incl. editing on Data/Table)?

@york_rff, could you let me know the server your are using so that we could have a look there? Or see if there is any issue with the server?

Projects | KoboToolbox

(Is is this what you were asking for?)

@york_rff, I confirm that the post I shared above (cascading select with select_one_from_file) works both with KoBoCollect android app as well as Enketo. I tested this with the HHI server as you say it does not work at your end.

Hence, would request you to give it another try. It should work.

OK, I took a closer look and managed to find the mistake: the lines on district and village needed to be deleted (if only the States_Code.csv is put to the server).

However, it brought me back to one of the error types already mentioned: I am able to collect and upload data with KoboCollect and I am able to open the saved forms in Enketo (Data/Table/pen-icon), but if I submit the changes, the submission fails:
“Authentication required. Please authenticate here in a different browser tab and try again.”
If I follow the link, I can login (“Username and password are stored. You can close this page now.”), but if I go back to the submission error page, I can get rid of the error message by clicking OK, but on “Submission”, the same error message returns.

Does it play a role that the error message says “ee” at the beginning, and not “kf” (kf.kobotoolbox.org)?

@york_rff, did you face this by yourself? It’s a different issue especially with the login. cascading select with select_one_from_file should not affect this issue.

My (yes: personal) experience is that the various error messages only appear, if I try various ways to make use of the selec_one_from_file. Was that you question? Otherwise, please rephrase.

Just another observation which makes me believe that the select_from_file feature is not stable: The last example (the one you shared: State_Code…) didn’t do what the documentation says: Using the same term in survey, choices and external file. But I went one step further: I deleted the list_name column of the csv – and it still works (except the non-saving issue). So select_from_file really seems to look only at choice_filter – if empty, it shows all entries in the “name” column of the csv and otherwise only the lines with the conditional value in the defined column of the csv. So all these descriptions are not really correct and maybe the scripts on the server aren’t either?

Could you share with us a screenshot (or maybe a short video recording) of the issue that you are facing? Maybe that would be helpful for us to understand your issue further better?

Sure: https://we.tl/t-ce2lbrTG6q

1 Like

@york_rff, thanks! The video was very helpful to understand your issue. Will have a look at my end. In the mean time would you mind trying this in a different browser to see that solves your issue?

@york_rff, I just made a try, and I confirm that I could edit it on my end. I tried this with the OCHA server using the Chrome browser.

Additional information: I tried it with this workaround discussed and shared here: