Select() function and all other related data import only accepting 3 columns in csv

I seem to be having a problem with pulling or searching for data in csv files on Kobo.

Instructions on XLSForm and other forums seem to indicate that you can have csv files with several columns, and this is why you have a way to indicate the column settings when you use the search() and pulldata() functions. However, Kobo only seems to accept 3 column csv files from me.

I spent a long time changing and reuploading a form I was building, tried every possible combination of settings and choices, and nothing worked. This includes uploading a few csv and xlsx files I found on this and other forums to test the pulldata() and search() functions, and they seemed to be broken, which was strange.

Finally, I simply removed all the columns from my csv except the ones necessary for the one function I was testing, and it worked. So to build the rest of the form, every time I needed a pulldata() search() or calculate function, I had to subset my original csv to the three columns I needed and uploaded an addtional csv to the media section. This seems very inefficient for large forms for, say, a baseline study you want to run again.

¿Am I doing something wrong or is this a limit in Kobo?

Thanks in advance.

Hi @bushidoenator,

Welcome to the community! Would you mind sharing with the community the xlsform and the csv file (with a long list of columns that is not supporting).

Maybe we could figure out some issues that has been affecting your file.

Have a great day!

Unfortunately, they comtain sensitive information of rural community members. ¿Do you have any clue of what sort of issues I should be sniffing for?

I got the form working, but with the three column workaround. Not been able to test all columns to see if any one in particular is messing with me.

Hi,
Can you create dummy data on the CSV and then send maybe with 20 rows so that I look at it?

Stephane

1 Like

Hello Stephane,

I have created the samples, but I do not seem to be able to upload files as I am a new user. However, I am attaching a google drive link.

https://drive.google.com/drive/folders/1SS-ntp8_hlgfOIcWzJND1A4AhLdFCqQp?usp=sharing

The CSV is a dummy that follows the exact format, original csv has almost 700 entries.

The survey also is the same as the real one, except I changed some choices to avoid revealing surveyor names. You will see that in the form I use “participantes.csv” and “master.csv”, which are just subsets of of the csv I uploaded to the sample folder as mentioned earlier.

I have gotten the web form to work, but KoBoCollect refuses to download the CSVs to the phones, and this phenomenon is true for my whole team.

Hi @bushidoenator,

Would you mind sharing it in the forum again. You should be able to share it now.

Have a great day!

Here you go!

sampleform.xlsx (14.3 KB) sample.csv (3.6 KB)

Hi @bushidoenator
Thanks for sharing the forms. Could you now be more specific about the areas that have problems by sharing screenshots of what you see and an explanation of what you had expected? This will help us better support you.

Stephane

1 Like

I would have to revert to an earlier version of the form, unfortunately, I save over previous versions and so I do not have the earlier versions on hand (unless you guys can go into the project and look at the previous versions, I did like 30 uploads while trying to figure out how to get it to work!). However, the issue was that when I set up the form to search() from the CSV, it would simply refuse to read the CSV and would not add the options to the Kobo form, pulldata() and select_one_from_file failed too. I tried every combination and method I could find, and even uploaded these sample forms from the ODK forums and they did not work either:

hhcensus_data.csv (2.2 KB) Sample_Baseline_2017.xlsx (42.7 KB)

Result is always the same: Form would load, but data from CSV would not work and just leave blanks in the fields it was supposed to fill.

After trying everything I could think of, I started to question the CSV itself, and it was not until I reduced the CSV to just the three columns I needed that it worked, and so my current form is using 2 separate csv files that are 3 column subsets of the original one that I uploaded earlier, and then I have each question refer to its own CSV. (which you can see in lines 5 and 7 of the form uploaded in previous comment “sampleform.xlsx”).

I was expecting to be able to upload one CSV with the whole baseline in it, and simply select the coumns I needed in the parameters for search() and pulldata(). Instead had to create separate documents.

Regardless, it would seem that I am maybe doing something wrong with the CSV files, because even though I managed to get the Enketo online form to work by subsetting, the CSV files still refuse to download into the KoBoCollect app. This is not exclusive to me, my whole team was unable to download the form into their KoBoCollect apps. We are all on latest version from Playstore (installed on Friday).

I am sorry I cannot give you screenshots of the webform failing, I will try to reproduce the problem this afternoon.

Thank you for all your support.

Now that my team has used the form to collect data in the Enketo version of the form, I tried to export it for checking, and the form will not export. The export fails.

edit1: added screenshot

edit2: automated report in Kobo is failing as well

image

edit3: when I went into the editor to make a change to the form, I got this error:

Hey I am bumping this thread because the issue has become rather urgent. I need to deliver these results, but I cannot download them. At this point the csv issue has become secondary to the fact I cannot access my data. ¿Any suggestions?

Hi @bushidoenator
Can you send your account details (name only) and project name on private message for us to look at it?

Of course, thank you! Coming right up!

edit: Sent!

Hi @bushidoenator,

Have you sent the account details as advised by @stephanealoo? If not could you please resend them again.

Have a great day!

Hey @Kal_Lam ! Yeah, I sent @stephanealoo my details. He suggested I read a couple of other threads in the forum.

These pertain specifically to the search issue in Kobo. I mentioned that my main issue right now is not the search issue. While it continues to puzzle me and I still cannot get the form to work on the KoboCollect app, the webform is up and running just fine, with the before mentioned workaround. My team has used it and collected several hundered responses.

The problem is that the data is there, and I can even edit responses in the data table, but the export to download keeps failing and I do not have a way to access this data we collected, and it is getting increasingly urgent that I access this data. The donors that maintain the project are starting to ask for it. I’ll go ahead and send you my account details too, in case there’s anything you can do to lend me a hand.

Thank you again for the follow-up.

Hi @bushidoenator
I have responded to your private message, could you please respond to it?

Stephane

1 Like

Hi @bushidoenator,

Regarding your first query:

Which version of the KoBoCollect android app are you using? Try using the latest version i.e. KoBoCollect v1.26.1 and you should be able to collect the same through your android app.

Regarding your second query:

This is a known bug already reported. You could follow it here:

However, there is a workaround for this issue. Have you tried downloading your dataset in the XLS (legacy) version (as shown in the image below)?


You should be able to download them smoothly.

Have a great day!

1 Like