Data Showing on the kobo portal but not in XLS downloaded file

User Support Data Management
,
1

In one of our ongoing projects, we are facing a unique issue. We are able to see the completed fill form on the kobo account under the data tab. However, on downloading the Xls file some data set is coming blank.

We started experiencing the issue after the data collection crossed the 22,000 sample mark, all our new data are coming blank.

PS: We are already using the extra space and extra submission services.

Kobo XLS file
Kobo XLS file1920Ă—1080 263 KB

Kobo web
Kobo web1917Ă—967 177 KB

2

Hello @nalinsharma_2004,
Could you provide some more details, please:

  • Did you change the form during data collection?
  • Do you use multiple languages?
  • Could you provide a screenshot with mapping cases (your screenshots seem to show different cases)?
  • Does your screenshots show labels or names (values)?
  • Which options do you use for the XLSX (Excel) download?
  • Can you see the data for these cases in KoBo modal view/edit?
2 Likes
3

@nalinsharma_2004, feel free with the queries reached out by @wroos! We shall then start investigating your issue. BTW, have you also tried downloading your data in XLS (legacy) format to see if they are there?

4

Hi Wroos,

The issue solved in a very strange way: I went to >Download >advance options > removed check from “Include media URLs”.

However below is my response to your queries:

  • Did you change the form during data collection? : No changes done in the form. Also, I want to highlight all questions are mandatory, thus XLS form should not be blank

  • Do you use multiple languages? No only one language

  • Could you provide a screenshot with mapping cases (your screenshots seem to show different cases)?: There is no way of tracking as UUID is also blank and each day close to 750 data being collected

  • Does your screenshots show labels or names (values)? labels

  • Which options do you use for the XLSX (Excel) download? XLS ( English - language)

  • Can you see the data for these cases in KoBo modal view/edit? In the kobo website under view/edit all data is complete and no missing info. However, in XLS file after 22,880 records 90% responses missing even for the mandatory questions.

  • The issue is solved but I am attaching the XLS file (partial data) for your investigation

1 Like
5

@nalinsharma_2004, :clap: :heart: :partying_face:

6

Hi @nalinsharma_2004, I’m glad you managed to find a workaround! Would you mind please sending me the following in a private message so that I can investigate the source of the issue:

  • server (humanitarian/non-humanitarian)
  • username

I removed the data export from your previous message as it may contain sensitive data.

2 Likes
7

@Josh
I am facing the same issue on one of my forms, on a self-hosted instance. The issue has been tested and reproducible on versions 2.021.45b and 2.022.24.

Unchecking Include media URLs resolves the issue. However, we would very much like to have the URLs in the report file.

Additional info:

  • There are over 33000 submissions in my problematic form. The issue starts at line 28181. As per @nalinsharma_2004 's screenshot here, his starts at line 25390.
  • On trying with another form that has over 48000 submissions, the problem does not occur.
  • This leads me to believe that it’s an edge case bug with forms having certain common but unknown characteristics, that are larger than a certain size.

It might be helpful to create an issue on Github.

8

Hi @ks_1, thank you for the additional details and I will create an issue on GitHub to track this :+1:

1 Like
9

@ks_1, @nalinsharma_2004 you can track this issue here:

1 Like