Are there works ongoing on the server which leads to loss of grouped data???
A few days ago, I had looked at a certain form (142050828) under “Edit” in DATA/Table and was able to see certain answers (group: Outcome 2 indicators) and for the same form they are gone now – and also don’t appear in the Power BI query!
Here are some observations which hopefully help to solve the very urgent issue:
The (complex) xlsform always had sub-groups within main groups and this worked well. A while ago, I have further structured the form (for better navigation within KoboCollect) by adding a THIRD layer in one of the sub-groups.
Only now, we discovered that the field list in Power BI (JSON query) only shows the respective fields in the original structure, i.e. “grouplevel1/grouplevel2/fieldname”, but not in the new one: “grouplevel1/grouplevel2/grouplevel3/fieldname”.
While the grouping works perfectly in KoboCollect,…
the whole second level group (where I had inserted the third level groups) is not exported when using DATA/Downloads
less than half of the fields are shown in the list when tapping “hide fields” in DATA/Table
the question list shown on “Edit” after selecting a certain form under DATA/Table shows all the questions, but from today without content (although some questions are “required”)
@york_rff, could you show us the structure of the group so that we could visualize it pictorially?
No. The data should be there. Maybe it should be some issues with the group that you lately added that affected the data structure. We will investigate to figure out your issue.
Good idea. To do this, I had to delete the “relevant” column (because the fields referred to deleted lines).
The form works well in KoboCollect (same like with the original).
The Data show well when opened under DATA/Table (different).
The Data show well when exported under DATA/Downloads, however, only the field names show in the header, not the group names like in the original project (e.g.: “lessonplan” instead of “sitevisit_form/ind_outcome3/lessonplan”). Is this a hint on the problem source?
I have continued to experiment and arrived step-by-step at the point that I uploaded and deployed the full xlsform of the troublemaking project to the test project – and it looks like everything is working perfectly.
Is it possible that your server software wasn’t able to digest the many changes to the xlsform over time, one of the last ones being that addition of a third level of groups?
What can we do to make the collected data fully accessible?
Additional observation: As already stated, all data seem to be there when using the URL https://kf.kobotoolbox.org/assets/xxx/submissions/?format=json – however, PowerBI has a problem to extract the respective group properly (workarounds seem to be discussed in some fora)… Now, a similar issue seems to apply to the Export feature under DATA/Downloads, just that it is more radical and supresses a whole second level group. As we downlaod backup copies from time to time, this remains an urgent issue.
No. I mean can you still reproduce this issue for another project? If yes, could you share us the steps to reproduce the issue so that we could test the same at our end and create a bug issue.
I think we found a way to make all data of the respective group available in Power BI (needed some PBI tweaks which I don’t understand, except that it has something to do with building formulas which untypically depend on more than the first 1000 datasets) – as all data are part of the JSON query.
With regard to the same data missing when using DATA/Downloads, I just detected that the advanced options box “all versions” unintentionally had been unchecked.
What looked like a systematic problem, only was a bad coincidence.
Take-away: If you collect a lot of data to be visualized in Power BI, be very careful when restructuring your data collection form!
