It seems that recently the list of internal variables (columns) and SPSS labels on KoBo download has been extended and changed in order:
Can anyone explain what they contain (and where the values come from)?
Is there any documentation (link) for this change? (Since which release?)
We would appreciate, if you extend the label / data export list that you only add variables at the END and keep the previous order because external programming does count on the given order (before KoBo changes).
Before we had, as far as we know only:
Side-note: Other internal labels are still missing as SPSS labels (from repeat groups/sheets), like
Also variable names starting with _ are NOT allowed in SPSS.
Hi @wroos the unofficial release notes for these changes can be found here:
This was a partial release only in production on OCHA thus far and will be included in the full release next week. The new default exported columns are included in this release and I apologize for the order change.
Regarding where _notes and _tags come from, they can be set on submissions through the /api/v1 are from legacy KoBoCAT features and have just not been part of the exports until now. We’ve begun including them in preparation for shutting down legacy exports: there is not a direct replacement for per-submission notes and tags, but we did not want anyone to lose access to anything they may have entered using those legacy features. [edit by @jnm]
With the upcoming changes, you will be able to filter which columns you want included in your export so you can just exclude all those that are irrelevant for your needs.
What do you mean by “correspond”? We do not intend or desire at this time for the table view to be an exact duplicate of the XLS exports—for example, the table view obviously has no provision for multiple worksheets and thus anything in the XLS export for linking parent rows to repeating-group child rows is completely irrelevant—but if you are claiming that something like _id in the table view does not match _id in the XLS export, then that is a significant bug.
That’s certainly a reasonable request, and as Josh said, we’re sorry for causing trouble with your integrations. That said, I’m uneasy guaranteeing that we’ll never, ever change the order of existing fields or insert something new between existing fields. Is there some way we can help you make your external programming less brittle?
Yes, e.g. you will use _index and _parent_index to join the repeat levels.
And others are interesting for monitoring, QA and analysis, like submission_time, submitted_by, _validation_status.
We also like to use _parent_index to link repeats in the Excel sheets when we specify QA/cleaning issues in Excel (e.g. all members of a household to the household level).
And _id or __uuid is helpful for us to validate that all repeat joins went well.
It would be helpful to have __version__ (see table view) also in the data download. This would make it easier to check for data issues with form changes (see community topics).
But our issue was more about the SPSS labels (sorry, not well explained, I updated the post now). We would like that all exported (internal) variables from the XLS data download are included also in the SPSS label download and have a naming (in export) corresponding to SPSS syntax. So, also for the repeats as their data will be imported to SPSS.
_status is a really old thing from KoBoCAT, and I can only be 99% sure of exactly what it does without ploughing a whole bunch of time into reading code. Here’s my conclusion, until proven otherwise, backed up by checking the values in the database: