API to get history of submission

Suggestion Box
1

I looking to use the API (/api/v2/assets/) to regularly get the latest data. Ideally, I would also get a history of changes to the data, but I have not figured out a way to get previous (changed) submissions via the API.

I am using /api/v2/assets/{asset_uid}/data/ to get the latest data. I want to get only data that is new or changed since my last pull of data. I first tried to query by _submission_time, but it seems like this is not updated on edit. I found that I can use meta/deprecatedID to get any submissions that have been edited.

/api/v2/assets/{asset_uid}/data/?query={"meta/deprecatedID": {"$exists": true}}

I can store the key:value (submission id : meta/deprecatedID) in order to determine which submissions changed.

I thought I might be able to use the meta/deprecatedID to trace the history of submissions. ?query={"_uuid": "7c4f3d6e-c306-40a5-8579-5ccdc27f2bfb"}

But this does not work for getting deprecated uuids. Once an edit occurs, and a new uuid is created, the API will not pull the previous version.

Is there a way to get the history/past versions of a submission via the API? Maybe some magic query parameter? :slight_smile:

Thanks!
Matt

2

_id parameter doesn’t change on edits, and it’s incremental. _uuid changes upon editing.

Using these 2, you can figure out which are the new records and which are the edited ones. :slightly_smiling_face:

2 Likes
3

Thanks for the reply. I am able to find which records have been edited, but from what I can tell, I cannot get a history of the edits. I was hoping I could use the old _uuid values to get previous versions of edited submissions.

4

Hi @mattlangeman, unfortunately this isn’t available through the API and, as far as I know, we don’t store the history of individual submissions, only the form itself. @jnm will be able to correct/elaborate here.

5

Ah… I doubt that’s possible. From what I can understand, every time you edit an instance, the older one is overwritten.

6

But maybe you can work around it.
If you enforce a workflow where you use the new duplicate function before editing, you’ll have a history of all the edits.
If you have a primary key column in your data, you’ll be able to link them. Or else you can enter the previous _id in your edits to create a kind of linked list.

7

Thanks @Josh and @ks_1 .

Having a complete history probably isn’t critical, but would be nice. If I’m pulling in data on a nightly basis, it would only miss changes if there are multiple in a day.

@ks_1 Regarding enforcing a workflow and using the new duplicate function, are you talking about this Create new submission based on existing submission (aka duplicate record) · Issue #2769 · kobotoolbox/kpi · GitHub

So instead of editing, a user would duplicate a submission and then change the duplicate version?

8

I decided to dig into the code a bit. It does look like the older submission is overwritten. However, a new InstanceHistory object is created with a copy of the old xml, old uuid, and link to Instance object. Guessing this history is not accessible via the API, but nice to know it is getting stored.

2 Likes
9

Right! Seems like it’d be nice to have in the future. Maybe something for Suggestion Box - KoboToolbox Community Forum

11

@jnm Sounds good. I’ll add a description of this feature over there.

@wroos The suggestion you linked to is a bit different than this one. That is for a history of a form/survey. This is for a history of submissions.

1 Like
12

Sorry @mattlangeman, you are right. I removed the previous post.
I understood that you are just looking for a filter for (new/updated) submissions.

There was another discussion about history (change log) PER submission: https://community.kobotoolbox.org/t/update-an-already-submitted-record-on-a-mobile-client/1745/9.

Maybe setting and using the validation_status or an extra variable might offer a workaround?

2 Likes
13

Is there an up to date API documentation for kobotoolbox ?