Edit a big chunk of data - Data Consistency

[randriar]

Dear all,

I am trying to edit my dataset with more than 10,000 submissions. One of my questions ask a name of country and the responses are different for countries like the US. For instance, I have ‘US’, ‘us’, ‘U.S’, ‘The US’, ‘United States’, ‘The United States of America’…
Is there a way to fix this without having to edit each submission manually? Can we do it through a Python , SQL scripts , or API?

Thank you so much in advance

[Kal_Lam]

Hi @randriar,

You could edit a submission as outlined in the post discussed previously:

Updating Records via the API

@palmernet After discussing with @jnm and based on our recent discussion I can confirm that it is in fact already possible to do these batch updates via the API, using the endpoint http://kc.kobotoolbox.org/api/v1/submissions. This is the same endpoint used to submit new records. As per OpenRosa, the POST request needs to specify the original UUID as well as a newly generated UUID to identify the edited version. An edit should therefore have <deprecatedID>my-old-submission-id</deprecatedID> and…

Have a great day!

[randriar]

Hi @Kal_Lam,

Thank you so much for the quick response. I’ll try it.

Related questions: It takes too long to update thousands of submissions, is there a quicker way to do it by accessing the database directly? For instance via SQL or any other database management tool.

Thank you again.

Have you nice week ahead!

[Kal_Lam]

Hi @randriar,

Maybe you could use the same api link and see if it works with the SQL?

Have a great day!

[stephanealoo]

Hi @randriar
What duration of time are you seeing and for what specific number of entries? Sometimes, the duration is definitely understandable given the amount of data.

@randriar

It takes too long to update thousands of submissions,

Stephane

[randriar]

I have multiple forms with ~10,000 submissions each. I was wondering if there is another way to access the data and make bulk editing.
It seems like REST API is the (may be?) only way to do it, but that would require my scripts to 1) identify the uuid of the submissions I want to edit OR 2) delete everything, fix the issue and re-upload all of the them back.

Thanks

[stephanealoo]

Hi @randriar
As you may have noticed, this is the only way that would do this for you. All the best on this

Stephane

[Kal_Lam]

There is a suggestion box with a feature request where users should be able to edit the response of a variable from several records simultaneously:

Being able to edit several records/delete several projects/archive several projects simultaneously

Hi There ! While dealing with several projects we were thinking about an interesting feature: being able to batch edit the responses to a form for multiple submissions in one go. For example for a systematic mistake, were you need to convert all answers “a” to “b” for a given form. This would now need to be done on a record by record basis or after exporting the data, which is a shame. We are using KOBO Humanitarian response. Similarly, it could be very useful to being able to …

Maybe you could VOTE this features request if you wish to see this in the upcoming days!