Changing Form Choices via API, need guidance

[paulsermon]

I need some guidance on updating question choices via the API. So far I have tried the following:

I have succesfully retrieved the form as xls, then edited the choices within the form with my new choices. I am stuck on where and how to replace the form with this new xls. Where do I upload a new form xls via the API?

I notice that I can apparently update assets via the API as detailed on the REST api page, but it does not detail how to adjust the choices. Can someone give me a some examples of updating choices via this endpoint?

[stephanealoo]

Hi @paulsermon
Welcome to the community forum. I am recommending the following topic which should walk you through the process

Uploading Data from XLS to the server

Hi @brighton_zocs, you can try using this (no longer supported) script: You can also try following these yet-to-be-published, draft tutorials on doing this through the API: Part 1 Part 2

Stephane

[paulsermon]

@stephanealoo Thanks, however these links detail updating submission data. I need to update the questions and their choices in the form itself.

I’ve edited my question test to make it clear that I am talking about updating the form itself.

[stephanealoo]

Hi @paulsermon
Have you searched through the forum? Do you mind checking if the discussion (don’t mind the title) on this topic would work for you?

Using API to upload media files?

@Josh It works beautifully! Many thanks. I did some experimenting and I found that ODK will download the new CSV file if you sync manually, even if you don’t redeploy the form. Here’s the final python script for anyone who wants to programmatically upload the CSV file for pulldata purposes: import requests import json KC_URL = 'https://<kc_url>/api/v1/' KF_URL = 'https://<kf_url>/api/v2/' TOKEN = '<token>' XFORM = <xform_id> FILE_FOLDER = r"<file_folder>" …

Stephane

[paulsermon]

I have indeed looked through the forum. There are many topics that come close to what we are trying to acheive, but are not quite the answer.

Unfortunately we cannot use the Media files route as our users need to be able to edit forms via the online form builder and the select_one_from_file method does not work there.

I’m specifically looking at a ways to upload the form structure by either: uploading an updated xls file, or updating the asset directly via json, or any other method that might allow me to edit choices.

[stephanealoo]

Hi @paulsermon
I am leaving this open for other users. I am howevering recommending the following for your reading. Kindly dont focus on the title but rather the content

Managing cascading select questions for a very large question choices

Hi @cchukwu I would like to propose an entire different approach to getting this form to work. This will however require that you do some work on your end to customize the form on the cascade selection using the following instructions. Approach 1: Using search () in the appearance column. NOTE: This method will only work on the android collect application. The original writeup can be found here (XLSForm Docs). This method does not need you to execute choice_filter for cascade questions …

Stephane

[Josh]

Hi @paulsermon, you can take either route to update your form.

1. Upload via API: I also couldn’t find documentation, but you can do the following. If your current project has the asset UID of aW5HJb2gDhpATTaAAHZdxB and you want to replace the current form with a new form of name new-form.xlsx (with a local path of /path/to):

curl -X POST https://kf.kobotoolbox.org/api/v2/imports/ \
  -F file=@/path/to/new-form.xlsx \
  -F destination=https://kf.kobotoolbox.org/api/v2/assets/aW5HJb2gDhpATTaAAHZdxB/ \
  -H "Authorization: Token <your secret token>"

You should get a JSON response from the request that looks something like:

{
  "uid": "it76ukZ57Ks3oSEYrgyTzC",
  "url": "https://kf.kobotoolbox.org/api/v2/imports/it76ukZ57Ks3oSEYrgyTzC/",
  "status": "processing"
}

2. You can update the form with a PATCH request directly to https://kf.kobotoolbox.org/api/v2/assets/aW5HJb2gDhpATTaAAHZdxB/, updating the content with your new form in JSON format.

Note that for either of these routes, you need to redeploy the project to make it live (you can do this through the API too). An example of this and other useful API tricks can be seen here:

KoBo API examples, using new KPI endpoints

This is copied from some private correspondence I had a while ago. Perhaps it’s useful to someone in this format, until more official documentation is developed for the new API In writing “new,” I’m contrasting KPI to the old KoBoCAT (/api/v1) API. KoBo API Examples I’ve used curl to demonstrate these API calls. You’ll notice that all my curl commands begin with curl --silent --user jnm_api:some-secret-phrase --header 'Accept: application/json' and end by being piped to python -m…

[paulsermon]

I’m afraid the content of that post does not help me with this issue at all.

[paulsermon]

@Josh Brilliant! This is exactly what I needed!

It seems I was close to acheiving this. The confusion lay around the destination parameter vs the actual endpoint to send the file to. I have now successfully uploaded a new modified form using XLS upload via the API.

I’m going to experiment using the json endpoint now, as that may be simpler for me than modifying xls in python.

Thanks for your help.

[paulsermon]

As a follow up, here is some examples of the python code I used to acheive this.

The rewriting of choices:

URL = 'https://kf.kobotoolbox.org/api/v2/'
TOKEN = config.KOBO_TOKEN
XFORM = form_id
choice_id = 'xxx' # id of choice in form. This is usually some random text. You can find it if you download the xls
new_choices = [
  'Choice 1',
  'Choice 2',
  'Choice 3',
]

headers = {'Authorization': f'Token {TOKEN}'}
asset_url = "%sassets/%s/" % (URL, XFORM)

# get form as json from kobo
response = requests.get(asset_url, headers=headers, params={'format': 'json'})

# Copy the content so we can edit it (some of the response seemed immutable)
existing_content = copy.deepcopy(response.json()['content'])
choices = existing_content['choices']

for i, choice in enumerate(choices[:]): # Iterating over a copy of choices and removing matches
    if choice['list_name'] == option_id:
        choices.remove(choice)

# Append new choices
for n in new_choices: 
    choices.append({'list_name': option_id, 'label': [n], 'name': n})

response = requests.patch(asset_url, headers=headers, data={'content': json.dumps(existing_content)})

Deploying the new version:

response = requests.get(asset_url, headers=headers, params={'format': 'json'})
version_to_deploy = response.json()['version_id']

response = requests.patch(asset_url + 'deployment/', headers=headers, data={'version_id': version_to_deploy})

[Kal_Lam]

Thank you for sharing this with the entire community, and making the community rich with documentation @paulsermon Expecting the same in the upcoming days too