KoBo API examples, using new KPI endpoints

jnm · March 21, 2019, 4:55am

This is copied from some private correspondence I had a while ago. Perhaps it’s useful to someone in this format, until more official documentation is developed for the new API In writing “new,” I’m contrasting KPI to the old KoBoCAT (/api/v1) API.

KoBo API Examples

I’ve used curl to demonstrate these API calls. You’ll notice that all my curl commands begin with curl --silent --user jnm_api:some-secret-phrase --header 'Accept: application/json' and end by being piped to python -m json.tool. The python call simply formats the JSON response to make it easier to read. The --silent flag to curl is not required, but it makes the output cleaner for the purposes of this demonstration. Basic authentication with --user username:password as I’ve done is fine, but you may also authenticate using a token retrieved at https://kobo.humanitarianresponse.info/token/ by sending the header Authorization: Token [your-token]. Including the header Accept: application/json for all requests is highly recommended to avoid having the API return “browseable” HTML that is not easily parsed.

Importing a form

Uploading the XLSForm

POST a new import as a multipart form. The necessary parameters are library=false and file, which cotains the XLSForm:

jnm@world:/tmp/api_demo$ curl --silent --user jnm_api:some-secret-phrase --header 'Accept: application/json' -X POST https://kobo.humanitarianresponse.info/imports/ --form library=false --form file=@kobo_1701_NW.xlsx | python -m json.tool
{
    "status": "processing",
    "uid": "iGYukk6NVEA64zwMsPtgRD",
    "url": "https://kobo.humanitarianresponse.info/imports/iGYukk6NVEA64zwMsPtgRD/"
}

Getting the resulting `asset` UID

The asset UID is given by messages.created[0].uid in the JSON response when GETting an import. Multiple GETs may be required until the server indicates "status": "complete" in the response:

jnm@world:/tmp/api_demo$ curl --silent --user jnm_api:some-secret-phrase --header 'Accept: application/json' https://kobo.humanitarianresponse.info/imports/iGYukk6NVEA64zwMsPtgRD/ | python -m json.tool
{
    "date_created": "2018-05-19T15:21:49.709161Z",
    "messages": {
        "created": [
            {
                "kind": "asset",
                "owner__username": "jnm_api",
                "summary": {
                    "columns": [
                        "type",
                        "name",
                        "label",
                        "required",
                        "select_from_list_name",
                        "constraint_message",
                        "constraint",
                        "default",
                        "appearance",
                        "choice_filter",
                        "hint",
                        "relevant"
                    ],
                    "default_translation": null,
                    "geo": false,
                    "labels": [
                        "\u062c \u0628\u064a\u0627\u0646\u0627\u062a \u062c\u0627\u0645\u0639 \u0627\u0644\u0645\u0639\u0644\u0648\u0645\u0627\u062a",
                        "**\u062c1 \u0631\u0642\u0645 \u0627\u0644\u0627\u0633\u062a\u0628\u064a\u0627\u0646**",
                        "**\u062c2 \u0627\u0644\u062a\u0627\u0631\u064a\u062e (\u064a\u0648\u0645/\u0634\u0647\u0631/\u0633\u0646\u0629) **",
                        "**\u062c3 \u0631\u0645\u0632 \u062c\u0627\u0645\u0639 \u0627\u0644\u0645\u0639\u0644\u0648\u0645\u0627\u062a **",
                        "**\u062c4 \u062c\u0646\u0633 \u062c\u0627\u0645\u0639 \u0627\u0644\u0645\u0639\u0644\u0648\u0645\u0627\u062a **"
                    ],
                    "languages": [
                        null
                    ],
                    "row_count": 1269
                },
                "uid": "apLBsTJ4JAReiAWQQQBKNZ"
            }
        ]
    },
    "status": "complete",
    "uid": "iGYukk6NVEA64zwMsPtgRD"
}

Working with existing forms

Deploying an `asset`

Construct a URL using the asset UID: https://kobo.humanitarianresponse.info/api/v2/assets/[your-asset-UID]/deployment/; then, POST with the form parameter active=true.

jnm@world:/tmp/api_demo$ curl --silent --user jnm_api:some-secret-phrase --header 'Accept: application/json' -X POST https://kobo.humanitarianresponse.info/api/v2/assets/apLBsTJ4JAReiAWQQQBKNZ/deployment/ --form active=true | python -m json.tool
{
    "active": true,
    "backend": "kobocat",
    "identifier": "https://kc.humanitarianresponse.info/jnm_api/forms/apLBsTJ4JAReiAWQQQBKNZ",
    "version_id": "vh9JnfrRNwVFjJZnSwRYqd"
}

Rename a form

Send a PATCH to the asset URL, i.e. https://kobo.humanitarianresponse.info/api/v2/assets/[your-asset-UID/, with the form parameter name=[Your New Name]. If your form is large, the response will be large as well! I have omitted the output below for that reason.

jnm@world:/tmp/api_demo$ curl --silent --user jnm_api:some-secret-phrase --header 'Accept: application/json' -X PATCH https://kobo.humanitarianresponse.info/api/v2/assets/apLBsTJ4JAReiAWQQQBKNZ/ --form name='My New Name' | python -m json.tool

If the form was already deployed, you must redeploy the form for the name change to appear in data collection clients.

Redeploy a form

First, retrieve the UID of the latest version of the asset by making a GET request to the asset URL. Again, if your form is large, the response will be large as well. Here, I use a simple grep to restrict the output to the property I want: version_id.

jnm@world:/tmp/api_demo$ curl --silent --user jnm_api:some-secret-phrase --header 'Accept: application/json' https://kobo.humanitarianresponse.info/api/v2/assets/apLBsTJ4JAReiAWQQQBKNZ/ | python -m json.tool | grep version_id
    "deployed_version_id": "vh9JnfrRNwVFjJZnSwRYqd",
    "version_id": "ve338oJQooAbkyar7qjukR",

Note that deployed_version_id is not equal to version_id, indicating that the form has been changed since the last deployment.

Now, send a PATCH to https://kobo.humanitarianresponse.info/api/v2/assets/[your-asset-UID]/deployment/ with the form parameter version_id set to the UID of the latest version of the form obtained above.

jnm@world:/tmp/api_demo$ curl --silent --user jnm_api:some-secret-phrase --header 'Accept: application/json' -X PATCH https://kobo.humanitarianresponse.info/api/v2/assets/apLBsTJ4JAReiAWQQQBKNZ/deployment/ --form version_id=ve338oJQooAbkyar7qjukR | python -m json.tool
{
    "active": true,
    "backend": "kobocat",
    "identifier": "https://kc.humanitarianresponse.info/jnm_api/forms/apLBsTJ4JAReiAWQQQBKNZ",
    "version_id": "ve338oJQooAbkyar7qjukR"
}

Clone a form

POST to https://kobo.humanitarianresponse.info/api/v2/assets/ with the form parameter clone_from set to the UID of the asset that should be cloned. You may also include a name parameter that sets the name of the new asset. Again, if you form is large, the response will be large. Just to make this demonstration easier to read, I pipe the output to tail:

jnm@world:/tmp/api_demo$ curl --silent --user jnm_api:some-secret-phrase --header 'Accept: application/json' -X POST https://kobo.humanitarianresponse.info/api/v2/assets/ --form clone_from=apLBsTJ4JAReiAWQQQBKNZ --form name='My Cloned Form' | python -m json.tool | tail
        "row_count": 1269
    },
    "tag_string": "",
    "uid": "a9Cufbcm2GsqD5oh7u5wXd",
    "url": "https://kobo.humanitarianresponse.info/api/v2/assets/a9Cufbcm2GsqD5oh7u5wXd/",
    "version_count": 1,
    "version_id": "vq7Xi8wmQYuB4BT3uBdxE3",
    "xform_link": "https://kobo.humanitarianresponse.info/api/v2/assets/a9Cufbcm2GsqD5oh7u5wXd/xform/",
    "xls_link": "https://kobo.humanitarianresponse.info/api/v2/assets/a9Cufbcm2GsqD5oh7u5wXd/xls/"
}

The new asset's UID is given by the uid property in the response.
The new asset will not be deployed automatically, even if the source asset was deployed.

Share a form

Sharing requires a POST to https://kobo.humanitarianresponse.info/api/v2/assets/[your-asset-UID]/permission-assignments/ with two form parameters:

permission: the URL, relative or absolute, identifying the permission to grant, e.g. /api/v2/permissions/change_asset/. This can be any assignable permission for an asset;
user: the URL, relative or absolute, identifying the user who will receive the new permission, e.g. /api/v2/users/jnm/. The last component of the URL is the username.

jnm@world:/tmp/api_demo$ curl --silent --user jnm_api:some-secret-phrase --header 'Accept: application/json' -X POST https://kobo.humanitarianresponse.info/api/v2/assets/a9Cufbcm2GsqD5oh7u5wXd/permission-assignments/ --form permission=/api/v2/permissions/change_asset/ --form user=/api/v2/users/jnm/ | python -m json.tool
{
    "url": "https://kobo.humanitarianresponse.info/api/v2/assets/aDwm9jvrYog3YXjWqYP3FE/permission-assignments/pYjuKYQ6QuiGtgLb4DfMPU/",
    "user": "https://kobo.humanitarianresponse.info/api/v2/users/jnm/",
    "permission": "https://kobo.humanitarianresponse.info/api/v2/permissions/change_asset/",
    "label": "Edit form"
}

prashant_sh · April 23, 2019, 6:49am

Is there an API endpoint to create users in kobo?

mskaraca · April 24, 2019, 6:35am

please… we need a complete documentation for the new API

jonathanjo · June 20, 2019, 10:48am

How do we get the submitted data for a form?

Document at http://support.kobotoolbox.org/articles/592398-api-and-rest-services describes a https://[kpi-url]/submissions/ endpoint, but apparently that doesn’t exist, returning HTML The requested URL /submissions/ was not found on this server.

Please could you give an example of how to get the submitted data for a given form – surely the most important action?

Many thanks

sawan · June 25, 2019, 8:08pm

I’m able to access my submitted data with the kc api at https://kc.humanitarianresponse.info/api/v1/data. Just add the form “id” to the url to get the submitted data for a specific form. It’s suggested to use the new kpi but I wasn’t able to find the kpi equivalent of it.

degami · August 5, 2019, 9:57am

Hi @jonathanjo,

It looks like you can get submissions for a form with a GET to https://[kpi-url]/assets/[uid]/submissions. This looks like the KPI equivalent to /api/v1/data/[uid], at least for now.

degami · August 5, 2019, 10:11am

Hi,

Thanks very much for this post. It’s good to get a head-start in moving our applications over to the new KPI.

A quick question: I’m working on how to update an existing form by importing a new XLS file. From looking at the front-end, it appears the correct call is a POST to https://[kpi-url]/imports/[uid] to overwrite the [uid] survey asset. Could you confirm if this will be part of the API, or if I’m doing something unsupported and there’s a better way I should try?

tomgr · November 26, 2019, 9:07am

Thank you for the post! Is there any way to create REST services for a deployed form via the API? I’m trying to integrate KoBo into another system, and this would be really useful.

maric.vladimir · December 1, 2019, 9:04am

Apparently endpoint https://[kpi-url]/imports/[uid] can be used to import a form definition.

Is there a way to import submissions through the V2 api?

Thanks in advance.

stephanealoo · December 10, 2019, 11:14am

Hi @maric.vladimir
Do you mean importing submissions into KoBoToolbox via APIs? If so, then I would say theoretically yes since the logic is already clear here
I may not have specifics of how to do that but I will let other users chime in

Regards
Stephane

maric.vladimir · December 16, 2019, 8:05pm

Thanks for the pointer @stephanealoo.

I am in fact looking for a way to import the data exported not from Collect or Web form, but from a Kobo API endpoint, be it /api/v1 or /exports/.

Is there a straightforward way of getting an archive from KoBo suitable for import through /bulk-submission-form?

stephanealoo · December 16, 2019, 8:15pm

Hi @maric.vladimir
We do not have an approach for importing data via API into KoBoToolbox . This is not currently supported on the platform. However we can see if other users have been able to do this on their end.

Stephane

jnm · December 20, 2019, 8:43pm

@tomgr, yes, you can manipulate REST Services through the API. Take a look at https://kf.kobotoolbox.org/api/v2/assets/[Your Asset UID]/hooks/.

AndrewM · January 8, 2020, 12:21pm

The format of the XML output from the /data endpoint seems odd.

https://kobo.humanitarianresponse.info/api/v2/assets/[AssetID]/data.xml

Produces the below, as expected

<?xml version="1.0" encoding="UTF-8" ?><root><count>2</count><next></next><previous></previous><results><item>

But within each item tag is an encoded string of text representing XML, rather than containing the XML directly:

>     &lt;aexRbJgmrYG3gwJExgGpan xmlns:jr=&quot;http://openrosa.org/javarosa&quot; xmlns:odk=&quot;http://www.opendatakit.org/xforms&quot; xmlns:orx=&quot;http://openrosa.org/xforms&quot; id=&quot;aexRbJgmrYG3gwJExgGpan&quot; version=&quot;vu67i8YdEnNeYNmv7QH5oC&quot;&gt;
>               &lt;formhub&gt;
>                 &lt;uuid&gt;e13d16e21e1f458386737162dd307795&lt;/uuid&gt;
>               &lt;/formhub&gt;
>               &lt;start&gt;2020-01-06T15:51:02.023-00:00&lt;/start&gt;

So we can’t easily access the data itself using XSL processing offered by the tool we use to consume the data.

Is this deliberate?

AndrewM · January 17, 2020, 5:21pm

In addition to the above, we also see occasional additional XML declarations within the output XML. Can this be fixed? Is this forum the only way of flagging issues with the XML V2 API?

nolive · January 24, 2020, 4:24pm

Hello @AndrewM,
Thank you for feedback.
Actually, there is no official support for XML format for data endpoint. The recommended format is JSON.
Having that said, we are aware that the XML rendering is not perfect and an issue was opened for that.
https://github.com/kobotoolbox/kpi/issues/2393.
We do not have an ETA for that, though.

tinok · January 31, 2020, 9:04pm

@AndrewM You could have a look at this option, which I’ve used to import data from Excel:

arthurday · April 17, 2020, 12:12am

Did you ever figure out how to create users via the API endpoint? I’ve tried, but keep getting at 500 error.

nat · July 5, 2021, 12:51am

Also x-reffing back to the UNHCR KoBo API usage documentation - exaomples of how to query submissions using the API, to return only submissions meeting specific criteria (rather than always getting all of them).’

sulimandanish · July 26, 2021, 6:43am

hey everyone is there any way to integrate kobo with odoo modules?