Edit survey through API

Hey all,

Editing a report via the Kobo API endpoint by authentication with the API Key only seems to work as long as I am also logged in as a user to the KoboToolbox web environment.

Many other API endpoints are accessible with the API Key only, and that used to be the case with the edit endpoint too in the past.

Does anyone know what I am missing?

My test form is here:
https://ee-eu.kobotoolbox.org/x/AoIhYvar

When using my API Key to post to this API endpoint:
https://eu.kobotoolbox.org/api/v2/assets/aR8XkRVtZD5GpsX8pkvzwv/data/553107085/enketo/edit/?return_url=false
the server does return a url, for example:
https://ee-eu.kobotoolbox.org/edit/rQUcb1BV?instance_id=ec9b11b8-4abd-442d-a98f-58034933341d&return_url=false

But sending that to a browser, e.g. Edge or Brave, gives this result:

Sometimes though, interestingly, the survey does load, but when I submit the form this error is shown:

Not allowed to post data to this data server.
Contact the survey administrator please. (403)

However, when I happen to be logged in to KoboToolbox, all works fine.

Much appreciating your help,
Bert

Welcome back to the community, @bert12! This post discussed previously should help you solve your issue:

Thank you @Kal_Lam for your continued support over the years! :+1:

But these related posts seem to deal with obtaining the url from the API, or with authenticating with the API, or with the format of the API request or with the duration of the validity of the API response.

In my case however I authenticate successfully with the API and I do receive a valid url back from the API and am able to automatically forward it to the browser within the time of validity.

My problem is that ONLY if I am also logged in to the web environment of Kobo, then the link that was returned from the API results in a correctly loading report that can be edited and succesfully re-submitted. But, if I am not logged in the web environment of Kobo, the link that was returned from the API in a browser shows the error from my message above.

Which is odd, since I already authenticated by API Key to obtain the edit url and also since my project is open to receive submissions without login.

To reproduce the issue, this little php script authenticates with the API in my project, gets the url to edit the report and forwards the url to the browser:
https://entrack.org/test/edit_test.php

Much appreciating your thoughts!

Hi,
@jnm, from your responses to other questions I think you have in-depth knowledge about this. Would be great if you can have a look.

Basically my issue is:
For editing a submission through the link given by the edit-endpoint of the API it seems like I have to authenticate twice: one time with the API to obtain the edit link, and a second time when opening the link in a browser.

The first authentication is always succesful, I receive the edit link back from the API and that url is in the exact same format as if I would edit a submission through the Kobo interface.
But then with sending that url to a browser it seems I need to authenticate again, as I get the error as shown in my post above.

Is there maybe a special authentication header that I should send along to the browser together with the edit url?

Hi all,

Have found a workaround that I would like to share with you.

It is helpful in case a report needs to be send back to the enumerator for edits while the enumerator has no Kobo user account.

Note that with a user account and as logged in user, the regular way to edit reports through the enketo interface by url through the kobo api is as described in “Update current submission” in: https://eu.kobotoolbox.org/api/v2/assets/{uid}/data/
But since that currently only seems to work if you happen to be logged in to kobo and since not every enumerator is logged into kobo, the workaround here is helpful.

In short:
Step 1. Use the api to get the content of the report
Step 2. Build a url to a new report and populate the report fields with the data from the old report
Step 3. Send the url to the enumerator for any edits to the report
Step 4. The enumerator submits that newly edited report
Step 5. Use the api to copy the edits of the enumerator to the old report

In detail:
Step 1. Use the api to get the content of the report:
A get request to your instance of the report, for example:
https://eu.kobotoolbox.org/api/v2/assets/{form uid}/data/{report _id}/?format=json

Step 2. Build a url to a new report and populate the report fields with the data from the old report:
Kobo forms can be pre-populated with data by passing the data to the url in this format:

https://ee.kobotoolbox.org/x/{enketo form id }?d [ / {kobo form id} / field_name_1 ]=value_1 & d [ / {kobo form id} / field_name_2 ]=value_2

For example, in practice:

https://ee.kobotoolbox.org/x/6K8reKvt?d[%2FaYY7X8SYjTRw7QNN29tAQ6%2FEntry1]=25&d[%2FaYY7X8SYjTRw7QNN29tAQ6%2FEntry2]=3521&d[%2FaYY7X8SYjTRw7QNN29tAQ6%2Fref]=This_form_overwrites_form:38302492

Here is a snippet in php of how to build the url automatically:

		// get the report from kobo:
		$old_report = get_report_from_kobo ($report_id);

		// exclude system markers like _ or / or "start" or "end" as these should not be changed:
		$exclude_these_fields = array();
		foreach ($old_report as $field => $key){
			if( substr($field, 0, 1) === "_" || strpos($field, "/") !== false || $field === "start" || $field === "end" )
				$exclude_these_fields[$field] = $key; 
		}	
		$old_report_cleaned = array_diff_key($old_report, $exclude_these_fields); 

		// construct the url:
		$url = "https://ee-eu.kobotoolbox.org/x/$form_id?";	
		$slash = "%2F";
		foreach($old_report_cleaned as $element => $value) {
			$url.= "d[" . $slash . $form_xml_uid . $slash . $element . "]=" . $value . "&";				
		}
		// conveniently add a variable that carries the old report ID for automated processing later on; this variable is stored in a kobo field of type hidden
		$url.= "d[" . $slash . $form_xml_uid . $slash . "sys_stamp]=" . $report_id;

Step 3. Send the url to the enumerator for any edits to the report:
The url can be easily sent for example by email

Step 4. Submit that newly edited report:
The enumerator can make the edits and submit the report

Step 5. Use the api to copy the edits of the enumerator to the old report:
With both the old report and the new report in hand, the values can be compared and changed on the side of the old report by using the v2 api endpoint:

		$url = "https://eu.kobotoolbox.org/api/v2/assets/$kb_form_xml_uid/data/bulk/";        
        $token = "your_confidential_token";  // when logged in to kobo, this token can be found at https://eu.kobotoolbox.org/token/?format=json 
        
        // Create an array with the data that needs the update
        $data = [
            'submission_ids' => [ strval( $report_id ) ],
            'data' => [ strval( $key ) => strval( $value ) ]
        ];        

        // Initialize cURL session
        $ch = curl_init($url);

        $payload = '{"payload":' . json_encode($data) . '}';
        // Set cURL options
        curl_setopt($ch, CURLOPT_CUSTOMREQUEST, 'PATCH');
        curl_setopt($ch, CURLOPT_POSTFIELDS, $payload);
        curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
        curl_setopt($ch, CURLOPT_HTTPHEADER, [
            'Authorization: Token ' . $token,
            'Content-Type: application/json'
        ]);

        // Run cURL request
        $response = curl_exec($ch);

        
        // Close cURL session
        curl_close($ch);

        // Check for errors
        if (curl_errno($ch)) {
            $this->write_logfile("edit report field: ", 'cURL error: ' . print_r( curl_error($ch), true ) );
        } else {
            $this->write_logfile("edit report field: ",  'Response: ' . $response ); 
        }

With the edits now done to the old report, the new report can be deleted and the same effect is reached as with edit report, but now without the enumerator having to be logged in to kobo.