Handling REST Service failures

User Support Data Management
1

Hello! :wave:

We are trying to build some automation to recover from REST Service notification failures.

After 3 retries, Kobo will send an email notification with the list of failed notifications.

This list includes the error and response code, as well as some ID, which we were hoping to be able to use to fetch individual failed submissions via API.

Unfortunately it looks like this unique id is not the form submission id, and we therefore cannot use this to retrieve the submission. It seems to be an id specific to the REST Service invocation (?).

Can someone explain what this id is appearing in the REST Service failure email notifications, and how this could help us identify the undelivered submissions?

Many thanks?

4

Hi @yjouanique, the UID you are referring to is the hook log UID that looks like hlGKQir5k8Ef2SMs7FNo7eH (starting with hl). You can associate this with the submission by the following:

  1. Find the UID of your REST Service at https://[kf_url]/api/v2/assets/[asset_uid]/hooks.json, for example: hdQBCLmJVzoVCJtFWPqtqk
  2. Make a request to https://[kf_url]/api/v2/assets/[asset_uid]/hooks/[hook_uid]/logs/[uid from the email].json

You will see the submission_id in the JSON response that looks something like this:

{
  "url": "https://kf.kobotoolbox.org/api/v2/assets/aZc6XxkHmEM4Xfb4cRX2Mb/hooks/hdQBCLmJVzoVCJtFWPqtqk/logs/hlGKQir5k8Ef2SMs7FNo7eH/",
  "uid": "hlGKQir5k8Ef2SMs7FNo7eH",
  "submission_id": 123406156,
  "tries": 4,
  "status": 0,
  "status_str": "Failed",
  "status_code": null,
  "message": "HTTPSConnectionPool(host='example.com', port=443): Max retries exceeded with url: /example (Caused by SSLError(SSLError(\"bad handshake: Error([('SSL routines', 'tls_process_server_certificate', 'certificate verify failed')])\")))",
  "date_modified": "2021-11-08T20:53:22.939297Z"
}
2 Likes
5

Great, this works very well! Many thanks!

This actually got me thinking: it looks like rather than manually copy-pasting the hook log id from the mail notification to a sheet, and then doing my own processing, I can probably just query the failed hooks from the API and just trigger the retry!

2 Likes
6

@yjouanique that’s correct :slightly_smiling_face:

1 Like
7

Actually after playing a bit with the API I felt that it wouldn’t let me do quite what I wanted, so I’ve created this new feature request: Hooks API improvements: filter and force retry · Issue #3536 · kobotoolbox/kpi · GitHub

I’ve worked around it, but it’s a little cumbersome right now, I hope these improvements can be considered…

1 Like
8

Thank you for the feature request :slightly_smiling_face:

1 Like
9

And now with a fresh PR: Status filter for hook log API by Yann-J · Pull Request #3595 · kobotoolbox/kpi · GitHub :octopus:

1 Like