Unable to download data from KoBoToolbox server using ODK Briefcase

Am facing another problem , when I pull the survey , it says with error , below is the screenshot . thank you for all the support

pull.jpeg

Hi @wb_cjc,

Please follows the steps as outlined below to download the data that is in the KoBoToolbox server using ODK Briefcase:

  • Pull>Configure (configure the ODK Briefcase by setting the URL, User Name & Password). URL is https://kc.kobotoolbox.org/user_name if you are using the HHI server and https://kc.humanitarianresponse.info/user_name if you are using the OCHA server. User Name is your KoBoToolbox User Account Name and the Password is the KoBoToolbox User Account Password.
  • Once you have set the configuration correctly, you will be displayed with all the project details that is in the KoBoToolbox Server. Now chose the project whose data you wish to download and press the Pull button (located at the right end corner). You will see Success under the Pull Status of that respective project that you have tried to download.
  • Now go to the Export tab which is just next to the Push tab. You should see the project that you pulled from the Pull tab. Select the project and then press Export. In the Export Status you will see Exported … submissions.
  • Now check the folder or the path that you configured to. The file should be exported there.

If you have already tried this earlier and still facing the issue, click the Reset button under the Pull tab.

GOOD DAY!

@Kal_Lam, again thank you so much , BUT

as you mentioned I have followed the steps and was able to see the projects and PULL , in doing so , it says " success with errors " when I open the error it said as in the below … THE ERROR PART IS IN quotation

Start pulling form and submissions
Start downloading form
Form downloaded
Start getting form manifest
Got the form manifest
Start getting submission IDs
Error getting batches of instance IDs: NOT FOUND
There are no submissions to download
Success with errors

even knowing the errors , I try to export and it just hang
looking forward for any suggestion .
Best
Tirsit

Hi @wb_cjc, @wroos,

In this case would you mind trying it again by first enabling the SETTINGS>Sharing>Anyone can view submissions made to this form from your KoBoToolbox Account (Survey Project).

Now repeat the steps mentioned above and you should be able to download the data. After downloading the data to your device, you could reverse back the change you made earlier (i.e. not making the data publicly available).

GOOD DAY!

Hi @Kal_Lam,

Thank you for your continued support .

I did change the setting of sharing , but the same problem persist , is there anything that I need to change in the XLSform setting t, I have tried to attach the XLSform but it said I am new user and was unable to do so. but I have attached the screenshot of my xls. let me know if I miss anything .

Untitled.jpeg
Thank you so much

Hi @wb_cjc,

I meant making your data public by checking out option 3 that has been outlined in the image below:

GOOD DAY!

@Kal_Lam , yes that is what exactly I did , and when the error still exists iI was think if the problem is also with the designed form submission ID or something

setting.jpeg

Hi @wb_cjc,

You have now done (i.e. the data are public). Try out the steps i have listed above and you should be able to get it in your briefcase.

I tried at my end and was able to download the data following the steps laid above. If it still does not work for you, would you mind providing your user name, password in a private message. I would try it at my end.

GOOD DAY!

Hi @wb_cjc,

I see three projects in your server. Which one would you like to pull?

GOOD DAY!

Hi ,

I would like to pull mainly the 1st phase and 2nd phase

1 Like

Hi @wb_cjc,

Just wanted to share with you that i was successful in pulling the data:
image

Was also able to export the data:
image

I shall shared with you the csv file in a private message shortly.

So, request you to simply follow the instructions (mentioned above) to get the data at your end.

GOOD DAY!

hi @Kal_Lam ,

I was able to pull successfully , what I forgot to write and was seeing errors earlier was , on the URL I didn’t enter the user name , so when I did it worked and pulled successfully . However export the data , it just hang or exporting is not successful , can you share me your export setting ?

i.e anything to edit on default Export Configuration ? particularly PEM file location ?
thanks
Tirsit

Hi, I was using ODK briefcase v1.17 and when I change to older version v1.11-beta it export the data without any problem .

@Kal_Lam , special tanks for all the support !!

1 Like

I played around a little with ODK briefcase on the kc server:

  1. When using the kc.kobotoolbox url without the username after the / I can see the forms I own, but I cannot pull them
  2. When using https://kc.kobotoolbox.com/myusername I can see the forms I created, I can pull the forms but not the media files like pictures, and I do not see the forms I was invited to with full privileges by the other users.
  3. When using https://kc.kobotoolbox.com/someooneelseusername (with my own username and password credentials) I can see all the forms they created on their account, public or not, but I cannot download them. I think it should not work that way.

Tested with ODK Briefcase 1.16.2 on Ubuntu.

1 Like

Hi @robinaudy,

Thanks for sharing your experiences to the entire community which the community should definitely benefit a lot. The concern you have raised (as bullet 3) is really concerning. Shall raise this in the team.

Thank you once again!

GOOD DAY!

I can confirm the same behaviour regarding the ability to download attachments when using ODK Briefcase 1.17.2 with the kc.kobotoolbox server. This causes particular problems with encrypted forms, as the form data appears to be sent as an attachment (.xml.enc) The .enc file is not downloaded with the ODK Briefcase log reporting:

2020-02-20 16:00:32,241 [ForkJoinPool-4-worker-2] ERROR o.o.b.p.a.PullFromAggregateTracker - Pull tutorial_encrypted - Error downloading attachment 1 of 1 of submission 2 of 2: HTTP 403 FORBIDDEN

The absence of the .enc file causes the attempts to decrypt and export the submissions to hang without error.

As suggested by @wb_cjc, using ODK Briefcase 1.11.3 allows the .enc file to be downloaded, and the decryption is successful.

I have tested a download and decryption of data from the test server (ODK Aggregate 1.7.0) described in this post. This works as expected with ODK Briefcase 1.17.2.

All the tests were on Windows 10

Andy

Hi Andy,
Thank you for highlighting the issue you have raised.

As you have noticed, this sounds to be an issue with a specific version of the ODK briefcase

The fact that the encryption works with the version 1.7.0 and not the later suggests that this is largely an issue with the ODK Briefcase as opposed to the KoBoToolBox encryption process. I have looked for similar issues on the ODK forum and would recommend searching for the same here.

Stephane

2 Likes

Hi Stephane,

Sorry, but I disagree, this appears to be an issue with how the kc.kobotoolbox server interacts with ODK Briefcase when the latter attempts to download attachments, This is an issue raised in the comment above:

This has an impact on encrypted forms as the form data is transferred as a separate attachment.

Results of my testing:

  • ODK Aggregate server (1.7.0 - not the latest version)
    Download and decryption works with ODK Briefcase 1.11.3 and 1.17.2 (latest)

  • kc.kobotoolbox server
    Download and decryption works with ODK Briefcase 1.11.3, but not 1.17.2

The results are the same whether the ODK or Kobo Android app is used to encrypt the form.

The inability to download media attachments from the kc.kobotoolbox server using ODK Briefcase 1.17.2 is only inconvenient as there are alternative approaches available. Being unable to download encrypted forms is more problematic as ODK Briefcase is the only tool available to decrypt forms as far as I know.

Andy

2 Likes

The inability to download encrypted forms with Briefcase from the kc.kobotoolbox server has been raised on the ODK Briefcase repository

1 Like

And the conclusion that the problem lies with KoBo has unfortunately not triggered any improvement.