BTW I also tried to use ODK Briefcase as mentioned in this thread, but had no luck.
I configured it correctly, as I was able to PULL form and submissions from KoboToolbox.
But was not able to PUSH, no forms appeared when I populated the PUSH screen with the same information I used in the PULL screen.
Just to complete the picture, the need for using the bulk submission came out of a debugging session with some KoboCollect devices that had unsent submissions. They were receiving a 500 error submitting their forms using KoboCollect, so I requested them to send me their files, their Android KoboCollect directory to me, so I could dig what was happening.
The submissions that were having issues were those that had heavy files (>10MB), so I attempted to send the submissions shrinking those files to a file size that was reasonable for Kobo toolbox API.
But no matter the size of the file, the failed. So I decided to create a new form and attempt to do submissions on this new form with just one field and the background audio. Still no luck, so that’s why I’m trying to debug this situation.
I’ve been playing around with this and it seems there may be an issue with the bulk-upload specifically around background-audio files (!?). Even without futzing around replacing audio files and editing XML, I have been unable to use the bulk-upload interface to re-submit a simple form with a short background-audio, after pulling it off the mobile device using adb. Whereas the identical submission comes up fine when I submit it directly from KoboCollect.
I’ll keep investigating but, suffice to say, “it’s not just you”
If you just want to get the audio file uploaded into Kobo, somehow, somewhere, you might also try a simple form with a single audio orfile upload widget, and attach your audio new file to that instead.
I presume from your description that you’ve already figured out how to pull off the too-large background audio files - off the devices with the failed submissions - and been able to shrink the files down to something more manageable?
If all you basically need are a way to get these new audio files up into your Kobo account, have you considered just creating a simple form/project with a single audio question, and firing up Enketo web interface to submit them all?
Or do you still need them to be re-associated intimately with the original form submission data? If that’s the case, you might also try replacing the too-large audio file in the submission folder on the physical device with your shrunk one - keeping the identical file name - and retrying the still pending submit from KoboCollect. Messy, but it might be an option.
It is not that.
The form has 72 questions, and the background audio is for auditting purposes.
I guess I could edit the form, add an audio field that is optional and at the end of the form, and upload it manually…
This would imply a usability issue, as audios could be find in one of two places, and our users will have to jump over that hoop to get to the audios.
I have not tried this myself yet.
In any case, it would be great to have this working properly still…
I did this in the past and it works, but I don’t have access to the devices so.I can’t do that maual process.
Understood. I wanted to make sure you at least have the ability to preserve the critical data, somewhere. Obviously, the goal is to capture the original submission in its entire former glory… The bulk upload should have worked but, as we can see, its misbehaving. Still investigating.
I dag into the kpi sourcecode a little bit…
I am not sure that this is related, but in the import functionality there are mentions about the images but not about audios or any other attachments like videos:
Could this be related to this?
This function is called for every submission that is sent in the zip file over the bulk submissions url. The trace is this one:
Thanks for the report; sorry about the trouble. I responded to your other topic, but I’ll cross-post here for convenience the part that links to an example of Python code that uploads submissions and attachments:
This is great, @nicopace, you are to be commended!
Couple minor suggestions:
Project_hash: This is your user folder inside KoboToolbox. If you have multiple projects set up, you will have several folders. There’s no way to identify them from the folder names alone, so you’ll need to browse inside each folder to determine which one contains your project.
Correct. Each project folder hash name is rather indecipherable… But, if you look inside each project folder - that is, if the user is connecting to multiple different projects in their KoboCollect - then there should be a single zero-length file that has the name of your project (!). So you can use this to reliably differentiate and locate submissions for different projects; eg
= 2021.2: /sdcard/Android/data/org.odk.collect.android/files/projects. Only accessible by Collect. The Project directories will contain a blank file with the same name as the Project itself.
Also, and as you might guess from above, you can also do all this from macOS too - you just need to install adb to do it [I note your instructions are specifically for Microsoft Windows: “On your computer, open the file manager, access the phone’s storage,…”]. There are some instructions for the Mac-only approach here:
If you have any issues using adb to pull stuff off an Android device onto a Mac please post back