QRealTime Plugin: Integration of Kobo with GIS

shivareddyiirs · November 4, 2019, 2:35pm

I am please to announce that QRealTime plugin now allows integration with kobo.
User can import, sync or create new forms from/into GIS layer.
This is experimental release, soon after initial user responses and comments, we would release regular version of plugin.
To install enable support for experimental plugins and install this plugin in QGIS

Looking for user feedback

for details visit
https://github.com/shivareddyiirs/QRealTime/tree/QRealTime4.0

stephanealoo · November 12, 2019, 5:19pm

Hi @shivareddyiirs
This is great news. I suggest you do some quick write up on how to do this and we can pick it up to make this more widely available for users to integrate. Good Job

Regards
Stephane

shivareddyiirs · April 23, 2020, 6:07am

Thanks @stephanealoo . This is is the link to the document.
https://github.com/shivareddyiirs/QRealTime/blob/QRealTime4.0/README.md

stephanealoo · November 13, 2019, 6:12am

Thanks so much

ruben_foquet · April 9, 2020, 1:49pm

Hi all,

I am grateful that this has been designed! I have downloaded the experimental plug-in in QGIS and I am testing the design of forms in QGIS shapefile layer. I have the following questions:

How do I best design constraints/dependencies/skip logic in my layer form, so that it shows up on KoboCollect?
Is there a way in QGIS to begin a group of questions, and end it, in order to show related questions on one screen in KoboCollect? Now, every new field is a new screen in KoboCollect.
After I designed my form in QGIS, brought it online to Kobo, filled one form on the KoboCollect on android and finally imported it in QGIS again, QGIS warns me that some errors were observed and added new fields to the layer, instead of using the predefined fields. In other words, the import of my data is not working properly.

> 2020-04-09T15:47:00 WARNING KoboToobox : Can’t find ‘ODKUUID’ field
*> *
> 2020-04-09T15:47:00 WARNING Python error : An error has occurred while executing Python code: See message log (Python Error) for more details.
*> *
> 2020-04-09T15:47:00 WARNING Traceback (most recent call last):
> File “C:/Users/MediaMonster/AppData/Roaming/QGIS/QGIS3\profiles\default/python/plugins\QRealTime\QRealTime.py”, line 278, in importData
> service.importData(layer,selectedForm,True)
> File “C:/Users/MediaMonster/AppData/Roaming/QGIS/QGIS3\profiles\default/python/plugins\QRealTime\QRealTime_dialog.py”, line 734, in importData
> self.collectData(layer,selectedForm,importData,self.layer_name,self.version,self.geoField)
> File “C:/Users/MediaMonster/AppData/Roaming/QGIS/QGIS3\profiles\default/python/plugins\QRealTime\QRealTime_dialog.py”, line 427, in collectData
> self.updateLayer(layer,remoteTable,geoField)
> File “C:/Users/MediaMonster/AppData/Roaming/QGIS/QGIS3\profiles\default/python/plugins\QRealTime\QRealTime_dialog.py”, line 506, in updateLayer
> with edit(layer):
> File “C:/OSGEO4~1/apps/qgis/./python\qgis\core\additions\edit.py”, line 38, in enter
> assert self.layer.startEditing()
> AssertionError

Alternatively, since I find it straightforward to design forms in KoboToolbox or in excel, I tested the import of forms from Kobo to QGIS. I see that skip logic (dependencies/constraints) are not integrated in the layer properties… also the possible answers in drop down lists, multiple choice questions, are not incorporated in the QGIS file. Am I missing something, or has this never been the plan?

Cheers,
Ruben

shivareddyiirs · April 9, 2020, 5:42pm

While importing the plugin it tries to edit layer and add features.
Please ensure your layer is not in editing mode before importing.

Even after doing above step, if import fails , please send the error message again.

By the way I would update code so that next time user would get message that layer is in editing mode.

Thanks for feedback.

Your feedback would make the kobotoolbox integration more stable.

shivareddyiirs · April 9, 2020, 5:52pm

The plan is to convert GIS layer ready for mobile data collection in single click.

Grouping attribute is possible in QGIS, I would try how can I translate to Xform. Good suggestion.

For advanced option like skip logic etc. the plugin is not suitable but irrespective of form design, data should be downloadable.

Feel free to suggest

Thanks

ruben_foquet · April 15, 2020, 2:09pm

Hi shivareddyiirs,

Thanks for the constructive feedback. I am happy I can contribute to this plug-in! The hint to avoid the layer being in edit-mode would be useful indeed. I assume that was my major constraint.

Grouping attributes would be useful in QGIS. I now decided to design forms in Kobotoolbox however, so that I can still use skip logic. I designed a simple form to start with (I have an xlsx form with the questions, though cannot upload it because I am a new user on this forum) but I am having some challenges: when filling in a test form in KoboCollect, I review the data in KoboToolbox on the DATA tab and all is fine. When I import the data in QGIS (after creating a new shapefile layer), not all the variables are showing up as fields and moreover, not all fields have values. I don’t understand what is going wrong.

I add a screenshot of the attribute info of one of my test points where you can see that 1) not all variables of the xlsx became fields and 2) quite a number of fields have no value (NULLL), though I did enter a value in KoboCollect which also shows up on the DATA tab of KoboToolbox.

It would be great to receive your feedback to potentially improve the plug-in or improve the procedures on how to work with it.

All the best,
Ruben

shivareddyiirs · April 22, 2020, 5:48pm

Hi,
Thanks for your feedback.

The problem is with the file format i.e. shapefile.

Shapefile don’t allow attribute name longer than 10 characters.

It means the name of all attributes should be smaller than 10 characters.

Solution:

Modify 'name**’** column’ in the survey to smaller value(less than 10 characters). It would not effect the user inteface in mobile.

Tip : Label corresponding to each field is shown to the user in mobile therefore if required longer description for any question it should be entered in ‘label’ while keeping name shorter.

ruben_foquet · April 21, 2020, 3:18pm

Hi!

Very useful feedback, thanks. The problem is solved. In my form design in KoboToolbox I edited the Data Column Name to contain names shorter than 11 characters:

It would be useful to write this in a notification somewhere, or develop a protocol?

Cheers,
Ruben

ruben_foquet · April 23, 2020, 6:07am

Hi,

I have now edited my form: I grouped all questions that belong together in KoboToolbox. Although my fields are shorter than 10 characters, I am not able to import it in QGIS. The only fields that pop up in QGIS are the metadata:

The error I receive is the following:

2020-04-22T11:58:51 WARNING Traceback (most recent call last):
File “C:/Users/MediaMonster/AppData/Roaming/QGIS/QGIS3\profiles\default/python/plugins\QRealTime\QRealTime.py”, line 278, in importData
service.importData(layer,selectedForm,True)
File “C:/Users/MediaMonster/AppData/Roaming/QGIS/QGIS3\profiles\default/python/plugins\QRealTime\QRealTime_dialog.py”, line 732, in importData
self.layer_name,self.version, self.geoField,self.fields= self.updateLayerXML(layer,xml)
File “C:/Users/MediaMonster/AppData/Roaming/QGIS/QGIS3\profiles\default/python/plugins\QRealTime\QRealTime_dialog.py”, line 757, in updateLayerXML
fieldType=attrib[‘type’]
KeyError: ‘type’

Any logical explanation for this behaviour?

All the best,
Ruben

shivareddyiirs · April 23, 2020, 6:10am

Hey,
Can you share the XForm or XLS form so that I can reproduce the error and investigate the problem.

It seems like a bug in the code

ruben_foquet · April 23, 2020, 6:25am

Sure, I add the XLS form in annex. For the moment, it remains the easiest for me to just export my data in Kobotoolbox as csv and import it in QGIS manually (for some reason he then allows fields with more than 10 characters?).

Thank you so much!
ExampleProject_Kobo_beekeeping.xlsx (10.3 KB)