Kobo Form to ArcGIS Online Feature Service

User Support Data Management
, ,
#1

Hi,

Looking for ideas around uploading data to my arcgis online account. At the moment I have designed a questionnaire and am looking to create a feature service and then have the data populated whilst data is entered in the kobo form online (Enketo). Anyone that has had experience uploading kobo form data to ArcGIS online directly please feel free to comment.

#2

Hi @Anare,

Welcome to the community! Have not used it but there should be a way-out like you pull data to excel power query as outlined in the support article Pulling Data Into Excel Power Query or pulling data to power bi as outlined in the support article Pulling Data Into PowerBI. These two support article should give you some clue on how it’s possible to link data from KoBoToolbox.

Have a great day!

1 Like
#3

Hi @Kal_Lam,

It would be great if ArcGIS Online had the same simple add url and enter password details section. But I have not seen that functionality yet. Am remaining hopeful that ArcGIS online developers would work out a option for this as is in PowerBI.

Thanks again for providing feedback on this Kal.

Cheers

Anare

1 Like
#4

Hi @Anare
Though we do not offer support on ArcGIS, I would suggest that you look at the following articles and see if they fit the purpose you need

https://pro.arcgis.com/en/pro-app/help/mapping/time/visualize-data-updating-using-live-mode.htm

Once you figure out the ArcGIS component, we can then look at the KoBoToolbox component.

Regards,
Stephane

1 Like
#5

Hi, recently I’ve used a Jupyter Notebook script (Python) to get data from KoBo via API and upload directly to ArcGIS Online using ArcGIS API. Notebooks were hosted and run in the Azure DataBricks but Esri is now offering Jupyter Notebooks too. The benefit of DataBricks was that we could easily add notebooks to a scheduler and run e.g. every hour or day to have near real-time data from KoBo in ArcGIS Online. Very simple and powerful!

1 Like
#6

Hi @janekb,

Thank you for sharing this with the entire community! :star_struck: This should help a lot of our community users!

Expecting your knowledge and expertise in the upcoming days as well.

#7

I’d really like to learn more about your architecture to implement something similar. Can you share your code, solution or some more technical documentation? Thank you.

1 Like
#8

Sure. Basically you need to:

  1. set up a new Notebook in DataBricks

  2. import ArcGIS library (unless you’ve already installed it in your cluster)
    dbutils.library.installPyPI("arcgis","1.6.2")

  3. log in to ArcGIS Online and get the layer you want to update (or create a new one if needed)
    #connect to GIS
    from arcgis.gis import GIS
    from arcgis.features import FeatureLayerCollection
    gis = GIS(url,user,passwd)
    item = gis.content.get(uuid)
    flayer_collection = FeatureLayerCollection.fromitem(item)

  4. Then use HTTP request to get data from Kobo:
    import requests
    from requests.auth import HTTPBasicAuth
    response = requests.get(url, auth=HTTPBasicAuth(user, pwd))

  5. Save the response into a CSV

  6. Call the overwrite() method from AGOL to update the layer:
    flayer_collection.manager.overwrite('file.csv')

Note: your CSV file from KoBo must match data schema of your feature layer in ArcGIS Online. If this is not the case, you need to manipulate the data before overwriting.

Hope that helps!

2 Likes
#9

Thanks @janekb for the amazing share

1 Like
#10

I honestly have no experience in DataBricks but will attempt to follow the steps. Thanks again @janekb. Not sure if possible but would you @janekb be willing to have a zoom call for the steps to follow?

1 Like
#11

Hi, as a first step I’d suggest getting an environment for hosting Jupyter Notebooks. This can be on you local computer or using one of the cloud platforms. Have a look here: https://www.dataschool.io/cloud-services-for-jupyter-notebook/
Hope that helps!

1 Like