How to filter choices by distance to geopoint?

Hey Community,
I’m trying to build an XLSForm for a small and simple survey. I want to collect data from every household in a district of a city. However, I don’t want to manually enter the street and house number every time I visit the next house. Instead, I want to set up a simple reverse address lookup.

To achieve this, I retrieve street names, house numbers, latitude, and longitude from Overpass Turbo and store them in the choices sheet.

Using a select_one question with a choice_filter, I aim to filter addresses near my current geolocation and select one from the list.

However, KoboToolbox does not accept my XLSForm. I have several versions that pass validation and are working correctly on ODK’s XLSForm validator, and some of them can be uploaded to KoboToolbox. But when I try to deploy the form, I always receive error messages like:

ODK Validate Errors: >> Something broke the parser. See above for a hint. XPath evaluation: cannot handle function 'distance' requires a field as the parameter. The following files failed validation: ${tmp5gn7did0} Result: Invalid

Can someone help me understand this issue? (I’m not allowed to upload a minimum example :frowning:)

Minimum Example:

survey-tab

type name label choice_filter
start start
geopoint location My Location
select_one address select_address Select Address distance(concat(${location},‘;’, coordinates)) <= 30
end end

choices-tab

list_name name label coordinates
address 10983245794 Am Paradeplatz 12 49.3864 8.6856089 0 0
address 10983245795 Am Paradeplatz 14 49.3865899 8.6855765 0 0
address 10983245796 Am Paradeplatz 16 49.3870185 8.6855279 0 0
address 10284427944 Am Paradeplatz 18 49.3872033 8.6854968 0 0

Sorry for the delay, this took a little investigation…

The good news is that you did everything right! Yes, your form is correct and will properly filter the 4 displayed location options based on how close they are to your current location. As you may have seen already, this works as expected if you run it against the XLSForm Online validator.

However, as you observed, if you try to deploy your form in Kobo you will get a validation error.

However, both Enketo and KoboCollect will quite happy consume a form today with a distance() function that has a non-field parameter (ie something different than ${location}, eg a calculation, like yours…)

What is happening here is that when you deploy an XLSForm form in Kobo it passes it thru pyxform to translate your original XLSForm spreadsheet into a raw XForm XML definition that the client - Enketo or KoboCollect - will actually consume. A number of checks are performed during this translation, and if any of them fail your form will fail to deploy with a validation error.

Interestingly, the validation check that is picking up this particular is a result of pyxform performing an initial pass thru of all the calculations in the form, just to see if any of them barf. Under normal circumstances, these calculations run continuously against any changes made when the user enters data into the form; so for this initial pass pyxform has to rely on the initial XForm XML instance data only, most of which are empty (unless you happen to have any default values specified in your form). And to perform these calculations, pyxform has to use javarosa, which is the XForm/XPath engine inside ODK Collect/KoboCollect. [Enketo uses a different XPath engine than javarosa, but for the most part they perform equivalently].

In this case, although both Enketo and KoboCollect’s XPath engine handle this flavor of the distance() function happily, Kobo is currently using a slightly earlier version of pyxform, which is correspondingly using a slightly earlier version of javarosa to run these calculations. And it is this earlier version of javarosa that doesnt yet support non-field arguments to the distance() function. Specifically, these were add in May 2024; here’s the PR: Add support for single string argument · getodk/javarosa@fb3557b · GitHub

So what is happening is Kobo is running pyxform to translate your XLSForm into an XForm for deployment, running validation to check for any errors, which is running javarosa once (only) against your choice_filter calculation (and using the initial empty form data to do so), which barfs on your distance() function, meaning validation fails - a false negative in this situation - meaning you can’t even deploy your form!

Fortunately, there is a trick to get around this (!). It relies on the fact that this pre-deployment javarosa calculation check relies on using the initial default form values to run these calculations [since you havent started filling in the form yet…]. So by ensuring that your choice_filter calculation is OK against default (ie null) values, you cant trick javarosa to not run it against real values(!). To accomplish this, I changed your form to

Initially ${location} will be empty until the user enters their location. Hence ${location} is null when pyxform runs its initial check, so javarosa just evaluates false() - which is perfectly valid - and never gets to see the new distance() function! [I also happen to use a slightly simpler form of the distance function that takes 2 geopoints, but that’s unrelated to the root issue]. Have a play with this new version of the form. I’ve been able to successfully deploy and run it in Kobo, both against Enketo and KoboCollect:

DistanceFilter2.xlsx (18.3 KB)

FYI these acrobatics will go away once Kobo updates to the latest version of pyxform, which will happen soon. Till then, you will have to perform trickery like this to fool our current version of pyxform into avoiding running its older version of javarosa against the new flavor(s) of distance() during its pre-deployment validation step.

[FWIW I knew fairly quickly what the problem was, but it took a bit to figure out how to work around it, for a choice_filter especially. Hence the delay. :wink:]

2 Likes

Ok, wow! Thanks a lot! I’m still pretty new to KoboToolbox and didn’t expect to dive so deep into the details with my very first project. I would have never figured this out on my own!

Maybe this example will help others set up a simple reverse address lookup using open data. :smiley:

So, thanks again for your time and help!

1 Like

I guess you got ‘lucky’… :wink:

Yours is a good example of how to do location-based filtering of data that is geo-referenced. So you did a good job getting it basically right the first time! Thanks for posting. :+1:

1 Like