Variable for csv filename in pull-data

Can a variable be used for the csv filename for pulldata?
When I use this (with variable fileName previously defined in survey):
pulldata(${fileName}, ‘plot_type’,‘plotid_key’,${PlotID})

the form deploys but gives the error ‘Failed to fetch’ when opened as though it doesn’t have the proper CSV filename.

However, it works without problem if I change it to use the static filename as in:
pulldata(‘myCsvFile’, ‘plot_type’,‘plotid_key’,${PlotID})

So, the csv file is not a problem, is properly formatted, can be loaded, and read.

I want to do this because I have 2 different csv files and which one I want to pull data from depends on a previous answer in my survey.

Welcome @nancyv,
Can you

  • check with a text variable or note what the ${filename} is
  • try to put it between ’ …’ or " … " in the pulldata function
1 Like

Thank you @wroos.

Yes, I have done both of those things:

  • The ${filename} variable is the name of the file (without the .csv extension - the same string that when put literally in the pulldata function works just fine).

  • Adding single (’…’) or double ("…") quotes in the pulldata function itself as in pulldata(’${filename}’,… or pulldata("${filename}",… results in the same ‘Failed to fetch’ error.

(Sorry, can’t attach example survey spreadsheet and data file because I am ‘new user’).

Hello @nancyv,
maybe @Kal_Lam can take over, please?

You might try with

  • pulldata( concat("’", ${filename}, “’”) ), … ), i.e. " ’ "
  • pulldata( jr://file-csv/path/to/${filename}.csv, …)

Ok thanks for the suggestions.

The first suggestion did not work. I tried it with single apostrophes (not ‘smart quote’ - right?)

The second suggestion didn’t work either but I am not sure I have the syntax correct. Where is the documentation on this?
I tried ‘pulldata( jr://file-csv/${filename}.csv, …)’ as I am not sure what the proper ‘path/to’ would be. My file lives in the ‘Media’ folder…
And, is concat not required as in:
pulldata(concat(‘jr://file-csv/’,${filename},’.csv’, …) or something similar?

Is this not a bug/issue where 'pulldatat(${filename}, …) isn’t being interpreted correctly? Can someone examine what is getting generated by the XLSForm to XForm converter from a simple example as in here ( I can provide a simple example when allowed by forum):

1 Like

Hello @nancyv,
here are the links
https://getodk.github.io/xforms-spec/#fn:pulldata
https://getodk.github.io/xforms-spec/#file-endpoints

You might

  • use your form and print it as pdf to see how ${filename} is replaced
  • show $filename}.in a note (or a textfield)
  • try another name, e.g. MyFileName

Hi @nancyv, @wroos, this is related to the discussion here about trying to include images dynamically.

Unfortunately, Enketo resolves the URLs (jr://...) on the backend and fetches the external data when the form is first loaded — therefore it needs to know the filename upfront. You can see this difference in the XML of the XLSForm, one with a dynamic filename and one fixed (example from the XLSForm docs on pulldata):

survey 1

type name label calculation
calculate filename ‘fruits’
calculate fruit pulldata(${filename}, ‘name’, ‘name_key’, ‘mango’)
note note_country The fruit ${fruit} is pulled csv data.

survey 2

type name label calculation
calculate fruit pulldata(‘fruits’, ‘name’, ‘name_key’, ‘mango’)
note note_country The fruit ${fruit} is pulled csv data.
@@ -14,10 +14,10 @@
           </meta>
         </aUJ4gsohDFa8wmxGiSs5Dg>
       </instance>
-      <instance id="${filename}" src="jr://file-csv/${filename}.csv"/>
+      <instance id="fruits" src="jr://file-csv/fruits.csv"/>
       <bind calculate="'fruits'" nodeset="/aUJ4gsohDFa8wmxGiSs5Dg/filename" type="string"/>
       <bind nodeset="/aUJ4gsohDFa8wmxGiSs5Dg/n" readonly="true()" type="string"/>
-      <bind calculate="pulldata( /aUJ4gsohDFa8wmxGiSs5Dg/filename , 'name', 'name_key', 'mango')" nodeset="/aUJ4gsohDFa8wmxGiSs5Dg/fruit" type="string"/>
+      <bind calculate="pulldata('fruits', 'name', 'name_key', 'mango')" nodeset="/aUJ4gsohDFa8wmxGiSs5Dg/fruit" type="string"/>
       <bind nodeset="/aUJ4gsohDFa8wmxGiSs5Dg/note_country" readonly="true()" type="string"/>
       <bind jr:preload="uid" nodeset="/aUJ4gsohDFa8wmxGiSs5Dg/meta/instanceID" readonly="true()" type="string"/>
     </model>

In the network tab of your browser’s dev tools, you can see that the form with a dynamic filename does not make a GET request for the data, but when the filename is declared upfront you can see:

Additionally, you can see the difference in how Enketo resolves these URLs in the response to the POST request made to /transform/xform:

@@ -14,5 +14,5 @@
             </meta>
         </aUJ4gsohDFa8wmxGiSs5Dg>
     </instance>
-    <instance id="${filename}" src="jr://file-csv/${filename}.csv"/>
+    <instance id="fruits" src="/media/get/https/kc.kobotoolbox.org/joshuaberetta/xformsMedia/970740/1564798.csv"/>
 </model>

So in summary, you will have to do something similar to what I recommended in the linked thread where you load all external data and then reference it inside the form in a dynamic way. Hope that helps :slightly_smiling_face:

2 Likes

Thank you @josh. You are correct that this does not work in Enketo and I have now been able to see that in the xml of the XLSForm. @wroos - it doesn’t matter what format used for the first arg of pulldata, it is used without translation.

I have worked around this for now using ‘if’ statements to pull from different files as suggested here:

This is really a workaround and not a solution as what I was looking for was a way to have my filename be set by default to a variable so I could easily change filenames. With this workaround, I still have the filename hardcoded in multiple/many places in my form.

I thought I saw elsewhere in the specs that variables could be used for all the args to pulldata…
Even if preloading, it seems that could be possible with a variable that is given a default value in the form (ie one that is not dependent upon user input).

BTW, I am not seeing /transform/xform in Chrome dev tools. Is this on form load?

1 Like