Calculate in dynamic repeat activated despite skip - giving error for online submission edit

Description

Calculate type inside a dynamic repeat (repeat_count) with a reference in relevant creates an error when the repeat is skipped. Case cannot be submitted online. So, also edit of submitted cases is no more possible.
The bug is found on new submission with Enketo and on re-editing/submit for data at the KoBo server (from Collect offline).

Steps to Reproduce

First example
Bug Edit01.xlsx (15.6 KB)

  1. Import and deploy XLSForm Bug Edit01
  2. Enter case with StopInterview != 2
  3. Submit case >> Error blocks flow

2nd Example (here we added even a redundant skip at the calculate)
Bug Edit02.xlsx (15.3 KB)

  1. Import and deploy XLSForm Bug Edit02
  2. Enter case with StopInterview != 2
  3. Submit case >> Error blocks flow

If at step 2 you
2. Enter case with StopInterview = 2
It works fine and you can submit.
But if you EDIT the submitted data case and set StopInterview != 2 it blocks with error like above.

Expected behavior

Skipped parts of forms and dynamic repeats should not create errors. Calculate should behave similar to integer.

Actual behavior

Example 1
Capture01

Example 2
Capture02

Additional details

The bug blocks our data cleaning process after a comprehensive survey. We urgently need to find a workaround.

All was fine with ODK offline work and submissions.

It works, when

  • the variable type is integer instead of calculate
  • the repeat_count is fixed (like 1) instead of dynamic ( ${HHMTotal} ).

In the second example, the calculate even has an additional (redundant) skip and the reference in the relevant addresses a simple variable before the repeat. Also, required true or not or additional grouping didn’t change the problem.

Configuration: OCHA Server, current Enketo and KoBoToolbox version, XLSForm, ODK Collect 1.29.3 (or 1.29.4) on Android smartphones.

1 Like

What are you using the stopinterview variable for, in terms of your process flow?

It’s just an extract of a comprehensive Household survey. For QA of listing/sampling, coverage … we also register basic data for non-contact, refusal etc. cases. Those cases skip almost all household and Individual modules and go to a finalisation and QA (Supervisor) module.
But for the KoBo problem above, feel free to substitute this by any other skip variable or even remove it and stay with the count filter alone.

Not sure if I understand the process properly. But can you not use the Validation feature instead? You could have control from the back-end to only accept the validation approved entries.

Hello,
Don’t worry about the process here. It’s not important for the bug report.

We are working in very remote areas, so data on server level comes to late. Often even telephone is not working, so centralised QA feedback is not an option. We work offline with supervisors in the enumuration teams. The supervisor checks the data before they are sent, which is an advantage as offline edit/correction is still possible. Also, send data are fully anonymised already.

Kind regards

@wroos, is this an issue only with Enketo or is this an issue with Collect android app too?

Collect (ODK) works fine. It’s Enketo & Toolbox (Edit/submit) submission

1 Like

Well it seems to be an issue with Enketo. Will go through it in detail and proceed accordingly.

@wroos
Is it running on your own server? If so, is it an older version?
I’m facing some form logic errors on Enketo too.

No.

Regards

This probably needs to be brought to attention of @martijnr

2 Likes

Hello @Kal_Lam., @martijnr,
Could you provide any info on the status of the issue, please. (GitHub?)

We cannot clean/edit submitted data, based on this bug. Maybe there is at least advice for a workaround, like using a full path reference?

Kind regards

I think this may have been an old pyxform bug (even though it doesn’t occur in Collect) or an old Enketo bug. It doesn’t seem to occur when using the latest published version of both (I haven’t tested old versions).

If it can be reproduced when using the latest version of both, please file an issue in enketo-core (KoBoToolbox QA, @Kal_Lam ).

1 Like

Hello @martijnr. @Kal_Lam,

That’s what we used (see above). Also, for the example provided.
Do you have an idea for a workaround. As we are blocked cleaning survey data (on server level).
Kind regards
Kind regards

That’s what we used

No, I don’t think so. Professional adopters of the Enketo and pyxform tools, such as KoBoToolbox, are always a little behind and should be because they do their own QA.

Not sure if pyxform version is visible anywhere but you can check the Enketo (Express) version by going to the root Enketo url, e.g. https://ee.humanitarianresponse.info/.

The Enketo Express releases are published here: enketo-express/CHANGELOG.md at master · enketo/enketo-express · GitHub

Hello @Kal_Lam. @martijnr,
Thanks for the answer and the links. We are normal KoBoToolbox and ODK users, using the latest system releases as they are available to the related public. And we try to report problems and errors based on this, here in the community.

As far as we can see - without digging into GitHub with about 30 source elements, sorry - the Enketo version available in KoBo is at the moment


This is the same as on the root Enketo url, e.g. https://ee.humanitarianresponse.info/ .

This seems to be two versions behind master Enketo


We cannot see in the change look, if the bug was solved in one of the new updates.
I am sorry, I didn’t well understood: Did Martin test this example already without problems? (2.6,1? 2.6.0?)

The support demand is, please:

  1. We found a bug in using Enketo from KoBoToolbox. See example.
  2. We reported it here, incl. a reproducible example (28-01-20)
  3. We had to stop (time-critical) data cleaning of a survey, based on this problem
  4. We are looking for a solution or at least a workaround, please.

Let us understand, please, who should do what now to advance?

Thanks in advance.
Kind regards

1 Like

30/01/21

Any news on this, please?

A useful piece of information here is that previewing these forms works just fine; it’s not until they’re deployed and opened for data collection that the problem appears. This points to an issue with the way KoBoCAT is using pyxform.

I tried upgrading Enketo to 2.6.1 while leaving KPI and KoBoCAT unchanged, and the behavior is identical: previewing works fine, but a form opened for data collection shows the errors.

3 Likes

Hi @martijnr, I’m sorry to report that there’s still a problem with this form when testing on a freshly-installed ODK Central instance:

As I mentioned in my last post, this is an odd one because:

It’s only the act of submitting that triggers the error.

I should note that Central is shipping Enketo 2.5.6 currently, but seeing as XML-generation discrepancies are now out of the equation—and I’ve already tested 2.6.1 with KoBoToolbox—I’m not attempting to upgrade Central’s Enketo at the moment.

My Central instance is running getodk/pyxform-http:v1.3.4, which uses pyxform 1.3.4. I’ll include the XML below, which was generated from Bug Edit01.xlsx in the first post.

<?xml version="1.0"?>
<h:html xmlns="http://www.w3.org/2002/xforms" xmlns:ev="http://www.w3.org/2001/xml-events" xmlns:h="http://www.w3.org/1999/xhtml" xmlns:jr="http://openrosa.org/javarosa" xmlns:odk="http://www.opendatakit.org/xforms" xmlns:orx="http://openrosa.org/xforms" xmlns:xsd="http://www.w3.org/2001/XMLSchema">
    <h:head>
        <h:title>Bug Edit01</h:title>
        <model odk:xforms-version="1.0.0">
            <itext>
                <translation default="true()" lang="English (en)">
                    <text id="/data/StopInterview:label">
                        <value>StopInterview</value>
                    </text>
                    <text id="/data/HHMTotal:label">
                        <value>How many persons are usually living in this household, in TOTAL?</value>
                    </text>
                    <text id="/data/IV_RP:label">
                        <value>INDIV</value>
                    </text>
                    <text id="/data/IV_RP/WorkPayment:label">
                        <value>What was the value of NAME's last PAYMENT (income) for work activities, in cash and kind? Total In USD</value>
                    </text>
                    <text id="/data/IV_RP/DaysPayment:label">
                        <value>How many DAYS in total did NAME work for this payment?</value>
                    </text>
                    <text id="/data/IV_RP/WorkPaymentPerDay:label">
                        <value>WorkPaymentPerDay</value>
                    </text>
                    <text id="/data/IV_RP/WorkPaymentChk:label">
                        <value>CHECK : NAME earned about <output value=" ../WorkPaymentPerDay "/> USD per day of work? Is this correct?</value>
                    </text>
                </translation>
            </itext>
            <instance>
                <data id="Bug_Edit01" version="2021012805">
                    <StopInterview/>
                    <HHMTotal/>
                    <IV_RP_count/>
                    <IV_RP jr:template="">
                        <WorkPayment/>
                        <DaysPayment/>
                        <WorkPaymentPerDay/>
                        <WorkPaymentChk/>
                    </IV_RP>
                    <IV_RP>
                        <WorkPayment/>
                        <DaysPayment/>
                        <WorkPaymentPerDay/>
                        <WorkPaymentChk/>
                    </IV_RP>
                    <meta>
                        <instanceID/>
                    </meta>
                </data>
            </instance>
            <bind nodeset="/data/StopInterview" type="int"/>
            <bind nodeset="/data/HHMTotal" relevant=" /data/StopInterview  = 2" required="true()" type="int"/>
            <bind calculate=" /data/HHMTotal " nodeset="/data/IV_RP_count" readonly="true()" type="string"/>
            <bind nodeset="/data/IV_RP" relevant=" /data/HHMTotal  &gt; 0 and  /data/StopInterview  = 2"/>
            <bind nodeset="/data/IV_RP/WorkPayment" required="true()" type="int"/>
            <bind nodeset="/data/IV_RP/DaysPayment" required="true()" type="int"/>
            <bind calculate="round( ../WorkPayment  div  ../DaysPayment , 0)" nodeset="/data/IV_RP/WorkPaymentPerDay" relevant=" ../DaysPayment  &gt; 0" type="string"/>
            <bind nodeset="/data/IV_RP/WorkPaymentChk" required="true()" type="string"/>
            <bind jr:preload="uid" nodeset="/data/meta/instanceID" readonly="true()" type="string"/>
        </model>
    </h:head>
    <h:body>
        <input ref="/data/StopInterview">
            <label ref="jr:itext('/data/StopInterview:label')"/>
        </input>
        <input ref="/data/HHMTotal">
            <label ref="jr:itext('/data/HHMTotal:label')"/>
        </input>
        <group ref="/data/IV_RP">
            <label ref="jr:itext('/data/IV_RP:label')"/>
            <repeat jr:count=" /data/IV_RP_count " nodeset="/data/IV_RP">
                <input ref="/data/IV_RP/WorkPayment">
                    <label ref="jr:itext('/data/IV_RP/WorkPayment:label')"/>
                </input>
                <input ref="/data/IV_RP/DaysPayment">
                    <label ref="jr:itext('/data/IV_RP/DaysPayment:label')"/>
                </input>
                <trigger ref="/data/IV_RP/WorkPaymentChk">
                    <label ref="jr:itext('/data/IV_RP/WorkPaymentChk:label')"/>
                </trigger>
            </repeat>
        </group>
    </h:body>
</h:html>
3 Likes

@Josh, there are a few “context node” issues in enketo-core, but I’m not sure if they’re related: Issues · enketo/enketo-core · GitHub.
Could you please have a look and file a new issue if needed? Discourse ought to let you see the Markdown of my previous post, which you could more-or-less paste into GitHub if a new issue ends up being the way to go. Thank you.

2 Likes