Introducing the R package robotoolbox

dickoa · February 23, 2023, 1:15pm

Thanks for your message, just to check if the error is linked to robotoolbox, can you install another package using remotes, let’s say the dm package (used by robotoolbox)

remotes::install_github("cynkra/dm")

Let me know if it works.
Thanks

Webex · February 23, 2023, 1:18pm

I received again the following error:

checking for file ‘C:\Users\xxx\AppData\Local\Temp\Rtmp02iuFI\remotes30f8173831c7\cynkra-dm-5ce497e/DESCRIPTION’ … OK
preparing ‘dm’:
checking DESCRIPTION meta-information … OK
checking for LF line-endings in source and make files and shell scripts
checking for empty or unneeded directories
building ‘dm_1.0.4.9003.tar.gz’
Warning: invalid package ‘C:/Users/janss131/AppData/Local/Temp/Rtmp02iuFI/file30f8626a73ac/dm_1.0.4.9003.tar.gz’
Error: ERROR: no packages specified
Warning: installation of package ‘C:/Users/xxx/AppData/Local/Temp/Rtmp02iuFI/file30f8626a73ac/dm_1.0.4.9003.tar.gz’ had non-zero exit status

dickoa · February 23, 2023, 1:26pm

You might have an issue with your system.
What OS and version of R are you using? Do you use onedrive (if OS = Windows)?

Webex · February 23, 2023, 1:52pm

OS: windows and R version: R-4.2.2. For R, I don’t use Onedrive.

~WRD000.jpg

dickoa · February 23, 2023, 2:00pm

Thanks, not sure exactly why you have this issue. I’m fishing here, but you can try to install rtools to make sure you can install packages from source (unless you have this version installed already)

https://cran.r-project.org/bin/windows/Rtools/rtools42/rtools.html

You install and restart the laptop/PC and try again to install robotoolbox.

I’m just waiting for an upstream bug on KPI already fixed on Github, but not yet on production. Once released, I’ll patch robotoolbox to release it to CRAN. You’ll have then a binary version of the package (no need to install from source).

Webex · March 1, 2023, 12:37pm

I had some trouble with R, but that has been solved now.

Now, I can come back to this problem. In file explorer of the home directory, I can find the 2 downloads for robotoolbox and cynkra/dm, but somehow R cannot detected/accesses the downloaded files even after installing rtools:

Very strange!

You mentioned: “I’m just waiting for an upstream bug on KPI already fixed on Github, but not yet on production. Once released, I’ll patch robotoolbox to release it to CRAN. You’ll have then a binary version of the package (no need to install from source).” Do you know when you might release the ‘robotoolbox’ package?

dickoa · March 1, 2023, 2:10pm

Hi @Webex

I don’t know when they’ll release the new KPI version on their production server (I think but I pretty soon). I’m sharing the issue if you’re curious

github.com/kobotoolbox/kpi

Allow non-owners to view asset versions

kobotoolbox:beta ← kobotoolbox:1164-asset-version-sharing

opened 03:48AM - 01 Nov 22 UTC

JacquelineMorrissette

+94 -8

## Checklist 1. [x] If you've added code that should be tested, add tests 2.… [x] If you've changed APIs, update (or create!) the documentation 3. [x] Ensure the tests pass 4. [x] Make sure that your code lints and that you've followed [our coding style](https://github.com/kobotoolbox/kpi/blob/master/CONTRIBUTING.md) 5. [x] Write a description of your work suitable for publishing on [our forum](https://community.kobotoolbox.org/tag/release-notes) 6. [x] Mention any related issues in this repository (as #ISSUE) and in other repositories (as kobotoolbox/other#ISSUE) 7. [ ] Open an issue in the [docs](https://github.com/kobotoolbox/docs/issues/new) if there are UI/UX changes ## Description Allows for users with the `View Asset` permission for the form to view the versions via API ## Related issues closes #1164

In the meantime, I’ll find a way to build binaries with the current devel version of robotoolbox before pushing to CRAN. I’m aiming for next week for a binary version of the package.

Best,
Ahmadou

Webex · March 2, 2023, 12:29pm

Thanks for sharing!

I have had assistance by a person who has more experience with R to solve the problem:

## install.packages("remotes")
remotes::install_gitlab("dickoa/robotoolbox")

First, he has installed Rtools in the proper way and followed by de-installing the packages devtools and remotes and re-installed them. If you already had installed the mentioned packages prior Rtools they may not detect Rtools and work with it.

MaddTitan · April 12, 2023, 8:21pm

This is truly amazing work @dickoa, much appreciated and will certainly be put to good use

jeanpo · May 18, 2023, 10:18pm

Hello Dan! thanks for this package, it pretty usefull, specially when you want to download multiple forms. I have a problem with the labels. In my first version of the form I used draft labels in a selec_one type of options. There are some test records with those old labels. Now I updated the form to a new version with the new labels, but the new records on the new form have the old labels. Is there a fix to this?

Correct labels:

jeanpo · May 18, 2023, 10:22pm

continue from my previous question

Old labels that still apply

Kal_Lam · May 19, 2023, 9:12am

Welcome to the community, @jeanpo! Pinging @dickoa here!

dickoa · May 19, 2023, 9:47am

Hi @jeanpo,

I hope all is good.
If understand correctly, for some select_one variables, the labels from the latest version of the form is not used. Is it possible to upgrade robotoolbox and all the packages used by robotoolbox, just to make sure that we’re working on the same version (1.2.9002).

# install.packages("pak")
pak::pkg_install("dickoa/robotoolbox")

Which version of KPI are you using?
The new released version comes with a fix allowing to pull info (including labels) from all version of the forms. More info here: Release Notes - version 2.023.12

If you still have this issue with the latest version of robotoolbox and KPI, I’ll try to reproduce the behavior on my own and work on a fix.

Thanks,
Ahmadou

jeanpo · May 19, 2023, 1:28pm

Hi @dickoa ! thanks for the answering so quick. I re installed the package with pak but I get the same issue How do I check the KPI? I dont know if that is the problem.

To further explain, it’s like you said, I have a select_one question and I changed the labels of the choices (but I didn’t change etiher namer o list_name). I.e: I changed “Acuerdo” to “Estoy de acuerdo”, but the level I get by downloading with the API is th eold one

dickoa · May 19, 2023, 4:03pm

If you’re using kf.kobotoolbox.org or kobo.humanitarianresponse.info, you probably have the latest version of KPI.
I created a form change_select_one_choices_v1.xlsx, collected 3 submissions and then changed the choices, and saved it as change_select_one_choices_v2.xlsx to collect 2 submissions.

library(tidyverse)
library(robotoolbox)
library(labelled)
uid <- "aswdwD1123rfcvssS13"
df <- kobo_data(uid)

glimpse(df)
## Rows: 4
## Columns: 16
## $ start                <chr> "2023-05-19T15:41:14.464-00:00", "2023-05-19T15:41:03.130-00:00", "…
## $ end                  <chr> "2023-05-19T15:41:24.887-00:00", "2023-05-19T15:41:14.447-00:00", "…
## $ today                <chr> "2023-05-19", "2023-05-19", "2023-05-19", "2023-05-19"
## $ full_name            <chr> "Aicha", "Fatma", "Ali", "Ahmadou"
## $ pet_type             <chr+lbl> "2", "4", "3", "1"
## $ vaccinated           <chr+lbl> "3", "2", "1", "2"
## $ `_id`                <int> 29098275, 29098272, 29098265, 29098264
## $ uuid                 <chr> "485f4004b28244228b674674a553e412", "485f4004b28244228b674674a553e4…
## $ `__version__`        <chr> "vTbjhgntR6d4h6ZvWahofi", "vTbjhgntR6d4h6ZvWahofi", "vKdkehR7oT9pse…
## $ instanceID           <chr> "uuid:f673e00c-682f-4e1c-bc10-0f21eb612de7", "uuid:6839306a-7ed…
## $ `_xform_id_string`   <chr> "a9Di9WtdgzLr6TZUMH4jWh", "a9Di9WtdgzLr6TZUMH4jWh", "a9Di9Wtdgz…
## $ `_uuid`              <chr> "f673e00c-682f-4e1c-bc10-0f21eb612de7", "6839306a-7ed6-486b-a661-5f…
## $ `_status`            <chr> "submitted_via_web", "submitted_via_web", "submitted_via_web", "sub…
## $ `_submission_time`   <chr> "2023-05-19T15:41:33", "2023-05-19T15:41:24", "2023-05-19T15:39:55"…
## $ `_validation_status` <int> NA, NA, NA, NA
## $ `_submitted_by`      <int> NA, NA, NA, NA

df$pet_type
## <labelled<character>[4]>: What's your favorite pet in this list?
## [1] 2 4 3 1

## Labels:
##  value  label
##      1  lapin
##      2 poulet
##      3  chien
##      4   chat
##      5 tortue

to_character(df$pet_type)
## [1] "poulet" "chat"   "chien"  "lapin"

As you can see (see attached), in the first form, I had chicken, cat, dog, turtle and rabbit but even for the 3 first submissions that used v1, I have the label of v2. I’m sure I’m missing something, but with this simple example, I can’t reproduce it.

change_select_one_choices_v1.xlsx (10.1 KB)
change_select_one_choices_v2.xlsx (10.1 KB)

jeanpo · May 19, 2023, 5:39pm

It could be something about the name or labels @dickoa ? Here I attach how I designed the choices (left side is the final form, right side are the old labels)

I found it weird too, because why it would apply the old labels even in the new submissions? Do I need to change something in settings?

dickoa · May 19, 2023, 6:13pm

@jeanpo Can you do

uid <- "xxxxxxxxxxxxxxxxxx"
form <- kobo_form(uid)
form |>
  filter(name == "var_name") |>
  pull(choices)

when I do it with my example, I have:

filter(form, name == "pet_type") |> pull(choices)

## # A tibble: 5 × 4
##   value_name value_label value_lang   value_version         
##   <chr>      <chr>       <chr>        <chr>                 
## 1 1          lapin       English (en) vTbjhgntR6d4h6ZvWahofi
## 2 2          poulet      English (en) vTbjhgntR6d4h6ZvWahofi
## 3 3          chien       English (en) vTbjhgntR6d4h6ZvWahofi
## 4 4          chat        English (en) vTbjhgntR6d4h6ZvWahofi
## 5 5          tortue      English (en) vTbjhgntR6d4h6ZvWahofi

Can you also do this:

kobo_asset_version_list(uid)
## # A tibble: 4 × 4
##   uid                    url   asset_deployed date_modified      
##   <chr>                  <chr> <lgl>          <dttm>             
## 1 vTbjhgntR6d4h6ZvWahofi http… TRUE           2023-05-19 15:40:40
## 2 vKdkehR7oT9psemeiE85ve http… TRUE           2023-05-19 15:39:28
## 3 vSmBN6KCy6rV6KbeGuH8g7 http… FALSE          2023-05-19 15:38:49
## 4 vQJkKVJLiPLQAFz2nbDXFQ http… FALSE          2023-05-19 15:38:48

Finally, what’s the output of this?

df <- kobo_data(uid)
df[["__version__"]]
table(df[["__version__"]])

jeanpo · May 19, 2023, 7:15pm

Sure! @dickoa
For the first test I get this:

# A tibble: 5 × 4
  value_name   value_label                       value_lang value_version
  <chr>        <chr>                             <chr>      <chr>
1 c_desacuerdo Estoy completamente en desacuerdo Labels     v3vXhLa2ybtehjJBSD5…
2 desacuerdo   Estoy en desacuerdo               Labels     v3vXhLa2ybtehjJBSD5…
3 neutral      Neutral al respecto               Labels     v3vXhLa2ybtehjJBSD5…
4 acuerdo      Estoy de acuerdo                  Labels     v3vXhLa2ybtehjJBSD5…
5 c_acuerdo    Estoy completamente de acuerdo    Labels     v3vXhLa2ybtehjJBSD5…

I notice that here I get the correct value_labels

For the second test:

kobo_asset_version_list(test_1)
# A tibble: 29 × 4
   uid                    url                 asset_deployed date_modified
   <chr>                  <chr>               <lgl>          <dttm>
 1 v3vXhLa2ybtehjJBSD5tS4 https://kf.kobotoo… TRUE           2023-05-19 17:29:43
 2 vRk5M94wQbA8zSe5i5yx6Z https://kf.kobotoo… TRUE           2023-05-18 15:45:44
 3 vQodqZjAoFxA4ZaWM6qDnQ https://kf.kobotoo… TRUE           2023-05-16 18:03:53
 4 v9hHzEBNqNSXau4sKVvyjV https://kf.kobotoo… TRUE           2023-05-16 13:54:06
 5 vCQgUYKVakzyiH8AuBVnbN https://kf.kobotoo… TRUE           2023-05-16 12:40:41
 6 v5ny6ZYYpXjvC6HC2wJagN https://kf.kobotoo… TRUE           2023-05-16 12:39:11
 7 vNPzMQWULexktsDxeTEMEd https://kf.kobotoo… TRUE           2023-05-15 18:30:27
 8 vET6wdEBub83gta9GDmu8v https://kf.kobotoo… TRUE           2023-05-15 18:28:52
 9 vGgPoBcKqK5gNZLqtpoc2W https://kf.kobotoo… TRUE           2023-05-15 18:27:10
10 vTX53FHrfYZrudBRcL8Bup https://kf.kobotoo… TRUE           2023-05-15 18:22:25

The versions list:

test_df[["__version__"]]
 [1] "vQodqZjAoFxA4ZaWM6qDnQ" "vQodqZjAoFxA4ZaWM6qDnQ" "vQodqZjAoFxA4ZaWM6qDnQ"
 [4] "vRk5M94wQbA8zSe5i5yx6Z" "vRk5M94wQbA8zSe5i5yx6Z" "vRk5M94wQbA8zSe5i5yx6Z"
 [7] "vRk5M94wQbA8zSe5i5yx6Z" "vRk5M94wQbA8zSe5i5yx6Z" "vRk5M94wQbA8zSe5i5yx6Z"
[10] "vRk5M94wQbA8zSe5i5yx6Z" "vQodqZjAoFxA4ZaWM6qDnQ" "vRk5M94wQbA8zSe5i5yx6Z"
[13] "vRk5M94wQbA8zSe5i5yx6Z" "vRk5M94wQbA8zSe5i5yx6Z" "vRk5M94wQbA8zSe5i5yx6Z"
[16] "vRk5M94wQbA8zSe5i5yx6Z" "vRk5M94wQbA8zSe5i5yx6Z" "vRk5M94wQbA8zSe5i5yx6Z"
[19] "vRk5M94wQbA8zSe5i5yx6Z" "vRk5M94wQbA8zSe5i5yx6Z" "vRk5M94wQbA8zSe5i5yx6Z"
[22] "vRk5M94wQbA8zSe5i5yx6Z" "vQodqZjAoFxA4ZaWM6qDnQ" "vRk5M94wQbA8zSe5i5yx6Z"
[25] "vRk5M94wQbA8zSe5i5yx6Z" "vRk5M94wQbA8zSe5i5yx6Z" "vRk5M94wQbA8zSe5i5yx6Z"
[28] "vRk5M94wQbA8zSe5i5yx6Z" "vRk5M94wQbA8zSe5i5yx6Z" "vRk5M94wQbA8zSe5i5yx6Z"
[31] "vRk5M94wQbA8zSe5i5yx6Z" "vRk5M94wQbA8zSe5i5yx6Z"
attr(,"label")
[1] "__version__"

And the table:

table(test_df[["__version__"]])

vQodqZjAoFxA4ZaWM6qDnQ vRk5M94wQbA8zSe5i5yx6Z

I got hype when the first test, so decided to try “to_factor” from labelled but got this:

> to_factor(test_df$relaciones_violencia_genero)
 [1] Acuerdo                     Acuerdo
 [3] Neutral                     Acuerdo
 [5] Completamente de acuerdo    Completamente de acuerdo
 [7] Desacuerdo                  Desacuerdo
 [9] Acuerdo                     Desacuerdo
[11] Desacuerdo                  Acuerdo
[13] Acuerdo                     Completamente de acuerdo
[15] Completamente de acuerdo    Acuerdo
[17] Completamente en desacuerdo Acuerdo
[19] Acuerdo                     Neutral
[21] Acuerdo                     Neutral
[23] Completamente en desacuerdo Desacuerdo
[25] Completamente de acuerdo    Completamente en desacuerdo
[27] Acuerdo                     Neutral
[29] Acuerdo                     Acuerdo
[31] Acuerdo                     Neutral
attr(,"label")
[1] Las relaciones en la escuela entre estudiantes hombres y mujeres es buena
5 Levels: Completamente en desacuerdo Desacuerdo Neutral ... Completamente de acuerdo```

The to_factor functions bring the old labels :(

abrahamreach · July 7, 2023, 10:55am

Hello,
Great Package.
I am trying to load the audit files through kobo_audit, and this error is showing:
Error in select():
! Can’t subset columns that don’t exist.
Column $autoname doesn’t exist.
Can someone explain?

dickoa · July 9, 2023, 11:15am

Hi @abrahamreach,

I hope all is good. Which version of the package are you using ?
Can you install the CRAN version and try again

install.packages("robotoolbox")

If it’s not working, I’ll see how to reproduce it and fix it. A minimal version of the xlsform would be useful to reproduce it.