Introducing the R package robotoolbox

Thanks for sharing!

I have had assistance by a person who has more experience with R to solve the problem:

## install.packages("remotes")
remotes::install_gitlab("dickoa/robotoolbox")

First, he has installed Rtools in the proper way and followed by de-installing the packages devtools and remotes and re-installed them. If you already had installed the mentioned packages prior Rtools they may not detect Rtools and work with it.

2 Likes

This is truly amazing work @dickoa, much appreciated and will certainly be put to good use

1 Like

Hello Dan! thanks for this package, it pretty usefull, specially when you want to download multiple forms. I have a problem with the labels. In my first version of the form I used draft labels in a selec_one type of options. There are some test records with those old labels. Now I updated the form to a new version with the new labels, but the new records on the new form have the old labels. Is there a fix to this?

Correct labels:

continue from my previous question

Old labels that still apply
image

Welcome to the community, @jeanpo! Pinging @dickoa here!

1 Like

Hi @jeanpo,

I hope all is good.
If understand correctly, for some select_one variables, the labels from the latest version of the form is not used. Is it possible to upgrade robotoolbox and all the packages used by robotoolbox, just to make sure that we’re working on the same version (1.2.9002).

# install.packages("pak")
pak::pkg_install("dickoa/robotoolbox")

Which version of KPI are you using?
The new released version comes with a fix allowing to pull info (including labels) from all version of the forms. More info here: Release Notes - version 2.023.12

If you still have this issue with the latest version of robotoolbox and KPI, I’ll try to reproduce the behavior on my own and work on a fix.

Thanks,
Ahmadou

1 Like

Hi @dickoa ! thanks for the answering so quick. I re installed the package with pak but I get the same issue :frowning: How do I check the KPI? I dont know if that is the problem.

To further explain, it’s like you said, I have a select_one question and I changed the labels of the choices (but I didn’t change etiher namer o list_name). I.e: I changed “Acuerdo” to “Estoy de acuerdo”, but the level I get by downloading with the API is th eold one

If you’re using kf.kobotoolbox.org or kobo.humanitarianresponse.info, you probably have the latest version of KPI.
I created a form change_select_one_choices_v1.xlsx, collected 3 submissions and then changed the choices, and saved it as change_select_one_choices_v2.xlsx to collect 2 submissions.

library(tidyverse)
library(robotoolbox)
library(labelled)
uid <- "aswdwD1123rfcvssS13"
df <- kobo_data(uid)

glimpse(df)
## Rows: 4
## Columns: 16
## $ start                <chr> "2023-05-19T15:41:14.464-00:00", "2023-05-19T15:41:03.130-00:00", "…
## $ end                  <chr> "2023-05-19T15:41:24.887-00:00", "2023-05-19T15:41:14.447-00:00", "…
## $ today                <chr> "2023-05-19", "2023-05-19", "2023-05-19", "2023-05-19"
## $ full_name            <chr> "Aicha", "Fatma", "Ali", "Ahmadou"
## $ pet_type             <chr+lbl> "2", "4", "3", "1"
## $ vaccinated           <chr+lbl> "3", "2", "1", "2"
## $ `_id`                <int> 29098275, 29098272, 29098265, 29098264
## $ uuid                 <chr> "485f4004b28244228b674674a553e412", "485f4004b28244228b674674a553e4…
## $ `__version__`        <chr> "vTbjhgntR6d4h6ZvWahofi", "vTbjhgntR6d4h6ZvWahofi", "vKdkehR7oT9pse…
## $ instanceID           <chr> "uuid:f673e00c-682f-4e1c-bc10-0f21eb612de7", "uuid:6839306a-7ed…
## $ `_xform_id_string`   <chr> "a9Di9WtdgzLr6TZUMH4jWh", "a9Di9WtdgzLr6TZUMH4jWh", "a9Di9Wtdgz…
## $ `_uuid`              <chr> "f673e00c-682f-4e1c-bc10-0f21eb612de7", "6839306a-7ed6-486b-a661-5f…
## $ `_status`            <chr> "submitted_via_web", "submitted_via_web", "submitted_via_web", "sub…
## $ `_submission_time`   <chr> "2023-05-19T15:41:33", "2023-05-19T15:41:24", "2023-05-19T15:39:55"…
## $ `_validation_status` <int> NA, NA, NA, NA
## $ `_submitted_by`      <int> NA, NA, NA, NA

df$pet_type
## <labelled<character>[4]>: What's your favorite pet in this list?
## [1] 2 4 3 1

## Labels:
##  value  label
##      1  lapin
##      2 poulet
##      3  chien
##      4   chat
##      5 tortue

to_character(df$pet_type)
## [1] "poulet" "chat"   "chien"  "lapin" 

As you can see (see attached), in the first form, I had chicken, cat, dog, turtle and rabbit but even for the 3 first submissions that used v1, I have the label of v2. I’m sure I’m missing something, but with this simple example, I can’t reproduce it.

change_select_one_choices_v1.xlsx (10.1 KB)
change_select_one_choices_v2.xlsx (10.1 KB)

1 Like

It could be something about the name or labels @dickoa ? Here I attach how I designed the choices (left side is the final form, right side are the old labels)


I found it weird too, because why it would apply the old labels even in the new submissions? Do I need to change something in settings?

@jeanpo Can you do

uid <- "xxxxxxxxxxxxxxxxxx"
form <- kobo_form(uid)
form |>
  filter(name == "var_name") |>
  pull(choices)

when I do it with my example, I have:

filter(form, name == "pet_type") |> pull(choices)

## # A tibble: 5 × 4
##   value_name value_label value_lang   value_version         
##   <chr>      <chr>       <chr>        <chr>                 
## 1 1          lapin       English (en) vTbjhgntR6d4h6ZvWahofi
## 2 2          poulet      English (en) vTbjhgntR6d4h6ZvWahofi
## 3 3          chien       English (en) vTbjhgntR6d4h6ZvWahofi
## 4 4          chat        English (en) vTbjhgntR6d4h6ZvWahofi
## 5 5          tortue      English (en) vTbjhgntR6d4h6ZvWahofi

Can you also do this:

kobo_asset_version_list(uid)
## # A tibble: 4 × 4
##   uid                    url   asset_deployed date_modified      
##   <chr>                  <chr> <lgl>          <dttm>             
## 1 vTbjhgntR6d4h6ZvWahofi http… TRUE           2023-05-19 15:40:40
## 2 vKdkehR7oT9psemeiE85ve http… TRUE           2023-05-19 15:39:28
## 3 vSmBN6KCy6rV6KbeGuH8g7 http… FALSE          2023-05-19 15:38:49
## 4 vQJkKVJLiPLQAFz2nbDXFQ http… FALSE          2023-05-19 15:38:48

Finally, what’s the output of this?

df <- kobo_data(uid)
df[["__version__"]]
table(df[["__version__"]])
1 Like

Sure! @dickoa
For the first test I get this:

# A tibble: 5 × 4
  value_name   value_label                       value_lang value_version
  <chr>        <chr>                             <chr>      <chr>
1 c_desacuerdo Estoy completamente en desacuerdo Labels     v3vXhLa2ybtehjJBSD5…
2 desacuerdo   Estoy en desacuerdo               Labels     v3vXhLa2ybtehjJBSD5…
3 neutral      Neutral al respecto               Labels     v3vXhLa2ybtehjJBSD5…
4 acuerdo      Estoy de acuerdo                  Labels     v3vXhLa2ybtehjJBSD5…
5 c_acuerdo    Estoy completamente de acuerdo    Labels     v3vXhLa2ybtehjJBSD5…

I notice that here I get the correct value_labels

For the second test:

kobo_asset_version_list(test_1)
# A tibble: 29 × 4
   uid                    url                 asset_deployed date_modified
   <chr>                  <chr>               <lgl>          <dttm>
 1 v3vXhLa2ybtehjJBSD5tS4 https://kf.kobotoo… TRUE           2023-05-19 17:29:43
 2 vRk5M94wQbA8zSe5i5yx6Z https://kf.kobotoo… TRUE           2023-05-18 15:45:44
 3 vQodqZjAoFxA4ZaWM6qDnQ https://kf.kobotoo… TRUE           2023-05-16 18:03:53
 4 v9hHzEBNqNSXau4sKVvyjV https://kf.kobotoo… TRUE           2023-05-16 13:54:06
 5 vCQgUYKVakzyiH8AuBVnbN https://kf.kobotoo… TRUE           2023-05-16 12:40:41
 6 v5ny6ZYYpXjvC6HC2wJagN https://kf.kobotoo… TRUE           2023-05-16 12:39:11
 7 vNPzMQWULexktsDxeTEMEd https://kf.kobotoo… TRUE           2023-05-15 18:30:27
 8 vET6wdEBub83gta9GDmu8v https://kf.kobotoo… TRUE           2023-05-15 18:28:52
 9 vGgPoBcKqK5gNZLqtpoc2W https://kf.kobotoo… TRUE           2023-05-15 18:27:10
10 vTX53FHrfYZrudBRcL8Bup https://kf.kobotoo… TRUE           2023-05-15 18:22:25

The versions list:

test_df[["__version__"]]
 [1] "vQodqZjAoFxA4ZaWM6qDnQ" "vQodqZjAoFxA4ZaWM6qDnQ" "vQodqZjAoFxA4ZaWM6qDnQ"
 [4] "vRk5M94wQbA8zSe5i5yx6Z" "vRk5M94wQbA8zSe5i5yx6Z" "vRk5M94wQbA8zSe5i5yx6Z"
 [7] "vRk5M94wQbA8zSe5i5yx6Z" "vRk5M94wQbA8zSe5i5yx6Z" "vRk5M94wQbA8zSe5i5yx6Z"
[10] "vRk5M94wQbA8zSe5i5yx6Z" "vQodqZjAoFxA4ZaWM6qDnQ" "vRk5M94wQbA8zSe5i5yx6Z"
[13] "vRk5M94wQbA8zSe5i5yx6Z" "vRk5M94wQbA8zSe5i5yx6Z" "vRk5M94wQbA8zSe5i5yx6Z"
[16] "vRk5M94wQbA8zSe5i5yx6Z" "vRk5M94wQbA8zSe5i5yx6Z" "vRk5M94wQbA8zSe5i5yx6Z"
[19] "vRk5M94wQbA8zSe5i5yx6Z" "vRk5M94wQbA8zSe5i5yx6Z" "vRk5M94wQbA8zSe5i5yx6Z"
[22] "vRk5M94wQbA8zSe5i5yx6Z" "vQodqZjAoFxA4ZaWM6qDnQ" "vRk5M94wQbA8zSe5i5yx6Z"
[25] "vRk5M94wQbA8zSe5i5yx6Z" "vRk5M94wQbA8zSe5i5yx6Z" "vRk5M94wQbA8zSe5i5yx6Z"
[28] "vRk5M94wQbA8zSe5i5yx6Z" "vRk5M94wQbA8zSe5i5yx6Z" "vRk5M94wQbA8zSe5i5yx6Z"
[31] "vRk5M94wQbA8zSe5i5yx6Z" "vRk5M94wQbA8zSe5i5yx6Z"
attr(,"label")
[1] "__version__"

And the table:

table(test_df[["__version__"]])

vQodqZjAoFxA4ZaWM6qDnQ vRk5M94wQbA8zSe5i5yx6Z

I got hype when the first test, so decided to try “to_factor” from labelled but got this:

> to_factor(test_df$relaciones_violencia_genero)
 [1] Acuerdo                     Acuerdo
 [3] Neutral                     Acuerdo
 [5] Completamente de acuerdo    Completamente de acuerdo
 [7] Desacuerdo                  Desacuerdo
 [9] Acuerdo                     Desacuerdo
[11] Desacuerdo                  Acuerdo
[13] Acuerdo                     Completamente de acuerdo
[15] Completamente de acuerdo    Acuerdo
[17] Completamente en desacuerdo Acuerdo
[19] Acuerdo                     Neutral
[21] Acuerdo                     Neutral
[23] Completamente en desacuerdo Desacuerdo
[25] Completamente de acuerdo    Completamente en desacuerdo
[27] Acuerdo                     Neutral
[29] Acuerdo                     Acuerdo
[31] Acuerdo                     Neutral
attr(,"label")
[1] Las relaciones en la escuela entre estudiantes hombres y mujeres es buena
5 Levels: Completamente en desacuerdo Desacuerdo Neutral ... Completamente de acuerdo```

The to_factor functions bring the old labels :(

Hello,
Great Package.
I am trying to load the audit files through kobo_audit, and this error is showing:
Error in select():
! Can’t subset columns that don’t exist.
:heavy_multiplication_x: Column $autoname doesn’t exist.
Can someone explain?

Hi @abrahamreach,

I hope all is good. Which version of the package are you using ?
Can you install the CRAN version and try again

install.packages("robotoolbox")

If it’s not working, I’ll see how to reproduce it and fix it. A minimal version of the xlsform would be useful to reproduce it.

1 Like

Great work @dickoa, congrats! This package help me a lot.

2 Likes

Quick question… When importing data using kobo_data(), is there a way to specify that a specific variable should be imported as a character and not numeric? I ask because, in my data, I have a question where I ask my enumerators to report questions not answered by respondents and to separate those numbers by commas (,). So for example, if the respondent did not answer questions 2,3 and 4, the entry for this question would be (2,3,4). However, in the import, kobo_data() interprets this as a numeric question and drops the commas (,) so that in the import, this shows as 234.

1 Like

Hi @fodhiambo,

That’s an interesting use case, let me see what can be done. The data is pulled from the API in json format and then parsed into a data.frame in R. I’ll see if during the JSON to data.frame we can allow certain data to be as character.
I’ll need a minimal form to reproduce the behavior to work on a fix. I’ll come back to you ASAP

1 Like

@dickoa, of course. Let me know how I can be of help on this. I am a new user(just newly registered :slight_smile:) , so cannot upload an xls form. But you can download here >>> xls_form.xlsx - Google Sheets

Can you try the dev version and tell me if it works as expected now ?

# install.packages("pak")
pak::pkg_install("dickoa/robotoolbox")

Don’t forget to restart your session with the new version of robotoolbox.

1 Like

@dickoa, thanks for this update; the import behaves as expected now. Once again, thank you so much

2 Likes

Thanks @fodhiambo for raising this issue, it’ll make the package more robust.

2 Likes