Introducing the R package robotoolbox

User Support Data Management
, , ,
1

Dear KoBo and R users,

Happy new year! I wanted to share with you a work-in-progress R package to access KoBoToolbox using v2 API: robotoolbox
I would be grateful if you can give it a try and provide feedbacks on how to improve it, particularly if something is not working as expected.

I didn’t implement unit tests yet and some features might not be available if you have a project with multiple form versions where questions and/org group names changed during the course of the project.

The main function of robotoolbox is kobo_data or kobo_submissions (alias) and to get the data you need to setup your account and the use your asset uid to read the data.

You’ll need to install this package using the remotes R package

## install.packages("remotes")
remotes::install_gitlab("dickoa/robotoolbox")

The first step is to get your API token using the kobo_token function.

library(robotoolbox)
token <- kobo_token(username = "cool_user_name", password = "myP@$$Word", 
                    url = "https://{kpi-url}/")

With the token (that you can also get from your account) you can now setup the robotoolbox package.

kobo_setup(url = "https://{kpi-url}", token = token)
kobo_settings() ## just to check
## <robotoolbox settings>
##    KoBoToolbox URL: https://{kpi-url}/
##    KoBoToolbox API Token: xxxxxxxxxxxxxxxxxxxxxxxxxx

Now you can access all your projects and read the data from the project you picked. You can list project using kobo_asset_list().

library(dplyr)
l <- kobo_asset_list() 
glimpse(l) # l is a data.frame with as many rows as projects
## $ uid            <chr> "b9kgvd7AXQCmo5qyUOBEl", "aRfJMpTSGRLzZ…"
## $ name           <chr> "Proj_A1", "Proj_A2", "Proj_A3", "Proj_A…"
## $ asset_type     <chr> "survey", "survey", "survey", "survey", …
## $ owner_username <chr> "xxxxxxxxxxxxxx", "xxxxxxxxxxxxxxx", "xx…"
## $ date_created   <dttm> 2020-04-27 20:34:23, 2020-04-27 21:21:1…
## $ date_modified  <dttm> 2021-06-17 01:52:57, 2021-06-17 01:52:5…
## $ submissions    <int> 2951, 2679, 2, 1, 0, 0, 287, 73, 0, 274,…

Let’s pick the first project and load it

uid <- l$uid[1]
asset <- kobo_asset(uid)
asset
## <robotoolbox asset>  b9agvd9AXQCmo5qyUOBEl
##   Asset Name: proj_A1
##   Asset Type: survey
##   Created: 2021-05-10 07:47:53
##   Last modified: 2021-08-16 12:35:50
##   Submissions: 941

With the asset you can now read your data

data <- kobo_data(asset)
glimpse(data)
## Rows: 941
## Columns: 17
## $ id                                                         <int> …
## $ start                                                      <dttm> …
## $ end                                                        <dttm> …
## $ today                                                      <date> …
## $ deviceid                                                   <chr> …
## $ test                                                       <chr+lbl> …
## $ round                                                      <date> …
## $ effective_date                                             <date> …
## $ collect_type                                               <chr+lbl> …
## $ covid_module                                               <chr+lbl> …
## $ country                                                    <chr+lbl> …
## $ interviewer_id                                             <chr> …
## $ respondent_is_major                                        <chr+lbl> …
## $ consent                                                    <chr+lbl> …
## $ admin_level_1                                              <chr+lbl> …
## $ admin_level_2                                              <chr+lbl> …
## $ admin_level_3                                              <chr+lbl> …

This package relies on the R package labelled to read labels into R. You can learn more here

Another feature, is the use of dm to handle repeating groups. More info in this vignette:

Still working on both the package and its documentation and again would love hear from you.

Best,
Ahmadou

11 Likes
2

Welcome back to the community, @dickoa & thank you for sharing this with the entire community! :clap: :heart: :books:

3

Wow, congrats.

2 Likes
4

Awesome, thanks. You can reach me at “mail at ahmadoudicko dot com”

1 Like
5

awesome, thanks a lot. Great package, I am downloading all the surveys (over 50) every day like this (see below). Any ideas on how to automatically import numbers as numbers and not characters?

get data from KoBo server

token = “xxxxxxx”

kobo_setup(url = “https://kobo.humanitarianresponse.info”, token = token)
kobo_settings()

surveys ← kobo_asset_list()

for (i in 1:length(surveys$uid)) {

print(i)
tryCatch( {
xx=kobo_data(surveys$uid[i])
assign(surveys$name[i],xx,pos = “.GlobalEnv”)
}
, error = function(e) {an.error.occured <<- TRUE})

}

6

Hi @titeuf_community

Can you clarify, what do you mean importing numbers as numbers?

Thanks

1 Like
7

I mean with numbers “as numeric”, basically what this function does: as.numeric()… I mean this is probably not very related to your package but would be easier if I can directly set this while importing the data.

8

You have columns in your data that are supposed to be numbers but are characters ?

1 Like
9

@titeuf_community do you want select_one with values coded as integer to behave like numeric values in R ? Can you share your form and run str(data) with data being the associated data loaded in R.

1 Like
10

thank you for your answer. I have realized that I accidentally coded the “don’t know” with “dont_know” instead of a number. so everything fine, I am very sorry about this… :slight_smile:

1 Like
11

I’m glad you sorted it out. I made some minor changes recently, don’t hesitate to upgrade.
Feedbacks are welcome on the package and its overall usability.
Thanks

1 Like
12

Hi there. Thank you for this package. It’s very useful. Unfortunately, for “specify other” type of questions, don’t get downloaded. Is there a way to fix this? Thank you for your help

1 Like
13

Hi @smaz

Do you have a minimal xlsform I can use to check? I have never noticed it, but if we fix it, I’ll add it to the unit test.

Thanks,
Ahmadou

1 Like
14

Hi @dickoa has there been any changes to this. The installation of remotes have worked however I get error in loading “robotoolbox”

Find picture below

image

15

Hi @nattie
It looks like the installation failed for some reason. Can you re-start R or try to re-install, then re-install it, and paste the warnings/error you have (if any)?

remotes::install_github("dickoa/robotoolbox")

Best,

1 Like
17

Hi @dickoa, thank for your interesting package. I have a question. In your tutorial, the result will give a data frame include date format. But in my result, it doesn’t have date format. Which argument do I need to specify in your function to have date format, or I will have to transform the data manually after read data to R.
Cheers,
Manh

18

Hi @manh_nguyen

It should normally turn it into date, but it’ll be great if I can have a look at the format of your date.
Can you share the format? e.g also the part of the xlsform where we have the date.

1 Like
19

tmp_form.xlsx (16.7 KB)
thanks for your quick response. here is my form and below the screen shot of the data i read to R. even the first 2 columns are not in date format like your tutorial.

Screen Shot 2022-12-16 at 20.37.25
Screen Shot 2022-12-16 at 20.37.25860×676 35.9 KB

20

Can you re-install it the package, I just pushed some changes to have dates back.
Let me know if it works.
Best,

1 Like
21

Fantastic @dickoa. one more favor if you can set up for the select multiple question can be read in 1 single column.
Thank you a lot for saving my time working on Kobotoolbox.
Cheers,
Manh

1 Like