I hope this message finds you well. I am currently conducting a Best-Worst Scaling (BWS) experimental test that involves 6 different patterns (or versions) of a questionnaire. In a BWS, respondents are asked to select the best and worst options from a set of attributes. In this case, I have 6 different versions of the test, each with variations in the attributes presented.
I am writing to inquire whether it is possible to randomly assign one of these 6 patterns to each respondent during the survey. My goal is for each respondent to be shown a randomly selected version of the questionnaire, rather than all respondents seeing the same version.
I have developed an initial form, but I am facing challenges with setting up the randomization, and it is not working as expected. Could you please assist me in configuring this randomization in a tool like Kobo or using XLSForm?
I think I understand you goal here, and there are probably a few different approaches you could take depending on the subtle nuances (which I cant quite determineâŠ)
Perhaps the simplest approach (but excuse me if I got this wrong) is that you basically want to present a randomized list of options/attributes to each user when they fill in your form, from which they are to select their best and worst option. If thatâs the case, then I might suggest going with using randomized select options, as described here and here. The only extra thing you need is to use the same random seed number for both the âbestâ and âworstâ select questions, so they both show the options in the same randomized order.
Have a look at this form and see if you can adapt it to your needs. Each time a user runs the form it will generate a different seed number, which will result in the options being displayed in a different order [I also added a check to make sure they dont select the same option for both, but thatâs somewhat outside the scope of your description].
There are also other approaches you might take. For example, if you already have a predetermined distinct set of (6?) permutations of questions, and you want to display a different set each time, then you can put each permutation in a sub-group and then use a random number 1âŠ6 in the relevant calculation for each group to determine which group to show, hiding the other 5 groups.
Or if you just have a set of 6 questions that you simply want to display in a different random order every time you run the form, then you can do this as well; see my earlier post here: Randomizing the order of questions - Showcase - ODK Forum]
Thank you very much for your clarifications and continued support. I am pleased to inform you that I have resolved the issue with the first question, which requires the respondent/user to select the most important and least important attributes for each question.
However, I regret to inform you that I was unable to fix the random selection or generation of one pattern out of six different patterns (every time the respondent uses the questionnaire). (Please refer to the attached sheet I developed, which unfortunately isnât functioning yet) I would greatly appreciate it if you could kindly review and help fix this issue.
and that you want to randomly select one of these different option sets to present to the user each time [as opposed to randomly shuffling the order the same options, which my previous post was accomplishing]. Correct?
If thatâs the case, then as I alluded to you an put each of these different select questions - or in your case, best and worst pair of questions - in a group, and then use a random number to select which group to display. This all has to happen in your main form (not in the choices sheet). Have a look at the following form as to how you would go about setting this up.
Note, because some/most of your options themselves differ between each permutation, you will have to setup a different choice list for each: fruit1, fruit2, fruit3. Although each choice list could still share some common options, eg âHigh priceâ, they are still different lists of options. These choice-lists basically become your 6 permutations (Iâve only done 3 for brevity)
Then in the main form randomly select which group to display - each group containing a different select question(s) referencing a different corresponding choice list. Although the select questionsâ labels can obviously be the same, these are all fundamentally still different form questions.
Hopefully thatâs enough to point you in the right direction.
Thank you very much for your valuable feedback and support. I have gained a lot from your clarifications, and Iâm pleased to inform you that the form is working very well following your instructions.
However, I would like to confirm one detail. As you know, I have 6 patterns, each consisting of 6 questions with 6 predetermined attributes. I suppose that when I run the survey, each user should receive one complete pattern (i.e., one set of 6 questions with 6 options per question). Am I correct in understanding this?
I am concerned that the randomization might apply to the 6 attributes within each question, which I do not want. I only need the randomization to ensure that each user receives a different pattern (not randomizing the attributes within each question).
To clarify, I have attached an Excel file with the 6 patterns. Your confirmation and further assistance would be greatly appreciated.
Correct. The random number at the beginning will determine which of the 6 groups to display, So it will show whatever options are assocaited with that groupâs choice list.
I am concerned that the randomization might apply to the 6 attributes within each question,
Nope, it wont. As provided in the second form above, there is no randomization of the options themselves [other than whatever process you might have used to generate your 6 permutationsâŠ]. Again, the group to display is randomly selected; and that question(s) will display whatever (unrandomized!) options youâve specified for its select question(s).
Thank you very very much for your helpful clarification. Itâs reassuring to know that the randomization only applies to the selection of the group and not the attributes within each question. This clears up my concern about the potential randomization of options.
I truly appreciate your detailed explanation and support. Thanks a million!
I apologize for reaching out again, but your assistance has been immensely helpful. To prevent any issues when pasting this form into the full questionnaire (since the column titles for âcalculationsâ and âconstraint messagesâ are missing from the XLSForm of the complete questionnaire), could you please advise on how best to align the newly developed sheetâs titles with the existing structure? I want to ensure everything is properly harmonized for a smooth and successful redeployment.
I hope this message finds you well. I am pleased to reach out to you again and apologize for any inconvenience. I have updated my questionnaire in line with your instructions using XLSForm; however, I encountered errors when attempting to deploy it in KoboToolbox. I would appreciate your help in identifying and addressing these issues.
Additionally, I would like to ensure that the randomization function will operate correctly before deployment and field data collection. Specifically, I am unsure if the âpartnersâ (1 to 6) need to remain within the same choices sheet alongside other options or be separated, as seen in my questionnaire.
Though I am still familiarizing myself with KoboToolbox, I have made substantial progress over the past two months. Your patience and guidance would be invaluable to me in completing this step.
(test)- 18102024-Patterns Randomization_kobo.xlsx
