Release Notes - version 2.022.08, 2.022.08a

Hello, everyone. We’ve just launched a new release (effective at 21:00 UTC on 17 March 2022) on both and

Components comprising this release:

  • KPI version 2.022.08
  • KoBoCAT version 2.022.08
  • Enketo Express version 2.8.1
  • Pyxform version 1.7.0

Security Fix

This release includes a fix for an issue discovered by a recent security audit. If the following conditions were all met:

  • A project allowed anonymous submissions;
  • A person knew of the existence of that project;
  • That person knew the UUID of an existing submission (which contains 32 random hexadecimal digits);
  • That person submitted specially-crafted XML referencing that UUID;

then that person could overwrite data in the existing submission identified by that UUID.

This release fixes the problem by ensuring that no kind of anonymous request can ever modify previously submitted data.

We encourage anyone who manages a public instance of KoboToolbox to upgrade as soon as possible. If you need assistance, please open a topic in the “Kobo On Your Own Server” category. Thank you.

New Features

Czech Translation

Thanks to our volunteer translators, the KoboToolbox user interface is now available in Czech! If you would like to make KoboToolbox accessible to more people around the world by adding more translations or helping the existing ones stay up-to-date, please refer to this forum post to get started.

Improved Submission Editing

There are two notable changes to submission editing:

  1. Attachments are now available to download from within Enketo while editing;
  2. A submission is now opened with the same version of the form that was used to submit it in the first place, not the latest version of the form. [This has been reverted temporarily because it broke some people’s workflows. A future release will include a user interface for choosing which version to use when editing.]

This was implemented in #3689 and #3707 and was tracked by #3654.

Synchronous Exports (via API only)

Synchronous CSV and XLSX exports are now available at /api/v2/assets/{asset_uid}/export-settings/{settings_uid}/data.csv (and …/data.xlsx). If an export is requested multiple times for the same settings, it will be regenerated only once every 5 minutes. Synchronous exports may fail for large projects, as their processing must complete within the web application server’s 2-minute timeout (compared to 30 minutes for asynchronous export tasks, which are available at /api/v2/assets/{asset_uid}/exports/).

This was implemented in #3586, #3700, and #3716. It was tracked by #3319.

OAuth2 API Authentication

Requests authenticated with OAuth2 can now access all API endpoints, as with other authentication mechanisms. This was implemented in #3590 and tracked in #3589.

Bug Fixes

PR Description Related Issues
#280 Fix handling of translated media
Addresses future handling of image::lang case and of past forms affected by the bug.
closes #115
closes #175
#281 Fix for nested repeats without fields in their section closes #279
#284 Fix translations and labels mismatch error message
Handle NoneType correctly and display error message correctly when there is a mismatch between the translations and labels.
closes #283
#786 #789 Fix edit permission evaluation
Forbid anonymous edits to ensure that only owners and those explicitly assigned edit permissions are allowed to edit submissions.

CSV submission imports using the csv_import endpoint may no longer set _submitted_by. This field will now always be overwritten with the username of the user performing the CSV endpoint.
#3516 Fix wrong language of sector label being used
Fixes a bug when form is created using one UI language, but then after switching to different one, the sector label displayed is from the first language.
Fixes #3514
Contains code from #3507
#3525 Fix print styles
Fixes print styles - mainly for reports and table view.
Fixes #3524
#3560 Fix missing draft navigation
The draft navigation and header editable title were missing for drafts that were just created.
Fixes #3559
#3565 Fix accidental truncating of labels
Fixes truncating of report view choice labels in the table below graph.
Fixes #3564
#3575 Fix filtering repeat groups in export
Fix export filtering for repeat groups.
closes #3324
#3583 Make sure the tags search field in the form builder’s library sidebar keeps working by updating configuration for the select control (react-select noOptionsMessage crash)
#3597 Don’t display past repeat group responses twice (and badly)
Changing a repeat group into regular group causes an additional column to appear in Table View with “[object Object]” as cell values. This is now fixed.
Fixes #3594
#3602 Fix export filtering with attachments
Include _attachments field when filtering fields for exports.
closes #3601
#3605 Fix submission bulk edit XML handling
Fixes submission bulk editing for grouped fields.
closes #3604
#3628 Fix Settings Media button
Fixes Asset > Settings > Media button not working.
#3631 Gracefully handle missing items when rewriting attachment URLs (fixes KeyError 500 error)
#3634 Fixed KobocatUser.sync bug
Fixed an issue with KobocatUser.sync which caused duplicate entries and 500 errors on login
#3641 Fix missing highlight of selected ui language in account dropdown
Minor fix.
#3666 Fix history animation and fix logo
Fix version history animation. Also update one last old logo occurence.
Related to #3630
#3679 Fix validation column disappearing
Fix validation column disppearing while hiding a different column through “Hide fields”.
Fixes #3612


PR Description Related Issues
Upgrade pyxform to v1.7.0
Upgrade pyxform with bug fixes and new allow-mock-accuracy parameter for geo questions (change log).
closes #777, #3549
#2718 Drop IE11 support
Three things here:
- Dropped IE11 from .browserslistrc (the file used for determining what browsers to support while building JS files)
- Created a redirect to kf.kobo.local/modern_browsers/ with a message about modern browsers (I copied the message from Enketo)
- Cleaned up some old IE-related things in templates
Fixes #2716
#3582 TableMediaPreview component style tweaks
This change makes the image or video be always fully visible regardless of screen size, i.e. we no longer display scrollbar, just make the image/video fit (useful for very tall and short images). We also introduce a missing loading spinner for images and audio files.
Part of #3567
#3625 Keep all existing export settings
Our exports API has more options available (submission_ids and query) than our UI allows to set. This fixes the issue of losing those additional settings when updating the export settings through UI.
Fixes #3603
#3630 Update logo
Updated logo throughout the app.
Part of #3529
#3633 Remove archive button when project not deployed
Only display the “ARCHIVE PROJECT” or “UNARCHIVE PROJECT” once the project has been deployed (not a draft).
closes #3632
#3683 Improve long label text styles for button component
Previously the text was wrapping in an ugly way.
Fixes #3682

Of Interest to Self-Hosters

PR Description Related Issues
#379 Bump NGINX version to 1.21
Add support to multi architecture docker image with official Python image.
#783 Set FILE_UPLOAD_PERMISSIONS to prevent 403 when NGINX accesses large submission attachments
This prevents a 403 error from occurring when accessing large files attached to submissions on self-hosted instances of KoboToolbox that use local disk storage.
#791 Delete attachments from storage when their submission are deleted
#3515 Add management command to sync permissions between KPI and KoBoCAT
Permissions cannot be set in KoBoCAT anymore. This management command is useful to synchronize permissions one last time to ensure all permissions are in sync between both applications.
Closes kobotoolbox/tasks#335
#3573 Site message edit fix
When editing site messages, Markdown editor is shown without raising a 500 error
Fixes #3572
#3588 Fix 500 errors returned when trying to delete a user from admin platform
When a user is deleted, related records (to be confirmed to delete too) are displayed before deletion.
It fixes this summary.
Fixes #3587
#3611 Make project and user metadata fields configurable
Allow superusers to configure project and user metadata fields
Closes #3554, closes #3555, closes #3556, closes #3599
#3629 Configurable metadata fixes
Unified the feel and looks of multiple forms throughout the app. Dropped phone and address fields from Account Settings.
This is a followup to #3611
Part of #3554
Closes #3629
#3650 Add required indicator to metadata fields and fix account settings values not being displayed in UI
#3657 Serve attachments with NGINX
Use X-Accel-Redirect header to let NGINX serve the attachments instead of Django.
#3711 Delay get export requests
Avoid bombarding backend with get export calls by randomizing and increasing the interval times.
Fixes #3706

Of Interest to Developers

PR Description Related Issues
#270 Apply black formatting to formpack library closes #268
#765 Fix 2 static file paths Fixes #764
#770 Update transifex-client for Python 3.9
#779 Digest authentication refactoring and legacy views clean-up
Code refactoring to centralize the calls of HttpDigestAuthenticator and DigestAuthentication classes.
#780 Deactivate single-factor authentication when 2FA is activated [2FA is an upcoming feature]
When users activate 2FA on their account (from KPI), Basic, Digest and Token authentications are deactivated and return a 401 error.
Related to kobotoolbox/kpi#3584
#781 Consistent uWSGI settings
#784 Reduce docker image size with multi stage build
#787 Run pip-sync before
Install Python dependencies before trying to run any Python scripts, fixing errors like ModuleNotFoundError: No module named 'django_digest' that previously required a full rebuild to resolve. Mirrors kobotoolbox/kpi#3607.
#790 Clean up deprecated code.
#793 Correct the versioned app registry and apply other, trivial migrations to quieten ./ makemigrations.
#3169 Fix npm audit errors
#3504 Use maintained vusion fork of webfonts-generator
Switch to a maintained fork of the webfonts-generator tool
#3540 Button component and Design System
Introduce a new Button component that covers all possible types, sizes and colors of buttons we will be using in UI.
Includes code from #3579
Fixes #3526
Part of #3548
#3545 Another batch of updated icons
A few more updated icon designs.
Part of #3305
#3574 Accounts sidebar
Adds new route for the new collection of account options. Sets the template for usage dashboard and security components
Part of #3563 #3097
#3576 Organize package json dependencies
Nothing interesting. Organize dependencies into dependencies and devDependencies for clarity sake.
#3579 Further update colors
The last part of updating the color palette.
#3580 Make uwsgi consistent with kobo-docker
Replaces the need for kobo-docker/kpi_uwsgi.ini at master · kobotoolbox/kobo-docker · GitHub
Related to Consistent uwsgi by bufke · Pull Request #781 · kobotoolbox/kobocat · GitHub
#3596 Add API endpoint to retrieve submission attachments
Created an endpoint to return attachments from KoBoCat and can convert audio and video files to MP3 formats for front end media player
closes #3384
#3598 removed translation wrappers from country and language lists
Stop translating country and language names for now to ease the burden on translators. These names will no longer appear in Transifex
#3607 Run pip-sync before
Install Python dependencies before trying to run any Python scripts, fixing errors like ModuleNotFoundError: No module named 'django_digest' that previously required a full rebuild to resolve.
#3615 Fix documentation typo
Just a simple typo
#3619 Standardize country and sector user metadata
Fixes an issue where sector and country details entered during account registration did not appear in the account settings page.
#3621 Upgrade react select and hack the unwanted scrollbar bug
Hackfixes a bug in New Project modal when opening a dropdown caused a scrollbar to appear unwantedly. Also includes an upgrade of react-select version.
This is a way around menuplacement auto in div with overflow not correct · Issue #4108 · JedWatson/react-select · GitHub
#3624 Wrapped select cleanup
Further code improvements for the WrappedSelect component.
Followup to #3621
#3626 Changed ugettext as _ to gettext as t
Replace ugettext as _ and ugettext_lazy as _ with gettext as t and ugettext as t
closes #3620
#3627 Add missing commas and semicolons in projectSettings.es6
Just code style cleanup.
#3645 Access attachments stored in KoBoCAT storage/database directly
Improve attachment endpoint by reading attachments directly from KoBoCAT database and storage.
#3655 Use subprocess instead of pydub to convert audio files
#3659 Port Django get_valid_filename utility to front-end code to open files with unicode characters and space
When a submission with an attachment comes in. Django renames the attachement file before saving the file in the storage.
- It removes letters with accent (and replaces then with their counterpart without accent)
- It replaces spaces with underscore
- It removes any others characters
#3668 Do not expand empty country or sector values in the current user API
#3673 Two-Factor Authentication (back end only)
#3684 Expose modified and disabled dates for users’ MFA applications in API
Add a new endpoint to get last modified and disabled (if any) date of the MFA services a user has.
Part of #3563
#3700 Fix unreleased synchronous export 500 error on insufficient permissions
Return 404 if user does not have sufficient permissions to create a synchronous export.
closes #3699
#3722 Set name correctly in project settings
Fixes an unreleased bug where the name of a new project was not properly read from the XLSForm file name or form_title setting.
Fixes #3703

As of now, 21:00 UTC on 21 March 2022, a patch has been deployed on both servers rolling back to the old editing behavior where submissions are always loaded in the latest version of the form. Technical details can be found here: Use latest deployed version of a form when editing a submission by noliveleger · Pull Request #3734 · kobotoolbox/kpi · GitHub.