Hello, everyone. We’ve just launched a new release (effective at 21:00 UTC on 17 March 2022) on both kf.kobotoolbox.org and kobo.humanitarianresponse.info.
Components comprising this release:
- KPI version 2.022.08
- KoBoCAT version 2.022.08
- Enketo Express version 2.8.1
- Pyxform version 1.7.0
Security Fix
This release includes a fix for an issue discovered by a recent security audit. If the following conditions were all met:
- A project allowed anonymous submissions;
- A person knew of the existence of that project;
- That person knew the UUID of an existing submission (which contains 32 random hexadecimal digits);
- That person submitted specially-crafted XML referencing that UUID;
then that person could overwrite data in the existing submission identified by that UUID.
This release fixes the problem by ensuring that no kind of anonymous request can ever modify previously submitted data.
We encourage anyone who manages a public instance of KoboToolbox to upgrade as soon as possible. If you need assistance, please open a topic in the “Kobo On Your Own Server” category. Thank you.
New Features
Czech Translation
Thanks to our volunteer translators, the KoboToolbox user interface is now available in Czech! If you would like to make KoboToolbox accessible to more people around the world by adding more translations or helping the existing ones stay up-to-date, please refer to this forum post to get started.
Improved Submission Editing
There are two notable changes to submission editing:
- Attachments are now available to download from within Enketo while editing;
-
A submission is now opened with the same version of the form that was used to submit it in the first place, not the latest version of the form.[This has been reverted temporarily because it broke some people’s workflows. A future release will include a user interface for choosing which version to use when editing.]
This was implemented in #3689 and #3707 and was tracked by #3654.
Synchronous Exports (via API only)
Synchronous CSV and XLSX exports are now available at /api/v2/assets/{asset_uid}/export-settings/{settings_uid}/data.csv (and …/data.xlsx). If an export is requested multiple times for the same settings, it will be regenerated only once every 5 minutes. Synchronous exports may fail for large projects, as their processing must complete within the web application server’s 2-minute timeout (compared to 30 minutes for asynchronous export tasks, which are available at /api/v2/assets/{asset_uid}/exports/).
This was implemented in #3586, #3700, and #3716. It was tracked by #3319.
OAuth2 API Authentication
Requests authenticated with OAuth2 can now access all API endpoints, as with other authentication mechanisms. This was implemented in #3590 and tracked in #3589.
Bug Fixes
| PR | Description | Related Issues |
|---|---|---|
| #280 | Fix handling of translated media Addresses future handling of image::lang case and of past forms affected by the bug. |
closes #115 closes #175 |
| #281 | Fix for nested repeats without fields in their section | closes #279 |
| #284 | Fix translations and labels mismatch error message Handle NoneType correctly and display error message correctly when there is a mismatch between the translations and labels. |
closes #283 |
| #786 #789 | Fix edit permission evaluation Forbid anonymous edits to ensure that only owners and those explicitly assigned edit permissions are allowed to edit submissions. CSV submission imports using the csv_import endpoint may no longer set _submitted_by. This field will now always be overwritten with the username of the user performing the CSV endpoint. |
|
| #3516 | Fix wrong language of sector label being used Fixes a bug when form is created using one UI language, but then after switching to different one, the sector label displayed is from the first language. |
Fixes #3514 Contains code from #3507 |
| #3525 | Fix print styles Fixes print styles - mainly for reports and table view. |
Fixes #3524 |
| #3560 | Fix missing draft navigation The draft navigation and header editable title were missing for drafts that were just created. |
Fixes #3559 |
| #3565 | Fix accidental truncating of labels Fixes truncating of report view choice labels in the table below graph. |
Fixes #3564 |
| #3575 | Fix filtering repeat groups in export Fix export filtering for repeat groups. |
closes #3324 |
| #3583 | Make sure the tags search field in the form builder’s library sidebar keeps working by updating configuration for the select control (react-select noOptionsMessage crash) | |
| #3597 | Don’t display past repeat group responses twice (and badly) Changing a repeat group into regular group causes an additional column to appear in Table View with “[object Object]” as cell values. This is now fixed. |
Fixes #3594 |
| #3602 | Fix export filtering with attachments Include _attachments field when filtering fields for exports. |
closes #3601 |
| #3605 | Fix submission bulk edit XML handling Fixes submission bulk editing for grouped fields. |
closes #3604 |
| #3628 | Fix Settings Media button Fixes Asset > Settings > Media button not working. |
|
| #3631 | Gracefully handle missing items when rewriting attachment URLs (fixes KeyError 500 error) |
|
| #3634 | Fixed KobocatUser.sync bug Fixed an issue with KobocatUser.sync which caused duplicate entries and 500 errors on login |
|
| #3641 | Fix missing highlight of selected ui language in account dropdown Minor fix. |
|
| #3666 | Fix history animation and fix logo Fix version history animation. Also update one last old logo occurence. |
Related to #3630 |
| #3679 | Fix validation column disappearing Fix validation column disppearing while hiding a different column through “Hide fields”. |
Fixes #3612 |
Changes
| PR | Description | Related Issues |
|---|---|---|
|
#778 #3558 |
Upgrade pyxform to v1.7.0Upgrade pyxform with bug fixes and new allow-mock-accuracy parameter for geo questions (change log). |
closes #777, #3549 |
| #2718 | Drop IE11 support Three things here: - Dropped IE11 from .browserslistrc (the file used for determining what browsers to support while building JS files)- Created a redirect to kf.kobo.local/modern_browsers/ with a message about modern browsers (I copied the message from Enketo) - Cleaned up some old IE-related things in templates |
Fixes #2716 |
| #3582 | TableMediaPreview component style tweaks This change makes the image or video be always fully visible regardless of screen size, i.e. we no longer display scrollbar, just make the image/video fit (useful for very tall and short images). We also introduce a missing loading spinner for images and audio files. |
Part of #3567 |
| #3625 | Keep all existing export settings Our exports API has more options available ( submission_ids and query) than our UI allows to set. This fixes the issue of losing those additional settings when updating the export settings through UI. |
Fixes #3603 |
| #3630 | Update logo Updated logo throughout the app. |
Part of #3529 |
| #3633 | Remove archive button when project not deployed Only display the “ARCHIVE PROJECT” or “UNARCHIVE PROJECT” once the project has been deployed (not a draft). |
closes #3632 |
| #3683 | Improve long label text styles for button component Previously the text was wrapping in an ugly way. |
Fixes #3682 |
Of Interest to Self-Hosters
| PR | Description | Related Issues |
|---|---|---|
| #379 | Bump NGINX version to 1.21 | |
|
#782 #3592 |
Add support to multi architecture docker image with official Python image. | |
| #783 | Set FILE_UPLOAD_PERMISSIONS to prevent 403 when NGINX accesses large submission attachmentsThis prevents a 403 error from occurring when accessing large files attached to submissions on self-hosted instances of KoboToolbox that use local disk storage. |
|
| #791 | Delete attachments from storage when their submission are deleted | |
| #3515 | Add management command to sync permissions between KPI and KoBoCAT Permissions cannot be set in KoBoCAT anymore. This management command is useful to synchronize permissions one last time to ensure all permissions are in sync between both applications. |
Closes kobotoolbox/tasks#335 |
| #3573 | Site message edit fix When editing site messages, Markdown editor is shown without raising a 500 error |
Fixes #3572 |
| #3588 | Fix 500 errors returned when trying to delete a user from admin platform When a user is deleted, related records (to be confirmed to delete too) are displayed before deletion. It fixes this summary. |
Fixes #3587 |
| #3611 | Make project and user metadata fields configurable Allow superusers to configure project and user metadata fields |
Closes #3554, closes #3555, closes #3556, closes #3599 |
| #3629 | Configurable metadata fixes Unified the feel and looks of multiple forms throughout the app. Dropped phone and address fields from Account Settings. |
This is a followup to #3611 Part of #3554 Closes #3629 |
| #3650 | Add required indicator to metadata fields and fix account settings values not being displayed in UI | |
| #3657 | Serve attachments with NGINX Use X-Accel-Redirect header to let NGINX serve the attachments instead of Django. |
|
| #3711 | Delay get export requests Avoid bombarding backend with get export calls by randomizing and increasing the interval times. |
Fixes #3706 |
Of Interest to Developers
| PR | Description | Related Issues |
|---|---|---|
| #270 | Apply black formatting to formpack library |
closes #268 |
| #765 | Fix 2 static file paths | Fixes #764 |
| #770 | Update transifex-client for Python 3.9 | |
| #779 | Digest authentication refactoring and legacy views clean-up Code refactoring to centralize the calls of HttpDigestAuthenticator and DigestAuthentication classes. |
|
| #780 | Deactivate single-factor authentication when 2FA is activated [2FA is an upcoming feature] When users activate 2FA on their account (from KPI), Basic, Digest and Token authentications are deactivated and return a 401 error. |
Related to kobotoolbox/kpi#3584 |
| #781 | Consistent uWSGI settings | |
| #784 | Reduce docker image size with multi stage build | |
| #787 | Run pip-sync before manage.pyInstall Python dependencies before trying to run any Python scripts, fixing errors like ModuleNotFoundError: No module named 'django_digest' that previously required a full rebuild to resolve. Mirrors kobotoolbox/kpi#3607. |
|
| #790 | Clean up deprecated code. | |
| #793 | Correct the versioned app registry and apply other, trivial migrations to quieten ./manage.py makemigrations. |
|
| #3169 | Fix npm audit errors | |
| #3504 | Use maintained vusion fork of webfonts-generator Switch to a maintained fork of the webfonts-generator tool |
|
| #3540 | Button component and Design System Introduce a new Button component that covers all possible types, sizes and colors of buttons we will be using in UI. |
Includes code from #3579 Fixes #3526 Part of #3548 |
| #3545 | Another batch of updated icons A few more updated icon designs. |
Part of #3305 |
| #3574 | Accounts sidebar Adds new route for the new collection of account options. Sets the template for usage dashboard and security components |
Part of #3563 #3097 |
| #3576 | Organize package json dependencies Nothing interesting. Organize dependencies into dependencies and devDependencies for clarity sake. |
|
| #3579 | Further update colors The last part of updating the color palette. |
|
| #3580 | Make uwsgi consistent with kobo-docker Replaces the need for kobo-docker/kpi_uwsgi.ini at master · kobotoolbox/kobo-docker · GitHub |
Related to Consistent uwsgi by bufke · Pull Request #781 · kobotoolbox/kobocat · GitHub |
| #3596 | Add API endpoint to retrieve submission attachments Created an endpoint to return attachments from KoBoCat and can convert audio and video files to MP3 formats for front end media player |
closes #3384 |
| #3598 | removed translation wrappers from country and language lists Stop translating country and language names for now to ease the burden on translators. These names will no longer appear in Transifex |
|
| #3607 | Run pip-sync before manage.pyInstall Python dependencies before trying to run any Python scripts, fixing errors like ModuleNotFoundError: No module named 'django_digest' that previously required a full rebuild to resolve. |
|
| #3615 | Fix documentation typo Just a simple typo |
|
| #3619 | Standardize country and sector user metadataFixes an issue where sector and country details entered during account registration did not appear in the account settings page. |
|
| #3621 | Upgrade react select and hack the unwanted scrollbar bug Hackfixes a bug in New Project modal when opening a dropdown caused a scrollbar to appear unwantedly. Also includes an upgrade of react-select version. |
This is a way around menuplacement auto in div with overflow not correct · Issue #4108 · JedWatson/react-select · GitHub |
| #3624 | Wrapped select cleanup Further code improvements for the WrappedSelect component. |
Followup to #3621 |
| #3626 | Changed ugettext as _ to gettext as t Replace ugettext as _ and ugettext_lazy as _ with gettext as t and ugettext as t
|
closes #3620 |
| #3627 | Add missing commas and semicolons in projectSettings.es6 Just code style cleanup. |
|
| #3645 | Access attachments stored in KoBoCAT storage/database directly Improve attachment endpoint by reading attachments directly from KoBoCAT database and storage. |
|
| #3655 | Use subprocess instead of pydub to convert audio files |
|
| #3659 | Port Django get_valid_filename utility to front-end code to open files with unicode characters and spaceWhen a submission with an attachment comes in. Django renames the attachement file before saving the file in the storage. - It removes letters with accent (and replaces then with their counterpart without accent) - It replaces spaces with underscore - It removes any others characters |
|
| #3668 | Do not expand empty country or sector values in the current user API |
|
| #3673 | Two-Factor Authentication (back end only) | |
| #3684 | Expose modified and disabled dates for users’ MFA applications in API Add a new endpoint to get last modified and disabled (if any) date of the MFA services a user has. |
Part of #3563 |
| #3700 | Fix unreleased synchronous export 500 error on insufficient permissions Return 404 if user does not have sufficient permissions to create a synchronous export. |
closes #3699 |
| #3722 | Set name correctly in project settings Fixes an unreleased bug where the name of a new project was not properly read from the XLSForm file name or form_title setting. |
Fixes #3703 |
