Release Notes - version 2.021.30

Hello KoBoToolbox Community! We are pleased to release version 2.021.30 with the following changes. This release:

  • has been deployed to as of 02:00 UTC on 17 August;
  • has been deployed to as of 03:15 UTC on 19 August.

While the work offered here has gone through a cycle of internal testing and improvement, please do not hesitate to report any problems that you find. This helps the community as a whole in addition to yourself. Thanks.

New features and enhancements

PR Description Related Issues
Dynamic Data Attachments

Adds UI for exporting/importing data across connected projects using xml-external
Supports #2767
Fixes #3019
Row Level Access
Extend row-level permission controls beyond viewing to editing, validating, and deleting
Closes #2768
#3234 Add support for background-audio in the table view.
Adds UI changes outlined in #3193 that implement table view and formbuilder support for background-audio recording
Fixes #3193
Fixes #3037
Allow for view-only access to submissions in Enketo UI.
Update submission /edit endpoint to /enketo/<action> and accept view or edit
Closes #2992
Part of #3322
Closes kobocat#671
formpack#249 Support choice from previous repeat answers for select_one
Handle the case of select_one ${question_name} to support XLSForm/pyxform#472.
Closes formpack#248
#3294 Allow anonymous users to export data from publicly-shared projects Closes #3293
#3177 Library upload asset type selector
Whenever user uploads one file (through NEW modal or by dropping on list) in My Library, it is possible to select the desired type, e.g. “template”.
Fixes #3142
#3274 Table view style improvements
New design for Table View. This PR contains mainly stylistic changes. Also includes an update for our Roboto font (added Roboto Mono).
Fixes #3198
#3409 Make authentication required more obvious in permissions UI
When “Authentication required to see forms and submit data” is not enabled, we display a warning message in “Share project” modal. We also make “Add submissions” always checked (and, following the permissions rules, “View form” too).
Fixes #2620

Bug fixes and other improvements

PR Description Related Issues
#3320 Change downloaded media file names to the original file name
Explicitly provides the media download link with the original name of the uploaded file
Fixes #3315
#3292 Allow viewing REST services settings for archived forms Fixes #3290
#3298 Bring back gray background for form-view
#3339 Responsive styles fix for icon buttons
#3340 Include new icons for media files
#3365 Fix text alignment when adding a new question inside a group
#3396 Style fix: sidebar tabs labels alignment and table submission actions alignment, for when there are only view icons.
#3325 Fix for filtered repeat group exports Closes #3324
#3308 Fix Table View’s class names.
#3322 Fix Table View crash when user has only view permissions
#3323 Fixed a bug where files with spaces in its name does not properly show in the single submission modal or table view.
#3328 Disallow matrix column names that contain certain characters (e.g. spaces), which caused serious problems for Enketo. Fixes #3326
#3301 Update formpack hash
Fix for dynamic select_* questions included in formpack update kobotoolbox/formpack#249.
Closes #3300
kobocat#757 Remove django-registration-redux
Removes account registration and password reset functions from KoBoCAT, which are handled by KPI in any KoBoToolbox installation. Fixes a 500 error when attempting to access the KoBoCAT login page, which usually only redirects to KPI but can be configured to display a form (see the KOBOFORM_LOGIN_AUTOREDIRECT setting).
Fixes kobocat#756
kobocat#755 Restore /api/v1/forms/<formid>/enketo endpoint
kobocat#738 Improve metadata content type validation
Content type is auto-detected when uploading form media files and can no longer be specified manually. Also improves content type detection when attaching media files by URL.
Closes kobocat#736
Closes kobocat#737
kobocat#742 Fix ZIP exports of files attached to submissions for large projects Hotfix for kobocat#475
kobocat#741 Update supported media types to broaden support for wav and zip mimetypes. Closes kobocat#740
#3398 Frontend QA fixes for 2.021.30 release
#3400 Fix problem where the “Manage project” permission could not be assigned Fixes #3399

Of interest to self-hosters

PR Description Related Issues
kobocat#732 Always remove deleted submissions entirely instead of merely marking them as deleted

When deleting submissions through the legacy view (but not via the API), a special marker was applied to the “deleted” submissions instead of removing them outright. This meant that every query to retrieve submitted data needed to filter out submissions having this marker. We now always remove submissions outright when deleting and therefore no longer need this extra filter, which caused high database overhead.
Closes kobocat#696
Related to kpi#3287
kobocat#744 Add data migration to purge deleted submissions

Automatically removes submissions that were already marked as deleted. Necessary for self-hosters so that the changes in kobocat#732 do not cause deleted submissions to resurface.

:warning: Includes a migration that could be slow on large databases.

If you run your own instance of KoBoToolbox with many thousands of submissions, consider using shell_plus inside the KoBoCAT container (kobo-install/ -cf exec kobocat bash) to purge deleted submissions using the code shown in kobocat#696 prior to upgrading to this release.
Closes kobocat#733
Related to kobocat#732
kobocat#685 Integrate the improved version description from kobocat#676 into the new dynamic data attachments feature. Related to kobocat#686
#3309 Submissions by Country Report
Creates a report for Submission-by-country which can be filtered by a start and end date at the /superuser_stats/country_report/ URI.
Fixes #3065
#3303 Django admin panel: Sorted the users by their join date and added a Join Date column Fixes #3069

Of interest to developers

PR Description Related Issues
#3317 Added a new UI component for dropdowns. Needed for #3199
Part of #3266
Refactor for Dynamic Data Attachments Part of #3057
Refactor Enketo endpoints for edit and view Related to #2992
Related to kobocat#671
#3327 Refactor some utils code
Refactor some components that use methods React will remove in 17 Part of #2677
#3321 Enum creation function for constants
New utility function for writing cleaner code.
Row Level Access
- Refactor deployment backend: Abstraction and variables renaming.
- Add Mongo and XML support to unit tests for Row Level Access.
Related to #2768
Related to #3115
#3367 Describe submissions API unit tests
Submission API tests are many and this helps (a little bit more) to understand what they are doing (until #3360 and #3359 are ready to go)
Part of #2900 #3123
Related to #3360 #3359
kobocat#739 Disable pre-delete signal for bulk_delete Closes kobocat#735
kobocat#750 Incorporate changes from the code review of kpi#2900 (row-level write permissions), with the aim of increasing code clarity and security. Related to kpi#2900
#3388 Fix failing mock submission tests
Flush the mock MongoDB in unit tests only when appropriate—not always—so that tests using multiple assets function properly
Related to #3387

As of 22:12 UTC on 25 August, we’ve released a few bug fixes to both servers as version 2.021.30a:

PR Description Related Issues
#3432 Fix blank screen when viewing REST Services details
#3414 Remove double slash in Enketo preview API call Fixes #3413
kobocat#761 Empty/corrupted superuser stats report fix
Make superusers be able to download stats reports.
Fixes kobocat#760

Thanks as always for being a part of the KoBoToolbox community. Good luck with your projects!