Release Notes - version 2.023.12

Announcements
1

Thank you to everyone who helped test this release. We are pleased to announce the deployment of 2.023.12 to:

  • kf.kobotoolbox.org at 01:30 UTC on 11 May 2023
  • kobo.humanitarianresponse.info at 02:30 UTC on 11 May 2023

The components comprising this release are as follows. See further below for detailed changes.

  • KPI version 2.023.12
  • KoBoCAT version 2.023.12
  • Pyxform version 1.9.0, unchanged from the previous release
  • Enketo Express:

Improvements

PR Description
kpi#4296 Add usage page to track current user stats
Allow user to check the following stats on the new route /account/usage:
- Monthly submissions
- Total storage (in GB)
- Monthly transcription minutes
- Monthly translation characters
kpi#4318
kpi#4396 		Add Hungarian, Japanese, and Persian interface translations
kpi#4373
kobocat#871 		Move project deletion to an asynchronous process so that large data sets can be removed without intervention from the Kobo team

Fixes

PR Description
kpi#4100 Allow non-owners to view asset versions
Allows for users with the View Asset permission for the form to view the versions via API
kpi#4279 kpi#4284 Fix app crashing when quickly switching routes between form sub routes
Fix crash when quickly switching form routes (Summary, Form, Data, and Settings tabs)
kpi#4290 Fix move to collection feature
All user owned collection now will appear in more actions menu under “Move to” section.
kpi#4297 Fix bottom links overlapping each other in login/registration form on mobile screens
kpi#4300 Fix ‘Organization Website’ not being saved in Account Settings.
kpi#4390 Don’t show errors when accessing forms anonymously
Removed error pop-ups that were accidentally displaying when viewing a form anonymously.
kobocat#862 Allow model to be used as a field name in a form.

Of interest to self hosters

Security improvement

If you host your own instance of KoboToolbox, you can enhance security by enforcing a Content Security Policy. To do so, set the environment variable ENABLE_CSP=True for both KPI and KoBoCAT. If you use kobo-install, create the kobo-docker/docker-compose.frontend.custom.yml file (if it doesn’t exist already) and make sure it has at least this content:

version: '3'

services:
  kpi:
    environment:
      - ENABLE_CSP=True

  kobocat:
    environment:
      - ENABLE_CSP=True

The CSP has been hardened in kpi#4311, and all inline JavaScript has been removed. In addition to the CSP, kpi#4311 instructs browsers to download form media files instead of rendering them when opened directly, as opposed to being embedded in a form.

Other changes

PR Description
kpi#4373
kobocat#871 		Bulk actions for users and projects
Allow users to bulk delete and bulk (un)archive their projects (API only).
Allow superusers to bulk undelete projects and bulk delete/remove users.
The deletion process is asynchronous to accommodate large projects.
kpi#4131 Set free tier thresholds in Constance configuration
Make free tier thresholds dynamic and expose values in the API (environment endpoint)
kpi#4226 Add name and email to user statistics report
Update User Statistics superuser report to include name and email for each user
kpi#4238
kobocat#864 		Make user session age configurable
Make the user session cookie customizable via kobo-install to fit the needs of organizations hosting their own instances
kpi#4242 Make activation email configurable
Make the activation email customizable from the admin interface
kpi#4246 Allow manual column resizing in Regional Projects Table
Columns can be resized in the Regional Projects Table by click-and-drag in the column headers.
kpi#4251 Allow filtering by country in projects list
Users can now select country by full name in projects filter list.
kpi#4281 Validate usernames the same way in Django Admin and registration form
Allow usernames between 2 to 30 characters in lowercase which can contain numbers and underscores.
The first character must be a letter.
kpi#4282 Always use autocomplete inputs for User field in Django Admin
Make user assignment to other objects more user-friendly.
kpi#4287 Add private social apps
Restrict visibility of private social application login.
kpi#4293 Allow disabling user editing SSO signup email address
Add settings option to disallow SSO users from editing signup email address.
kpi#4299 Store UI language for usage statistics
Stores currently selected language on /me endpoint for internal statistics on language usage.
kpi#4304 Add users UI language to reports
kpi#4322 Switch to Google Analytics 4
Internal code upgrade that allows us to keep making educated decisions based on user activity.
kpi#4332 Fix Log in as user button doing nothing with new (stricter) CSP
Overload package template to remove inline Javascript
kpi#4352 Restore the ability to configure which user interface translations are enabled via the DJANGO_LANGUAGE_CODES environment variable
kpi#4363 Fix welcome message for signup template
Make the signup template honor the custom welcome message again
kpi#4384
kobocat#871 		Fix override of S3Boto3StorageFile to work with new version of django-storages and make the override less brittle
kpi#4391 Fix dangling comma in SSO signup form, minor language changes
Update SSO signup form to fix hanging comma when user’s SSO account username not available. Minor language changes

Of interest to developers

PR Description
kpi#3984 Create endpoint for total and daily submission counts
Create an API endpoint for daily and total counts of submissions
kpi#4239 Cleanup permissions checking methods
Move permission checking methods from mixins to a separate file and update all the places that use them.
kpi#4265 Fix python build failure during CI unit-tests
Make GitHub Actions pass on pip uWSGI dependency install
kpi#4267 Avoid redundant building steps on container start-up
Remove manual calls of npm run copy-fonts (because npm post-install calls it automatically) and also copy pip-dependencies.txt in the last build step to avoid calling pip-sync when the dependencies are the same.
kpi#4275 update dj-stripe version
Update the version of dj-stripe for a bug-fix
kpi#4278 Add API endpoint for Stripe Products
Add API endpoint for Stripe Products. Used to increase limits on public servers such as kf.kobotoolbox.org
kpi#4289 Fix error of management command populate_asset_date_deployed not populating all deployed assets.
Use the first deployed version of Asset objects to populate the date_deployed field.
kpi#4294 Add prometheus django and flower for observability
Adds optional celery monitoring via flower and metrics endpoint for prometheus
kpi#4298 Deduplicate identical types
kpi#4302 Fix saving and validation of extra user metadata
User metadata fields that are left empty in the user interface are now sent to the back end as empty strings (or false, in the case of require_auth) instead of being omitted from the request data altogether. The back end now validates only fields that are present in the request data, allowing for partial updates of extra_details instead of requiring that all fields be included with every request.
kpi#4308 Install procps inside the Docker image for the benefit of systems administrators
kpi#4316 Update the Update Translations Strings github action
Update the update_translations_strings github action to use the new Transifex CLI and to work with main branch name change
kpi#4323 Faster frontend builds
Improve frontend build speed.
kpi#4327 Use new field date_deployed to retrieve deployed assets faster
Remove subquery on AssetVersion to filter deployed assets.
kpi#4328 Reorganize the Asset class methods alphabetically
Improve readability of the file by sorting methods alphabetically and keep Kobo team python coding style consistent.
kpi#4336 Change 500 response to 400 when invalid source passed to v1 export API
Requesting an export from the v1 API (/exports/) and providing an invalid source, e.g. an asset that had been deleted, a 500 error was previously returned. This change causes a 400 error to be returned instead in this scenario. Note that some malformed source URLs may cause a 404 error to be returned; that was an existing behavior and is not affected by this change.
kpi#4358 Run prettier on processing code (but on beta)
kpi#4360 DeBEMize processing components
Internal cleanup of code.
kpi#4365 Django storages to 1.13.2
Fixes a regression where uploaded assets with Azure Blob Storage may not work.
kpi#4368 Improve processing code documentation
This is internal code documentation improvements for Single Processing route code.
kpi#4369 Remove gruntserver management command
Removed the guntserver management command because it was old and no longer used
kpi#4370 Remove the remove_duplicate_assetversions management command
Removes the remove_duplicate_assetversions management command because it is old and unused
kpi#4381 Bypass unreliable assertNumQueries
kpi#4388 Fix comment in .browserslistrc
It contained an outdated URL
kpi#4389 Add date_deployed to /api/v2/assets endpoint
kpi#4401 Apply SSO copy edits from #4391 to release branch
kpi#4405 Add do-nothing migrations to appease Django
kpi#4406 Add unique together migration for emailcontent
kpi#4439 Ensure the surveyCompanionStore is imported correctly
Fixes (unreleased) problem with Library importing (by drag & drop) in Form Builder. Fixes kpi#4436
kpi#4434 Make in-app messages fetch immediately once the user is logged in
Fixed an (unreleased) issue that prevented in-app messages from displaying until the “Help” icon was clicked.
kobocat#870 Upgrade django-storages to 1.13.2
kobocat#867
kobocat#873 		Open legacy export download links in a new tab
Allows legacy exports within a KPI to continue working when CSP is enabled
kobocat#866 Install procps inside the Docker image for the benefit of systems administrators
kobocat#865 Match KPI GitHub Actions pytest workflow
kobocat#874 Replace slashes with dashes in Docker image tags
kobocat#872 Fix override of S3Boto3StorageFile to work with new version of django-storages and make the override less brittle
1 Like