I have a form that should be shared with a few hundred users. I noticed that sharing a form (that is already shared with a large number of users) is very slow. It takes about a minute and during that time the CPU usage is 100%.
The form itself has about twenty questions, and the server has a decent amount of memory and CPU cores.
Has anyone else encountered such an issue and is there a way to speed up the sharing?
Hi @djole,
Welcome to the community! Kindly please be informed that the sharing of a project as outlined in the support article Managing Permissions should not take up that CPU usage as you have experienced. Could you kindly try this out in a different device and see if the same happens.
Yes the Managing Permissions is definately a manual process but kindly please be informed that if you have once assigned a user to any of a project previously you should be able to assign the Managing Permissions quickly by using the Copy team from another project feature.
Have a great day!
Thank you @Kal_Lam. Access rights are shared as described. I will test with the same settings on another system.
Best regards
And just to add some information to what my colleague @djole posted: this is on a custom installation of Kobo on a separate server. The CPU utilization issue is on the server.
Here are the servers specs:
OS: Ubuntu 18.04
Memory: 16384 MiB
CPU Cores: 8 cores
Kernel: latest
Size: 20 GiB
1 Like
@raph @djole Could you post a snippet of the API code you use to assign permissions and what you’re seeing on the server side that indicates CPU usage is related to the permission update?
Could you confirm please that if you use the same script to assign a clone of the same form with a second user that this issue doesn’t happen?
1 Like
I don’t have a code at the moment because most of the users weren’t added by us. I just added a few new users and tried to share a project with them just as it is described here. It even took a while to load the list of users with whom the project already was shared. At that point, CPU utilization looks like this:
When I selected the permissions for a new user and clicked on “grant permissions” CPU utilization looks like this:
At some point, the form is cloned and the permissions are copied from the old form, but this behavior has remained the same.
To be sure, what happens when you create a brand new project and share it with a single other user? Does the same CPU spike occur?
It’s possible that the process of retrieving current permissions gets slower the more users a project has been shared with. The API request is [kpi-url]/api/v2/assets/[asset-id]/permission-assignments/.
If you need to add this many users to a single project one way would be to use the API instead to add additional permissions. For example, you can POST
[{"user":"https://kf.kobotoolbox.org/api/v2/users/[username]/","permission":"https://kf.kobotoolbox.org/api/v2/permissions/view_asset/"}]
to [kpi-url]/api/v2/assets/[asset-id]/permission-assignments/bulk to add a view_asset permission for a single user. Using this JSON format you can add any number of users (of course only if you’re authenticated as the project owner).
1 Like
Hello,
If this is really an 8-core server, the CPU usage doesn’t seem particularly high: uwsgi using 80% of a single core and postgres using around 20% would leave the other 7 cores essentially idle. If the permissions assignment code is inefficient for large numbers of users, which is certainly plausible, I’d expect to see uwsgi and/or postgres running at 100% continuously as reported by top until the assignment completes, since there are other cores available to service other tasks.
Could you describe the type of disk storage used by this server and show the beginning of the top output so that we can see the load, memory statistics, and CPU wa (the time spent by the CPU waiting on disk I/O)?
1 Like
Thank you all, I am not in a position to investigate this case further. However, once all users were added and permissions were set, there were no other problems.
1 Like
OK, thanks for following up.
1 Like
The OP has opened a GitHub issue for this: https://github.com/kobotoolbox/kpi/issues/2779
3 Likes
