Uploading users in bulk in Django

At the moment, it seems like the only option is to create each user individually. For a recent project, we had to create 1,700 users; thankfully, we have the technical capacity to add the users directly into the database, but this is not accessible to most, and even for us it is not scalable. As a result, we’d love to see an option to bulk import users from a spreadsheet.

Can you share how you directly added the users in the database?

1 Like

@djole is there anything you can share about the process of adding users into the database?

Hi @raph, thanks for the suggestion. Is it common for you to create so many users for a project?

If you are using a self-hosted instance of Kobo, you can do this through the Django shell in the KPI container, however I don’t think we have a public API endpoint to achieve this yet.

1 Like

This was done by our partners on one project and we assisted them. They used Kobo Api and changed something directly in the PostgreSQL database.
It didn’t go very smoothly, we had problems until we entered all the users and their rights.

If you have access to the hosted server:

  • You can get into the kpi container doing:
    docker exec -it <kpi> bash
  • Then enter the Django shell:
    ./manage.py shell_plus
  • Then from inside the ipython shell, you can create a user doing the following:
    User.objects.create(username='<some new user>')
  • Or if you are creating a whole bunch of users:
for user in users:
    User.objects.create(username=user)

(Additionally you can assign permissions to those users as you create them through the shell — perhaps listing usernames and permissions in your spreadsheet, importing that and iterating through as you create and assign, etc. I can send through some sample code if that will be useful)

3 Likes

Yes, please, this would be very helpful.

Hi @djole,

Lets assume you have some projects already created and are wanting to create users and give them permissions to particular projects. You could create a spreadsheet with the following structure for creating new users (let’s call it users.csv):

username first_name last_name email
johndoe John Doe johndoe@gmail.com

And a second sheet with your projects and permissions for each user (let’s call it permissions.csv):

username project_uid permission
johndoe qWeRtyUiOpAsDfGh view_asset
johndoe pOiKjUhdwzkvkue6h view_submissions

You could then do something like this in the kpi Django shell (if you use pandas, you’ll have to pip install it):

import pandas as pd
# we need to let Django handle the password hashing
from django.contrib.auth.hashers import make_password

users = pd.read_csv('users.csv')
permissions = pd.read_csv('permissions.csv')

# let's create some users with default passwords equal to their username
for i, row in users.iterrows():
    User.objects.create(
		username=row['username'],
		first_name=row['first_name'],
		last_name=row['last_name'],
		email=row['email'],
		password=make_password(row['username'])

# let's assign permissions
for i, row in permissions.iterrows():
    try:
        asset = Asset.objects.get(uid=row['project_uid'])
    except Asset.DoesNotExist:
        print(row['project_uid'])
        continue
    try:
        user = User.objects.get(username=row['username'])
    except User.DoesNotExist:
        print(row['username'])
        continue
    asset.assign_perm(user, row['permission'])

You can find a list of the assignable permissions here and note that some permissions are implied by others, which you can see here.

Hope that helps :slightly_smiling_face:

2 Likes

Thanks a lot!

1 Like