Migration issue kobo-docker

Amar_Kamthe · October 21, 2016, 10:04am

Hi All,

We already are using the kobo-docker for shelter associates to capture slum level data. They are around 10 thousand records captured. The kobo-docker is the old one with redeploy functionality. As in this week there was a new release done by kobotoolbox where there was a fix done for option and cascading issue, I did a docker-compose pull . And after completely the process I did a docker-compose up. The docker-compose up command fails, below is the stacktrace.

kpi_1 | File “manage.py”, line 10, in
kpi_1 | execute_from_command_line(sys.
argv)
kpi_1 | File “/usr/local/lib/python2.7/dist-packages/django/core/management/init.py”, line 354, in execute_from_command_line
kpi_1 | utility.execute()
kpi_1 | File “/usr/local/lib/python2.7/dist-packages/django/core/management/init.py”, line 346, in execute
kpi_1 | self.fetch_command(subcommand).run_from_argv(self.argv)
kpi_1 | File “/usr/local/lib/python2.7/dist-packages/django/core/management/base.py”, line 394, in run_from_argv
kpi_1 | self.execute(*args, **cmd_options)
kpi_1 | File “/usr/local/lib/python2.7/dist-packages/django/core/management/base.py”, line 445, in execute
kpi_1 | output = self.handle(*args, **options)
kpi_1 | File “/usr/local/lib/python2.7/dist-packages/django/core/management/commands/syncdb.py”, line 25, in handle
kpi_1 | call_command(“migrate”, **options)
kpi_1 | File “/usr/local/lib/python2.7/dist-packages/django/core/management/init.py”, line 120, in call_command
kpi_1 | return command.execute(*args, **defaults)
kpi_1 | File “/usr/local/lib/python2.7/dist-packages/django/core/management/base.py”, line 445, in execute
kpi_1 | output = self.handle(*args, **options)
kpi_1 | File “/usr/local/lib/python2.7/dist-packages/django/core/management/commands/migrate.py”, line 222, in handle
kpi_1 | executor.migrate(targets, plan, fake=fake, fake_initial=fake_initial)
kpi_1 | File “/usr/local/lib/python2.7/dist-packages/django/db/migrations/executor.py”, line 110, in migrate
kpi_1 | self.apply_migration(states[migration], migration, fake=fake, fake_initial=fake_initial)
kpi_1 | File “/usr/local/lib/python2.7/dist-packages/django/db/migrations/executor.py”, line 148, in apply_migration
kpi_1 | state = migration.apply(state, schema_editor)
kpi_1 | File “/usr/local/lib/python2.7/dist-packages/django/db/migrations/migration.py”, line 115, in apply
kpi_1 | operation.database_forwards(self.app_label, schema_editor, old_state, project_state)
kpi_1 | File “/usr/local/lib/python2.7/dist-packages/django/db/migrations/operations/special.py”, line 183, in database_forwards
kpi_1 | self.code(from_state.apps, schema_editor)
kpi_1 | File “/srv/src/kpi/kpi/migrations/0015_assetversion.py”, line 38, in copy_reversion_to_assetversion
kpi_1 | apps.get_model(‘reversion’, ‘Version’),
kpi_1 | File “/srv/src/kpi/kpi/management/commands/populate_assetversions.py”, line 49, in populate_assetversions
kpi_1 | _replace_deployment_ids(_AssetVersion, _Asset)
kpi_1 | File “/srv/src/kpi/kpi/management/commands/populate_assetversions.py”, line 121, in _replace_deployment_ids
kpi_1 | asset.save(adjust_content=False, create_version=False)
kpi_1 | File “/usr/local/lib/python2.7/dist-packages/django/utils/decorators.py”, line 145, in inner
kpi_1 | return func(*args, **kwargs)
kpi_1 | File “/srv/src/kpi/kpi/models/object_permission.py”, line 245, in save
kpi_1 | super(ObjectPermissionMixin, self).save(*args, **kwargs)
kpi_1 | TypeError: save() got an unexpected keyword argument ‘adjust_content’

After debugging I found that the migration script 0015 which is there in KPI fails to migrate with the “Save() got an unexpected keyword argument ‘adjust_content’” error

The new release works if I do not take the old database (where there are 10 thousand records ). How do I keep my old data with the new table changes done in KPI? Any support on this issue.

Regards,
Amar

Donald_Lobo · October 21, 2016, 3:32pm

i reproduced this also on the latest version of kobodocker using a really small sample db

lobo

···

On Friday, October 21, 2016 at 3:04:05 AM UTC-7, Amar Kamthe wrote:

Hi All,

We already are using the kobo-docker for shelter associates to capture slum level data. They are around 10 thousand records captured. The kobo-docker is the old one with redeploy functionality. As in this week there was a new release done by kobotoolbox where there was a fix done for option and cascading issue, I did a docker-compose pull . And after completely the process I did a docker-compose up. The docker-compose up command fails, below is the stacktrace.

kpi_1 | File “manage.py”, line 10, in
kpi_1 | execute_from_command_line(sys.
argv)
kpi_1 | File “/usr/local/lib/python2.7/dist-packages/django/core/management/init.py”, line 354, in execute_from_command_line
kpi_1 | utility.execute()
kpi_1 | File “/usr/local/lib/python2.7/dist-packages/django/core/management/init.py”, line 346, in execute
kpi_1 | self.fetch_command(subcommand).run_from_argv(self.argv)
kpi_1 | File “/usr/local/lib/python2.7/dist-packages/django/core/management/base.py”, line 394, in run_from_argv
kpi_1 | self.execute(*args, **cmd_options)
kpi_1 | File “/usr/local/lib/python2.7/dist-packages/django/core/management/base.py”, line 445, in execute
kpi_1 | output = self.handle(*args, **options)
kpi_1 | File “/usr/local/lib/python2.7/dist-packages/django/core/management/commands/syncdb.py”, line 25, in handle
kpi_1 | call_command(“migrate”, **options)
kpi_1 | File “/usr/local/lib/python2.7/dist-packages/django/core/management/init.py”, line 120, in call_command
kpi_1 | return command.execute(*args, **defaults)
kpi_1 | File “/usr/local/lib/python2.7/dist-packages/django/core/management/base.py”, line 445, in execute
kpi_1 | output = self.handle(*args, **options)
kpi_1 | File “/usr/local/lib/python2.7/dist-packages/django/core/management/commands/migrate.py”, line 222, in handle
kpi_1 | executor.migrate(targets, plan, fake=fake, fake_initial=fake_initial)
kpi_1 | File “/usr/local/lib/python2.7/dist-packages/django/db/migrations/executor.py”, line 110, in migrate
kpi_1 | self.apply_migration(states[migration], migration, fake=fake, fake_initial=fake_initial)
kpi_1 | File “/usr/local/lib/python2.7/dist-packages/django/db/migrations/executor.py”, line 148, in apply_migration
kpi_1 | state = migration.apply(state, schema_editor)
kpi_1 | File “/usr/local/lib/python2.7/dist-packages/django/db/migrations/migration.py”, line 115, in apply
kpi_1 | operation.database_forwards(self.app_label, schema_editor, old_state, project_state)
kpi_1 | File “/usr/local/lib/python2.7/dist-packages/django/db/migrations/operations/special.py”, line 183, in database_forwards
kpi_1 | self.code(from_state.apps, schema_editor)
kpi_1 | File “/srv/src/kpi/kpi/migrations/0015_assetversion.py”, line 38, in copy_reversion_to_assetversion
kpi_1 | apps.get_model(‘reversion’, ‘Version’),
kpi_1 | File “/srv/src/kpi/kpi/management/commands/populate_assetversions.py”, line 49, in populate_assetversions
kpi_1 | _replace_deployment_ids(_AssetVersion, _Asset)
kpi_1 | File “/srv/src/kpi/kpi/management/commands/populate_assetversions.py”, line 121, in _replace_deployment_ids
kpi_1 | asset.save(adjust_content=False, create_version=False)
kpi_1 | File “/usr/local/lib/python2.7/dist-packages/django/utils/decorators.py”, line 145, in inner
kpi_1 | return func(*args, **kwargs)
kpi_1 | File “/srv/src/kpi/kpi/models/object_permission.py”, line 245, in save
kpi_1 | super(ObjectPermissionMixin, self).save(*args, **kwargs)
kpi_1 | TypeError: save() got an unexpected keyword argument ‘adjust_content’

After debugging I found that the migration script 0015 which is there in KPI fails to migrate with the “Save() got an unexpected keyword argument ‘adjust_content’” error

The new release works if I do not take the old database (where there are 10 thousand records ). How do I keep my old data with the new table changes done in KPI? Any support on this issue.

Regards,
Amar

Esmail_Fadae · October 21, 2016, 4:35pm

Hi, Amar. The problem relates to a database migration command that will need to be executed to use your old records with the new code. We’re working on a fix and will let you know when it’s ready.

Regards,
Esmail

···

On Friday, October 21, 2016 at 6:04:05 AM UTC-4, Amar Kamthe wrote:

Hi All,

We already are using the kobo-docker for shelter associates to capture slum level data. They are around 10 thousand records captured. The kobo-docker is the old one with redeploy functionality. As in this week there was a new release done by kobotoolbox where there was a fix done for option and cascading issue, I did a docker-compose pull . And after completely the process I did a docker-compose up. The docker-compose up command fails, below is the stacktrace.

kpi_1 | File “manage.py”, line 10, in
kpi_1 | execute_from_command_line(sys.
argv)
kpi_1 | File “/usr/local/lib/python2.7/dist-packages/django/core/management/init.py”, line 354, in execute_from_command_line
kpi_1 | utility.execute()
kpi_1 | File “/usr/local/lib/python2.7/dist-packages/django/core/management/init.py”, line 346, in execute
kpi_1 | self.fetch_command(subcommand).run_from_argv(self.argv)
kpi_1 | File “/usr/local/lib/python2.7/dist-packages/django/core/management/base.py”, line 394, in run_from_argv
kpi_1 | self.execute(*args, **cmd_options)
kpi_1 | File “/usr/local/lib/python2.7/dist-packages/django/core/management/base.py”, line 445, in execute
kpi_1 | output = self.handle(*args, **options)
kpi_1 | File “/usr/local/lib/python2.7/dist-packages/django/core/management/commands/syncdb.py”, line 25, in handle
kpi_1 | call_command(“migrate”, **options)
kpi_1 | File “/usr/local/lib/python2.7/dist-packages/django/core/management/init.py”, line 120, in call_command
kpi_1 | return command.execute(*args, **defaults)
kpi_1 | File “/usr/local/lib/python2.7/dist-packages/django/core/management/base.py”, line 445, in execute
kpi_1 | output = self.handle(*args, **options)
kpi_1 | File “/usr/local/lib/python2.7/dist-packages/django/core/management/commands/migrate.py”, line 222, in handle
kpi_1 | executor.migrate(targets, plan, fake=fake, fake_initial=fake_initial)
kpi_1 | File “/usr/local/lib/python2.7/dist-packages/django/db/migrations/executor.py”, line 110, in migrate
kpi_1 | self.apply_migration(states[migration], migration, fake=fake, fake_initial=fake_initial)
kpi_1 | File “/usr/local/lib/python2.7/dist-packages/django/db/migrations/executor.py”, line 148, in apply_migration
kpi_1 | state = migration.apply(state, schema_editor)
kpi_1 | File “/usr/local/lib/python2.7/dist-packages/django/db/migrations/migration.py”, line 115, in apply
kpi_1 | operation.database_forwards(self.app_label, schema_editor, old_state, project_state)
kpi_1 | File “/usr/local/lib/python2.7/dist-packages/django/db/migrations/operations/special.py”, line 183, in database_forwards
kpi_1 | self.code(from_state.apps, schema_editor)
kpi_1 | File “/srv/src/kpi/kpi/migrations/0015_assetversion.py”, line 38, in copy_reversion_to_assetversion
kpi_1 | apps.get_model(‘reversion’, ‘Version’),
kpi_1 | File “/srv/src/kpi/kpi/management/commands/populate_assetversions.py”, line 49, in populate_assetversions
kpi_1 | _replace_deployment_ids(_AssetVersion, _Asset)
kpi_1 | File “/srv/src/kpi/kpi/management/commands/populate_assetversions.py”, line 121, in _replace_deployment_ids
kpi_1 | asset.save(adjust_content=False, create_version=False)
kpi_1 | File “/usr/local/lib/python2.7/dist-packages/django/utils/decorators.py”, line 145, in inner
kpi_1 | return func(*args, **kwargs)
kpi_1 | File “/srv/src/kpi/kpi/models/object_permission.py”, line 245, in save
kpi_1 | super(ObjectPermissionMixin, self).save(*args, **kwargs)
kpi_1 | TypeError: save() got an unexpected keyword argument ‘adjust_content’

After debugging I found that the migration script 0015 which is there in KPI fails to migrate with the “Save() got an unexpected keyword argument ‘adjust_content’” error

The new release works if I do not take the old database (where there are 10 thousand records ). How do I keep my old data with the new table changes done in KPI? Any support on this issue.

Regards,
Amar

Esmail_Fadae · October 24, 2016, 7:33pm

Hi, Amar. Apologies for the delay in getting back to you. We have a fix in the latest kpi image, but before putting it to use, I’d strongly recommend making a backup of your Postgres database.

To do so, first make sure your postgres container is running with:
docker-compose up -d postgres

``

Next, execute the backup with:
docker exec -it kobodocker_postgres_1 bash -c “su -c ‘pg_dump --compress=1 --format=custom kobotoolbox’ postgres > /srv/backups/$(date +%Y.%m.%d.%H_%M_%S)__kobotoolbox.pgrestore”

``

Now you should be able to see your backup with:
ls -lh backups/postgres

``

If that all went well, you’re ready to pull the latest image and run it. Please note that the first time you run the image, it will take some time to migrate your existing DB depending on how large it is. I’d recommend doing a run with minimal services this first time with:
docker-compose up -d kpi && docker-compose logs -f

``

Once the migration’s complete, you’ll be able to use the container normally. Let us know how it goes.

Regards,
Esmail

···

On Friday, October 21, 2016 at 12:35:04 PM UTC-4, Esmail Fadae wrote:

Hi, Amar. The problem relates to a database migration command that will need to be executed to use your old records with the new code. We’re working on a fix and will let you know when it’s ready.

Regards,
Esmail

On Friday, October 21, 2016 at 6:04:05 AM UTC-4, Amar Kamthe wrote:

Hi All,

We already are using the kobo-docker for shelter associates to capture slum level data. They are around 10 thousand records captured. The kobo-docker is the old one with redeploy functionality. As in this week there was a new release done by kobotoolbox where there was a fix done for option and cascading issue, I did a docker-compose pull . And after completely the process I did a docker-compose up. The docker-compose up command fails, below is the stacktrace.

kpi_1 | File “manage.py”, line 10, in
kpi_1 | execute_from_command_line(sys.
argv)
kpi_1 | File “/usr/local/lib/python2.7/dist-packages/django/core/management/init.py”, line 354, in execute_from_command_line
kpi_1 | utility.execute()
kpi_1 | File “/usr/local/lib/python2.7/dist-packages/django/core/management/init.py”, line 346, in execute
kpi_1 | self.fetch_command(subcommand).run_from_argv(self.argv)
kpi_1 | File “/usr/local/lib/python2.7/dist-packages/django/core/management/base.py”, line 394, in run_from_argv
kpi_1 | self.execute(*args, **cmd_options)
kpi_1 | File “/usr/local/lib/python2.7/dist-packages/django/core/management/base.py”, line 445, in execute
kpi_1 | output = self.handle(*args, **options)
kpi_1 | File “/usr/local/lib/python2.7/dist-packages/django/core/management/commands/syncdb.py”, line 25, in handle
kpi_1 | call_command(“migrate”, **options)
kpi_1 | File “/usr/local/lib/python2.7/dist-packages/django/core/management/init.py”, line 120, in call_command
kpi_1 | return command.execute(*args, **defaults)
kpi_1 | File “/usr/local/lib/python2.7/dist-packages/django/core/management/base.py”, line 445, in execute
kpi_1 | output = self.handle(*args, **options)
kpi_1 | File “/usr/local/lib/python2.7/dist-packages/django/core/management/commands/migrate.py”, line 222, in handle
kpi_1 | executor.migrate(targets, plan, fake=fake, fake_initial=fake_initial)
kpi_1 | File “/usr/local/lib/python2.7/dist-packages/django/db/migrations/executor.py”, line 110, in migrate
kpi_1 | self.apply_migration(states[migration], migration, fake=fake, fake_initial=fake_initial)
kpi_1 | File “/usr/local/lib/python2.7/dist-packages/django/db/migrations/executor.py”, line 148, in apply_migration
kpi_1 | state = migration.apply(state, schema_editor)
kpi_1 | File “/usr/local/lib/python2.7/dist-packages/django/db/migrations/migration.py”, line 115, in apply
kpi_1 | operation.database_forwards(self.app_label, schema_editor, old_state, project_state)
kpi_1 | File “/usr/local/lib/python2.7/dist-packages/django/db/migrations/operations/special.py”, line 183, in database_forwards
kpi_1 | self.code(from_state.apps, schema_editor)
kpi_1 | File “/srv/src/kpi/kpi/migrations/0015_assetversion.py”, line 38, in copy_reversion_to_assetversion
kpi_1 | apps.get_model(‘reversion’, ‘Version’),
kpi_1 | File “/srv/src/kpi/kpi/management/commands/populate_assetversions.py”, line 49, in populate_assetversions
kpi_1 | _replace_deployment_ids(_AssetVersion, _Asset)
kpi_1 | File “/srv/src/kpi/kpi/management/commands/populate_assetversions.py”, line 121, in _replace_deployment_ids
kpi_1 | asset.save(adjust_content=False, create_version=False)
kpi_1 | File “/usr/local/lib/python2.7/dist-packages/django/utils/decorators.py”, line 145, in inner
kpi_1 | return func(*args, **kwargs)
kpi_1 | File “/srv/src/kpi/kpi/models/object_permission.py”, line 245, in save
kpi_1 | super(ObjectPermissionMixin, self).save(*args, **kwargs)
kpi_1 | TypeError: save() got an unexpected keyword argument ‘adjust_content’

After debugging I found that the migration script 0015 which is there in KPI fails to migrate with the “Save() got an unexpected keyword argument ‘adjust_content’” error

The new release works if I do not take the old database (where there are 10 thousand records ). How do I keep my old data with the new table changes done in KPI? Any support on this issue.

Regards,
Amar

Amar_Kamthe · October 25, 2016, 1:22pm

Esmail : Thanks for the backup commands and the fix for migration issue. I was able to run kobo-docker with successful migration.

After completing the process I could see new columns added to kpi_asset and new table kpi_assetversion. When I login to the KPI application I do not see the forms that were created by the user. But I could see the deployed survey’s in the projects section. Also the “view data in table” gives 500 error. Any help on this?

Regards,
Amar

···

On Tuesday, October 25, 2016 at 1:03:24 AM UTC+5:30, Esmail Fadae wrote:

Hi, Amar. Apologies for the delay in getting back to you. We have a fix in the latest kpi image, but before putting it to use, I’d strongly recommend making a backup of your Postgres database.

To do so, first make sure your postgres container is running with:
docker-compose up -d postgres

``

Next, execute the backup with:
docker exec -it kobodocker_postgres_1 bash -c “su -c ‘pg_dump --compress=1 --format=custom kobotoolbox’ postgres > /srv/backups/$(date +%Y.%m.%d.%H_%M_%S)__kobotoolbox.pgrestore”

``

Now you should be able to see your backup with:
ls -lh backups/postgres

``

If that all went well, you’re ready to pull the latest image and run it. Please note that the first time you run the image, it will take some time to migrate your existing DB depending on how large it is. I’d recommend doing a run with minimal services this first time with:
docker-compose up -d kpi && docker-compose logs -f

``

Once the migration’s complete, you’ll be able to use the container normally. Let us know how it goes.

Regards,
Esmail

On Friday, October 21, 2016 at 12:35:04 PM UTC-4, Esmail Fadae wrote:

Hi, Amar. The problem relates to a database migration command that will need to be executed to use your old records with the new code. We’re working on a fix and will let you know when it’s ready.

Regards,
Esmail

On Friday, October 21, 2016 at 6:04:05 AM UTC-4, Amar Kamthe wrote:

Hi All,

We already are using the kobo-docker for shelter associates to capture slum level data. They are around 10 thousand records captured. The kobo-docker is the old one with redeploy functionality. As in this week there was a new release done by kobotoolbox where there was a fix done for option and cascading issue, I did a docker-compose pull . And after completely the process I did a docker-compose up. The docker-compose up command fails, below is the stacktrace.

kpi_1 | File “manage.py”, line 10, in
kpi_1 | execute_from_command_line(sys.
argv)
kpi_1 | File “/usr/local/lib/python2.7/dist-packages/django/core/management/init.py”, line 354, in execute_from_command_line
kpi_1 | utility.execute()
kpi_1 | File “/usr/local/lib/python2.7/dist-packages/django/core/management/init.py”, line 346, in execute
kpi_1 | self.fetch_command(subcommand).run_from_argv(self.argv)
kpi_1 | File “/usr/local/lib/python2.7/dist-packages/django/core/management/base.py”, line 394, in run_from_argv
kpi_1 | self.execute(*args, **cmd_options)
kpi_1 | File “/usr/local/lib/python2.7/dist-packages/django/core/management/base.py”, line 445, in execute
kpi_1 | output = self.handle(*args, **options)
kpi_1 | File “/usr/local/lib/python2.7/dist-packages/django/core/management/commands/syncdb.py”, line 25, in handle
kpi_1 | call_command(“migrate”, **options)
kpi_1 | File “/usr/local/lib/python2.7/dist-packages/django/core/management/init.py”, line 120, in call_command
kpi_1 | return command.execute(*args, **defaults)
kpi_1 | File “/usr/local/lib/python2.7/dist-packages/django/core/management/base.py”, line 445, in execute
kpi_1 | output = self.handle(*args, **options)
kpi_1 | File “/usr/local/lib/python2.7/dist-packages/django/core/management/commands/migrate.py”, line 222, in handle
kpi_1 | executor.migrate(targets, plan, fake=fake, fake_initial=fake_initial)
kpi_1 | File “/usr/local/lib/python2.7/dist-packages/django/db/migrations/executor.py”, line 110, in migrate
kpi_1 | self.apply_migration(states[migration], migration, fake=fake, fake_initial=fake_initial)
kpi_1 | File “/usr/local/lib/python2.7/dist-packages/django/db/migrations/executor.py”, line 148, in apply_migration
kpi_1 | state = migration.apply(state, schema_editor)
kpi_1 | File “/usr/local/lib/python2.7/dist-packages/django/db/migrations/migration.py”, line 115, in apply
kpi_1 | operation.database_forwards(self.app_label, schema_editor, old_state, project_state)
kpi_1 | File “/usr/local/lib/python2.7/dist-packages/django/db/migrations/operations/special.py”, line 183, in database_forwards
kpi_1 | self.code(from_state.apps, schema_editor)
kpi_1 | File “/srv/src/kpi/kpi/migrations/0015_assetversion.py”, line 38, in copy_reversion_to_assetversion
kpi_1 | apps.get_model(‘reversion’, ‘Version’),
kpi_1 | File “/srv/src/kpi/kpi/management/commands/populate_assetversions.py”, line 49, in populate_assetversions
kpi_1 | _replace_deployment_ids(_AssetVersion, _Asset)
kpi_1 | File “/srv/src/kpi/kpi/management/commands/populate_assetversions.py”, line 121, in _replace_deployment_ids
kpi_1 | asset.save(adjust_content=False, create_version=False)
kpi_1 | File “/usr/local/lib/python2.7/dist-packages/django/utils/decorators.py”, line 145, in inner
kpi_1 | return func(*args, **kwargs)
kpi_1 | File “/srv/src/kpi/kpi/models/object_permission.py”, line 245, in save
kpi_1 | super(ObjectPermissionMixin, self).save(*args, **kwargs)
kpi_1 | TypeError: save() got an unexpected keyword argument ‘adjust_content’

After debugging I found that the migration script 0015 which is there in KPI fails to migrate with the “Save() got an unexpected keyword argument ‘adjust_content’” error

The new release works if I do not take the old database (where there are 10 thousand records ). How do I keep my old data with the new table changes done in KPI? Any support on this issue.

Regards,
Amar

Esmail_Fadae · October 26, 2016, 12:43pm

Hi, Amar. Sorry to hear that didn’t work. Could you navigate to the KPI API’s assets endpoint (i.e. ${YOUR_KPI_URL}/assets) and let us know if your forms are listed there? Also, any network or console errors from your browser’s inspector that might show up when navigating to KPI normally might help.

Regarding the 500 error, can you confirm that KoBoCAT is running and you’re able to access it in your browser?

···

On Tuesday, October 25, 2016 at 9:22:16 AM UTC-4, Amar Kamthe wrote:

Esmail : Thanks for the backup commands and the fix for migration issue. I was able to run kobo-docker with successful migration.

After completing the process I could see new columns added to kpi_asset and new table kpi_assetversion. When I login to the KPI application I do not see the forms that were created by the user. But I could see the deployed survey’s in the projects section. Also the “view data in table” gives 500 error. Any help on this?

Regards,
Amar

On Tuesday, October 25, 2016 at 1:03:24 AM UTC+5:30, Esmail Fadae wrote:

Hi, Amar. Apologies for the delay in getting back to you. We have a fix in the latest kpi image, but before putting it to use, I’d strongly recommend making a backup of your Postgres database.

To do so, first make sure your postgres container is running with:
docker-compose up -d postgres

``

Next, execute the backup with:
docker exec -it kobodocker_postgres_1 bash -c “su -c ‘pg_dump --compress=1 --format=custom kobotoolbox’ postgres > /srv/backups/$(date +%Y.%m.%d.%H_%M_%S)__kobotoolbox.pgrestore”

``

Now you should be able to see your backup with:
ls -lh backups/postgres

``

If that all went well, you’re ready to pull the latest image and run it. Please note that the first time you run the image, it will take some time to migrate your existing DB depending on how large it is. I’d recommend doing a run with minimal services this first time with:
docker-compose up -d kpi && docker-compose logs -f

``

Once the migration’s complete, you’ll be able to use the container normally. Let us know how it goes.

Regards,
Esmail

On Friday, October 21, 2016 at 12:35:04 PM UTC-4, Esmail Fadae wrote:

Hi, Amar. The problem relates to a database migration command that will need to be executed to use your old records with the new code. We’re working on a fix and will let you know when it’s ready.

Regards,
Esmail

On Friday, October 21, 2016 at 6:04:05 AM UTC-4, Amar Kamthe wrote:

Hi All,

We already are using the kobo-docker for shelter associates to capture slum level data. They are around 10 thousand records captured. The kobo-docker is the old one with redeploy functionality. As in this week there was a new release done by kobotoolbox where there was a fix done for option and cascading issue, I did a docker-compose pull . And after completely the process I did a docker-compose up. The docker-compose up command fails, below is the stacktrace.

kpi_1 | File “manage.py”, line 10, in
kpi_1 | execute_from_command_line(sys.
argv)
kpi_1 | File “/usr/local/lib/python2.7/dist-packages/django/core/management/init.py”, line 354, in execute_from_command_line
kpi_1 | utility.execute()
kpi_1 | File “/usr/local/lib/python2.7/dist-packages/django/core/management/init.py”, line 346, in execute
kpi_1 | self.fetch_command(subcommand).run_from_argv(self.argv)
kpi_1 | File “/usr/local/lib/python2.7/dist-packages/django/core/management/base.py”, line 394, in run_from_argv
kpi_1 | self.execute(*args, **cmd_options)
kpi_1 | File “/usr/local/lib/python2.7/dist-packages/django/core/management/base.py”, line 445, in execute
kpi_1 | output = self.handle(*args, **options)
kpi_1 | File “/usr/local/lib/python2.7/dist-packages/django/core/management/commands/syncdb.py”, line 25, in handle
kpi_1 | call_command(“migrate”, **options)
kpi_1 | File “/usr/local/lib/python2.7/dist-packages/django/core/management/init.py”, line 120, in call_command
kpi_1 | return command.execute(*args, **defaults)
kpi_1 | File “/usr/local/lib/python2.7/dist-packages/django/core/management/base.py”, line 445, in execute
kpi_1 | output = self.handle(*args, **options)
kpi_1 | File “/usr/local/lib/python2.7/dist-packages/django/core/management/commands/migrate.py”, line 222, in handle
kpi_1 | executor.migrate(targets, plan, fake=fake, fake_initial=fake_initial)
kpi_1 | File “/usr/local/lib/python2.7/dist-packages/django/db/migrations/executor.py”, line 110, in migrate
kpi_1 | self.apply_migration(states[migration], migration, fake=fake, fake_initial=fake_initial)
kpi_1 | File “/usr/local/lib/python2.7/dist-packages/django/db/migrations/executor.py”, line 148, in apply_migration
kpi_1 | state = migration.apply(state, schema_editor)
kpi_1 | File “/usr/local/lib/python2.7/dist-packages/django/db/migrations/migration.py”, line 115, in apply
kpi_1 | operation.database_forwards(self.app_label, schema_editor, old_state, project_state)
kpi_1 | File “/usr/local/lib/python2.7/dist-packages/django/db/migrations/operations/special.py”, line 183, in database_forwards
kpi_1 | self.code(from_state.apps, schema_editor)
kpi_1 | File “/srv/src/kpi/kpi/migrations/0015_assetversion.py”, line 38, in copy_reversion_to_assetversion
kpi_1 | apps.get_model(‘reversion’, ‘Version’),
kpi_1 | File “/srv/src/kpi/kpi/management/commands/populate_assetversions.py”, line 49, in populate_assetversions
kpi_1 | _replace_deployment_ids(_AssetVersion, _Asset)
kpi_1 | File “/srv/src/kpi/kpi/management/commands/populate_assetversions.py”, line 121, in _replace_deployment_ids
kpi_1 | asset.save(adjust_content=False, create_version=False)
kpi_1 | File “/usr/local/lib/python2.7/dist-packages/django/utils/decorators.py”, line 145, in inner
kpi_1 | return func(*args, **kwargs)
kpi_1 | File “/srv/src/kpi/kpi/models/object_permission.py”, line 245, in save
kpi_1 | super(ObjectPermissionMixin, self).save(*args, **kwargs)
kpi_1 | TypeError: save() got an unexpected keyword argument ‘adjust_content’

After debugging I found that the migration script 0015 which is there in KPI fails to migrate with the “Save() got an unexpected keyword argument ‘adjust_content’” error

The new release works if I do not take the old database (where there are 10 thousand records ). How do I keep my old data with the new table changes done in KPI? Any support on this issue.

Regards,
Amar

Amar_Kamthe · October 27, 2016, 1:38pm

Esmail :
Using ${YOUR_KPI_URL}/assets I could see all the forms. Debugging this issue, what I found was on login there is ``${YOUR_KPI_URL}/assets?q=survey_type:survey url which returns zero records. Unfortunately there is no error display/logged in file as the URL successfully works with 200 status code. The response is {'count':0, 'next':null, 'previous':null, 'results':[]} I navigated directly to below listed API's,

``${YOUR_KPI_URL}/assets?q=survey_type:survey which returned zero records whereas
``` ${YOUR_KPI_URL}/assets returned all the forms`
${YOUR_KPI_URL}/assets?survey_type=survey this returns all the forms
If I create a form using KPI this shows up in all the above listed API’s, but the old forms are not displayed.

`About the KoBoCAT issue : It is running, I could login to kobocat app and could see the forms that were deployed. I could even navigate to one of the deployed project where I see a screen of links to submission, Photos, and Add Data. Here when I click on “View data in table” which is there under submission it gives 500 error and a message
“If this is significant error that needs to be fixed, please send a support request to sup...@kobotoolbox.org with
information about what triggered this error.”
There is no error found in log files.

Waiting for you support.

Regards,
Amar
`

···

On Wednesday, October 26, 2016 at 6:13:39 PM UTC+5:30, Esmail Fadae wrote:

Hi, Amar. Sorry to hear that didn’t work. Could you navigate to the KPI API’s assets endpoint (i.e. ${YOUR_KPI_URL}/assets) and let us know if your forms are listed there? Also, any network or console errors from your browser’s inspector that might show up when navigating to KPI normally might help.

Regarding the 500 error, can you confirm that KoBoCAT is running and you’re able to access it in your browser?

On Tuesday, October 25, 2016 at 9:22:16 AM UTC-4, Amar Kamthe wrote:

Esmail : Thanks for the backup commands and the fix for migration issue. I was able to run kobo-docker with successful migration.

After completing the process I could see new columns added to kpi_asset and new table kpi_assetversion. When I login to the KPI application I do not see the forms that were created by the user. But I could see the deployed survey’s in the projects section. Also the “view data in table” gives 500 error. Any help on this?

Regards,
Amar

On Tuesday, October 25, 2016 at 1:03:24 AM UTC+5:30, Esmail Fadae wrote:

Hi, Amar. Apologies for the delay in getting back to you. We have a fix in the latest kpi image, but before putting it to use, I’d strongly recommend making a backup of your Postgres database.

To do so, first make sure your postgres container is running with:
docker-compose up -d postgres

``

Next, execute the backup with:
docker exec -it kobodocker_postgres_1 bash -c “su -c ‘pg_dump --compress=1 --format=custom kobotoolbox’ postgres > /srv/backups/$(date +%Y.%m.%d.%H_%M_%S)__kobotoolbox.pgrestore”

``

Now you should be able to see your backup with:
ls -lh backups/postgres

``

If that all went well, you’re ready to pull the latest image and run it. Please note that the first time you run the image, it will take some time to migrate your existing DB depending on how large it is. I’d recommend doing a run with minimal services this first time with:
docker-compose up -d kpi && docker-compose logs -f

``

Once the migration’s complete, you’ll be able to use the container normally. Let us know how it goes.

Regards,
Esmail

On Friday, October 21, 2016 at 12:35:04 PM UTC-4, Esmail Fadae wrote:

Hi, Amar. The problem relates to a database migration command that will need to be executed to use your old records with the new code. We’re working on a fix and will let you know when it’s ready.

Regards,
Esmail

On Friday, October 21, 2016 at 6:04:05 AM UTC-4, Amar Kamthe wrote:

Hi All,

We already are using the kobo-docker for shelter associates to capture slum level data. They are around 10 thousand records captured. The kobo-docker is the old one with redeploy functionality. As in this week there was a new release done by kobotoolbox where there was a fix done for option and cascading issue, I did a docker-compose pull . And after completely the process I did a docker-compose up. The docker-compose up command fails, below is the stacktrace.

kpi_1 | File “manage.py”, line 10, in
kpi_1 | execute_from_command_line(sys.
argv)
kpi_1 | File “/usr/local/lib/python2.7/dist-packages/django/core/management/init.py”, line 354, in execute_from_command_line
kpi_1 | utility.execute()
kpi_1 | File “/usr/local/lib/python2.7/dist-packages/django/core/management/init.py”, line 346, in execute
kpi_1 | self.fetch_command(subcommand).run_from_argv(self.argv)
kpi_1 | File “/usr/local/lib/python2.7/dist-packages/django/core/management/base.py”, line 394, in run_from_argv
kpi_1 | self.execute(*args, **cmd_options)
kpi_1 | File “/usr/local/lib/python2.7/dist-packages/django/core/management/base.py”, line 445, in execute
kpi_1 | output = self.handle(*args, **options)
kpi_1 | File “/usr/local/lib/python2.7/dist-packages/django/core/management/commands/syncdb.py”, line 25, in handle
kpi_1 | call_command(“migrate”, **options)
kpi_1 | File “/usr/local/lib/python2.7/dist-packages/django/core/management/init.py”, line 120, in call_command
kpi_1 | return command.execute(*args, **defaults)
kpi_1 | File “/usr/local/lib/python2.7/dist-packages/django/core/management/base.py”, line 445, in execute
kpi_1 | output = self.handle(*args, **options)
kpi_1 | File “/usr/local/lib/python2.7/dist-packages/django/core/management/commands/migrate.py”, line 222, in handle
kpi_1 | executor.migrate(targets, plan, fake=fake, fake_initial=fake_initial)
kpi_1 | File “/usr/local/lib/python2.7/dist-packages/django/db/migrations/executor.py”, line 110, in migrate
kpi_1 | self.apply_migration(states[migration], migration, fake=fake, fake_initial=fake_initial)
kpi_1 | File “/usr/local/lib/python2.7/dist-packages/django/db/migrations/executor.py”, line 148, in apply_migration
kpi_1 | state = migration.apply(state, schema_editor)
kpi_1 | File “/usr/local/lib/python2.7/dist-packages/django/db/migrations/migration.py”, line 115, in apply
kpi_1 | operation.database_forwards(self.app_label, schema_editor, old_state, project_state)
kpi_1 | File “/usr/local/lib/python2.7/dist-packages/django/db/migrations/operations/special.py”, line 183, in database_forwards
kpi_1 | self.code(from_state.apps, schema_editor)
kpi_1 | File “/srv/src/kpi/kpi/migrations/0015_assetversion.py”, line 38, in copy_reversion_to_assetversion
kpi_1 | apps.get_model(‘reversion’, ‘Version’),
kpi_1 | File “/srv/src/kpi/kpi/management/commands/populate_assetversions.py”, line 49, in populate_assetversions
kpi_1 | _replace_deployment_ids(_AssetVersion, _Asset)
kpi_1 | File “/srv/src/kpi/kpi/management/commands/populate_assetversions.py”, line 121, in _replace_deployment_ids
kpi_1 | asset.save(adjust_content=False, create_version=False)
kpi_1 | File “/usr/local/lib/python2.7/dist-packages/django/utils/decorators.py”, line 145, in inner
kpi_1 | return func(*args, **kwargs)
kpi_1 | File “/srv/src/kpi/kpi/models/object_permission.py”, line 245, in save
kpi_1 | super(ObjectPermissionMixin, self).save(*args, **kwargs)
kpi_1 | TypeError: save() got an unexpected keyword argument ‘adjust_content’

After debugging I found that the migration script 0015 which is there in KPI fails to migrate with the “Save() got an unexpected keyword argument ‘adjust_content’” error

The new release works if I do not take the old database (where there are 10 thousand records ). How do I keep my old data with the new table changes done in KPI? Any support on this issue.

Regards,
Amar

Esmail_Fadae · October 27, 2016, 6:38pm

Okay; so the KPI issue appears to be due to an update to the Whoosh schema in our code. Until we come up with something automatic, the manual workaround (which might take some time to complete) is:
docker-compose exec -it kobodocker_kpi_1 python manage.py rebuild_index --noinput

``

Regarding the KoBoCAT issue, turn on KOBOCAT_DJANGO_DEBUG=True in envfiles/kobocat.txt and let us know what exception is occurring.

···

On Thursday, October 27, 2016 at 9:38:35 AM UTC-4, Amar Kamthe wrote:

Esmail :
Using ${YOUR_KPI_URL}/assets I could see all the forms. Debugging this issue, what I found was on login there is ``${YOUR_KPI_URL}/assets?q=survey_type:survey url which returns zero records. Unfortunately there is no error display/logged in file as the URL successfully works with 200 status code. The response is {'count':0, 'next':null, 'previous':null, 'results':[]} I navigated directly to below listed API's,

``${YOUR_KPI_URL}/assets?q=survey_type:survey which returned zero records whereas

``` ${YOUR_KPI_URL}/assets returned all the forms`

${YOUR_KPI_URL}/assets?survey_type=survey this returns all the forms
If I create a form using KPI this shows up in all the above listed API’s, but the old forms are not displayed.

`About the KoBoCAT issue : It is running, I could login to kobocat app and could see the forms that were deployed. I could even navigate to one of the deployed project where I see a screen of links to submission, Photos, and Add Data. Here when I click on “View data in table” which is there under submission it gives 500 error and a message
“If this is significant error that needs to be fixed, please send a support request to sup...@kobotoolbox.org with
information about what triggered this error.”
There is no error found in log files.

Waiting for you support.

Regards,
Amar
`
On Wednesday, October 26, 2016 at 6:13:39 PM UTC+5:30, Esmail Fadae wrote:

Hi, Amar. Sorry to hear that didn’t work. Could you navigate to the KPI API’s assets endpoint (i.e. ${YOUR_KPI_URL}/assets) and let us know if your forms are listed there? Also, any network or console errors from your browser’s inspector that might show up when navigating to KPI normally might help.

Regarding the 500 error, can you confirm that KoBoCAT is running and you’re able to access it in your browser?

On Tuesday, October 25, 2016 at 9:22:16 AM UTC-4, Amar Kamthe wrote:

Esmail : Thanks for the backup commands and the fix for migration issue. I was able to run kobo-docker with successful migration.

After completing the process I could see new columns added to kpi_asset and new table kpi_assetversion. When I login to the KPI application I do not see the forms that were created by the user. But I could see the deployed survey’s in the projects section. Also the “view data in table” gives 500 error. Any help on this?

Regards,
Amar

On Tuesday, October 25, 2016 at 1:03:24 AM UTC+5:30, Esmail Fadae wrote:

Hi, Amar. Apologies for the delay in getting back to you. We have a fix in the latest kpi image, but before putting it to use, I’d strongly recommend making a backup of your Postgres database.

To do so, first make sure your postgres container is running with:
docker-compose up -d postgres

``

Next, execute the backup with:
docker exec -it kobodocker_postgres_1 bash -c “su -c ‘pg_dump --compress=1 --format=custom kobotoolbox’ postgres > /srv/backups/$(date +%Y.%m.%d.%H_%M_%S)__kobotoolbox.pgrestore”

``

Now you should be able to see your backup with:
ls -lh backups/postgres

``

If that all went well, you’re ready to pull the latest image and run it. Please note that the first time you run the image, it will take some time to migrate your existing DB depending on how large it is. I’d recommend doing a run with minimal services this first time with:
docker-compose up -d kpi && docker-compose logs -f

``

Once the migration’s complete, you’ll be able to use the container normally. Let us know how it goes.

Regards,
Esmail

On Friday, October 21, 2016 at 12:35:04 PM UTC-4, Esmail Fadae wrote:

Hi, Amar. The problem relates to a database migration command that will need to be executed to use your old records with the new code. We’re working on a fix and will let you know when it’s ready.

Regards,
Esmail

On Friday, October 21, 2016 at 6:04:05 AM UTC-4, Amar Kamthe wrote:

Hi All,

We already are using the kobo-docker for shelter associates to capture slum level data. They are around 10 thousand records captured. The kobo-docker is the old one with redeploy functionality. As in this week there was a new release done by kobotoolbox where there was a fix done for option and cascading issue, I did a docker-compose pull . And after completely the process I did a docker-compose up. The docker-compose up command fails, below is the stacktrace.

kpi_1 | File “manage.py”, line 10, in
kpi_1 | execute_from_command_line(sys.
argv)
kpi_1 | File “/usr/local/lib/python2.7/dist-packages/django/core/management/init.py”, line 354, in execute_from_command_line
kpi_1 | utility.execute()
kpi_1 | File “/usr/local/lib/python2.7/dist-packages/django/core/management/init.py”, line 346, in execute
kpi_1 | self.fetch_command(subcommand).run_from_argv(self.argv)
kpi_1 | File “/usr/local/lib/python2.7/dist-packages/django/core/management/base.py”, line 394, in run_from_argv
kpi_1 | self.execute(*args, **cmd_options)
kpi_1 | File “/usr/local/lib/python2.7/dist-packages/django/core/management/base.py”, line 445, in execute
kpi_1 | output = self.handle(*args, **options)
kpi_1 | File “/usr/local/lib/python2.7/dist-packages/django/core/management/commands/syncdb.py”, line 25, in handle
kpi_1 | call_command(“migrate”, **options)
kpi_1 | File “/usr/local/lib/python2.7/dist-packages/django/core/management/init.py”, line 120, in call_command
kpi_1 | return command.execute(*args, **defaults)
kpi_1 | File “/usr/local/lib/python2.7/dist-packages/django/core/management/base.py”, line 445, in execute
kpi_1 | output = self.handle(*args, **options)
kpi_1 | File “/usr/local/lib/python2.7/dist-packages/django/core/management/commands/migrate.py”, line 222, in handle
kpi_1 | executor.migrate(targets, plan, fake=fake, fake_initial=fake_initial)
kpi_1 | File “/usr/local/lib/python2.7/dist-packages/django/db/migrations/executor.py”, line 110, in migrate
kpi_1 | self.apply_migration(states[migration], migration, fake=fake, fake_initial=fake_initial)
kpi_1 | File “/usr/local/lib/python2.7/dist-packages/django/db/migrations/executor.py”, line 148, in apply_migration
kpi_1 | state = migration.apply(state, schema_editor)
kpi_1 | File “/usr/local/lib/python2.7/dist-packages/django/db/migrations/migration.py”, line 115, in apply
kpi_1 | operation.database_forwards(self.app_label, schema_editor, old_state, project_state)
kpi_1 | File “/usr/local/lib/python2.7/dist-packages/django/db/migrations/operations/special.py”, line 183, in database_forwards
kpi_1 | self.code(from_state.apps, schema_editor)
kpi_1 | File “/srv/src/kpi/kpi/migrations/0015_assetversion.py”, line 38, in copy_reversion_to_assetversion
kpi_1 | apps.get_model(‘reversion’, ‘Version’),
kpi_1 | File “/srv/src/kpi/kpi/management/commands/populate_assetversions.py”, line 49, in populate_assetversions
kpi_1 | _replace_deployment_ids(_AssetVersion, _Asset)
kpi_1 | File “/srv/src/kpi/kpi/management/commands/populate_assetversions.py”, line 121, in _replace_deployment_ids
kpi_1 | asset.save(adjust_content=False, create_version=False)
kpi_1 | File “/usr/local/lib/python2.7/dist-packages/django/utils/decorators.py”, line 145, in inner
kpi_1 | return func(*args, **kwargs)
kpi_1 | File “/srv/src/kpi/kpi/models/object_permission.py”, line 245, in save
kpi_1 | super(ObjectPermissionMixin, self).save(*args, **kwargs)
kpi_1 | TypeError: save() got an unexpected keyword argument ‘adjust_content’

After debugging I found that the migration script 0015 which is there in KPI fails to migrate with the “Save() got an unexpected keyword argument ‘adjust_content’” error

The new release works if I do not take the old database (where there are 10 thousand records ). How do I keep my old data with the new table changes done in KPI? Any support on this issue.

Regards,
Amar

Amar_Kamthe · October 28, 2016, 2:53pm

Esmail

Thanks for the support. The migration issue is now resolved.

KoBoCAT issue : I was able to figure out the problem. As you know I took old database from our live instance of docker. And I was using the dump locally, after making the change you suggested I could debugged the error and it was found that the csv files were missing. Sorry to bother you, this was because I forgot to copy those csv file from live kobocat instance. After copying the csv files to media folder the issue was resolved.

Regards,
Amar

···

On Friday, October 28, 2016 at 12:08:50 AM UTC+5:30, Esmail Fadae wrote:

Okay; so the KPI issue appears to be due to an update to the Whoosh schema in our code. Until we come up with something automatic, the manual workaround (which might take some time to complete) is:
docker-compose exec -it kobodocker_kpi_1 python manage.py rebuild_index --noinput

``

Regarding the KoBoCAT issue, turn on KOBOCAT_DJANGO_DEBUG=True in envfiles/kobocat.txt and let us know what exception is occurring.

On Thursday, October 27, 2016 at 9:38:35 AM UTC-4, Amar Kamthe wrote:

Esmail :
Using ${YOUR_KPI_URL}/assets I could see all the forms. Debugging this issue, what I found was on login there is ``${YOUR_KPI_URL}/assets?q=survey_type:survey url which returns zero records. Unfortunately there is no error display/logged in file as the URL successfully works with 200 status code. The response is {'count':0, 'next':null, 'previous':null, 'results':[]} I navigated directly to below listed API's,

``${YOUR_KPI_URL}/assets?q=survey_type:survey which returned zero records whereas

``` ${YOUR_KPI_URL}/assets returned all the forms`

${YOUR_KPI_URL}/assets?survey_type=survey this returns all the forms
If I create a form using KPI this shows up in all the above listed API’s, but the old forms are not displayed.

`About the KoBoCAT issue : It is running, I could login to kobocat app and could see the forms that were deployed. I could even navigate to one of the deployed project where I see a screen of links to submission, Photos, and Add Data. Here when I click on “View data in table” which is there under submission it gives 500 error and a message
“If this is significant error that needs to be fixed, please send a support request to sup...@kobotoolbox.org with
information about what triggered this error.”
There is no error found in log files.

Waiting for you support.

Regards,
Amar
`
On Wednesday, October 26, 2016 at 6:13:39 PM UTC+5:30, Esmail Fadae wrote:

Hi, Amar. Sorry to hear that didn’t work. Could you navigate to the KPI API’s assets endpoint (i.e. ${YOUR_KPI_URL}/assets) and let us know if your forms are listed there? Also, any network or console errors from your browser’s inspector that might show up when navigating to KPI normally might help.

Regarding the 500 error, can you confirm that KoBoCAT is running and you’re able to access it in your browser?

On Tuesday, October 25, 2016 at 9:22:16 AM UTC-4, Amar Kamthe wrote:

Esmail : Thanks for the backup commands and the fix for migration issue. I was able to run kobo-docker with successful migration.

After completing the process I could see new columns added to kpi_asset and new table kpi_assetversion. When I login to the KPI application I do not see the forms that were created by the user. But I could see the deployed survey’s in the projects section. Also the “view data in table” gives 500 error. Any help on this?

Regards,
Amar

On Tuesday, October 25, 2016 at 1:03:24 AM UTC+5:30, Esmail Fadae wrote:

Hi, Amar. Apologies for the delay in getting back to you. We have a fix in the latest kpi image, but before putting it to use, I’d strongly recommend making a backup of your Postgres database.

To do so, first make sure your postgres container is running with:
docker-compose up -d postgres

``

Next, execute the backup with:
docker exec -it kobodocker_postgres_1 bash -c “su -c ‘pg_dump --compress=1 --format=custom kobotoolbox’ postgres > /srv/backups/$(date +%Y.%m.%d.%H_%M_%S)__kobotoolbox.pgrestore”

``

Now you should be able to see your backup with:
ls -lh backups/postgres

``

If that all went well, you’re ready to pull the latest image and run it. Please note that the first time you run the image, it will take some time to migrate your existing DB depending on how large it is. I’d recommend doing a run with minimal services this first time with:
docker-compose up -d kpi && docker-compose logs -f

``

Once the migration’s complete, you’ll be able to use the container normally. Let us know how it goes.

Regards,
Esmail

On Friday, October 21, 2016 at 12:35:04 PM UTC-4, Esmail Fadae wrote:

Hi, Amar. The problem relates to a database migration command that will need to be executed to use your old records with the new code. We’re working on a fix and will let you know when it’s ready.

Regards,
Esmail

On Friday, October 21, 2016 at 6:04:05 AM UTC-4, Amar Kamthe wrote:

Hi All,

We already are using the kobo-docker for shelter associates to capture slum level data. They are around 10 thousand records captured. The kobo-docker is the old one with redeploy functionality. As in this week there was a new release done by kobotoolbox where there was a fix done for option and cascading issue, I did a docker-compose pull . And after completely the process I did a docker-compose up. The docker-compose up command fails, below is the stacktrace.

kpi_1 | File “manage.py”, line 10, in
kpi_1 | execute_from_command_line(sys.
argv)
kpi_1 | File “/usr/local/lib/python2.7/dist-packages/django/core/management/init.py”, line 354, in execute_from_command_line
kpi_1 | utility.execute()
kpi_1 | File “/usr/local/lib/python2.7/dist-packages/django/core/management/init.py”, line 346, in execute
kpi_1 | self.fetch_command(subcommand).run_from_argv(self.argv)
kpi_1 | File “/usr/local/lib/python2.7/dist-packages/django/core/management/base.py”, line 394, in run_from_argv
kpi_1 | self.execute(*args, **cmd_options)
kpi_1 | File “/usr/local/lib/python2.7/dist-packages/django/core/management/base.py”, line 445, in execute
kpi_1 | output = self.handle(*args, **options)
kpi_1 | File “/usr/local/lib/python2.7/dist-packages/django/core/management/commands/syncdb.py”, line 25, in handle
kpi_1 | call_command(“migrate”, **options)
kpi_1 | File “/usr/local/lib/python2.7/dist-packages/django/core/management/init.py”, line 120, in call_command
kpi_1 | return command.execute(*args, **defaults)
kpi_1 | File “/usr/local/lib/python2.7/dist-packages/django/core/management/base.py”, line 445, in execute
kpi_1 | output = self.handle(*args, **options)
kpi_1 | File “/usr/local/lib/python2.7/dist-packages/django/core/management/commands/migrate.py”, line 222, in handle
kpi_1 | executor.migrate(targets, plan, fake=fake, fake_initial=fake_initial)
kpi_1 | File “/usr/local/lib/python2.7/dist-packages/django/db/migrations/executor.py”, line 110, in migrate
kpi_1 | self.apply_migration(states[migration], migration, fake=fake, fake_initial=fake_initial)
kpi_1 | File “/usr/local/lib/python2.7/dist-packages/django/db/migrations/executor.py”, line 148, in apply_migration
kpi_1 | state = migration.apply(state, schema_editor)
kpi_1 | File “/usr/local/lib/python2.7/dist-packages/django/db/migrations/migration.py”, line 115, in apply
kpi_1 | operation.database_forwards(self.app_label, schema_editor, old_state, project_state)
kpi_1 | File “/usr/local/lib/python2.7/dist-packages/django/db/migrations/operations/special.py”, line 183, in database_forwards
kpi_1 | self.code(from_state.apps, schema_editor)
kpi_1 | File “/srv/src/kpi/kpi/migrations/0015_assetversion.py”, line 38, in copy_reversion_to_assetversion
kpi_1 | apps.get_model(‘reversion’, ‘Version’),
kpi_1 | File “/srv/src/kpi/kpi/management/commands/populate_assetversions.py”, line 49, in populate_assetversions
kpi_1 | _replace_deployment_ids(_AssetVersion, _Asset)
kpi_1 | File “/srv/src/kpi/kpi/management/commands/populate_assetversions.py”, line 121, in _replace_deployment_ids
kpi_1 | asset.save(adjust_content=False, create_version=False)
kpi_1 | File “/usr/local/lib/python2.7/dist-packages/django/utils/decorators.py”, line 145, in inner
kpi_1 | return func(*args, **kwargs)
kpi_1 | File “/srv/src/kpi/kpi/models/object_permission.py”, line 245, in save
kpi_1 | super(ObjectPermissionMixin, self).save(*args, **kwargs)
kpi_1 | TypeError: save() got an unexpected keyword argument ‘adjust_content’

After debugging I found that the migration script 0015 which is there in KPI fails to migrate with the “Save() got an unexpected keyword argument ‘adjust_content’” error

The new release works if I do not take the old database (where there are 10 thousand records ). How do I keep my old data with the new table changes done in KPI? Any support on this issue.

Regards,
Amar