Transfer data from kobotoolbox to docker

Hello,

We try to transfer data from kobotoolbox to docker, all form that we transfer are working fine but when we create new form in kpi and trying to upload new form data from kobocollect it shows an error(csrf token invalid).

Following Steps that we have done to transfer data

  1. Upload forms from kobotoolbox dkobo to docker kpi.

  2. Transfer Postgres data in logger_instance from kobotoolbox to docker (according to changes that required now in logger_instance)

  3. Transfer MOngo data in formhub from kobotoolbox to docker (according to changes that required now in mongodb)

Is there some thing we are missing while transferring data.

Regards,

Sitaram

Hello,

Just to give more context on this issue:

  1. No error is displayed in the UI. “CRSR token invalid” error is shown in the browser console.

  2. In the fresh install of docker instance, form creation is working as expected. So it might be possible that some steps might have been missed in our migration script.

  3. Here is the link to migration script: Shelter/scripts/pyscript.py at master · ShelterAssociates/Shelter · GitHub

It would be really great if someone can help us in resolving this issue.

Regards,

Kurund

···

On Monday, August 22, 2016 at 1:14:04 PM UTC+5:30, Sitaram Sawant wrote:

Hello,

We try to transfer data from kobotoolbox to docker, all form that we transfer are working fine but when we create new form in kpi and trying to upload new form data from kobocollect it shows an error(csrf token invalid).

Following Steps that we have done to transfer data

  1. Upload forms from kobotoolbox dkobo to docker kpi.
  1. Transfer Postgres data in logger_instance from kobotoolbox to docker (according to changes that required now in logger_instance)
  1. Transfer MOngo data in formhub from kobotoolbox to docker (according to changes that required now in mongodb)

Is there some thing we are missing while transferring data.

Regards,

Sitaram

Hello. Can you confirm that you’ve followed the setup procedure described in the kobo-docker repo and that the basic functions of your site are working for new forms first?

···

On Thursday, August 25, 2016 at 6:02:42 AM UTC-4, Kurund Jalmi wrote:

Hello,

Just to give more context on this issue:

  1. No error is displayed in the UI. “CRSR token invalid” error is shown in the browser console.
  1. In the fresh install of docker instance, form creation is working as expected. So it might be possible that some steps might have been missed in our migration script.
  1. Here is the link to migration script: https://github.com/ShelterAssociates/Shelter/blob/master/scripts/pyscript.py

It would be really great if someone can help us in resolving this issue.

Regards,

Kurund

On Monday, August 22, 2016 at 1:14:04 PM UTC+5:30, Sitaram Sawant wrote:

Hello,

We try to transfer data from kobotoolbox to docker, all form that we transfer are working fine but when we create new form in kpi and trying to upload new form data from kobocollect it shows an error(csrf token invalid).

Following Steps that we have done to transfer data

  1. Upload forms from kobotoolbox dkobo to docker kpi.
  1. Transfer Postgres data in logger_instance from kobotoolbox to docker (according to changes that required now in logger_instance)
  1. Transfer MOngo data in formhub from kobotoolbox to docker (according to changes that required now in mongodb)

Is there some thing we are missing while transferring data.

Regards,

Sitaram

Hello Esmail,

We have followed all steps of kobo-docker. Everything is working fine, if we do not migrate data. Once we migrate data from old Kobotoolbox instance to new kob-docker using script specified by Kurund above. We start seeing the data to the appropriate survey form and we can even edit using enketo app. The “CSRF token error” appears only when we create new form or upload data from kobo-collect app.

if we delete the migrated data everything works fine.

Regards,

Sitaram Sawant

···

On Friday, August 26, 2016 at 10:57:25 PM UTC+5:30, Esmail Fadae wrote:

Hello. Can you confirm that you’ve followed the setup procedure described in the kobo-docker repo and that the basic functions of your site are working for new forms first?

On Thursday, August 25, 2016 at 6:02:42 AM UTC-4, Kurund Jalmi wrote:

Hello,

Just to give more context on this issue:

  1. No error is displayed in the UI. “CRSR token invalid” error is shown in the browser console.
  1. In the fresh install of docker instance, form creation is working as expected. So it might be possible that some steps might have been missed in our migration script.
  1. Here is the link to migration script: https://github.com/ShelterAssociates/Shelter/blob/master/scripts/pyscript.py

It would be really great if someone can help us in resolving this issue.

Regards,

Kurund

On Monday, August 22, 2016 at 1:14:04 PM UTC+5:30, Sitaram Sawant wrote:

Hello,

We try to transfer data from kobotoolbox to docker, all form that we transfer are working fine but when we create new form in kpi and trying to upload new form data from kobocollect it shows an error(csrf token invalid).

Following Steps that we have done to transfer data

  1. Upload forms from kobotoolbox dkobo to docker kpi.
  1. Transfer Postgres data in logger_instance from kobotoolbox to docker (according to changes that required now in logger_instance)
  1. Transfer MOngo data in formhub from kobotoolbox to docker (according to changes that required now in mongodb)

Is there some thing we are missing while transferring data.

Regards,

Sitaram

Kobo-team:

any chance someone can take a look at this and advise us please. This issue is blocking the rollout on a wider basis

I suspect there is some data being transferred that the code does a check on. Happy to get on the phone/skype to discuss details. We are releasing all the work we do as open source also and will blog about it

thank you for your advice and help

lobo

···

On Monday, August 29, 2016 at 5:48:26 AM UTC-7, Sitaram Sawant wrote:

Hello Esmail,

We have followed all steps of kobo-docker. Everything is working fine, if we do not migrate data. Once we migrate data from old Kobotoolbox instance to new kob-docker using script specified by Kurund above. We start seeing the data to the appropriate survey form and we can even edit using enketo app. The “CSRF token error” appears only when we create new form or upload data from kobo-collect app.

if we delete the migrated data everything works fine.

Regards,

Sitaram Sawant

On Friday, August 26, 2016 at 10:57:25 PM UTC+5:30, Esmail Fadae wrote:

Hello. Can you confirm that you’ve followed the setup procedure described in the kobo-docker repo and that the basic functions of your site are working for new forms first?

On Thursday, August 25, 2016 at 6:02:42 AM UTC-4, Kurund Jalmi wrote:

Hello,

Just to give more context on this issue:

  1. No error is displayed in the UI. “CRSR token invalid” error is shown in the browser console.
  1. In the fresh install of docker instance, form creation is working as expected. So it might be possible that some steps might have been missed in our migration script.
  1. Here is the link to migration script: https://github.com/ShelterAssociates/Shelter/blob/master/scripts/pyscript.py

It would be really great if someone can help us in resolving this issue.

Regards,

Kurund

On Monday, August 22, 2016 at 1:14:04 PM UTC+5:30, Sitaram Sawant wrote:

Hello,

We try to transfer data from kobotoolbox to docker, all form that we transfer are working fine but when we create new form in kpi and trying to upload new form data from kobocollect it shows an error(csrf token invalid).

Following Steps that we have done to transfer data

  1. Upload forms from kobotoolbox dkobo to docker kpi.
  1. Transfer Postgres data in logger_instance from kobotoolbox to docker (according to changes that required now in logger_instance)
  1. Transfer MOngo data in formhub from kobotoolbox to docker (according to changes that required now in mongodb)

Is there some thing we are missing while transferring data.

Regards,

Sitaram

this issue has been resolved by the team. We’ll send an update in a few hours

lobo

···

On Wednesday, August 31, 2016 at 7:41:10 AM UTC-7, Donald Lobo wrote:

Kobo-team:

any chance someone can take a look at this and advise us please. This issue is blocking the rollout on a wider basis

I suspect there is some data being transferred that the code does a check on. Happy to get on the phone/skype to discuss details. We are releasing all the work we do as open source also and will blog about it

thank you for your advice and help

lobo

On Monday, August 29, 2016 at 5:48:26 AM UTC-7, Sitaram Sawant wrote:

Hello Esmail,

We have followed all steps of kobo-docker. Everything is working fine, if we do not migrate data. Once we migrate data from old Kobotoolbox instance to new kob-docker using script specified by Kurund above. We start seeing the data to the appropriate survey form and we can even edit using enketo app. The “CSRF token error” appears only when we create new form or upload data from kobo-collect app.

if we delete the migrated data everything works fine.

Regards,

Sitaram Sawant

On Friday, August 26, 2016 at 10:57:25 PM UTC+5:30, Esmail Fadae wrote:

Hello. Can you confirm that you’ve followed the setup procedure described in the kobo-docker repo and that the basic functions of your site are working for new forms first?

On Thursday, August 25, 2016 at 6:02:42 AM UTC-4, Kurund Jalmi wrote:

Hello,

Just to give more context on this issue:

  1. No error is displayed in the UI. “CRSR token invalid” error is shown in the browser console.
  1. In the fresh install of docker instance, form creation is working as expected. So it might be possible that some steps might have been missed in our migration script.
  1. Here is the link to migration script: https://github.com/ShelterAssociates/Shelter/blob/master/scripts/pyscript.py

It would be really great if someone can help us in resolving this issue.

Regards,

Kurund

On Monday, August 22, 2016 at 1:14:04 PM UTC+5:30, Sitaram Sawant wrote:

Hello,

We try to transfer data from kobotoolbox to docker, all form that we transfer are working fine but when we create new form in kpi and trying to upload new form data from kobocollect it shows an error(csrf token invalid).

Following Steps that we have done to transfer data

  1. Upload forms from kobotoolbox dkobo to docker kpi.
  1. Transfer Postgres data in logger_instance from kobotoolbox to docker (according to changes that required now in logger_instance)
  1. Transfer MOngo data in formhub from kobotoolbox to docker (according to changes that required now in mongodb)

Is there some thing we are missing while transferring data.

Regards,

Sitaram

Thanks lobo – please do send the solution / issue.

Support for custom deployment is not as quick as we would like, but we are working on it. We initially thought most use would be through the hosted solutions…

Good luck!

···

this issue has been resolved by the team. We’ll send an update in a few hours

lobo

On Wednesday, August 31, 2016 at 7:41:10 AM UTC-7, Donald Lobo wrote:

Kobo-team:

any chance someone can take a look at this and advise us please. This issue is blocking the rollout on a wider basis

I suspect there is some data being transferred that the code does a check on. Happy to get on the phone/skype to discuss details. We are releasing all the work we do as open source also and will blog about it

thank you for your advice and help

lobo

On Monday, August 29, 2016 at 5:48:26 AM UTC-7, Sitaram Sawant wrote:

Hello Esmail,

We have followed all steps of kobo-docker. Everything is working fine, if we do not migrate data. Once we migrate data from old Kobotoolbox instance to new kob-docker using script specified by Kurund above. We start seeing the data to the appropriate survey form and we can even edit using enketo app. The “CSRF token error” appears only when we create new form or upload data from kobo-collect app.

if we delete the migrated data everything works fine.

Regards,

Sitaram Sawant

On Friday, August 26, 2016 at 10:57:25 PM UTC+5:30, Esmail Fadae wrote:

Hello. Can you confirm that you’ve followed the setup procedure described in the kobo-docker repo and that the basic functions of your site are working for new forms first?

On Thursday, August 25, 2016 at 6:02:42 AM UTC-4, Kurund Jalmi wrote:

Hello,

Just to give more context on this issue:

  1. No error is displayed in the UI. “CRSR token invalid” error is shown in the browser console.
  1. In the fresh install of docker instance, form creation is working as expected. So it might be possible that some steps might have been missed in our migration script.
  1. Here is the link to migration script: https://github.com/ShelterAssociates/Shelter/blob/master/scripts/pyscript.py

It would be really great if someone can help us in resolving this issue.

Regards,

Kurund

On Monday, August 22, 2016 at 1:14:04 PM UTC+5:30, Sitaram Sawant wrote:

Hello,

We try to transfer data from kobotoolbox to docker, all form that we transfer are working fine but when we create new form in kpi and trying to upload new form data from kobocollect it shows an error(csrf token invalid).

Following Steps that we have done to transfer data

  1. Upload forms from kobotoolbox dkobo to docker kpi.
  1. Transfer Postgres data in logger_instance from kobotoolbox to docker (according to changes that required now in logger_instance)
  1. Transfer MOngo data in formhub from kobotoolbox to docker (according to changes that required now in mongodb)

Is there some thing we are missing while transferring data.

Regards,

Sitaram

You received this message because you are subscribed to the Google Groups “Kobo Users” group.

To unsubscribe from this group and stop receiving emails from it, send an email to kobo-users+unsu...@googlegroups.com.

To post to this group, send email to
kobo-...@googlegroups.com.

Visit this group at
https://groups.google.com/group/kobo-users
.

For more options, visit
https://groups.google.com/d/optout
.

There were 2 issues that we found missing during our upgrade process:

  1. When we ran the script it created records in logger_instance table of Postgres and MongoDB. But the sequence value was not set correctly for logger_instance id_seq table. It was using old sequence value.

  2. There was a mismatch of version Code and Survey type which is referenced to logger_instance according to kobocat.

After correcting above migrated instances seems to be working as expected.

Here is the updated script that we used for the migration process: Shelter/scripts/pyscript.py at master · ShelterAssociates/Shelter · GitHub

Hope it helps.

Kurund

···

On Wednesday, August 31, 2016 at 9:16:34 PM UTC+5:30, Vinck, Patrick wrote:

Thanks lobo – please do send the solution / issue.

Support for custom deployment is not as quick as we would like, but we are working on it. We initially thought most use would be through the hosted solutions…

Good luck!

From: kobo...@googlegroups.com [mailto:kobo-...@googlegroups.com] On Behalf Of Donald Lobo

Sent: Wednesday, August 31, 2016 10:56 AM

To: Kobo Users kobo...@googlegroups.com

Subject: [KoBo Users] Re: Transfer data from kobotoolbox to docker

this issue has been resolved by the team. We’ll send an update in a few hours

lobo

On Wednesday, August 31, 2016 at 7:41:10 AM UTC-7, Donald Lobo wrote:

Kobo-team:

any chance someone can take a look at this and advise us please. This issue is blocking the rollout on a wider basis

I suspect there is some data being transferred that the code does a check on. Happy to get on the phone/skype to discuss details. We are releasing all the work we do as open source also and will blog about it

thank you for your advice and help

lobo

On Monday, August 29, 2016 at 5:48:26 AM UTC-7, Sitaram Sawant wrote:

Hello Esmail,

We have followed all steps of kobo-docker. Everything is working fine, if we do not migrate data. Once we migrate data from old Kobotoolbox instance to new kob-docker using script specified by Kurund above. We start seeing the data to the appropriate survey form and we can even edit using enketo app. The “CSRF token error” appears only when we create new form or upload data from kobo-collect app.

if we delete the migrated data everything works fine.

Regards,

Sitaram Sawant

On Friday, August 26, 2016 at 10:57:25 PM UTC+5:30, Esmail Fadae wrote:

Hello. Can you confirm that you’ve followed the setup procedure described in the kobo-docker repo and that the basic functions of your site are working for new forms first?

On Thursday, August 25, 2016 at 6:02:42 AM UTC-4, Kurund Jalmi wrote:

Hello,

Just to give more context on this issue:

  1. No error is displayed in the UI. “CRSR token invalid” error is shown in the browser console.
  1. In the fresh install of docker instance, form creation is working as expected. So it might be possible that some steps might have been missed in our migration script.
  1. Here is the link to migration script: https://github.com/ShelterAssociates/Shelter/blob/master/scripts/pyscript.py

It would be really great if someone can help us in resolving this issue.

Regards,

Kurund

On Monday, August 22, 2016 at 1:14:04 PM UTC+5:30, Sitaram Sawant wrote:

Hello,

We try to transfer data from kobotoolbox to docker, all form that we transfer are working fine but when we create new form in kpi and trying to upload new form data from kobocollect it shows an error(csrf token invalid).

Following Steps that we have done to transfer data

  1. Upload forms from kobotoolbox dkobo to docker kpi.
  1. Transfer Postgres data in logger_instance from kobotoolbox to docker (according to changes that required now in logger_instance)
  1. Transfer MOngo data in formhub from kobotoolbox to docker (according to changes that required now in mongodb)

Is there some thing we are missing while transferring data.

Regards,

Sitaram

You received this message because you are subscribed to the Google Groups “Kobo Users” group.

To unsubscribe from this group and stop receiving emails from it, send an email to kobo-users+...@googlegroups.com.

To post to this group, send email to
kob...@googlegroups.com.

Visit this group at
https://groups.google.com/group/kobo-users
.

For more options, visit
https://groups.google.com/d/optout
.