Data search issue after form modification — old records not appearing in Data Table search

Hi everyone,

We are running a self-hosted KoboToolbox instance on our own server and recently encountered a search issue in the Data tab.

Here’s what happens:

After we modify a form (for example, adding a new skip logic condition or changing a group’s relevance),

We can still see all the old submissions in the dataset,

But when we use the search function in the Data Table, only the new submissions (those collected after the form update) appear in search results.

The older records are not deleted — they simply don’t show up when searching.

It seems like the older data is stored correctly in the database but is not indexed or searchable after the form structure changes.

:small_blue_diamond: Environment details:

Self-hosted KoboToolbox (Docker installation)

PostgreSQL + Elasticsearch setup (default from kobo-install)

The issue started right after a form modification and redeployment.

:small_blue_diamond: What we’ve tried:

Verified that all records still exist in PostgreSQL

Confirmed that only the new records appear in Elasticsearch queries

Suspect this is caused by schema mismatch between old and new form versions

:small_blue_diamond: Question:

Is this expected behavior after updating an XForm?

How can we rebuild or refresh the search index so that older records become searchable again in the Data Table?

Is there a safe command or recommended way (via manage.py reindex or similar) to do this on a live production environment?

Any guidance or best practices would be greatly appreciated.

Thanks in advance for your help! :folded_hands:

@hoseinnematolahi, would you mind trying to replicate the issue on one of our public KoboToolbox servers to see if the same behaviour occurs?