Duplicate data in search

[Isak-Kallini]

Description

I think this is because the daily sync is run in each instance at the same time (eg. 4 instances in the screenshot)

Steps to reproduce

No response

Further information

No response

[alfredgrip]

Is there any way to ensure only one instance runs this? @danieladugyan

[alfredgrip]

I think this could have been solved if we didn't generate ids when putting data in meilisearch, but instead use our primary keys from Prisma. But this haven't worked for me, since I think some of the ids we use for some tables aren't compatible with Meilisearch. Meilisearch likes uuids, but for Positions we use like dsek.infu.dwww.mdlm, and I suspect that Meilisearch doens't like this form of ids.
If we didn't generate ids, Meilisearch would see that we are sending duplicate data and not create four different entries.

[alfredgrip]

I think this could have been solved if we didn't generate ids when putting data in meilisearch, but instead use our primary keys from Prisma. But this haven't worked for me, since I think some of the ids we use for some tables aren't compatible with Meilisearch. Meilisearch likes uuids, but for Positions we use like dsek.infu.dwww.mdlm, and I suspect that Meilisearch doens't like this form of ids. If we didn't generate ids, Meilisearch would see that we are sending duplicate data and not create four different entries.

Just confirmed this:

The document id must be an integer or a string. If the id is a string, it can only contain alphanumeric characters (a-z, A-Z, 0-9), hyphens (-), and underscores (_).

https://www.meilisearch.com/docs/learn/getting_started/primary_key#formatting-the-document-id
So it cannot contain dots . Should be an easy fix, but it's a shame that we have to do this conversion.

[alfredgrip]

Added a commit in #662 which should solve this temporarily, but ideally it should be run on only one instance.

[alfredgrip]

Maybe something like this could solve this issue?
https://greenydev.com/blog/pm2-cron-job-multiple-instances/#solution-1-setting-app-name-in-pm2-instances
It should be relevant for all cron jobs we do, not only syncing with Meilisearch.

@danieladugyan