Find a better mechanism for sharing file_index.db between dias and theremin

[anjakefala]

Both Dias and theremin need access to marimba:/var/lib/dias/file_index.db. To add complexity, theremin is currently on agogo.

At the moment, Dias writes to marimba:/var/lib/dias/file_index.db. This is copied to gong:dias/file_index.db once-a-day. Theremin then accesses the database, in read-only, at gong:dias/file_index.db.

The concerns with this method are:

• theremin only gets the updated dias products once-a-day. This defeats the intention of having dias run multiple times a day. Also, it might try to access files that do not exist anymore.
• if someone is trying to access the database while it changes underneath them, then theremin might crash, or send back a garbage result. It is possible that its copy could also be corrupted.

We would like a better solution.

The concerns for any solution are:

• the effect of simultaneous connections to the database, and possibilities of corruption
• issues with connecting over an NFS

Alternatives:

• Place the primary file_index.db on gong and both sensitivity and agogo access it
  • "Did you do any research about consistency issues when accessing a sqlite db file over NFS from multiple hosts? NFS isn't great at locking, and has other issues so I'd be quite worried about this. Can you have a look into this, and post in here what you find out?"
• Backup from marimba:file_index.db to gong:file_index.db more frequently
  • Would need to ensure a consistent copy, while working with sqlite's locking
• keep the data_index.db for the analyzer's output files, and do not move that to the tracker
  • makes the tracker less useful
• move dias or theremin to be on the same host, i.e. move theremin to marimba; or (less likely I think) move dias to agogo.
• move the file index tables into chimedb. This is probably the best solution.