reindex also when embedding parameters change

[oatmealm]

I've deleted the db to reindex documents to see the effect of different settings for chunking and overlap (for local files, my literature is in the humanities mostly, clean markdown or text files). I think it might be useful to detect that and re-index also when the changes were made to the config file?

[jkitchin]

I will have to think about how to do that. The only thing that comes to mind is to store an md5 hash somewhere. That is too sensitive though, you don't want to trigger reindexing just because you change a space or email or something, and it isn't very future proof if choosing an embedding model gets more flexible.

It is also not necessary to delete the db. I think you would just iterate over the rows, get the text, and update the embedding. I added a function to do that.