Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Database changes #21

Open
phirestalker opened this issue Oct 4, 2024 · 3 comments
Open

Database changes #21

phirestalker opened this issue Oct 4, 2024 · 3 comments
Labels
enhancement New feature or request

Comments

@phirestalker
Copy link

When i change the default algos on -update for the first time, does it keep those settings for future -update, or do I need to specify each time?

I recently made the mistake of using only dct for some new images I added to the directory. I then tried a new run of update with dct+fdct, but it did nothing. Can you add a way to update the database with new algos?

These are the cases I am thinking and the outcomes I would expect (you may have covered some of these).

  1. I choose only some algos on first index.
    Subsequent update commands should use the same settings unless otherwise specified.
  2. I specify the algos I want to use
    It should check the hashes (or whatever) for each file to see if all algos are present. if not, it should add the algos for the files that are missing. This would require checking algos before file existence so that files are not skipped when found on the system (will unfortunately slow down scanning)
  3. I want to remove some algos from the index to prune the size.
    I would love some options to see what algos are in use for the index currently, and to remove some algos from the index

Also, slightly related. I have some issues with graphical programs started from the command line ICE authority because of Whonix. When it crashes a lot has to be done again. It would be awesome if the matches could be saved to disk while it is processing and able to pick up where it left off. This is especially pertinent for template matching which takes the better part of a day on my lousy system.

I would love to hear your thoughts on this and/or any ideas it might have sparked.

@phirestalker
Copy link
Author

I ran -list-index-params on the pictures index and it listed all algos. This means that subsequent calls of update without specifying algos does indeed use them all. As of right now, I guess I will need to start over to get only the algos I want for all images. Once I have a good command with the options I want, I will copy it somewhere to use for every update.

So that gives some more information for the initial question anyway.

@scrubbbbs scrubbbbs added the enhancement New feature or request label Oct 17, 2024
@scrubbbbs
Copy link
Owner

My overall thought is, most of the time you just enable all algos because the space/time savings are not significant; the only niche use cases are:

  • You have well over 100,000 images to scan (enable DCT only)
  • You don't want to index videos

With that said, I think a few "simple" changes would cover your use cases.

  1. Add -sync or something that works exactly like -update, additionally adding/removing algos as specified.
  2. If any file is not indexed for the query algo, then the soft warning could add "please use -i.algos dct+fdct -sync"

@phirestalker
Copy link
Author

With that said, I think a few "simple" changes would cover your use cases.

  1. Add -sync or something that works exactly like -update, additionally adding/removing algos as specified.
  2. If any file is not indexed for the query algo, then the soft warning could add "please use -i.algos dct+fdct -sync"

That should work great.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

2 participants