Manual updating sources doesn't work (or just keep cached version)

[kam193]

Describe the bug
Since the last changes to source updates, it looks like manual triggering an update doesn't work any more. I suspect it just keeps the cached version, what I wouldn't expect when manually triggering an update. I don't have any logs to confirm my suspicion at the moment.

To Reproduce
Steps to reproduce the behavior:

1. Have a source (e.g. YARA rules) that you control and that has been updated in AL some time ago.
2. Change something in the source.
3. Wait a few minutes to ensure no caching happens in between.
4. Go to the update sources page and manually trigger updating your selected source.
5. Observe quick "Skipped" result. You can also find signatures and confirm there was nothing updated.

To confirm that there was something to update, you can change something in the source configuration. Then you will see that it updates.

Expected behavior
When manually triggering an update, I would expect to ignore any cache.

Screenshots

Environment (please complete the following information if pertinent):

• Assemblyline Version: 4.5.0.67
• I've observed the same behaviour in newest YARA as well as custom services.

Additional context

[cccs-rs]

So manually triggering an update, basically tasks the updater to run outside of it's normal sync interval so it's possible for it to use the cached version of the update if nothing has changed in the remote server.

The button was introduced because we have analysts that contribute YARA rules to a repository and we wanted to provide a way for them to trigger an update to pull in new rules without setting a small interval to hammer the remote server.

There is a way to ignore caching in Update Source configuration in the UI but that setting will be kept for future updates as well, both scheduled and manual.

[cccs-rs]

It's possible what we're using to check for caching with the remote server could be wrong and that could explain why it thinks nothing has changed.

[kam193]

The button was introduced because we have analysts that contribute YARA rules to a repository and we wanted to provide a way for them to trigger an update to pull in new rules without setting a small interval to hammer the remote server.

It was exactly the case. I've updated my rules hosted in a private Github repository, and tried to trigger the update manually a few times. I didn't work until I set ignoring cache in the settings (but - I wouldn't be too sure that this exact setting helped or just changing something in the source's settings).

However, while it was a persistent behaviour across different services when I reported it, I have to say it worked correctly a few days later. I will keep an eye on this and come back once I know more about the circumstances