Upload did not work for Finish last night

[ice1e0]

Finish was uploaded yesterday eve

But it was not fully processed this morning, so, it looks like that the processing broke. I have uploaded it today again and it worked now.

The messages are taken from the Telegram channel "BCCM - BMM Exporters".

[ice1e0]

The first upload of finish worked:

My current guess is that no writing checks are used and one writing operation deleted the information from the other. At the same time, Khasi was uploaded to BMM according to the Telegram channel:

[ice1e0]

Related issues:

• Upload didn't work for one language #41
• Unstable when uploading audio files to BMM/Fra Kåre #39

[ice1e0]

P.s. it could have happened that the BMM update/processing happened at the same time as titles where updated in the old BMM site

[kkuepper]

@KillerX can you give an update about the findings of your research?

[ice1e0]

I have a new case from today:

Hungarian was uploaded around 18.20 and the user confirmend that he got an email that it was uploaded. But when I send the status message at 21:17 o'clock, it was not uploaded.

Note that I updated the titles via the BMM Admin website tonight. I cannot guarantee that this happened exactly during that time. My current guess is that the status got somehow overwritten.

My current suggestion to potentially fix this would be to use a concurrency control tactic and let at least one update command fail when updating data in the database. Then nothing gets overwritten and the user can react on the error message.
@kkuepper @KillerX

[KillerX]

Original (fin - 115007) - This was a timeout in my side. We are working on a restructuring of the infra locally that should fix this cases, planned 2nd weekend in Jan 2025.

Everything else: I am unable to see any errors on my side. I did a closer investigation of the HU 115013 from yesterday, but I am unable to see any issues on MB side. All the flows completed w/o errors and the output I can see seems correct.

[ice1e0]

@KillerX i do not think there is a mayor issue when a system runs into a timeout. The mayor issue i see here is that the user do not get the feedback that something went wrong. The telegram chat shows that everything went fine and the user was informed by email that the workflow worked. That should not happen.

[KillerX]

@KillerX i do not think there is a mayor issue when a system runs into a timeout. The mayor issue i see here is that the user do not get the feedback that something went wrong. The telegram chat shows that everything went fine and the user was informed by email that the workflow worked. That should not happen.

On the first issue reported the system never sent any messages that the process completed! It also makes sense that no error message was ever sent in that case as the backoff became to large and it was waiting for next try (a few days later). This later part (too large backoff) has been tweaked after this.

Issue 2,3 are different, as stated in the previous comment.

[kkuepper]

I looked at the logs for the issue with hungarian (track_115013):
mediabanken last updated at 18:20:24 and Leo updated it 6 seconds later at 18:20:30 accidentally deleting hungarian again.
I guess one solution would be to show an error message to Leo "there's been changes in the meantime and we couldn't save the changes"

[KillerX]

@kkuepper it's probably better than just silently discarding uploaded files...

[ice1e0]

@kkuepper i don’t know what database framework is used but I guess something like concurrency conflicts logic as EF Core uses could help here: https://learn.microsoft.com/en-us/ef/core/saving/concurrency?tabs=data-annotations
I think it filters the update with the last value of the concurrency column and when the update the affects zero rows, it throws an exception. Just an idea there might be pros and cons to such a logic and it might be difficult to implement

[kkuepper]

for now we decided to focus on working on automating the Fra Kåre publishing process instead of introducing concurrency handling.
(We use RavenDB as our database)