Support for reservation and retry of failed jobs

[ieu]

We recently switched to yii2-queue and noticed that failed jobs was removed once it reached max retry count. Sometimes job fails due to bugs in code instead of external resource failure. Removing is misleading as it makes failures unseen from developer. and we will lose message from bugs that is hard to reproduce. Indeed developers are free to implement such things on their own. I think it is so basic and worth being part of yii2-queue, like what Laravel queue does.

[smiffsoft]

This is an absolute must have feature. I've had to write a custom queue because sometimes jobs fail (usually when connecting to 3rd party APIs). I'd need them to stay in the queue to be executed at a later time, not removed once max attempts have been reached.

[bizley]

Is anyone eager to prepare PR with it? It would be amazing.

[smiffsoft]

I went a completely different route, it's not based on yii2-queue at all, so I wouldn't be of any use here. This only became obvious recently after converting my code to yii2-queue and then realising some jobs were being lost even though they hadn't actually worked. In my queue the jobs remain and are periodically retried until cleared successfully.

[rob006]

You can't already control whether job will retry using RetrayableJobInterface::canRetry() - if you want to try infinitely, you can always return true.

[smiffsoft]

Maybe I've misunderstood it, but that would leave it in the queue forever even if the job is successful.

Here's how my solution works, hopefully this makes sense:

Each job has 10 attempts to run.
If it runs, remove from queue.
If not, increment attempt and try again until it hits 10. When it hits 10 it's flagged as a failing job and an alert gets sent to someone. The job stays in the queue so it can be investigated. If the underlying issue is resolved, the fail flag is reset and attempts go back to 0 and the queue will pick it up again as normal.

In the yii2-queue documentation it says:

The attempts option sets max number of attempts. If attempts are over, and the job isn't done, it will be removed from a queue as completed.

Does this mean, if I don't set attempts to anything in the config, it'll keep trying forever every ttr seconds? This might actually achieve what I'm hoping for, albeit I'd prefer to limit the number of attempts in reality.

[rob006]

Maybe I've misunderstood it, but that would leave it in the queue forever even if the job is successful.

canRetry() is called only on failure, so it won't retry finished jobs.

For more sophisticated solution you may use EVENT_AFTER_ERROR event at queue level and store failed job somewhere. Then you could repush jobs from there, but you need to handle it yourself.

yii2-queue/src/Queue.php

Lines 288 to 298 in 7d12073

	public function handleError(ExecEvent $event)
	{
	$event->retry = $event->attempt < $this->attempts;
	if ($event->error instanceof InvalidJobException) {
	$event->retry = false;
	} elseif ($event->job instanceof RetryableJobInterface) {
	$event->retry = $event->job->canRetry($event->attempt, $event->error);
	}
	$this->trigger(self::EVENT_AFTER_ERROR, $event);
	return !$event->retry;
	}

Does this mean, if I don't set attempts to anything in the config, it'll keep trying forever every ttr seconds?

If you don't configure attempts count, then default value will be used (which is 1, so no retries at all). You could set it to PHP_INT_MAX , which should be pretty close to "infinite retries".

[smiffsoft]

Ahh, I see. I think for my purposes using the canRetry() with a large "attempts" and add some sort of alerts in there to spot any jobs that have failed X times, that would solve my issue, I can have a periodic check of the queue status too to see if there's any still in there. I did think about the event handlers but it looked far more complex.

[silverslice]

I wrote extension to save failed jobs in database and manage them later.