[Question] queue_job when multiple odoo servers are used with load balancing (and single postgres)

[JordiBForgeFlow]

In the scenario where multiple odoo servers are used with load balancing, and a single postgres server is used, will all of the odoo servers try to run all the jobs?

[sbidoul]

@JordiBForgeFlow you need to make sure that only one Odoo instance has queue_job in --load / server_wide_modules. This is very important.

Alternatively, you can launch a separate job runner process with something like #409.

[sbidoul]

See also #256

[github-actions]

There hasn't been any activity on this issue in the past 6 months, so it has been marked as stale and it will be closed automatically if no further activity occurs in the next 30 days.
If you want this issue to never become stale, please ask a PSC member to apply the "no stale" label.

[daemonblade]

@sbidoul having to tweak the configuration for one instance can become an issue when we have to deploy to a variable number of Odoo instances. Is it not possible to namescope the NOTIFY payload with host/main-odoo-pid/dbname to allow multiple job-runners to be active?

[sbidoul]

@sbidoul having to tweak the configuration for one instance can become an issue when we have to deploy to a variable number of Odoo instances. Is it not possible to namescope the NOTIFY payload with host/main-odoo-pid/dbname to allow multiple job-runners to be active?

@daemonblade in such situations, I don't add queue_job to server_wide_modules, and instead start the job runner independently, so all Odoo instances can share the same config.

Multiple job runners is tricky because it is currently all based on the queue being in memory.

[b1zzu]

Hi @sbidoul how can I run the queue_job independently? I've tried this PYTHONPATH=/opt/odoo/ python3 -m queue_job.jobrunner but doesn't work, it fails with this error:

AssertionError: Invalid import of queue_job.models.base.Base, it should start with 'odoo.addons'.

[sbidoul]

Try python3 -m odoo.addons.queue_job.jobrunner.

You'll need to add {queue-job-repo-dir}/setup/queue_job to PYTHONPATH or install odoo-addon-queue_job with pip.

[daemonblade]

I've reviewed our logs for multi-node systems with queue_job. We have one instance configured with queue_job in server_wide_modules. What I have noticed is that the instances that do NOT have queue_job in server_wide_modules will never run any jobs at all. Is this intended?

[LordSouza]

Try python3 -m odoo.addons.queue_job.jobrunner.

You'll need to add {queue-job-repo-dir}/setup/queue_job to PYTHONPATH or install odoo-addon-queue_job with pip.

this 'odoo-addon-queue_job' is asking for python>=3.10 or higher, but we use 3.8 and when I try to use the PYTHONPATH it doesn't work because the job_queue path is different than odoo.addons, is there a way to run as a external module?

[sbidoul]

With #668 we will not need to worry about this, as queue job will manage a leader election automatically. The only constraint will be that all instances manage the same list of databases.

[daemonblade]

Will this commit allow the same configuration file for all Odoo instances, or will it require that only ONE instance have queue_job in server_wide_modules?

[sbidoul]

Yes, same config on all instances will work.