Skip to content

Comments

feat: migrate from OCA queue_job to job_worker#55

Open
jeremi wants to merge 6 commits into19.0from
migrate-odoo-job-worker
Open

feat: migrate from OCA queue_job to job_worker#55
jeremi wants to merge 6 commits into19.0from
migrate-odoo-job-worker

Conversation

@jeremi
Copy link
Member

@jeremi jeremi commented Feb 18, 2026

Summary

  • Replace queue_job dependency with job_worker across all 7 spp_* modules (spp_area, spp_dci_server, spp_demo, spp_drims, spp_gis_report, spp_programs, spp_starter_social_registry)
  • Replace all odoo.addons.queue_job imports with odoo.addons.job_worker
  • Delete queue.job.channel XML data and the queue_job_channel.py model from spp_programs
  • Add queue.limit data files for spp_area, spp_dci_server; migrate spp_programs queue_data.xml
  • Update test contexts from test_queue_job_no_delay to queue_job__no_delay
  • Update menu/action external ID references to job_worker.*
  • Remove stale ODOO_QUEUE_JOB_CHANNELS env var from entrypoint, docker-compose, README, and .env example
  • Update Docker config to use odoo-job-worker module and CLI worker
  • Update lint scripts to whitelist job_worker models including queue.limit

Dependencies

Requires odoo-job-worker with the group().on_done() barrier feature (feat/group-on-done-barrier branch).

Test plan

  • CI passes (lint, unit tests)
  • spp_programs cycle/eligibility/entitlement/payment managers run jobs via group().on_done() correctly
  • spp_area area import jobs enqueue and execute
  • spp_dci_server async routes and bulk upload delay correctly
  • spp_demo demo data generator batches via group
  • Docker production compose starts queue worker with python -m odoo.addons.job_worker.cli

@gemini-code-assist
Copy link

Summary of Changes

Hello @jeremi, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request undertakes a significant refactoring effort to replace the existing OCA queue_job module with the new job_worker module for asynchronous task processing. This migration impacts several core modules and infrastructure configurations, ensuring that all background jobs, from area imports and DCI transactions to demo data generation and GIS report refreshes, now leverage the capabilities of the job_worker system. The change aims to standardize and potentially enhance the reliability and management of asynchronous operations across the application, aligning with the new module's features like the group().on_done() barrier.

Highlights

  • Dependency Migration: Replaced the queue_job dependency with job_worker across all seven spp_* modules, including spp_area, spp_dci_server, spp_demo, spp_drims, spp_gis_report, spp_programs, and spp_starter_social_registry.
  • Codebase Updates: Updated all odoo.addons.queue_job imports to odoo.addons.job_worker and adjusted references to queue_job in various configurations, docstrings, and comments.
  • Configuration and Data Changes: Deleted queue.job.channel XML data and the queue_job_channel.py model from spp_programs. Added queue.limit data files for spp_area and spp_dci_server, and migrated spp_programs' queue_data.xml to the new queue.limit model.
  • Testing Context Adjustment: Updated test contexts from test_queue_job_no_delay to queue_job__no_delay to align with the job_worker module's naming convention for bypassing job delays in tests.
  • Docker and Environment Configuration: Removed stale ODOO_QUEUE_JOB_CHANNELS environment variables from entrypoint.sh, docker-compose.yml, README, and .env examples. Updated Docker configurations to include the odoo-job-worker module and use its CLI worker for background job processing.
  • Linting and Security Rules: Modified lint scripts (.pre-commit-config.yaml, scripts/lint/check_naming.py, scripts/lint/check_performance.py) to whitelist job_worker models (including queue.limit) and update performance check suggestions to refer to job_worker.
Changelog
  • .pre-commit-config.yaml
    • Updated pre-commit hook exclusions to reference job_worker instead of queue_job for Python and XML files.
  • .semgrep/odoo-security.yml
    • Updated Semgrep rule message and suggestion to recommend job_worker for batch operations instead of queue_job.
  • docker-compose.yml
    • Removed the ODOO_QUEUE_JOB_CHANNELS environment variable configuration.
  • docker/.env.production.example
    • Removed the ODOO_QUEUE_JOB_CHANNELS entry from the example production environment file.
  • docker/Dockerfile
    • Added odoo-job-worker to the list of modules to download during the Docker build process.
  • docker/README.md
    • Removed the ODOO_QUEUE_JOB_CHANNELS entry from the environment variable table.
  • docker/docker-compose.production.yml
    • Updated the comment for the queue worker service to reflect the use of job_worker.
    • Removed ODOO_WORKERS, ODOO_CRON_THREADS, and QUEUE_JOB_CHANNELS environment variables from the queue-worker service.
    • Changed the queue-worker command to python -m odoo.addons.job_worker.cli for standalone execution.
  • docker/entrypoint.sh
    • Removed the ODOO_QUEUE_JOB_CHANNELS environment variable setup.
  • docker/odoo.conf.template
    • Added odoo-job-worker to the addons_path.
    • Removed queue_job from server_wide_modules.
    • Removed the entire [queue_job] configuration section.
  • scripts/lint/check_naming.py
    • Whitelisted job_worker module in MODULE_WHITELIST.
    • Whitelisted queue.limit model in THIRD_PARTY_MODEL_WHITELIST and updated comments to refer to job_worker.
  • scripts/lint/check_performance.py
    • Updated comments, violation messages, and suggestions to refer to job_worker instead of queue_job for batch processing recommendations.
  • spp_area/manifest.py
    • Updated module dependency from queue_job to job_worker.
    • Replaced data/queue_job_channel.xml with data/queue_limit_data.xml.
  • spp_area/data/queue_job_channel.xml
    • Removed the queue.job.channel definition for area_import.
  • spp_area/data/queue_limit_data.xml
    • Added a queue.limit record for area_import with a limit of 1.
  • spp_area/models/area_import.py
    • Updated import statement for group from queue_job.delay to job_worker.delay.
    • Changed the _area_import_channel from root.area_import to area_import.
  • spp_area/tests/common.py
    • Updated test context comment to refer to job_worker.
    • Changed test_queue_job_no_delay to queue_job__no_delay in test context.
  • spp_area/tests/test_area.py
    • Updated test context comment to refer to job_worker.
    • Changed test_queue_job_no_delay to queue_job__no_delay in test context.
  • spp_area/tests/test_area_tags.py
    • Changed test_queue_job_no_delay to queue_job__no_delay in test context.
  • spp_area/tests/test_res_partner.py
    • Changed test_queue_job_no_delay to queue_job__no_delay in test context.
  • spp_area_hdx/wizards/hdx_cod_import_wizard.py
    • Updated comment to suggest job_worker for large imports instead of queue_job.
  • spp_base_common/models/ir_module_module.py
    • Updated the menu XML ID reference for the job queue from queue_job.menu_queue_job_root to job_worker.menu_queue_job_root.
  • spp_dci_server/manifest.py
    • Updated module dependency from queue_job to job_worker.
    • Replaced data/queue_job_channel.xml with data/queue_limit_data.xml.
  • spp_dci_server/data/queue_job_channel.xml
    • Removed the queue.job.channel definition for dci.
  • spp_dci_server/data/queue_limit_data.xml
    • Added a queue.limit record for dci with a limit of 2 and a rate limit of 10.
  • spp_dci_server/models/subscription.py
    • Updated docstrings to refer to job_worker.
    • Changed job channel from root.dci to dci and added a timeout parameter to with_delay calls.
    • Updated method docstring to indicate calls by job_worker.
  • spp_dci_server/models/transaction.py
    • Updated docstrings and comments to refer to job_worker.
    • Updated help text for job_uuid and job_state fields.
    • Updated method docstrings to indicate calls by job_worker.
    • Changed job channel from root.dci to dci and added a timeout parameter to with_delay calls.
  • spp_dci_server/routers/async_router.py
    • Updated docstrings and comments to refer to job_worker.
    • Changed job channel from root.dci to dci and added a timeout parameter to with_delay calls.
  • spp_dci_server/routers/bulk_upload.py
    • Changed job channel from root.dci to dci and added a timeout parameter to with_delay calls.
  • spp_dci_server/routers/callbacks.py
    • Changed job channel from root.dci to dci and added a timeout parameter to with_delay calls.
  • spp_dci_server/tests/test_transaction.py
    • Updated comment to refer to job_worker when mocking with_delay.
  • spp_demo/manifest.py
    • Updated module dependency from queue_job to job_worker.
  • spp_demo/models/demo_data_generator.py
    • Updated import statement for group from queue_job.delay to job_worker.delay.
  • spp_demo/tests/test_apps_wizard.py
    • Changed test_queue_job_no_delay to queue_job__no_delay in test context.
  • spp_demo/tests/test_demo_data_generator.py
    • Changed test_queue_job_no_delay to queue_job__no_delay in test context.
  • spp_demo/tests/test_demo_stories.py
    • Changed test_queue_job_no_delay to queue_job__no_delay in test context.
  • spp_demo/tests/test_res_config_settings.py
    • Changed test_queue_job_no_delay to queue_job__no_delay in test context.
  • spp_demo/tests/test_res_country.py
    • Changed test_queue_job_no_delay to queue_job__no_delay in test context.
  • spp_demo/tests/test_res_partner.py
    • Changed test_queue_job_no_delay to queue_job__no_delay in test context.
  • spp_drims/manifest.py
    • Updated module dependency from queue_job to job_worker.
  • spp_drims_sl_demo/wizard/drims_demo_generator.py
    • Updated comment to refer to job_worker for bypassing delays.
  • spp_gis/tests/test_area_import_raw.py
    • Changed test_queue_job_no_delay to queue_job__no_delay in test context.
  • spp_gis/tests/test_geo_fields.py
    • Changed test_queue_job_no_delay to queue_job__no_delay in test context.
  • spp_gis_report/manifest.py
    • Updated module dependency from queue_job to job_worker.
  • spp_gis_report/controllers/geojson.py
    • Updated comment to indicate job_worker is used for background processing.
  • spp_gis_report/models/gis_report.py
    • Updated docstrings and comments to refer to job_worker for background processing.
  • spp_gis_report/tests/common.py
    • Changed test_queue_job_no_delay to queue_job__no_delay in test context.
  • spp_gis_report/tests/test_area_ext.py
    • Changed test_queue_job_no_delay to queue_job__no_delay in test context.
  • spp_gis_report/tests/test_gis_report.py
    • Changed test_queue_job_no_delay to queue_job__no_delay in test context.
  • spp_gis_report/tests/test_gis_report_data.py
    • Changed test_queue_job_no_delay to queue_job__no_delay in test context.
  • spp_gis_report/tests/test_gis_report_wizard.py
    • Changed test_queue_job_no_delay to queue_job__no_delay in test context.
  • spp_hazard/tests/common.py
    • Changed test_queue_job_no_delay to queue_job__no_delay in test context.
  • spp_hide_menus_base/models/ir_module_module.py
    • Updated the menu XML ID reference for the job queue from queue_job.menu_queue_job_root to job_worker.menu_queue_job_root.
  • spp_programs/manifest.py
    • Added job_worker as a module dependency.
    • Enabled data/queue_data.xml by uncommenting it.
    • Removed the oca_data_manual entry for queue_data.xml.
  • spp_programs/data/queue_data.xml
    • Migrated queue.job.channel records (channel_root_program, channel_cycle, channel_eligibility_manager, channel_program_manager) to queue.limit records (limit_cycle, limit_eligibility_manager, limit_program_manager), removing parent_id and adding limit and rate_limit fields.
  • spp_programs/models/init.py
    • Removed the commented-out import for queue_job_channel.
  • spp_programs/models/job_relate_mixin.py
    • Updated the XML ID reference for the queue job action from queue_job.action_queue_job to job_worker.action_queue_job.
  • spp_programs/models/managers/cycle_manager_base.py
    • Updated import statement for group from queue_job.delay to job_worker.delay and removed the try-except ImportError block.
    • Removed root_program prefix from job channel names (e.g., root_program.cycle became cycle).
  • spp_programs/models/managers/eligibility_manager.py
    • Updated import statement for group from queue_job.delay to job_worker.delay and removed the try-except ImportError block.
    • Removed root_program prefix from job channel names (e.g., root_program.eligibility_manager became eligibility_manager).
  • spp_programs/models/managers/entitlement_manager_base.py
    • Updated import statement for group from queue_job.delay to job_worker.delay and removed the try-except ImportError block.
  • spp_programs/models/managers/entitlement_manager_cash.py
    • Updated import statement for group from queue_job.delay to job_worker.delay.
  • spp_programs/models/managers/entitlement_manager_inkind.py
    • Updated import statement for group from queue_job.delay to job_worker.delay.
  • spp_programs/models/managers/payment_manager.py
    • Updated import statement for group from queue_job.delay to job_worker.delay and removed the try-except ImportError block.
  • spp_programs/models/managers/program_manager.py
    • Updated import statement for group from queue_job.delay to job_worker.delay and removed the try-except ImportError block.
    • Removed root_program prefix from job channel names (e.g., root_program.program_manager became program_manager).
  • spp_programs/models/queue_job_channel.py
    • Removed the custom queue.job.channel model.
  • spp_programs/tests/test_cycle_auto_approve_fund_check.py
    • Changed test_queue_job_no_delay to queue_job__no_delay in test context.
  • spp_starter_social_registry/manifest.py
    • Updated module dependency from queue_job to job_worker.
Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature Command Description
Code Review /gemini review Performs a code review for the current pull request in its current state.
Pull Request Summary /gemini summary Provides a summary of the current pull request in its current state.
Comment @gemini-code-assist Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help /gemini help Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Footnotes

  1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution.

@codecov
Copy link

codecov bot commented Feb 18, 2026

Codecov Report

❌ Patch coverage is 75.00000% with 6 lines in your changes missing coverage. Please review.
✅ Project coverage is 67.52%. Comparing base (5ac7496) to head (b99f560).
⚠️ Report is 2 commits behind head on 19.0.

Files with missing lines Patch % Lines
spp_programs/models/managers/cycle_manager_base.py 25.00% 3 Missing ⚠️
spp_programs/models/job_relate_mixin.py 0.00% 1 Missing ⚠️
...pp_programs/models/managers/eligibility_manager.py 50.00% 1 Missing ⚠️
spp_programs/models/managers/program_manager.py 50.00% 1 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             19.0      #55      +/-   ##
==========================================
- Coverage   71.31%   67.52%   -3.79%     
==========================================
  Files         299      380      +81     
  Lines       23618    30117    +6499     
==========================================
+ Hits        16844    20338    +3494     
- Misses       6774     9779    +3005     
Flag Coverage Δ
endpoint_route_handler ?
fastapi ?
spp_alerts ?
spp_api_v2 89.14% <ø> (ø)
spp_api_v2_change_request 66.61% <ø> (ø)
spp_api_v2_cycles 65.45% <ø> (ø)
spp_api_v2_data 48.67% <ø> (ø)
spp_api_v2_entitlements 68.43% <ø> (?)
spp_api_v2_products 64.39% <ø> (?)
spp_api_v2_service_points 63.12% <ø> (?)
spp_api_v2_vocabulary 43.70% <ø> (?)
spp_approval 43.40% <ø> (?)
spp_area 85.75% <100.00%> (?)
spp_area_hdx 88.83% <100.00%> (?)
spp_audit 69.47% <ø> (?)
spp_base_common 92.81% <ø> (ø)
spp_programs 49.64% <53.84%> (+0.07%) ⬆️
spp_security 51.08% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Copy link

@gemini-code-assist gemini-code-assist bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request successfully migrates the OpenSPP platform from the OCA queue_job module to the job_worker module. The changes are comprehensive, covering module dependencies, Docker configurations, CI/CD scripts, and application logic across multiple modules. The migration adopts a flat channel structure using the queue.limit model and leverages new features like group().on_done() for job barriers. The feedback provided focuses on ensuring consistency in the new API usage and identifying minor opportunities for robustness in the migrated code.

# Override command to run queue job worker
command: ["odoo", "gevent", "--limit-time-real=0"]
# Run job_worker standalone runner process
command: ["python", "-m", "odoo.addons.job_worker.cli"]

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

high

The command for the queue-worker service has been updated to use the job_worker CLI. Verify that this standalone process has access to all necessary environment variables for database connection, as it no longer runs as part of the main Odoo process.

Comment on lines 320 to 325
self.with_delay(
channel="root.dci",
channel="dci",
eta=delay,
timeout=60,
description=f"DCI Retry Callback {self.transaction_id}",
)._retry_callback()

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

The timeout parameter is added to the with_delay call. Ensure that the job_worker module's with_delay implementation correctly handles this parameter as a job-level execution timeout, especially since the internal requests.post call already has a 10s timeout.

)
main_job = group(*jobs)
main_job.on_done(self.delayable(channel="root_program.cycle").mark_check_eligibility_as_done(cycle))
main_job.on_done(self.delayable(channel="cycle").mark_check_eligibility_as_done(cycle))

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

The migration to group().on_done() is a significant improvement for handling job barriers. Ensure that the cycle recordset passed as an argument to mark_check_eligibility_as_done is correctly serialized and deserialized by the job_worker runner, particularly if it contains multiple records.

Replace queue_job dependency with job_worker across all spp_* modules.

- Update __manifest__.py depends in 7 modules
- Replace all odoo.addons.queue_job imports with job_worker
- Delete queue.job.channel XML data and Python model
- Add queue.limit data files for spp_area and spp_dci_server
- Migrate spp_programs queue_data.xml to queue.limit records
- Update test contexts from test_queue_job_no_delay to queue_job__no_delay
- Update menu/action references to job_worker external IDs
- Update lint scripts to whitelist job_worker model names
- Update Docker and pre-commit config for job_worker
@jeremi jeremi force-pushed the migrate-odoo-job-worker branch from 45f0b34 to 70cf079 Compare February 18, 2026 16:51
In Odoo 19, SQL constraint violations (UNIQUE, NOT NULL) raise
psycopg2.IntegrityError in tests, not ValidationError. The conversion
to ValidationError only happens in the RPC service layer, not during
direct ORM calls. Align with OCB's own test patterns.
The two-phase initialization for fresh starts with demo modules was
still referencing queue_job. Update to job_worker to match the migration.
The local test script was missing odoo-job-worker in the addons-path,
same issue that was previously fixed in the CI workflow files.
…del refs in access CSV

The ir.model.access.csv files referenced detail models from
spp_change_request_v2 without the module prefix, causing
"No matching record found for external id" errors during
fresh installation (e.g. MIS demo). Add spp_change_request_v2.
prefix to all model_id:id references.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant