Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[IMP] queue_job: Add split method #658

Open
wants to merge 3 commits into
base: 14.0
Choose a base branch
from

Conversation

paradoxxxzero
Copy link

@paradoxxxzero paradoxxxzero commented Jun 4, 2024

This is a draft PR trying an alternative approach for #566

  class ResPartner(models.Model):
      # ...

      def copy_all_partners(self):
          # Duplicate all partners in batches of 30:
          self.with_delay(split=30).copy()

    # or

     def copy_all_partners(self):
          # Duplicate all partners in batches of 30:
          self.delayable().set(description='Copy Partners').copy().split(30).delay()

  # ...
  self.env['res.partner'].search([], limit=1000).copy_all_partners()

The split function on Delayable returns a DelayableGroup.

This is an attempt and I am open to suggestions

@OCA-git-bot
Copy link
Contributor

Hi @guewen,
some modules you are maintaining are being modified, check this out!

"""

total_records = len(self.recordset)
count = 1 + total_records // size
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For a division without remainder, won't this return one too many?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, indeed. I have rewritten the loop and added some tests.

count = 1 + total_records // size

delayables = []
for batch in range(count):
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Perhaps simpler to let range do more of the work?

for start in range(0, total_records, size):

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good idea

group.delay()
return [d._generated_job for d in group._delayables]

delayable.delay()
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this necessary? Couldn't a developer just call delayable.split() themselves?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Removed

Copy link
Contributor

@amh-mw amh-mw left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@paradoxxxzero paradoxxxzero marked this pull request as ready for review June 4, 2024 14:32
@paradoxxxzero
Copy link
Author

The coverage does not seem to include the unittest :/

@amh-mw
Copy link
Contributor

amh-mw commented Jun 4, 2024

The coverage does not seem to include the unittest :/

Import the new test in tests/__init__.py

@paradoxxxzero paradoxxxzero force-pushed the 14.0-imp-queue_job-split branch 2 times, most recently from facdc08 to 46f344e Compare June 4, 2024 15:12
@paradoxxxzero
Copy link
Author

paradoxxxzero commented Jun 4, 2024

They weren't run as they were unitest.TestCase, I changed them to TransactionCase for odoo to run them.

@paradoxxxzero
Copy link
Author

I added a chain parameter to split into a chain instead of a group.

Is there anything lacking for this to be merged?

@amh-mw
Copy link
Contributor

amh-mw commented Jul 2, 2024

Is there anything lacking for this to be merged?

Just the attention of @OCA/connector-maintainers

@OCA-git-bot
Copy link
Contributor

This PR has the approved label and has been created more than 5 days ago. It should therefore be ready to merge by a maintainer (or a PSC member if the concerned addon has no declared maintainer). 🤖

1 similar comment
@OCA-git-bot
Copy link
Contributor

This PR has the approved label and has been created more than 5 days ago. It should therefore be ready to merge by a maintainer (or a PSC member if the concerned addon has no declared maintainer). 🤖

Copy link
Contributor

@simahawk simahawk left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice, thanks! Just some minor remarks :)

@guewen ping :)

# pylint: disable=odoo-addons-relative-import
from odoo.addons.queue_job.delay import Delayable, DelayableGraph


class TestDelayable(unittest.TestCase):
class TestDelayable(common.TransactionCase):
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There's no transaction required here. Use TestCase or BaseCase

from odoo.addons.queue_job.delay import Delayable


class TestDelayableSplit(common.TransactionCase):
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

same regarding the base class

Comment on lines +538 to +540
"""Split the Delayable into a DelayableGroup or DelayableChain
if `chain` is True containing batches of size `size`
"""
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
"""Split the Delayable into a DelayableGroup or DelayableChain
if `chain` is True containing batches of size `size`
"""
"""Split the Delayables.
Use `DelayableGroup` or `DelayableChain`
if `chain` is True containing batches of size `size`
"""

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants