Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

exclude "reserved" jobs in Autopopulate.populate #1107

Merged
merged 4 commits into from
Jan 29, 2024
Merged

exclude "reserved" jobs in Autopopulate.populate #1107

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
ed3fd03
exclude "reserved" jobs in `Autopopulate.populate`
ttngu207 Sep 6, 2023
7415a08
update test, CHANGELOG
ttngu207 Sep 6, 2023
3f111ae
bugfix in `test_populate_exclude_error_and_ignore_jobs`
ttngu207 Dec 4, 2023
4680325
Merge remote-tracking branch 'upstream/master' into exclude_reserved_…
Dec 4, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions CHANGELOG.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,7 @@
- Fixed - Updated set_password to work on MySQL 8 - PR [#1106](https://github.com/datajoint/datajoint-python/pull/1106)
- Added - Missing tests for set_password - PR [#1106](https://github.com/datajoint/datajoint-python/pull/1106)
- Changed - Returning success count after the .populate() call - PR [#1050](https://github.com/datajoint/datajoint-python/pull/1050)
- Fixed - `Autopopulate.populate` excludes `reserved` jobs in addition to `ignore` and `error` jobs

### 0.14.1 -- Jun 02, 2023
- Fixed - Fix altering a part table that uses the "master" keyword - PR [#991](https://github.com/datajoint/datajoint-python/pull/991)
Expand Down
4 changes: 2 additions & 2 deletions datajoint/autopopulate.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -207,12 +207,12 @@ def handler(signum, frame):

keys = (self._jobs_to_do(restrictions) - self.target).fetch("KEY", limit=limit)

# exclude "error" or "ignore" jobs
# exclude "error", "ignore" or "reserved" jobs
if reserve_jobs:
exclude_key_hashes = (
jobs
& {"table_name": self.target.table_name}
& 'status in ("error", "ignore")'
& 'status in ("error", "ignore", "reserved")'
).fetch("key_hash")
keys = [key for key in keys if key_hash(key) not in exclude_key_hashes]

Expand Down
8 changes: 5 additions & 3 deletions tests_old/test_autopopulate.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -75,17 +75,19 @@ def test_populate_exclude_error_and_ignore_jobs(self):
assert_true(self.subject, "root tables are empty")
assert_false(self.experiment, "table already filled?")

keys = self.experiment.key_source.fetch("KEY", limit=2)
keys = self.experiment.key_source.fetch("KEY", limit=3)
for idx, key in enumerate(keys):
if idx == 0:
schema.schema.jobs.ignore(self.experiment.table_name, key)
else:
elif idx == 1:
schema.schema.jobs.error(self.experiment.table_name, key, "")
else:
schema.schema.jobs.reserve(self.experiment.table_name, key)

self.experiment.populate(reserve_jobs=True)
assert_equal(
len(self.experiment.key_source & self.experiment),
len(self.experiment.key_source) - 2,
len(self.experiment.key_source) - 3,
)

def test_allow_direct_insert(self):
Expand Down