Skip to content

Commit

Permalink
ensure last updates are saved
Browse files Browse the repository at this point in the history
  • Loading branch information
prenner committed Jan 10, 2025
1 parent d7f207e commit c9376ad
Showing 1 changed file with 6 additions and 3 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -310,7 +310,10 @@ def log(event: str) -> None:
# Our entry estimatation (based on the cursor parameters)
# occasionally seem to over-estimate
log(f"est {entry.estimated_entries_in_range} entries")
elif i % 100 == 0:

updates.extend(entry.updates)

if i % 100 == 0:
# If we get down to one second, we can potentially be
# fetching an arbitrary large amount of data in one go,
# so store the checkpoint occasionally
Expand All @@ -324,8 +327,8 @@ def log(event: str) -> None:
),
)
current_next_fetch = entry.next
break
updates.extend(entry.updates)
updates = []

else: # AKA a successful fetch
# If we're hovering near the single-fetch limit for a period
# of time, we can likely safely expand our range.
Expand Down

0 comments on commit c9376ad

Please sign in to comment.