Skip to content

reddit firehose: time-box the read so each pass fits its 60 s slot - #281

Merged
ralyodio merged 1 commit into
mainfrom
reddit-firehose-timebox
Nov 9, 2026
Merged

ralyodio merged 1 commit into
mainfrom
reddit-firehose-timebox

Conversation

@ralyodio

@ralyodio ralyodio commented Nov 9, 2026

Copy link
Copy Markdown
Contributor

Follow-up to #280: passes overran the 60 s grid slot and idled until the next one, halving throughput. Read phase is now time-boxed at 40 s.

🤖 Generated with Claude Code

A pass that ran 67-87 s missed its slot on the daemon's fixed grid and
waited for the next one: ~6 pages a minute against Reddit's ~9, so lag
grew ~40 s a minute. Reading now stops at 40 s (gap 3.5 s, up to 30
pages), leaving the slot for the writes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@sh1pt-actions-fleet
sh1pt-actions-fleet Bot force-pushed the reddit-firehose-timebox branch from 6528f3b to dfdc9c0 Compare November 9, 2026 20:42
@ralyodio
ralyodio merged commit 8553ca9 into main Nov 9, 2026
3 checks passed
@ralyodio
ralyodio deleted the reddit-firehose-timebox branch November 9, 2026 20:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant