skynet/small-scripts
2
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

idle-draft: fill all free worker slots, not one per completion

Browse Source 
try_submit() returned after a single submission and was only called at
startup and once per completion, capping real concurrency at 1 task
regardless of the parallel setting — the ThreadPoolExecutor pool was
sized but never filled. Loop until every free slot is filled or no
eligible candidate remains, per spec §10 (wording sharpened to make the
whole-pool semantics explicit). No stub-claude harness exists yet to
test dispatch concurrency end-to-end; verified live against the real
queue (36 candidates, parallel=20).

Claude-Session: https://claude.ai/code/session_01Lgv4Qn82boNFC1jn8QXSNw
This commit is contained in:
Paul O'Reilly
2026-08-03 07:29:52 +12:00
parent bdd10cbbc1
commit 56c621713d
2 changed files with 23 additions and 18 deletions
Download Patch File Download Diff File Expand all files Collapse all files

35
scripts/idle-draft
View File

@@ -1369,23 +1369,26 @@ def run_dispatch(args, config: dict, repo: Path, state_path: Path) -> int:
def try_submit(): def try_submit():
nonlocal dispatched_count nonlocal dispatched_count
if args.once and dispatched_count >= 1:
return
nonlocal queue nonlocal queue
queue = build_ready_queue(repo, config, state, in_flight) while len(futures) < max(1, parallel):
for cand in queue: if args.once and dispatched_count >= 1:
provider, _ = select_provider(cand["work_type"], config, gates, credential_ok) return
if provider is None: queue = build_ready_queue(repo, config, state, in_flight)
continue for cand in queue:
cand = dict(cand) provider, _ = select_provider(cand["work_type"], config, gates, credential_ok)
cand["provider"] = provider if provider is None:
cand["before_pct"] = gates.get(provider, {}).get("seven_day_pct") continue
in_flight.add(cand["item_key"]) cand = dict(cand)
dispatched_count += 1 cand["provider"] = provider
log_event(repo, f"dispatch: {cand['item_key']} work_type={cand['work_type']} provider={provider}") cand["before_pct"] = gates.get(provider, {}).get("seven_day_pct")
fut = executor.submit(run_task, repo, config, cand, defaults["max_turns"], defaults["task_timeout"]) in_flight.add(cand["item_key"])
futures[fut] = cand dispatched_count += 1
return log_event(repo, f"dispatch: {cand['item_key']} work_type={cand['work_type']} provider={provider}")
fut = executor.submit(run_task, repo, config, cand, defaults["max_turns"], defaults["task_timeout"])
futures[fut] = cand
break
else:
return
try_submit() try_submit()

6
specs/idle-draft.spec.md
View File

@@ -218,8 +218,10 @@ next cycle); the walk continues to the next candidate.
Up to `parallel` tasks run concurrently (`concurrent.futures.ThreadPoolExecutor`). Up to `parallel` tasks run concurrently (`concurrent.futures.ThreadPoolExecutor`).
Each completion is handled **serially** in the main thread (state updates and git Each completion is handled **serially** in the main thread (state updates and git
commits never race). On every completion — and before the very first dispatch — the commits never race). On every completion — and before the very first dispatch — the
gates (§6) are recomputed from a fresh probe (§5); a task is submitted to fill a free gates (§6) are recomputed from a fresh probe (§5); tasks are then submitted until
worker slot only while an eligible `(item, work_type, provider)` triple remains. The **every** free worker slot is filled or no eligible `(item, work_type, provider)`
triple remains — one completion may therefore trigger multiple submissions, and the
initial call before the first dispatch fills the whole pool, not one slot. The
loop exits (dispatch mode, non-dryrun) when no eligible candidate remains. `--once` loop exits (dispatch mode, non-dryrun) when no eligible candidate remains. `--once`
dispatches at most one task total, then exits without waiting for further slots. dispatches at most one task total, then exits without waiting for further slots.