Lyra/docs/superpowers/specs/2026-07-15-press-controls-design.md

Press controls + inter-job delay — design

Date: 2026-07-15

Problem

The monitor can enqueue a large backlog of acquisition jobs at once. The worker drains them serially with no pacing (only a 2s idle poll when the queue is empty) and offers no runtime control over the queue — the only per-job action on the Floor is Retry (on needs_attention). The user wants:

1. A configurable delay between downloads to spread sustained Qobuz load over time.
2. Per-item controls on the Floor ("the Press"): pause and remove a queued item.
3. Whole-press controls: pause/resume, clear the queue, and a force-stop of the currently-downloading album.

Constraints (from the existing code)

• The worker is a single, strictly-serial process (worker/lyra_worker/main.py:256-298). One job runs to completion in a blocking run_pipeline call before the loop does anything else.
• Config is read fresh every loop iteration (main.py:262, get_config), so any Config change takes effect within ~2s with no restart.
• An in-flight download cannot be cleanly interrupted — run_pipeline and the adapter download are synchronous/blocking with only an on_progress callback, no cancel token. The only interruption is process death, after which reclaim_stuck_jobs (claim.py:8-24) requeues the in-flight job at startup and clear_staging_root (main.py:233) wipes the partial download.
• claim_next (claim.py:27-57) claims the oldest state='requested' job.
• Deleting a Request cascades to its Job and Candidate rows (schema.prisma:51,66); the linked MonitoredRelease is SetNull, not cascaded.
• enqueue_due (monitor.py:115-147) re-enqueues a monitored release once it has no live (non-terminal) job and its lastSearchedAt is older than retryIntervalHours (default 6). So a removed/cleared monitored item reappears later on the normal cadence unless ignored.
• The web container runs prisma migrate deploy on startup (web/docker-entrypoint.sh), so a new migration auto-applies on deploy.

Decisions (confirmed with the user)

• Active item: add a global force-kill ("Stop now") that aborts the in-flight download via worker process-restart. Per-item controls act only on queued items.
• Remove / Clear semantics: cancel-for-now. Manual one-off requests are gone for good; monitored/wanted releases stay wanted and may reappear on the monitor's normal cadence. Permanent skip remains the existing per-release Ignore.
• Pause scope: downloads only. Pausing halts claiming/downloading; the monitor keeps finding and enqueueing releases (they pile up in the queue) and Resume drains them.

Data model

• New column Job.paused Boolean @default(false) — a Prisma migration (add_job_paused). Auto-applies on deploy.
• Config keys (key/value Config rows, no migration):
  • floor.paused — "true"/"false"; global press pause.
  • floor.jobDelaySeconds — integer string, default "0"; delay between downloads.
  • floor.killRequested — "true"/"false"; one-shot force-kill signal, cleared by the worker on consumption.

Worker changes

worker/lyra_worker/main.py and worker/lyra_worker/claim.py.

1. Inter-job delay. After finish_job (main.py:297), read floor.jobDelaySeconds from the already-fresh config and sleep that many seconds before the next loop iteration. Sleep in small (~2s) chunks so the loop stays responsive; the force-kill watcher runs on a separate thread and works regardless. Parse with the existing _int-style guard (invalid → 0). A value of 0 preserves today's back-to-back behavior.

2. Global pause. Immediately before claim_next (main.py:286), if floor.paused is truthy (_TRUE idiom), skip claiming: time.sleep(IDLE_SLEEP); continue. Because the loop is serial, any in-flight job has already finished before this check — pause stops the next download. Monitor sweep / scan / discovery (above the claim in the loop body) continue to run, so the queue keeps filling while paused.

3. Per-item pause. claim_next WHERE gains AND NOT "paused". A paused queued job is skipped by the claim but remains state='requested', so it is not "lost" and resumes claiming as soon as paused is cleared. reclaim_stuck_jobs is unaffected (it only touches in-flight states).

4. Force-kill watcher. A daemon thread started in run_forever (modeled on _liveness_heartbeat, main.py:216-226) polls floor.killRequested on its own DB connection every ~1.5s. When set, it clears the flag (commit) and calls os._exit(0). The container restart policy brings the worker back; startup clear_staging_root + reclaim_stuck_jobs wipe the partial download and requeue the aborted job. The "Stop now" action also sets floor.paused=true, so the worker returns idle rather than instantly re-downloading the aborted album; the user resumes when ready.

Infrastructure (ops repo)

The prod worker's Docker restart policy is currently no (docker inspect → RestartPolicy=no), so process death — from force-kill or any crash — leaves it down. The force-kill design requires restart: unless-stopped on the worker service in the ops repo (/opt/git/vm-download/docker/lyra/compose.yaml). The in-repo docker-compose.yml already has this (docker-compose.yml:53); this change brings prod in line and is good hygiene independent of this feature. This is a one-line change in a separate repo and must be applied before the force-kill feature is usable in prod.

API (web) — follows existing retry / library DELETE patterns

• GET /api/floor → { paused: boolean, jobDelaySeconds: number } (reads the Config rows). Used by the Floor to render the paused banner and control states.
• POST /api/floor with { action }:
  • "pause" / "resume" → upsert floor.paused.
  • "clear" → delete every Request whose Job.state = 'requested' (queued, not yet claimed). Cascade removes the Jobs and Candidates. The in-flight and terminal jobs are untouched. Returns the number cleared.
  • "stop" → upsert floor.paused='true' and floor.killRequested='true'.
• POST /api/requests/[id]/pause with { paused: boolean } → set Job.paused. Guard: the job must be state='requested' (else 400, matching the existing retry route), since pausing an in-flight job has no effect.
• DELETE /api/requests/[id] → delete the Request (cascade Job + Candidates). Guard: only when Job.state = 'requested', so the active download is never touched (that is the job of Stop now). Mirrors DELETE /api/library/[id].

Web UI

web/src/app/queue.tsx and web/src/app/_ui/job-row.tsx; styling via existing .btn variants in design.css.

• Press header (above the queue list):
  • Pause the press / Resume toggle (from GET /api/floor).
  • Clear queue — confirm dialog showing the queued count; calls POST /api/floor {action:"clear"}.
  • Stop now ⚠ — rendered only when a job is actively downloading (derivable from the existing /api/requests data: any job in matching/matched/downloading/tagging). Confirm dialog ("aborts the current download and pauses the press"); calls POST /api/floor {action:"stop"}.
  • A "Press paused — N queued" banner/chip when paused.
• Per-item controls (queued rows only — job.state === 'requested'), placed in the row note slot as .btn sm ghost:
  • Pause / Resume → POST /api/requests/[id]/pause.
  • Remove → DELETE /api/requests/[id].
  • A per-item paused row shows a subtle "paused" chip and the Resume button.
• Active-download and needs_attention rows keep their current rendering (Retry unchanged).
• Polling cadence stays 2s; GET /api/floor is fetched alongside the existing three calls in refresh().

Settings

Add a numeric field "Delay between downloads (seconds)" (default 0, min 0) to the settings form (web/src/app/settings/settings-form.tsx), persisted to floor.jobDelaySeconds via the existing settings config-write path (the same mechanism used for the other numeric/monitor settings). Include short helptext explaining it paces sustained Qobuz load.

Testing

• Worker (pytest, worker/tests/):
  • claim_next skips paused=true queued jobs and still claims the next unpaused one.
  • Global pause: the loop skips claiming when floor.paused is set (unit-test the claim-gate helper).
  • Delay: the delay value is read and applied (unit-test the parse/gate helper; keep the sleep injectable/mockable).
  • Kill watcher: when floor.killRequested is set, the watcher clears the flag and triggers exit (mock os._exit, assert flag cleared + exit called).
• Web (route tests, mirroring requests/[id]/retry/route.test.ts and library/bulk/route.test.ts):
  • POST /api/floor for each action; clear deletes only requested jobs; stop sets both flags.
  • POST /api/requests/[id]/pause toggles Job.paused; rejects non-requested.
  • DELETE /api/requests/[id] removes a queued Request; refuses an in-flight/terminal job.
  • GET /api/floor returns the Config-derived state.

Edge cases

• Remove/Clear of monitored items reappear later by design (cancel-for-now). No extra handling; the per-release Ignore is the permanent path.
• Kill with nothing downloading still restarts the worker (brief blip). Minimized by only showing Stop now while a download is active.
• Delay during pause: the delay sleeps only between jobs; pause is checked before each claim, so a paused press never starts a delayed job.
• Race: clicking per-item Pause/Remove on a job that was just claimed — the guards (state='requested') reject it; the UI only renders those buttons on queued rows, and the 2s refresh reconciles.

Out of scope

• Resuming a partially-downloaded album (no download-resume support; force-killed jobs re-download from scratch).
• Reordering the queue / per-item priority.
• Any change to the per-release Ignore or the wanted list UI.