diff --git a/docs/superpowers/specs/2026-07-15-press-controls-design.md b/docs/superpowers/specs/2026-07-15-press-controls-design.md new file mode 100644 index 0000000..fa6464b --- /dev/null +++ b/docs/superpowers/specs/2026-07-15-press-controls-design.md @@ -0,0 +1,179 @@ +# Press controls + inter-job delay — design + +Date: 2026-07-15 + +## Problem + +The monitor can enqueue a large backlog of acquisition jobs at once. The worker drains +them serially with **no pacing** (only a 2s idle poll when the queue is empty) and offers +**no runtime control** over the queue — the only per-job action on the Floor is Retry (on +`needs_attention`). The user wants: + +1. A **configurable delay between downloads** to spread sustained Qobuz load over time. +2. **Per-item controls** on the Floor ("the Press"): pause and remove a queued item. +3. **Whole-press controls**: pause/resume, clear the queue, and a force-stop of the + currently-downloading album. + +## Constraints (from the existing code) + +- The worker is a **single, strictly-serial process** (`worker/lyra_worker/main.py:256-298`). + One job runs to completion in a blocking `run_pipeline` call before the loop does anything + else. +- **Config is read fresh every loop iteration** (`main.py:262`, `get_config`), so any Config + change takes effect within ~2s with no restart. +- An **in-flight download cannot be cleanly interrupted** — `run_pipeline` and the adapter + download are synchronous/blocking with only an `on_progress` callback, no cancel token. The + only interruption is process death, after which `reclaim_stuck_jobs` (`claim.py:8-24`) + requeues the in-flight job at startup and `clear_staging_root` (`main.py:233`) wipes the + partial download. +- `claim_next` (`claim.py:27-57`) claims the oldest `state='requested'` job. +- Deleting a `Request` cascades to its `Job` and `Candidate` rows (`schema.prisma:51,66`); the + linked `MonitoredRelease` is `SetNull`, not cascaded. +- `enqueue_due` (`monitor.py:115-147`) re-enqueues a monitored release once it has no live + (non-terminal) job and its `lastSearchedAt` is older than `retryIntervalHours` (default 6). + So a removed/cleared monitored item reappears later on the normal cadence unless `ignored`. +- The web container runs `prisma migrate deploy` on startup (`web/docker-entrypoint.sh`), so a + new migration auto-applies on deploy. + +## Decisions (confirmed with the user) + +- **Active item:** add a global **force-kill** ("Stop now") that aborts the in-flight download + via worker process-restart. Per-item controls act only on queued items. +- **Remove / Clear semantics:** **cancel-for-now**. Manual one-off requests are gone for good; + monitored/wanted releases stay wanted and may reappear on the monitor's normal cadence. + Permanent skip remains the existing per-release **Ignore**. +- **Pause scope:** **downloads only**. Pausing halts claiming/downloading; the monitor keeps + finding and enqueueing releases (they pile up in the queue) and Resume drains them. + +## Data model + +- **New column** `Job.paused Boolean @default(false)` — a Prisma migration + (`add_job_paused`). Auto-applies on deploy. +- **Config keys** (key/value `Config` rows, no migration): + - `floor.paused` — `"true"`/`"false"`; global press pause. + - `floor.jobDelaySeconds` — integer string, default `"0"`; delay between downloads. + - `floor.killRequested` — `"true"`/`"false"`; one-shot force-kill signal, cleared by the + worker on consumption. + +## Worker changes + +`worker/lyra_worker/main.py` and `worker/lyra_worker/claim.py`. + +1. **Inter-job delay.** After `finish_job` (`main.py:297`), read `floor.jobDelaySeconds` from + the already-fresh `config` and sleep that many seconds before the next loop iteration. + Sleep in small (~2s) chunks so the loop stays responsive; the force-kill watcher runs on a + separate thread and works regardless. Parse with the existing `_int`-style guard (invalid → + 0). A value of 0 preserves today's back-to-back behavior. + +2. **Global pause.** Immediately before `claim_next` (`main.py:286`), if `floor.paused` is + truthy (`_TRUE` idiom), skip claiming: `time.sleep(IDLE_SLEEP); continue`. Because the loop + is serial, any in-flight job has already finished before this check — pause stops the + *next* download. Monitor sweep / scan / discovery (above the claim in the loop body) + continue to run, so the queue keeps filling while paused. + +3. **Per-item pause.** `claim_next` `WHERE` gains `AND NOT "paused"`. A paused queued job is + skipped by the claim but remains `state='requested'`, so it is not "lost" and resumes + claiming as soon as `paused` is cleared. `reclaim_stuck_jobs` is unaffected (it only touches + in-flight states). + +4. **Force-kill watcher.** A daemon thread started in `run_forever` (modeled on + `_liveness_heartbeat`, `main.py:216-226`) polls `floor.killRequested` on its own DB + connection every ~1.5s. When set, it clears the flag (commit) and calls `os._exit(0)`. The + container restart policy brings the worker back; startup `clear_staging_root` + + `reclaim_stuck_jobs` wipe the partial download and requeue the aborted job. The "Stop now" + action also sets `floor.paused=true`, so the worker returns **idle** rather than instantly + re-downloading the aborted album; the user resumes when ready. + +## Infrastructure (ops repo) + +The prod worker's Docker restart policy is currently **`no`** (`docker inspect` → +`RestartPolicy=no`), so process death — from force-kill *or any crash* — leaves it down. The +force-kill design requires `restart: unless-stopped` on the `worker` service in the ops repo +(`/opt/git/vm-download/docker/lyra/compose.yaml`). The in-repo `docker-compose.yml` already has +this (`docker-compose.yml:53`); this change brings prod in line and is good hygiene +independent of this feature. This is a one-line change in a separate repo and must be applied +before the force-kill feature is usable in prod. + +## API (web) — follows existing `retry` / `library DELETE` patterns + +- **`GET /api/floor`** → `{ paused: boolean, jobDelaySeconds: number }` (reads the Config + rows). Used by the Floor to render the paused banner and control states. +- **`POST /api/floor`** with `{ action }`: + - `"pause"` / `"resume"` → upsert `floor.paused`. + - `"clear"` → delete every `Request` whose `Job.state = 'requested'` (queued, not yet + claimed). Cascade removes the Jobs and Candidates. The in-flight and terminal jobs are + untouched. Returns the number cleared. + - `"stop"` → upsert `floor.paused='true'` and `floor.killRequested='true'`. +- **`POST /api/requests/[id]/pause`** with `{ paused: boolean }` → set `Job.paused`. Guard: + the job must be `state='requested'` (else `400`, matching the existing retry route), since + pausing an in-flight job has no effect. +- **`DELETE /api/requests/[id]`** → delete the `Request` (cascade Job + Candidates). Guard: + only when `Job.state = 'requested'`, so the active download is never touched (that is the + job of `Stop now`). Mirrors `DELETE /api/library/[id]`. + +## Web UI + +`web/src/app/queue.tsx` and `web/src/app/_ui/job-row.tsx`; styling via existing `.btn` +variants in `design.css`. + +- **Press header** (above the queue list): + - `Pause the press` / `Resume` toggle (from `GET /api/floor`). + - `Clear queue` — confirm dialog showing the queued count; calls `POST /api/floor + {action:"clear"}`. + - `Stop now ⚠` — rendered only when a job is actively downloading (derivable from the + existing `/api/requests` data: any job in `matching`/`matched`/`downloading`/`tagging`). + Confirm dialog ("aborts the current download and pauses the press"); calls `POST /api/floor + {action:"stop"}`. + - A "Press paused — N queued" banner/chip when `paused`. +- **Per-item controls** (queued rows only — `job.state === 'requested'`), placed in the row + `note` slot as `.btn sm ghost`: + - `Pause` / `Resume` → `POST /api/requests/[id]/pause`. + - `Remove` → `DELETE /api/requests/[id]`. + - A per-item paused row shows a subtle "paused" chip and the `Resume` button. +- Active-download and `needs_attention` rows keep their current rendering (Retry unchanged). +- Polling cadence stays 2s; `GET /api/floor` is fetched alongside the existing three calls in + `refresh()`. + +## Settings + +Add a numeric field **"Delay between downloads (seconds)"** (default 0, min 0) to the settings +form (`web/src/app/settings/settings-form.tsx`), persisted to `floor.jobDelaySeconds` via the +existing settings config-write path (the same mechanism used for the other numeric/monitor +settings). Include short helptext explaining it paces sustained Qobuz load. + +## Testing + +- **Worker** (pytest, `worker/tests/`): + - `claim_next` skips `paused=true` queued jobs and still claims the next unpaused one. + - Global pause: the loop skips claiming when `floor.paused` is set (unit-test the claim-gate + helper). + - Delay: the delay value is read and applied (unit-test the parse/gate helper; keep the + sleep injectable/mockable). + - Kill watcher: when `floor.killRequested` is set, the watcher clears the flag and triggers + exit (mock `os._exit`, assert flag cleared + exit called). +- **Web** (route tests, mirroring `requests/[id]/retry/route.test.ts` and + `library/bulk/route.test.ts`): + - `POST /api/floor` for each action; `clear` deletes only `requested` jobs; `stop` sets both + flags. + - `POST /api/requests/[id]/pause` toggles `Job.paused`; rejects non-`requested`. + - `DELETE /api/requests/[id]` removes a queued Request; refuses an in-flight/terminal job. + - `GET /api/floor` returns the Config-derived state. + +## Edge cases + +- **Remove/Clear of monitored items** reappear later by design (cancel-for-now). No extra + handling; the per-release Ignore is the permanent path. +- **Kill with nothing downloading** still restarts the worker (brief blip). Minimized by only + showing `Stop now` while a download is active. +- **Delay during pause:** the delay sleeps only *between* jobs; pause is checked before each + claim, so a paused press never starts a delayed job. +- **Race:** clicking per-item Pause/Remove on a job that was just claimed — the guards + (`state='requested'`) reject it; the UI only renders those buttons on queued rows, and the + 2s refresh reconciles. + +## Out of scope + +- Resuming a partially-downloaded album (no download-resume support; force-killed jobs + re-download from scratch). +- Reordering the queue / per-item priority. +- Any change to the per-release Ignore or the wanted list UI.