Lyra/docs/superpowers/specs/2026-07-11-lyra-library-manager-design.md

Lyra — Library Manager (Design)

Date: 2026-07-11 Status: Approved design, ready for implementation planning Slice: 2 of 3 (Library manager)

Overview

Slice 2 turns Lyra from a one-shot acquisition tool into a standing library manager: you follow artists, monitor the specific releases you want, and Lyra keeps working in the background to obtain and upgrade them. It is Lidarr-style monitoring built directly on the slice 1 acquisition engine — the monitor's only output is the same Request + Job a manual request produces, so the entire existing pipeline is reused unchanged.

Two loops are equally central and share one monitoring engine:

1. Watched artists → auto-grab. Follow an artist, browse their discography, and monitor the releases you want. Optionally flip a per-artist toggle so future releases auto-monitor as they appear.
2. Wanted list → keep retrying. Every monitored release without an acceptable copy is "wanted" and is re-attempted on a schedule until a good copy lands — and, within a window, upgraded to better quality.

Goals

• Follow artists and monitor releases at release-group granularity; obtain and keep them current automatically.
• Reuse the slice 1 pipeline verbatim — monitoring enqueues normal jobs and reads their outcomes; the pipeline itself is untouched.
• Never silently drop a want: a monitored release without a copy stays wanted and keeps being retried.
• Keep sources and secrets in the worker. The web app performs only read-only, credential-free MusicBrainz metadata lookups for interactive search/browse.
• Fully developable and testable with zero real credentials via a fake MusicBrainz browser and seeded Postgres.

Non-goals (this slice)

• Notifications / webhooks on grab or needs-attention (a later slice).
• Discovery / recommendations (slice 3).
• A release-calendar / upcoming view.
• Multi-user; this remains a single-user home-server tool.
• Any change to how downloading, tagging, or the six-stage pipeline work.

Scope

In scope: watched artists (live MusicBrainz search + follow), a per-artist discography detail view with monitor toggles, a per-artist "auto-monitor future releases" toggle, a background monitor sweep (discovery + retry/upgrade), a cross-artist wanted list, and the outcome-reconcile that feeds job results back into monitoring state.

Out of scope: notifications, discovery, calendar. Deferred hardening from slice 1 is unaffected and remains deferred.

Key decisions

• Default is passive. Following an artist marks their entire discography unmonitored. You opt in per release. A per-artist autoMonitorFuture toggle (default off) makes genuinely-new releases auto-monitor from the moment it is enabled.
• No type filtering when auto-monitoring. When autoMonitorFuture is on, all official release-group types (albums, EPs, singles, live, compilations, …) auto-monitor; you unmonitor the noise by hand. ("Everything, filter later.")
• Quality upgrades within a window. A quality cutoff and an upgrade window (N days) govern re-searching: below cutoff and within the window → keep chasing better quality; at/above cutoff, or window expired → frozen (done).
• One table for discography and wanted list. A "wanted" item is simply a monitored release-group without a copy at cutoff. No separate wanted table, no sync problem.
• Monitor lives in the worker. A scheduled sweep runs inside the existing worker loop. The web app owns CRUD + display, plus read-only MusicBrainz search for the interactive picker.

Architecture

No new containers. Two additions to the existing three-container stack:

• Worker gains a monitor. Beside the job-claim loop, a cheap ~60s tick runs monitor.sweep(); after each pipeline run the loop calls monitor.reconcile(). All scheduled MusicBrainz work and all enqueuing live here, next to the pipeline it feeds.
• Web gains monitoring UI + read-only MB reads. New pages/routes for watched artists, discography detail, and the wanted list, plus a thin MusicBrainz read module used only for the live search/browse picker.

┌───────────────────────────┐   Postgres    ┌──────────────────────────────┐
│  Next.js app (UI + API)   │──────────────▶│  Python worker               │
│                           │  requests/    │                              │
│  /artists  /artists/[id]  │  jobs +       │  claim → run_pipeline (S1)   │
│  /wanted   live MB picker │  monitor rows │  monitor.sweep()  (~60s tick)│
│                           │◀──────────────│  monitor.reconcile(job)      │
│  read-only MB search ─────┼──▶ MusicBrainz│  scheduled MB discovery ─────┼─▶ MB
└───────────────────────────┘  (metadata)   └──────────────────────────────┘

Separation of concerns. The web reasons about watched artists, monitored releases, requests, and jobs; the worker reasons about candidates, files, and scheduled acquisition. The web's only external reach is read-only, cred-free MusicBrainz metadata for the picker — every download source and every credential remains worker-only, as does the scheduled discovery poll and all pipeline-time MB resolution.

Data model

Two new tables, one new nullable column, and config keys in the existing key/value Config table (no migration for config).

WatchedArtist
  id
  mbid                (unique)                 # MusicBrainz artist MBID
  name
  autoMonitorFuture   Bool  default false      # auto-monitor new releases going forward
  monitorFrom         DateTime default now()   # boundary for "future" when the toggle is on
  lastPolledAt        DateTime?                 # throttles the discovery poll per artist
  createdAt
  monitoredReleases   MonitoredRelease[]

MonitoredRelease                                # discography row AND wanted-list row
  id
  watchedArtistId     FK?                       # null = standalone manual wanted add
  artistMbid, artistName                        # denormalized: standalone rows need no join
  rgMbid              (unique)                   # release-group MBID
  album
  primaryType         String?                    # "Album" | "EP" | "Single" | ...
  secondaryTypes      String[]                   # ["Live", "Compilation", ...]
  firstReleaseDate    String?                    # "YYYY-MM-DD" as MB reports it
  monitored           Bool  default false        # does the sweep act on it?
  state               enum(wanted|grabbed|fulfilled)  default wanted
  currentQualityClass Int?                        # quality of the copy we have (null = none)
  firstGrabbedAt      DateTime?                   # drives the upgrade window
  lastSearchedAt      DateTime?                   # throttles retry
  createdAt

Request  (existing) gains:
  monitoredReleaseId  String?                     # set when the monitor enqueues; null for manual

Config keys (worker-read, via existing Config):

key	default	meaning
monitor.enabled	false	master switch; off → worker behaves exactly as slice 1
monitor.pollIntervalHours	24	how often discovery re-browses each artist
monitor.retryIntervalHours	6	how often a wanted item is re-searched
monitor.qualityCutoff	2	the qualityClass at/above which a release is "done"
monitor.upgradeWindowDays	14	days after first grab to keep chasing upgrades

qualityClass (from slice 1's quality.py): 3 = hi-res lossless, 2 = CD lossless, 1 = lossy. The default cutoff 2 means "any lossless satisfies; only a lossy copy stays wanted and keeps chasing an upgrade."

How rows come to exist

• Follow (live picker): the web already fetched the artist + discography from MusicBrainz to render the picker. On Follow it writes the WatchedArtist and persists the discography as MonitoredRelease rows, all unmonitored. The detail view is therefore instant.
• Background discovery: the worker's poll re-browses each watched artist and inserts any newly appearing release-groups. If autoMonitorFuture is on and the release is new relative to monitorFrom, it is inserted monitored + wanted; otherwise unmonitored.
• Standalone wanted add: adding an arbitrary album to the wanted list (not via a followed artist) resolves it to a release-group and writes a single MonitoredRelease with watchedArtistId = null, monitored = true.

The monitor (worker)

New module worker/lyra_worker/monitor.py. It uses an MbBrowser seam (below) and plain SQL over the shared connection; scheduling is driven entirely by timestamps in the tables, so the loop tick is just a cheap gate.

Scheduling

run_forever() records the last tick time; each loop iteration, if monitor.enabled and ≥ ~60s elapsed, it calls monitor.sweep(conn, browser, cfg) before continuing to claim jobs. After every run_pipeline it calls monitor.reconcile(conn, job_id). One process, one loop, no new infra.

sweep() — two phases

Phase 1 — Discovery (per artist where now - lastPolledAt > pollIntervalHours):

• browser.browse_release_groups(artist.mbid) → release-group list.
• For each rg not already a MonitoredRelease, insert it. monitored = artist.autoMonitorFuture and firstReleaseDate > artist.monitorFrom; state wanted. (All types eligible — no filtering.)
• Set lastPolledAt = now.

Phase 2 — Retry / enqueue (rows with monitored = true, state != fulfilled):

• Skip rows with an active (non-terminal) job for the same release.
• Skip / satisfy via dedupe: if a LibraryItem at/above cutoff already exists → set fulfilled.
• For state = grabbed: if now - firstGrabbedAt >= upgradeWindowDays or currentQualityClass >= cutoff → set fulfilled and skip; else it is an upgrade candidate.
• For due rows (lastSearchedAt null or older than retryIntervalHours): create Request(artist, album, monitoredReleaseId=id) + Job(state=requested); set lastSearchedAt = now. The slice 1 pipeline takes it from there.

reconcile(job_id) — feed outcomes back

After the pipeline finishes a job that has a monitoredReleaseId:

• imported: set currentQualityClass from the winning candidate and firstGrabbedAt (if null); set state = fulfilled if qualityClass >= cutoff, else grabbed.
• needs_attention: leave state = wanted (already has lastSearchedAt, so it waits out the retry interval before the next attempt).

Reconcile is a separate call in the main loop, so the pipeline code is untouched and the whole feature is inert when monitor.enabled = false.

MonitoredRelease state machine

discovery / follow
   ├─(autoMonitorFuture & new)─► monitored + wanted ─┐
   └─(default / back-cat)──────► unmonitored + wanted │  user toggle → monitored
                                                       ▼
   wanted ──sweep enqueues──► [job] ──imported < cutoff──► grabbed ──window expired / at cutoff──► fulfilled
      ▲                          │                            │
      └─────needs_attention──────┘        imported ≥ cutoff───┴──────────────────────────────► fulfilled

Unmonitor at any time → dormant (the sweep ignores it). A want is never dropped; it is retried on its interval until satisfied.

MusicBrainz access

The worker's existing MusicBrainzResolver handles pipeline-time album resolution. Monitoring adds a small browsing seam.

Worker — MbBrowser protocol (real impl via musicbrainzngs, lazily imported; faked in tests):

• search_artist(name) -> list[ArtistHit] — {mbid, name, disambiguation}.
• browse_release_groups(artist_mbid) -> list[ReleaseGroupInfo] — {rgMbid, title, primaryType, secondaryTypes, firstReleaseDate}.

Web — read-only MB read module for the live picker only: artist search and discography browse, returning the same shapes the web persists as MonitoredRelease rows on Follow. No credentials; read-only; metadata only.

Web UI & API

Three surfaces, thin over the two tables:

• /artists — watched artists (name, #monitored / #have), Add artist (live search picker), remove. Per-artist autoMonitorFuture toggle.
• /artists/[id] — the discography: every MonitoredRelease for the artist, each with a status badge (Have@quality / Wanted / Dormant), a monitor toggle, and a Search now button (enqueues immediately).
• /wanted — cross-artist list of everything monitored and not fulfilled (includes standalone manual adds and needs-attention retries), with last-searched time and Search now. Plus Add to wanted (resolve an arbitrary artist+album → standalone monitored row).

API routes (Next.js):

• GET /api/mb/artists?q= — live artist search (read-only MB).
• GET /api/mb/artists/[mbid]/releases — discography browse (read-only MB).
• POST /api/artists — follow: create WatchedArtist + persist discography as unmonitored MonitoredRelease rows.
• GET /api/artists, GET /api/artists/[id], DELETE /api/artists/[id], PATCH /api/artists/[id] (toggle autoMonitorFuture).
• GET /api/wanted — monitored, not-fulfilled rows across artists.
• POST /api/wanted — standalone add (resolve + create row).
• PATCH /api/releases/[id] — toggle monitored.
• POST /api/releases/[id]/search — enqueue now (Request + Job).

"Search now" and standalone adds create the same Request + Job the monitor does (with monitoredReleaseId set), so reconcile handles their outcomes too.

Integration touch-point: upgrades must replace

Quality upgrades re-import an album that is already in the library. LibraryItem is @@unique([artist, album]), so the import stage must upsert the row and overwrite the on-disk files rather than error on a duplicate. Current behavior will be verified during planning; if it errors today, the minimal import-upsert change is included in this slice. This is the only pipeline-adjacent change; the six stages otherwise stand unchanged.

Error handling & edge cases

Situation	Handling
New release not yet on any source	Job → needs_attention; row stays wanted; retried each interval until a copy appears (natural pre-order behavior).
Artist name ambiguous	The live picker shows MB disambiguation; the user chooses. No blind auto-match.
Duplicate discovery	MonitoredRelease.rgMbid is unique; re-browsing never duplicates rows.
Piled-up jobs	Sweep skips a release that already has a non-terminal job.
Already in library	Dedupe against LibraryItem; at/above cutoff → fulfilled without enqueuing.
Upgrade never improves	Upgrade window expires → fulfilled (frozen); no endless re-searching.
MB unreachable during sweep	Discovery for that artist is skipped this tick; retried next poll. Web picker surfaces the error to the user.
Monitor disabled	monitor.enabled = false → no sweep, no reconcile effects; worker behaves exactly as slice 1.

Testing strategy

Same philosophy as slice 1 — logic is testable with zero real credentials.

• Monitor unit tests — FakeMbBrowser (canned release-group lists) + seeded Postgres. Assert: discovery sets monitored correctly (default off; on only when autoMonitorFuture and new-vs-monitorFrom); sweep enqueues only due rows; respects poll/retry intervals; skips releases with active jobs; applies cutoff and upgrade-window transitions; dedupes against LibraryItem.
• Reconcile tests — a finished job in each end state updates its MonitoredRelease correctly (imported below/above cutoff; needs_attention).
• Web API route tests — artists/wanted/releases CRUD and toggles against seeded Postgres; the MB read routes tested against a stubbed MB read module.
• Opt-in live test (LYRA_LIVE_TESTS) — browse a real artist's discography and search a real artist name.
• Backward-compat — with monitor.enabled = false, the full existing worker suite passes unchanged and no monitoring side effects occur.

Goal: the entire library manager is built and tested on a laptop with fakes and a seeded database; real MusicBrainz enters only for the opt-in live test.

Notes: after this slice

Slice 2 makes Lyra a standing library manager: follow artists, monitor exactly the releases you want (or auto-monitor future ones per artist), and let the background sweep obtain and upgrade them through the slice 1 pipeline, with a wanted list that never forgets. Deferred to later: notifications, a release calendar, and slice 3 (discovery), each with its own spec → plan → implementation cycle.