Lyra/docs/superpowers/specs/2026-07-11-lyra-library-scan-design.md

Lyra — Library Scan (Design)

Date: 2026-07-11 Status: Approved design, ready for implementation planning Scope: Ingest an existing on-disk library into Lyra's have + monitor state.

Problem / Goal

Lyra only "knows about" albums it downloaded (via LibraryItem/MonitoredRelease rows). Albums already on disk — from prior downloads whose DB rows were lost, or a pre-existing collection — are invisible: not shown as owned, not monitored, and re-downloadable. A library scan walks /music, matches each album to MusicBrainz, and records it as have (LibraryItem) and on monitor (MonitoredRelease, fulfilled), with the artist followed (WatchedArtist), so the library manager reflects the real collection.

Decisions (from brainstorming)

• Identification: folder names first (Artist/Album (Year)/), falling back to embedded tags (mutagen) when the folder doesn't yield an artist+album.
• MusicBrainz matching: resolve every album; skip (log) anything MB can't match — only matched albums are recorded (nothing half-recorded).
• Trigger: a manual "Scan library" button (no auto-scan on startup — avoids re-hitting MB for the whole library on every boot).
• Artists: scanned artists become followed (WatchedArtist, autoMonitorFuture=false) so they appear under /artists.

Non-goals

• No downloading, tagging, or file moves — scan is read-only on /music.
• No auto-scan on startup; no filesystem watching.
• No multi-disc/compilation special handling; each album folder is one album.
• No re-tagging or re-organizing existing files.

Architecture

The scan runs in the worker (it owns the /music mount, the MusicBrainz resolver, and mutagen). The worker has no HTTP server, so the UI triggers it through the database:

[Settings "Scan library" button] --POST /api/scan--> Config: scan.requested = "true"
                                                          │
[worker loop] --sees scan.requested--> scan_library() --> upserts LibraryItem/MonitoredRelease/WatchedArtist
                                                          └─> Config: scan.result = "imported X, skipped Y", clears scan.requested
[Settings] --GET /api/scan--> reads Config: scan.result   (status line)

This mirrors the existing monitor-tick pattern (the worker loop already reads config and does periodic work). No new transport.

The scan (worker/lyra_worker/scan.py)

scan_library(conn, resolver, dest_root="/music") -> ScanResult(imported, skipped). For each Artist/Album[ (Year)]/ directory under dest_root (skipping .staging and dot-dirs):

1. Identify artist + album + year:
   • Parse the folder names: artist = parent dir; album, year = parse "Name (YYYY)" (year optional).
   • If the folder parse yields a blank artist or album, read the first audio file's embedded tags (mutagen: artist, album) as a fallback.
   • Skip folders with no audio files.
2. Quality: read the first audio file via mutagen (bits_per_sample, sample_rate, lossless-by-extension) → build a Quality → quality_class (3/2/1). This is the recorded currentQualityClass/LibraryItem.qualityClass.
3. Resolve (artist, album) via the MusicBrainz resolver → canonical artist/album/year + rgMbid + artistMbid. No match → skip (increment skipped, log).
4. Record (idempotent upserts):
   • WatchedArtist — INSERT ... ON CONFLICT (mbid) DO NOTHING when artistMbid is present (autoMonitorFuture=false, monitorFrom=now). If artistMbid is blank, the release is monitored standalone (watchedArtistId=null).
   • MonitoredRelease — INSERT ... ON CONFLICT (rgMbid) DO UPDATE SET monitored=true, state='fulfilled', currentQualityClass=<q>, firstGrabbedAt=COALESCE(existing, now), watchedArtistId=<artist>. ("on monitor")
   • LibraryItem — INSERT ... ON CONFLICT (artist, album) DO NOTHING (path=album_dir, source "scan", format, qualityClass). Never overwrites a real download's provenance. ("have")
   • Count as imported when a new LibraryItem (or monitored record) is created.

Re-scanning is safe: existing rows are updated-or-left, never duplicated.

Data-model touch-points

• MBTarget gains rg_mbid: str = "" and artist_mbid: str = ""; the MusicBrainzResolver already computes the release-group and artist-credit — it fills these (they are currently discarded). This also lets pipeline jobs link to release-groups later.
• LibraryItem.requestId becomes nullable (String?). A scanned "have" record has no originating Request; making the FK optional lets scanned items exist without a synthetic request (which would otherwise flood the queue view). Downloads still set requestId. Additive, backward-compatible migration (DROP NOT NULL).
• Config keys: scan.requested (web sets "true", worker clears) and scan.result (worker writes a human summary + timestamp; Settings displays it). Plain (non-secret) Config rows.

Everything else reuses existing tables (WatchedArtist, MonitoredRelease, LibraryItem).

Web

• POST /api/scan → upsert Config scan.requested="true" → 202.
• GET /api/scan → return Config scan.result (or "never run").
• Settings gains a "Scan library" button + a status line (last result), following the existing settings-form style.

Worker wiring

run_forever()'s loop, alongside the monitor tick: if config["scan.requested"] is truthy, run scan_library(conn, resolver, DEST_ROOT), write scan.result, and clear scan.requested. Wrapped in try/except (log, rollback) so a scan error can't kill the worker — consistent with the monitor/pipeline guards.

Testing

Offline, zero real MB/downloads:

• scan_library unit tests with a temp /music tree, a fake resolver (canned MBTarget incl. rg_mbid/artist_mbid, or None to force a skip), and a seeded DB:
  • folder-name identification; tag fallback when the folder is blank; skip folders with no audio.
  • quality read (a real tiny FLAC fixture, or a fake quality reader seam) → correct quality_class.
  • a matched album creates LibraryItem (have) + MonitoredRelease (monitored, fulfilled) + WatchedArtist (followed); an unmatched album is skipped and records nothing.
  • idempotent re-scan — running twice doesn't duplicate rows or flip a real download's source to scan.
• Resolver extension — the opt-in live resolver test asserts rg_mbid/artist_mbid are populated; offline construction stays lazy.
• Web — POST/GET /api/scan route tests against seeded lyra_test.
• Backward-compat — the nullable-requestId migration doesn't break existing pipeline/import tests; the full worker + web suites stay green.

Notes

Reading audio quality via mutagen is the one place the scan touches file internals; the mutagen import stays lazy (as in the tagger). After this lands: click Scan library, and the existing /music albums (e.g. Continuum, Sob Rock) show up as have + monitored with their real quality, and their artists appear under /artists. Deferred: auto-scan/watching, multi-disc handling, cover/tag repair.