# Lyra Library Scan Implementation Plan
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
**Goal:** A manual "Scan library" action that walks `/music`, matches each album to MusicBrainz, and records it as **have** (`LibraryItem`) + **on monitor** (`MonitoredRelease`, fulfilled) with the artist **followed** (`WatchedArtist`) — so the library manager reflects what's already on disk.
**Architecture:** The scan runs in the worker (owns `/music`, the MB resolver, mutagen), triggered via a `Config` flag the web sets. A new `scan.py` walks the library, probes each album's tags/quality via an injectable `AudioProbe` seam, resolves it against MusicBrainz (extended to return `rgMbid`/`artistMbid`), and idempotently upserts the rows. `LibraryItem.requestId` becomes nullable so scanned "have" records need no originating request. The web adds a Settings button + `POST/GET /api/scan`.
**Tech Stack:** Python 3.12 worker (psycopg, mutagen lazy), Prisma/Postgres, Next.js 15 web. Tests: pytest with a fake `AudioProbe` + fake resolver + temp `/music` tree + seeded `lyra_test`; vitest for the web routes. Zero real MusicBrainz/downloads offline.
## Global Constraints
- **Tests MUST target `lyra_test`** (a guard in conftest/setup refuses non-`*_test` DBs). Worker: `DATABASE_URL=postgresql://lyra:lyra@localhost:5432/lyra_test .venv/bin/python -m pytest` (or leave `DATABASE_URL` unset — conftest defaults to `lyra_test`). Web: `npm test` (already points at `lyra_test`). Run worker cmds from `/home/jonathan/Projects/lyra/worker` using `.venv/bin/python`; web cmds from `/home/jonathan/Projects/lyra/web`.
- Worker: psycopg raw SQL against Prisma tables (quoted PascalCase, camelCase columns, `gen_random_uuid()::text`, SQL-side `now()`), enum literals as plain strings, `text[]` as Python lists.
- The scan is READ-ONLY on `/music` — no downloads, tagging, moves, or file writes. mutagen is imported lazily inside the real probe (construction stays offline).
- Idempotent: re-scanning updates-or-leaves rows, never duplicates, and never flips a real download's `LibraryItem.source` to `"scan"`.
- Backward compatibility: the nullable-`requestId` migration is additive; every existing worker + web test stays green. `MBTarget` gains defaulted fields (existing constructions unaffected).
- Every task ends with a commit.
## Shared interfaces (defined by tasks below)
```python
# worker/lyra_worker/types.py (Task 1): MBTarget gains rg_mbid: str = "", artist_mbid: str = ""
# worker/lyra_worker/_musicbrainz.py (Task 1): resolver fills rg_mbid + artist_mbid
# worker/lyra_worker/probe.py (Task 3)
@dataclass(frozen=True)
class ProbeResult: artist: str; album: str; fmt: str; quality_class: int
class AudioProbe(Protocol):
def probe(self, path: str) -> ProbeResult: ...
# worker/lyra_worker/scan.py (Task 3)
@dataclass(frozen=True)
class ScanResult: imported: int; skipped: int
def scan_library(conn, resolver, probe: AudioProbe, dest_root: str = "/music") -> ScanResult: ...
# worker/lyra_worker/registry.py (Task 3): build_probe() -> AudioProbe
# worker/lyra_worker/main.py (Task 4): loop runs scan_library when Config "scan.requested" is truthy
# web: POST /api/scan (set scan.requested), GET /api/scan (read scan.result) (Task 5); Settings button (Task 6)
```
---
### Task 1: MBTarget + resolver expose `rgMbid` / `artistMbid`
**Files:**
- Modify: `worker/lyra_worker/types.py`
- Modify: `worker/lyra_worker/_musicbrainz.py`
- Test: `worker/tests/test_types_mbtarget.py`
- Test: `worker/tests/test_musicbrainz_live.py` (extend the opt-in live assertion)
**Interfaces:**
- Produces: `MBTarget.rg_mbid: str = ""`, `MBTarget.artist_mbid: str = ""`; `MusicBrainzResolver.resolve` fills both from the matched release-group and its artist-credit.
- [ ] **Step 1: Write the failing test**
`worker/tests/test_types_mbtarget.py`:
```python
from lyra_worker.types import MBTarget
def test_mbtarget_has_rg_and_artist_mbid_defaults():
t = MBTarget(artist="A", album="B")
assert t.rg_mbid == "" and t.artist_mbid == ""
t2 = MBTarget(artist="A", album="B", rg_mbid="rg1", artist_mbid="a1")
assert t2.rg_mbid == "rg1" and t2.artist_mbid == "a1"
```
- [ ] **Step 2: Run it to verify it fails**
```bash
cd /home/jonathan/Projects/lyra/worker
.venv/bin/python -m pytest tests/test_types_mbtarget.py -v
```
Expected: FAIL — `MBTarget` has no `rg_mbid`.
- [ ] **Step 3: Add the fields**
In `worker/lyra_worker/types.py`, add two fields to `MBTarget` (after `tracklist`):
```python
tracklist: tuple[str, ...] = ()
rg_mbid: str = ""
artist_mbid: str = ""
```
- [ ] **Step 4: Fill them in the resolver**
In `worker/lyra_worker/_musicbrainz.py`, in `resolve`, capture the artist MBID next to the name, and the release-group id, and pass both to `MBTarget(...)`.
Where `canonical_artist` is computed from the credit, also capture the id:
```python
canonical_artist = artist
artist_mbid = ""
credit = rg.get("artist-credit")
if isinstance(credit, list) and credit and isinstance(credit[0], dict):
_artist = credit[0].get("artist") or {}
canonical_artist = _artist.get("name", artist)
artist_mbid = _artist.get("id", "") or ""
```
And in the returned `MBTarget(...)`, add:
```python
return MBTarget(
artist=canonical_artist,
album=canonical_album,
track_count=track_count,
total_duration_s=total_duration_s,
year=year,
tracklist=titles,
rg_mbid=rg.get("id", "") or "",
artist_mbid=artist_mbid,
)
```
- [ ] **Step 5: Extend the opt-in live test**
In `worker/tests/test_musicbrainz_live.py`, in the existing live resolve test (the one gated by `LYRA_LIVE_TESTS`), add after the target is resolved:
```python
assert target.rg_mbid # a real release-group MBID
assert target.artist_mbid # a real artist MBID
```
(If the test's variable is not named `target`, use whatever it binds the resolved `MBTarget` to.)
- [ ] **Step 6: Run the offline tests + full suite**
```bash
cd /home/jonathan/Projects/lyra/worker
.venv/bin/python -m pytest tests/test_types_mbtarget.py -v
.venv/bin/python -m pytest -q
```
Expected: the new test passes; full suite green (live test still skipped); output pristine.
- [ ] **Step 7: Commit**
```bash
cd /home/jonathan/Projects/lyra
git add worker/lyra_worker/types.py worker/lyra_worker/_musicbrainz.py worker/tests/test_types_mbtarget.py worker/tests/test_musicbrainz_live.py
git commit -m "feat: MBTarget exposes rgMbid + artistMbid from the resolver"
```
---
### Task 2: `LibraryItem.requestId` nullable + test cleanup
**Files:**
- Modify: `web/prisma/schema.prisma`
- Create: (generated) migration under `web/prisma/migrations/`
- Modify: `worker/tests/conftest.py` (clean `LibraryItem`)
- Modify: `web/src/test/setup.ts` (clean `libraryItem`)
- Test: `worker/tests/test_library_item_optional_request.py`
**Interfaces:**
- Produces: `LibraryItem.requestId` is optional; a `LibraryItem` can be inserted with `requestId = NULL`. Test fixtures clean `LibraryItem` (scanned rows have no `Request` to cascade from).
- [ ] **Step 1: Write the failing test**
`worker/tests/test_library_item_optional_request.py`:
```python
def test_library_item_can_have_null_request(conn):
with conn.cursor() as cur:
cur.execute(
'INSERT INTO "LibraryItem" (id, "requestId", artist, album, path, source, format, '
'"qualityClass", "importedAt") '
"VALUES (gen_random_uuid()::text, NULL, 'A', 'B', '/m/A/B', 'scan', 'FLAC', 2, now()) "
"RETURNING id"
)
lib_id = cur.fetchone()[0]
conn.commit()
with conn.cursor() as cur:
cur.execute('SELECT "requestId" FROM "LibraryItem" WHERE id = %s', (lib_id,))
assert cur.fetchone()[0] is None
```
- [ ] **Step 2: Run it to verify it fails**
```bash
cd /home/jonathan/Projects/lyra/worker
.venv/bin/python -m pytest tests/test_library_item_optional_request.py -v
```
Expected: FAIL — `null value in column "requestId" violates not-null constraint`.
- [ ] **Step 3: Make the relation optional**
In `web/prisma/schema.prisma`, `model LibraryItem`, change the request field + fk to optional:
```prisma
request Request? @relation(fields: [requestId], references: [id], onDelete: Cascade)
requestId String? @unique
```
- [ ] **Step 4: Generate + apply the migration (into lyra_test)**
```bash
cd /home/jonathan/Projects/lyra/web
DATABASE_URL=postgresql://lyra:lyra@localhost:5432/lyra_test npx prisma migrate dev --name library_item_optional_request
```
Expected: a migration with `ALTER TABLE "LibraryItem" ALTER COLUMN "requestId" DROP NOT NULL;` is created and applied to `lyra_test`. (The live `lyra` DB gets it via `migrate deploy` on the next `docker compose up --build`.)
- [ ] **Step 5: Clean `LibraryItem` in the test fixtures**
Scanned `LibraryItem`s have `requestId = NULL`, so they no longer cascade-delete when `Request` rows are cleared. Add explicit cleanup.
In `worker/tests/conftest.py`, in BOTH cleanup blocks of the `conn` fixture, add a `LibraryItem` delete after `Request`:
```python
cur.execute('DELETE FROM "Job"')
cur.execute('DELETE FROM "Request"')
cur.execute('DELETE FROM "LibraryItem"')
cur.execute('DELETE FROM "MonitoredRelease"')
cur.execute('DELETE FROM "WatchedArtist"')
cur.execute('DELETE FROM "Config"')
```
In `web/src/test/setup.ts`, add `libraryItem` to the `beforeEach` cleanup (after `request`):
```ts
await prisma.job.deleteMany();
await prisma.request.deleteMany();
await prisma.libraryItem.deleteMany();
await prisma.monitoredRelease.deleteMany();
await prisma.watchedArtist.deleteMany();
await prisma.config.deleteMany();
```
- [ ] **Step 6: Run the test + full worker suite + web suite**
```bash
cd /home/jonathan/Projects/lyra/worker
.venv/bin/python -m pytest tests/test_library_item_optional_request.py -v
.venv/bin/python -m pytest -q
cd /home/jonathan/Projects/lyra/web
npm test --silent 2>&1 | tail -4
```
Expected: new test passes; full worker suite green; web suite green (the `libraryItem` cleanup is harmless to existing tests).
- [ ] **Step 7: Commit**
```bash
cd /home/jonathan/Projects/lyra
git add web/prisma/schema.prisma web/prisma/migrations worker/tests/conftest.py web/src/test/setup.ts worker/tests/test_library_item_optional_request.py
git commit -m "feat: LibraryItem.requestId nullable (scanned haves need no request) + test cleanup"
```
---
### Task 3: The scan — `AudioProbe` seam + `scan_library`
**Files:**
- Create: `worker/lyra_worker/probe.py` (seam)
- Create: `worker/lyra_worker/_probe.py` (real `MutagenProbe`)
- Create: `worker/lyra_worker/scan.py`
- Modify: `worker/lyra_worker/registry.py` (`build_probe`)
- Modify: `worker/lyra_worker/adapters/fakes.py` (add `FakeAudioProbe`)
- Test: `worker/tests/test_scan.py`
- Test: `worker/tests/test_probe_live.py` (opt-in)
**Interfaces:**
- Consumes: `MBTarget` (with `rg_mbid`/`artist_mbid`), an `MbResolver`, `AudioProbe`.
- Produces: `ProbeResult`, `AudioProbe` (Protocol), `MutagenProbe`, `FakeAudioProbe`, `ScanResult`, `scan_library(conn, resolver, probe, dest_root)`, `registry.build_probe()`.
- [ ] **Step 1: Write the failing test**
`worker/tests/test_scan.py`:
```python
from lyra_worker.probe import ProbeResult
from lyra_worker.scan import scan_library
from lyra_worker.types import MBTarget
class FakeProbe:
"""Returns a canned ProbeResult regardless of path (tests control identification via folders)."""
def __init__(self, artist="", album="", fmt="FLAC", quality_class=3):
self._r = ProbeResult(artist=artist, album=album, fmt=fmt, quality_class=quality_class)
def probe(self, path):
return self._r
class FakeResolver:
"""resolve(artist, album) -> MBTarget or None, keyed by (artist, album)."""
def __init__(self, table):
self._t = dict(table)
def resolve(self, artist, album):
return self._t.get((artist, album))
def _album(tmp_path, artist, album_folder, files=("01 Track.flac",)):
d = tmp_path / artist / album_folder
d.mkdir(parents=True)
for f in files:
(d / f).write_bytes(b"")
return d
def _counts(conn):
with conn.cursor() as cur:
out = {}
for t in ("LibraryItem", "MonitoredRelease", "WatchedArtist"):
cur.execute(f'SELECT count(*) FROM "{t}"')
out[t] = cur.fetchone()[0]
return out
def test_scan_records_have_monitored_and_follows_artist(conn, tmp_path):
_album(tmp_path, "John Mayer", "Continuum (2006)")
resolver = FakeResolver({
("John Mayer", "Continuum"): MBTarget(
artist="John Mayer", album="Continuum", year=2006, rg_mbid="rg1", artist_mbid="a1"),
})
result = scan_library(conn, resolver, FakeProbe(quality_class=3), dest_root=str(tmp_path))
assert (result.imported, result.skipped) == (1, 0)
assert _counts(conn) == {"LibraryItem": 1, "MonitoredRelease": 1, "WatchedArtist": 1}
with conn.cursor() as cur:
cur.execute('SELECT source, "qualityClass" FROM "LibraryItem" WHERE artist=%s AND album=%s',
("John Mayer", "Continuum"))
assert cur.fetchone() == ("scan", 3)
cur.execute('SELECT monitored, state, "currentQualityClass" FROM "MonitoredRelease" WHERE "rgMbid"=%s',
("rg1",))
assert cur.fetchone() == (True, "fulfilled", 3)
cur.execute('SELECT "autoMonitorFuture" FROM "WatchedArtist" WHERE mbid=%s', ("a1",))
assert cur.fetchone()[0] is False
def test_scan_skips_unmatched_albums(conn, tmp_path):
_album(tmp_path, "Obscure", "Nope (1999)")
result = scan_library(conn, FakeResolver({}), FakeProbe(), dest_root=str(tmp_path))
assert (result.imported, result.skipped) == (0, 1)
assert _counts(conn) == {"LibraryItem": 0, "MonitoredRelease": 0, "WatchedArtist": 0}
def test_scan_falls_back_to_tags_when_folder_does_not_resolve(conn, tmp_path):
_album(tmp_path, "GARBLE", "CD1") # folder names won't resolve
resolver = FakeResolver({
("Radiohead", "In Rainbows"): MBTarget(
artist="Radiohead", album="In Rainbows", year=2007, rg_mbid="rg9", artist_mbid="a9"),
})
probe = FakeProbe(artist="Radiohead", album="In Rainbows", quality_class=3) # embedded tags
result = scan_library(conn, resolver, probe, dest_root=str(tmp_path))
assert (result.imported, result.skipped) == (1, 0)
with conn.cursor() as cur:
cur.execute('SELECT count(*) FROM "MonitoredRelease" WHERE "rgMbid"=%s', ("rg9",))
assert cur.fetchone()[0] == 1
def test_scan_skips_folders_without_audio(conn, tmp_path):
d = tmp_path / "Artist" / "Album"
d.mkdir(parents=True)
(d / "cover.jpg").write_bytes(b"") # no audio
result = scan_library(conn, FakeResolver({}), FakeProbe(), dest_root=str(tmp_path))
assert (result.imported, result.skipped) == (0, 0)
def test_scan_is_idempotent(conn, tmp_path):
_album(tmp_path, "John Mayer", "Continuum (2006)")
resolver = FakeResolver({
("John Mayer", "Continuum"): MBTarget(
artist="John Mayer", album="Continuum", year=2006, rg_mbid="rg1", artist_mbid="a1"),
})
scan_library(conn, resolver, FakeProbe(quality_class=3), dest_root=str(tmp_path))
second = scan_library(conn, resolver, FakeProbe(quality_class=3), dest_root=str(tmp_path))
assert second.imported == 0 # already recorded
assert _counts(conn) == {"LibraryItem": 1, "MonitoredRelease": 1, "WatchedArtist": 1}
```
- [ ] **Step 2: Run it to verify it fails**
```bash
cd /home/jonathan/Projects/lyra/worker
.venv/bin/python -m pytest tests/test_scan.py -v
```
Expected: FAIL — `No module named 'lyra_worker.scan'`.
- [ ] **Step 3: Add the probe seam**
`worker/lyra_worker/probe.py`:
```python
from dataclasses import dataclass
from typing import Protocol
@dataclass(frozen=True)
class ProbeResult:
artist: str
album: str
fmt: str
quality_class: int
class AudioProbe(Protocol):
def probe(self, path: str) -> ProbeResult:
"""Read an audio file's artist/album tags + format/quality."""
...
```
`worker/lyra_worker/_probe.py`:
```python
import os
from lyra_worker.probe import ProbeResult
from lyra_worker.quality import quality_class
from lyra_worker.types import Quality
_LOSSLESS_EXT = {".flac", ".wav", ".alac", ".aiff", ".aif"}
class MutagenProbe:
"""Real AudioProbe via mutagen (imported lazily; not unit-tested offline)."""
def probe(self, path: str) -> ProbeResult:
import mutagen
ext = os.path.splitext(path)[1].lower()
audio = mutagen.File(path, easy=True)
artist = album = ""
bit_depth = sample_rate = None
if audio is not None:
artist = (audio.get("artist") or [""])[0]
album = (audio.get("album") or [""])[0]
info = getattr(audio, "info", None)
bit_depth = getattr(info, "bits_per_sample", None)
sample_rate = getattr(info, "sample_rate", None)
q = quality_class(Quality(
fmt=ext.lstrip(".").upper(),
lossless=ext in _LOSSLESS_EXT,
bit_depth=bit_depth,
sample_rate=sample_rate,
))
return ProbeResult(artist=artist, album=album, fmt=ext.lstrip(".").upper(), quality_class=q)
```
Append to `worker/lyra_worker/adapters/fakes.py`:
```python
from lyra_worker.probe import ProbeResult
class FakeAudioProbe:
"""In-memory AudioProbe for tests."""
def __init__(self, artist="", album="", fmt="FLAC", quality_class=3):
self._r = ProbeResult(artist=artist, album=album, fmt=fmt, quality_class=quality_class)
def probe(self, path: str) -> ProbeResult:
return self._r
```
- [ ] **Step 4: Implement `scan_library`**
`worker/lyra_worker/scan.py`:
```python
import os
import re
from dataclasses import dataclass
import psycopg
from lyra_worker.probe import AudioProbe
_AUDIO_EXT = {".flac", ".mp3", ".m4a", ".opus", ".ogg", ".aac", ".wav"}
_YEAR = re.compile(r"^(.*) \((\d{4})\)$")
@dataclass(frozen=True)
class ScanResult:
imported: int
skipped: int
def _parse_album_dir(name: str) -> str:
m = _YEAR.match(name)
return m.group(1) if m else name
def _first_audio(album_path: str) -> str | None:
for name in sorted(os.listdir(album_path)):
if os.path.splitext(name)[1].lower() in _AUDIO_EXT:
return os.path.join(album_path, name)
return None
def _upsert(conn, target, quality_class: int, path: str) -> bool:
"""Record have + monitored + followed for a resolved album. Returns True if newly imported."""
with conn.cursor() as cur:
watched_id = None
if target.artist_mbid:
cur.execute(
'INSERT INTO "WatchedArtist" (id, mbid, name, "autoMonitorFuture", "monitorFrom", "createdAt") '
"VALUES (gen_random_uuid()::text, %s, %s, false, now(), now()) "
'ON CONFLICT (mbid) DO NOTHING',
(target.artist_mbid, target.artist),
)
cur.execute('SELECT id FROM "WatchedArtist" WHERE mbid = %s', (target.artist_mbid,))
row = cur.fetchone()
watched_id = row[0] if row else None
if target.rg_mbid:
cur.execute(
'INSERT INTO "MonitoredRelease" (id, "watchedArtistId", "artistMbid", "artistName", '
'"rgMbid", album, "secondaryTypes", monitored, state, "currentQualityClass", '
'"firstGrabbedAt", "createdAt") '
"VALUES (gen_random_uuid()::text, %s, %s, %s, %s, %s, '{}', true, 'fulfilled', %s, now(), now()) "
'ON CONFLICT ("rgMbid") DO UPDATE SET monitored = true, state = \'fulfilled\', '
' "currentQualityClass" = EXCLUDED."currentQualityClass", '
' "firstGrabbedAt" = COALESCE("MonitoredRelease"."firstGrabbedAt", now()), '
' "watchedArtistId" = COALESCE("MonitoredRelease"."watchedArtistId", EXCLUDED."watchedArtistId")',
(watched_id, target.artist_mbid, target.artist, target.rg_mbid, target.album, quality_class),
)
cur.execute(
'INSERT INTO "LibraryItem" (id, "requestId", artist, album, path, source, format, '
'"qualityClass", "importedAt") '
"VALUES (gen_random_uuid()::text, NULL, %s, %s, %s, 'scan', %s, %s, now()) "
'ON CONFLICT (artist, album) DO NOTHING',
(target.artist, target.album, path, "FLAC", quality_class),
)
newly = cur.rowcount == 1 # a new LibraryItem means this album wasn't recorded before
conn.commit()
return newly
def scan_library(conn: psycopg.Connection, resolver, probe: AudioProbe, dest_root: str = "/music") -> ScanResult:
"""Walk `{dest_root}/{Artist}/{Album}/`, resolve each album against MusicBrainz, and record
matched albums as have + monitored + followed. Unmatched or audio-less folders are skipped."""
imported = 0
skipped = 0
if not os.path.isdir(dest_root):
return ScanResult(0, 0)
for artist_name in sorted(os.listdir(dest_root)):
artist_path = os.path.join(dest_root, artist_name)
if artist_name.startswith(".") or not os.path.isdir(artist_path):
continue
for album_folder in sorted(os.listdir(artist_path)):
album_path = os.path.join(artist_path, album_folder)
if not os.path.isdir(album_path):
continue
audio = _first_audio(album_path)
if audio is None:
continue # no audio → not an album
album_name = _parse_album_dir(album_folder)
target = resolver.resolve(artist_name, album_name) if artist_name and album_name else None
if target is None: # fall back to the file's embedded tags
pr = probe.probe(audio)
if pr.artist and pr.album and (pr.artist, pr.album) != (artist_name, album_name):
target = resolver.resolve(pr.artist, pr.album)
if target is None:
skipped += 1
continue
q = probe.probe(audio).quality_class
if _upsert(conn, target, q, album_path):
imported += 1
return ScanResult(imported, skipped)
```
Add to `worker/lyra_worker/registry.py`:
```python
from lyra_worker._probe import MutagenProbe
from lyra_worker.probe import AudioProbe
def build_probe() -> AudioProbe:
"""The audio probe used by the library scan."""
return MutagenProbe()
```
- [ ] **Step 5: Run the scan tests to verify they pass**
```bash
cd /home/jonathan/Projects/lyra/worker
.venv/bin/python -m pytest tests/test_scan.py -v
```
Expected: PASS — all five tests green.
- [ ] **Step 6: Add the opt-in real-probe test**
`worker/tests/test_probe_live.py`:
```python
import os
import pytest
pytestmark = pytest.mark.skipif(
not (os.environ.get("LYRA_LIVE_TESTS") and os.environ.get("LYRA_SAMPLE_AUDIO")),
reason="reads a real audio file; set LYRA_LIVE_TESTS=1 + LYRA_SAMPLE_AUDIO=/path/to/a.flac to run",
)
def test_probe_reads_a_real_file():
from lyra_worker._probe import MutagenProbe
r = MutagenProbe().probe(os.environ["LYRA_SAMPLE_AUDIO"])
assert r.quality_class in (1, 2, 3)
assert r.fmt
```
- [ ] **Step 7: Verify lazy import + full suite**
```bash
cd /home/jonathan/Projects/lyra/worker
.venv/bin/python -c "import sys; from lyra_worker._probe import MutagenProbe; from lyra_worker.scan import scan_library; print('import ok; mutagen loaded:', 'mutagen' in sys.modules)"
.venv/bin/python -m pytest -q
```
Expected: prints `import ok; mutagen loaded: False`; full suite green, `test_probe_live.py` skipped, output pristine.
- [ ] **Step 8: Commit**
```bash
cd /home/jonathan/Projects/lyra
git add worker/lyra_worker/probe.py worker/lyra_worker/_probe.py worker/lyra_worker/scan.py worker/lyra_worker/registry.py worker/lyra_worker/adapters/fakes.py worker/tests/test_scan.py worker/tests/test_probe_live.py
git commit -m "feat: library scan — probe seam + scan_library (have + monitored + followed)"
```
---
### Task 4: Wire the scan trigger into the worker loop
**Files:**
- Modify: `worker/lyra_worker/main.py`
- Test: `worker/tests/test_scan_trigger.py`
**Interfaces:**
- Consumes: `scan_library`, `build_probe`, the `Config` table.
- Produces: `run_forever()` reads config once per loop; when `Config` `scan.requested` is truthy it runs `scan_library`, writes a `scan.result` summary, and clears `scan.requested`. A `_run_scan(conn, resolver, probe)` helper is unit-tested.
- [ ] **Step 1: Write the failing test**
`worker/tests/test_scan_trigger.py`:
```python
from lyra_worker.main import _run_scan
from lyra_worker.types import MBTarget
class FakeProbe:
def probe(self, path):
from lyra_worker.probe import ProbeResult
return ProbeResult(artist="", album="", fmt="FLAC", quality_class=3)
class FakeResolver:
def resolve(self, artist, album):
return MBTarget(artist=artist, album=album, rg_mbid="rg1", artist_mbid="a1")
def _set(conn, key, value):
with conn.cursor() as cur:
cur.execute(
'INSERT INTO "Config"(key, value, secret, "updatedAt") VALUES (%s, %s, false, now()) '
'ON CONFLICT (key) DO UPDATE SET value = EXCLUDED.value, "updatedAt" = now()',
(key, value),
)
conn.commit()
def _get(conn, key):
with conn.cursor() as cur:
cur.execute('SELECT value FROM "Config" WHERE key = %s', (key,))
row = cur.fetchone()
return row[0] if row else None
def test_run_scan_writes_result_and_clears_flag(conn, tmp_path):
(tmp_path / "John Mayer" / "Continuum (2006)").mkdir(parents=True)
(tmp_path / "John Mayer" / "Continuum (2006)" / "01.flac").write_bytes(b"")
_set(conn, "scan.requested", "true")
_run_scan(conn, FakeResolver(), FakeProbe(), dest_root=str(tmp_path))
assert _get(conn, "scan.requested") == "false" # flag cleared
assert "imported" in (_get(conn, "scan.result") or "") # summary written
with conn.cursor() as cur:
cur.execute('SELECT count(*) FROM "LibraryItem"')
assert cur.fetchone()[0] == 1
```
- [ ] **Step 2: Run it to verify it fails**
```bash
cd /home/jonathan/Projects/lyra/worker
.venv/bin/python -m pytest tests/test_scan_trigger.py -v
```
Expected: FAIL — `cannot import name '_run_scan'`.
- [ ] **Step 3: Add `_run_scan` + wire the loop**
In `worker/lyra_worker/main.py`:
(a) Update imports:
```python
from lyra_worker.registry import build_adapters, build_browser, build_probe, build_resolver, build_tagger
from lyra_worker.scan import scan_library
```
(b) Add the helper (module level):
```python
_TRUE = {"1", "true", "yes", "on"}
def _set_config(conn, key: str, value: str) -> None:
with conn.cursor() as cur:
cur.execute(
'INSERT INTO "Config"(key, value, secret, "updatedAt") VALUES (%s, %s, false, now()) '
'ON CONFLICT (key) DO UPDATE SET value = EXCLUDED.value, "updatedAt" = now()',
(key, value),
)
conn.commit()
def _run_scan(conn, resolver, probe, dest_root: str = DEST_ROOT) -> None:
result = scan_library(conn, resolver, probe, dest_root)
_set_config(conn, "scan.result", f"imported {result.imported}, skipped {result.skipped}")
_set_config(conn, "scan.requested", "false")
print(f"worker: library scan done — {result.imported} imported, {result.skipped} skipped", flush=True)
```
(c) Build the probe and check the flag each loop. Replace `mcfg = MonitorConfig.from_config(get_config(conn))` with a single config read, and add the scan check after the monitor tick:
```python
probe = build_probe()
...
while True:
config = get_config(conn)
mcfg = MonitorConfig.from_config(config)
now = time.monotonic()
if mcfg.enabled and now - last_tick >= MONITOR_TICK_SECONDS:
... # unchanged monitor sweep block
last_tick = now
if str(config.get("scan.requested", "")).strip().lower() in _TRUE:
try:
_run_scan(conn, resolver, probe)
except Exception as e: # a scan error must never kill the worker
print(f"worker: library scan failed: {e}", flush=True)
conn.rollback()
_set_config(conn, "scan.requested", "false")
job_id = claim_next(conn)
... # unchanged
```
(`build_probe` is added to the imports from `registry`.)
- [ ] **Step 4: Run the trigger test to verify it passes**
```bash
cd /home/jonathan/Projects/lyra/worker
.venv/bin/python -m pytest tests/test_scan_trigger.py -v
```
Expected: PASS.
- [ ] **Step 5: Verify main imports + full suite**
```bash
cd /home/jonathan/Projects/lyra/worker
.venv/bin/python -c "from lyra_worker import main; print('ok')"
.venv/bin/python -m pytest -q
```
Expected: `ok`; full suite green, output pristine.
- [ ] **Step 6: Commit**
```bash
cd /home/jonathan/Projects/lyra
git add worker/lyra_worker/main.py worker/tests/test_scan_trigger.py
git commit -m "feat: worker runs the library scan when Config scan.requested is set"
```
---
### Task 5: Web `/api/scan` routes
**Files:**
- Create: `web/src/app/api/scan/route.ts`
- Test: `web/src/app/api/scan/route.test.ts`
**Interfaces:**
- Consumes: `prisma`, the `Config` table.
- Produces: `POST /api/scan` upserts `Config` `scan.requested="true"` → 202 `{requested:true}`; `GET /api/scan` returns `{ result, requested }` from `Config` (`result` = `scan.result` value or `null`; `requested` = whether `scan.requested` is `"true"`).
- [ ] **Step 1: Write the failing test**
`web/src/app/api/scan/route.test.ts`:
```ts
import { describe, it, expect } from "vitest";
import { prisma } from "@/lib/db";
import { POST, GET } from "./route";
describe("scan API", () => {
it("POST sets scan.requested and GET reflects it", async () => {
const res = await POST(new Request("http://localhost/api/scan", { method: "POST" }));
expect(res.status).toBe(202);
expect((await res.json()).requested).toBe(true);
const row = await prisma.config.findUnique({ where: { key: "scan.requested" } });
expect(row?.value).toBe("true");
const g = await GET();
const body = await g.json();
expect(body.requested).toBe(true);
expect(body.result).toBeNull(); // no scan.result yet
});
it("GET returns the worker's scan.result when present", async () => {
await prisma.config.create({ data: { key: "scan.result", value: "imported 3, skipped 1" } });
const body = await (await GET()).json();
expect(body.result).toBe("imported 3, skipped 1");
});
});
```
- [ ] **Step 2: Run it to verify it fails**
```bash
cd /home/jonathan/Projects/lyra/web
DATABASE_URL=postgresql://lyra:lyra@localhost:5432/lyra_test npx vitest run api/scan 2>&1 | tail -5
```
Expected: FAIL — cannot resolve `./route`.
- [ ] **Step 3: Implement the route**
`web/src/app/api/scan/route.ts`:
```ts
import { prisma } from "@/lib/db";
export async function POST() {
await prisma.config.upsert({
where: { key: "scan.requested" },
create: { key: "scan.requested", value: "true", secret: false },
update: { value: "true" },
});
return Response.json({ requested: true }, { status: 202 });
}
export async function GET() {
const [requested, result] = await Promise.all([
prisma.config.findUnique({ where: { key: "scan.requested" } }),
prisma.config.findUnique({ where: { key: "scan.result" } }),
]);
return Response.json({
requested: requested?.value === "true",
result: result?.value ?? null,
});
}
```
- [ ] **Step 4: Run the test to verify it passes**
```bash
cd /home/jonathan/Projects/lyra/web
DATABASE_URL=postgresql://lyra:lyra@localhost:5432/lyra_test npx vitest run api/scan 2>&1 | tail -5
```
Expected: PASS — both tests green.
- [ ] **Step 5: Run the full web suite**
```bash
cd /home/jonathan/Projects/lyra/web
npm test --silent 2>&1 | tail -4
```
Expected: PASS — every web test green.
- [ ] **Step 6: Commit**
```bash
cd /home/jonathan/Projects/lyra
git add web/src/app/api/scan
git commit -m "feat: /api/scan — request a library scan + read its result"
```
---
### Task 6: Settings "Scan library" button
**Files:**
- Modify: `web/src/app/settings/settings-form.tsx`
**Interfaces:**
- Consumes: `POST /api/scan`, `GET /api/scan`. UI — verified by `npx tsc --noEmit`.
- [ ] **Step 1: Add the scan control to the settings form**
In `web/src/app/settings/settings-form.tsx`, add scan state + handlers inside `SettingsForm` (alongside the existing state), and render a "Scan library" section below the existing `` tag (still inside the component's returned fragment — wrap the form and this new block in a `<>`…`` if needed), add:
```tsx
Library
Scan /music and record what's already on disk as owned + monitored.
{scanResult ?
Last scan: {scanResult}
: null}
```
If the component currently `return`s a single ``, change it to `return (<>{}...);` — i.e. wrap the existing form and the new `` in a fragment. Do not change any existing form fields.
- [ ] **Step 2: Typecheck**
```bash
cd /home/jonathan/Projects/lyra/web
npx tsc --noEmit
```
Expected: no type errors.
- [ ] **Step 3: Run the web suite (no regressions)**
```bash
cd /home/jonathan/Projects/lyra/web
npm test --silent 2>&1 | tail -4
```
Expected: PASS — existing tests unaffected (no component tests for the settings form; route tests still green).
- [ ] **Step 4: Commit**
```bash
cd /home/jonathan/Projects/lyra
git add web/src/app/settings/settings-form.tsx
git commit -m "feat: Settings 'Scan library' button + last-result status"
```
---
## Notes: library scan complete
With this plan, clicking **Scan library** in Settings walks `/music`, resolves each album against
MusicBrainz, and records matched albums as **have** (`LibraryItem`), **on monitor**
(`MonitoredRelease`, fulfilled, with real quality), and **followed** (`WatchedArtist`) — skipping
anything MusicBrainz can't match. Re-scans are idempotent. After it lands: re-`up --build` (applies
the nullable-`requestId` migration to the live DB), click Scan, and confirm *Continuum*/*Sob Rock*
appear as have + monitored under their artist. Deferred: auto-scan/filesystem-watching, multi-disc
handling, and repairing/re-tagging existing files.
```