From decb6302d47164310a25ae000bd7deed563abeae Mon Sep 17 00:00:00 2001 From: Jonathan Date: Sun, 12 Jul 2026 01:06:56 +0200 Subject: [PATCH] docs: mark slice 3 (discovery) done + record deferred todos MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit README: discovery status → done (ListenBrainz-native, no Spotify); off-by-default note for the discovery sweep; architecture mentions MusicBrainz+ListenBrainz. Plan: check off final verification (suites green, whole-branch review passed, live-API algorithm fix), append the review's deferred items to Notes. Co-Authored-By: Claude Opus 4.8 (1M context) --- README.md | 34 +++++++++++++------ .../plans/2026-07-11-lyra-discovery.md | 21 +++++++++--- 2 files changed, 41 insertions(+), 14 deletions(-) diff --git a/README.md b/README.md index dc0377f..92cd38a 100644 --- a/README.md +++ b/README.md @@ -2,8 +2,10 @@ Self-hosted, multi-source music acquisition and library tool — a replacement for Lidarr + NZBget. Lyra discovers, ranks, downloads, tags, and organizes music across -sources of differing quality: **Qobuz** (lossless/hi-res), **Soulseek** (P2P), -**YouTube** (universal lossy fallback), and **Spotify** (discovery/metadata only). +sources of differing quality: **Qobuz** (lossless/hi-res), **Soulseek** (P2P), and +**YouTube** (universal lossy fallback), with recommendation-driven discovery via +**ListenBrainz** (Spotify's recommendation/related-artists APIs were retired for new +apps, so discovery is MetaBrainz-native and MBID-clean — no Spotify). ## Status @@ -19,14 +21,25 @@ Built in three slices: auto-grabs through the acquisition pipeline. Plus a **library scan** that ingests existing on-disk albums as have + monitored + followed, and a per-artist studio-only / show-all release-type filter. -3. **Discovery** — ⬜ **not started**. Spotify/recommendation-driven finding of new music. +3. **Discovery** — ✅ **done**. Recommendation-driven finding of new music via + **ListenBrainz** similar-artists (MBID-native), behind a pluggable + `SimilaritySource` interface. A background sweep (and a "Discover now" button) + aggregates artists similar to the ones you follow into a manual review feed of + suggested **artists** and **albums** at `/discover`; **Follow** and **Want** + reuse the slice-2 follow / wanted flows, **Dismiss** is permanent. Plus a + web-side seed-search box. No Spotify (its recommendation APIs are dead for new + apps). -**Operational notes:** the monitor is **off by default** — set `monitor.enabled=true` -(+ optional `monitor.*` tuning) in the `Config` table to activate auto-grab/upgrade. -Credentials are entered in Settings and persist across `docker compose up -d --build` -rebuilds (don't use `down -v`). Tests run against a separate `lyra_test` database (a -guard blocks the live DB). Deferred hardening + the full roadmap live in the -implementation-plan "Notes" sections under [`docs/superpowers/plans/`](docs/superpowers/plans/). +**Operational notes:** the monitor and the discovery sweep are both **off by +default** — set `monitor.enabled=true` (+ optional `monitor.*` tuning) in the +`Config` table to activate auto-grab/upgrade, and `discover.enabled=true` (or +Settings → Discovery, + optional `discover.*` tuning) to activate the scheduled +discovery sweep. The `/discover` "Discover now" button and seed-search work with the +sweep off. Credentials are entered in Settings and persist across +`docker compose up -d --build` rebuilds (don't use `down -v`). Tests run against a +separate `lyra_test` database (a guard blocks the live DB). Deferred hardening + the +full roadmap live in the implementation-plan "Notes" sections under +[`docs/superpowers/plans/`](docs/superpowers/plans/). ## Architecture @@ -34,7 +47,8 @@ Three Docker containers on a home server: - **Next.js app** — UI + API (search, request, queue, progress, settings). - **Python worker** — all source integration (streamrip, yt-dlp, slskd client, - beets/mutagen), running the six-stage acquisition pipeline. + mutagen; MusicBrainz + ListenBrainz for metadata and discovery), running the + six-stage acquisition pipeline plus the background monitor and discovery sweeps. - **slskd** — off-the-shelf headless Soulseek daemon. **Postgres** is the shared source of truth; a shared `/music` volume is the library. diff --git a/docs/superpowers/plans/2026-07-11-lyra-discovery.md b/docs/superpowers/plans/2026-07-11-lyra-discovery.md index d6b26f3..62055e6 100644 --- a/docs/superpowers/plans/2026-07-11-lyra-discovery.md +++ b/docs/superpowers/plans/2026-07-11-lyra-discovery.md @@ -2137,10 +2137,18 @@ Co-Authored-By: Claude Opus 4.8 (1M context) " ## Final verification -- [ ] **Full worker suite:** `cd worker && python -m pytest -q` → all pass. -- [ ] **Full web suite:** `cd web && npm test` → all pass. -- [ ] **Web build:** `cd web && npm run build` → succeeds, `/discover` route present. -- [ ] **Manual smoke (real app):** `docker compose up -d --build` (never `down -v`). Follow an artist or run a scan so `WatchedArtist` seeds exist, open `/discover`, click **Discover now**, confirm suggestions appear; verify **Follow** creates a watched artist, **Want** adds a wanted release, **Dismiss** removes a suggestion and it does not reappear on the next run. **Confirm the ListenBrainz algorithm string and response field names in `_listenbrainz.py` / `listenbrainz.ts` against the live API** — adjust if the shapes differ (see the NOTE in `_listenbrainz.py`). +- [x] **Full worker suite:** `cd worker && python -m pytest -q` → **153 passed, 7 skipped**. +- [x] **Full web suite:** `cd web && npm test` → **62 passed** (19 files). +- [x] **Web build:** `cd web && npm run build` → succeeds, `/discover` route present; `tsc --noEmit` clean. +- [x] **Whole-branch review** (opus) → Ready to merge: YES, no Critical/Important; all cross-cutting contracts verified. +- [x] **ListenBrainz algorithm verified against the live API (2026-07-12).** The planned string + (`…contribution_5_threshold_15_limit_50…`) was **not a permitted endpoint enum** — every lookup + 400'd, making discovery inert. Corrected to + `session_based_days_7500_session_300_contribution_3_threshold_10_limit_100_filter_True_skip_30` + in both `_listenbrainz.py` and `listenbrainz.ts` (commit `d2c581d`); confirmed end-to-end (real + `ListenBrainzSource` returns 100 similar artists for Radiohead, parser shape correct). The response + fields the parser reads (`artist_mbid`, `name`, `comment`, `score`) match the live shape. +- [ ] **Manual smoke (real app), post-deploy:** `docker compose up -d --build web worker` (never `down -v`). Follow an artist or run a scan so `WatchedArtist` seeds exist, open `/discover`, click **Discover now**, confirm suggestions appear; verify **Follow** creates a watched artist, **Want** adds a wanted release, **Dismiss** removes a suggestion and it does not reappear on the next run. (Reset `WatchedArtist.lastDiscoveredAt = NULL` to re-run against the same seeds — the 168h throttle otherwise reports `artists 0`.) ## Notes / deferred @@ -2148,3 +2156,8 @@ Co-Authored-By: Claude Opus 4.8 (1M context) " - **Album popularity ranking** — v1 picks the most-recent core release; a later refinement can rank by ListenBrainz popular-recordings listen counts. - **Discovery is synchronous in the worker loop** (like the scan): bounded by `maxSeeds`, but a large sweep briefly blocks job-claim. Chunking mirrors the scan's deferred item. - **Seed scope** is all `WatchedArtist` MBIDs; a "top artists only" knob is possible if API load becomes an issue. +- **Non-cumulative score on partial re-sweep** (from whole-branch review): `_upsert_artist` sets `score = EXCLUDED.score` (replace, not accumulate). Because seeds are throttled by `lastDiscoveredAt`, a sweep where only some seeds are due recomputes a candidate's aggregate from just those seeds, so a pending suggestion's `score`/`seedCount` can shrink until all seeds refresh in the same window. Benign for a feed; revisit if scores look jumpy. +- **Trigger + tick coincidence** (from review): a pending `discover.requested` firing in the same loop iteration as the scheduled tick can run discovery twice. Idempotent upserts make this wasteful, not incorrect. +- **`discover.listenBrainzUrl`** config key (base-URL override read in `registry.py`) has no Settings UI — a hidden/undocumented override, not dead code. +- **Follow is not transactional** — `POST /api/discover/[id]` follow does `WatchedArtist.create` + `MonitoredRelease.createMany` as separate awaits (identical to the pre-existing `/api/artists` POST); single-user home server, low risk. +- **`_upsert_artist` per-`(seed, source)` similarPerSeed slice** — the top-K cut is applied per (seed, source) pair, not per seed overall; only matters once a second live `SimilaritySource` is added.