docs: slice 3 discovery design — MetaBrainz-native (ListenBrainz), suggestion feed

Spotify similarity/new-release APIs are dead for new apps (Nov 2024 + Feb 2026),
so discovery is built on ListenBrainz similar-artists (MBID-native) behind a
pluggable SimilaritySource interface. Manual review feed of similar artists +
albums; Follow/Want reuse slice-2 operations, Dismiss is permanent. Off by default.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

docs/superpowers/specs/2026-07-11-lyra-discovery-design.md

@@ -0,0 +1,319 @@
+# Lyra — Discovery (Design)
+
+**Date:** 2026-07-11
+**Status:** Approved design, ready for implementation planning
+**Slice:** 3 of 3 (Discovery)
+
+## Overview
+
+Slice 3 makes Lyra surface music you **don't already follow**. Slices 1–2
+obtain and monitor artists/releases you already know about; discovery closes the
+loop by recommending *new* artists and albums similar to what's in your library,
+then hands each accepted suggestion straight back into the slice-2 machinery.
+
+Discovery is deliberately **suggestion-only and manual**: a background sweep (and
+an on-demand button) populates a review feed of candidate artists and albums; you
+**Follow** an artist (creating a `WatchedArtist` with its discography tracked but
+**unmonitored**, exactly like the slice-2 follow) or **Want** an album (creating a
+`monitored` wanted `MonitoredRelease`, exactly like the slice-2 wanted list), or
+**Dismiss** it. Discovery never grabs anything itself — its only outputs are the
+exact rows the existing follow / wanted flows already create, and actual grabbing
+still happens only through the slice-2 monitor (off by default). The entire
+acquisition + monitoring pipeline is reused unchanged.
+
+### The source reality that shapes this slice
+
+The original acquisition-engine spec reserved **Spotify** for "discovery/metadata
+only." That premise no longer holds for a newly-registered app:
+
+- **Nov 2024** — Spotify deprecated **Related Artists**, **Recommendations**,
+  audio-features, audio-analysis and featured-playlists for any *new* app (new
+  apps receive 403/404; no waitlist, no path back).
+- **Feb 2026** — Spotify additionally removed **Get New Releases**, Get Artist's
+  Top Tracks and Get Several Albums.
+
+So Spotify can supply **none** of discovery's signals to a fresh app. Lyra is
+already **MusicBrainz-native** (every artist/release is an MBID), so discovery is
+instead built on **MetaBrainz**: **ListenBrainz similar-artists** (same family as
+MusicBrainz, returns MBIDs directly — zero fuzzy matching) as the engine, with
+MusicBrainz for the album sweep. Spotify is dropped entirely from this slice.
+
+### Goals
+
+- Recommend new **artists** (and standout **albums**) similar to the library, at
+  MBID granularity, with no personal-account / listening-history dependency.
+- Reuse slice-2 verbatim: **Follow** creates the same `WatchedArtist` +
+  unmonitored discography the follow flow does; **Want** creates the same
+  `monitored` wanted `MonitoredRelease` the wanted list does. No new grab logic.
+- Manual review only — nothing is followed or grabbed without explicit approval;
+  **dismissed suggestions never resurface**.
+- Keep the design source-agnostic behind a pluggable `SimilaritySource` interface
+  (mirroring the slice-1 `adapters/` registry), so a second source (Last.fm) or
+  tag-weighting can drop in later without touching the core.
+- Fully developable and testable with **zero credentials** (ListenBrainz labs API
+  is unauthenticated) via a fake `SimilaritySource` and seeded Postgres.
+
+### Non-goals (this slice)
+
+- **No Spotify** anywhere (its similarity/new-release APIs are dead for new apps).
+- **No auto-follow / auto-grab** — manual review only.
+- **No personal listening data / OAuth** (no ListenBrainz user account required).
+- **No second similarity source (Last.fm) and no genre-tag re-weighting** in this
+  slice — both are left as drop-in extensions behind `SimilaritySource`.
+- No notifications; no multi-user. Downloading, tagging, and the six-stage
+  pipeline are untouched.
+
+## Scope
+
+**In scope:** a `SimilaritySource` engine (ListenBrainz), a worker discovery run
+that aggregates similar artists across your library's artists and derives album
+suggestions, a `DiscoverySuggestion` review feed, per-suggestion Follow / Want /
+Dismiss actions that reuse slice-2 operations, an interactive seed-search box, a
+scheduled sweep + "Discover now" trigger (off by default), and Discovery settings.
+
+**Out of scope:** everything under Non-goals. Deferred hardening from slices 1–2
+is unaffected and remains deferred.
+
+## Key decisions
+
+- **Web reads, worker persists** — the same split slice 2 established. The
+  interactive **seed-search box** calls ListenBrainz directly from the web
+  (unauthenticated, read-only, results not persisted), exactly as `/api/mb/artists`
+  does live MusicBrainz search today. The **scheduled sweep** and **"Discover
+  now"** run in the worker and persist `DiscoverySuggestion` rows.
+- **Suggestion-only, manual.** Discovery emits suggestions; the *user* turns a
+  suggestion into a `WatchedArtist` (Follow) or a wanted `MonitoredRelease`
+  (Want). Discovery has no path that grabs audio.
+- **Dismissed is permanent.** A `dedupeKey` unique column plus keeping
+  dismissed/followed/wanted rows means the sweep's conditional upsert can never
+  revert them to `pending` — dismissed suggestions never come back.
+- **Off by default**, like the monitor: `discover.enabled` unset ⇒ no background
+  sweep. The on-demand button and seed-search still work regardless.
+- **Pluggable engine.** One source (ListenBrainz) ships now behind a
+  `SimilaritySource` Protocol + `build_similarity_sources()` registry, matching
+  the existing `SourceAdapter` / `build_adapters` pattern.
+
+## Architecture
+
+```
+                       WEB (read-only, no secrets)          WORKER (persists)
+  /discover page  ─┬─ GET /api/discover        (feed) ───▶ DiscoverySuggestion (read)
+                   ├─ GET /api/discover/search  ──────────▶ MB search + ListenBrainz
+                   │        (interactive seed, not stored)   (direct, unauthenticated)
+                   ├─ POST /api/discover/run  ────────────▶ Config discover.requested=true
+                   └─ POST /api/discover/[id] (action) ───▶ Follow → WatchedArtist
+                                                            Want   → MonitoredRelease
+                                                            Dismiss→ status=dismissed
+
+  worker loop tick (discover.enabled, every discover.intervalHours; or on
+  discover.requested):
+    run_discovery(conn, sources, browser, cfg):
+      seeds = WatchedArtist.mbid  (throttled by lastDiscoveredAt, capped maxSeeds)
+      for seed × source:  similar_artists(mbid)  → aggregate
+      filter out already-followed / dismissed
+      upsert ARTIST suggestions (conditional on status='pending')
+      for top artists: MbBrowser release-groups → core-release filter →
+                       drop library dupes → upsert ALBUM suggestions
+```
+
+## Data model
+
+One new table + two enums, plus one column on `WatchedArtist`.
+
+```prisma
+enum DiscoveryKind    { artist album }
+enum SuggestionStatus { pending followed wanted dismissed }
+
+model DiscoverySuggestion {
+  id               String   @id @default(cuid())
+  kind             DiscoveryKind
+  artistMbid       String
+  artistName       String
+  rgMbid           String?          // album kind only; null for artist
+  album            String?
+  primaryType      String?
+  secondaryTypes   String[]
+  firstReleaseDate String?
+  score            Float            // aggregated similarity across seeds
+  seedCount        Int              // how many of your artists pointed here
+  sources          String[]         // e.g. ["listenbrainz"] (pluggable)
+  status           SuggestionStatus @default(pending)
+  dedupeKey        String   @unique // kind + ':' + artistMbid + ':' + (rgMbid ?? "")
+  createdAt        DateTime @default(now())
+  updatedAt        DateTime @updatedAt
+}
+```
+
+Plus on `WatchedArtist`:
+
+```prisma
+  lastDiscoveredAt DateTime?   // per-seed throttle, mirrors lastPolledAt
+```
+
+**Why `dedupeKey`:** Postgres treats `NULL`s as distinct in a composite unique
+index, so a `@@unique([kind, artistMbid, rgMbid])` would let duplicate
+`kind=artist` rows through (their `rgMbid` is null). A single non-null
+`dedupeKey` string gives a clean `ON CONFLICT` target for the upsert.
+
+**Conditional upsert (the dismissed-is-permanent guarantee):**
+
+```sql
+INSERT INTO "DiscoverySuggestion" (... , status, "dedupeKey", ...)
+VALUES (... , 'pending', %s, ...)
+ON CONFLICT ("dedupeKey") DO UPDATE
+  SET score = EXCLUDED.score,
+      "seedCount" = EXCLUDED."seedCount",
+      sources = EXCLUDED.sources,
+      "updatedAt" = now()
+WHERE "DiscoverySuggestion".status = 'pending';
+```
+
+A row already `followed` / `wanted` / `dismissed` fails the `WHERE` and is left
+untouched — it never returns to the feed.
+
+## Engine — `worker/lyra_worker/similarity/`
+
+Mirrors `worker/lyra_worker/adapters/`.
+
+- **`base.py`** — Protocol:
+  ```python
+  @dataclass(frozen=True)
+  class SimilarArtist:
+      mbid: str
+      name: str
+      score: float
+
+  @runtime_checkable
+  class SimilaritySource(Protocol):
+      name: str
+      def health(self) -> bool: ...
+      def similar_artists(self, mbid: str) -> list[SimilarArtist]: ...
+  ```
+- **`_listenbrainz.py`** — `ListenBrainzSource`: GETs the ListenBrainz Labs
+  similar-artists dataset (`labs.api.listenbrainz.org`, unauthenticated), maps the
+  response to `SimilarArtist` (MBIDs come back native). `health()` is a
+  reachability check. A base-URL override reads from config
+  (`discover.listenBrainzUrl`, default the public host).
+- **`registry.build_similarity_sources(config)`** — health-filtered list, exactly
+  like `build_adapters`. `build_similarity_sources` is added to `registry.py`.
+
+## Discovery run — `worker/lyra_worker/discovery.py`
+
+`DiscoveryConfig.from_config(config)` parses the `discover.*` keys (same shape as
+`MonitorConfig`). `run_discovery(conn, sources, browser, cfg) -> DiscoveryResult`:
+
+1. **Seeds** = `WatchedArtist.mbid` where `lastDiscoveredAt IS NULL OR <
+   now() - intervalHours`, capped at `maxSeeds`.
+2. For each seed × source → `similar_artists(mbid)`, keep top `similarPerSeed`.
+3. **Aggregate** per candidate MBID: `score = Σ similarity`, `seedCount =`
+   distinct seeds pointing here, `sources =` union of contributing source names.
+4. **Filter** candidates already in `WatchedArtist` (already followed) and any
+   existing dismissed row; drop below `minScore`.
+5. **Upsert artist suggestions** (conditional-on-pending, above).
+6. **Album derivation:** for the top surfaced artists, use the existing
+   **`MbBrowser`** to list their release-groups, keep only **core releases** via
+   the existing `release-filter` predicate (reused from slice 2), drop any already
+   in `LibraryItem` or `MonitoredRelease`, and upsert up to `albumsPerArtist`
+   (default 1, most-recent core) **album** suggestions.
+7. Stamp `lastDiscoveredAt = now()` on each processed seed. Return counts.
+
+Errors from a single source or seed are logged and skipped — a discovery failure
+must never kill the worker loop (same discipline as monitor/scan).
+
+## Worker loop integration — `worker/lyra_worker/main.py`
+
+- Build similarity sources at startup: `sources = build_similarity_sources(config)`.
+- Add a **discover tick** beside the monitor tick: when `discover.enabled` and the
+  interval has elapsed, call `run_discovery(...)` inside a try/except that rolls
+  back and logs on failure. Off by default.
+- Handle the one-shot **`discover.requested`** flag exactly like `scan.requested`:
+  run `run_discovery`, write `discover.result` (`"artists N, albums M"`), clear
+  `discover.requested`.
+
+## Config keys (`discover.*`, mirroring `monitor.*`)
+
+| Key | Default | Meaning |
+|---|---|---|
+| `discover.enabled` | `false` | master switch for the scheduled sweep |
+| `discover.intervalHours` | `168` | sweep cadence (weekly) + per-seed throttle window |
+| `discover.maxSeeds` | `50` | seed artists processed per sweep |
+| `discover.similarPerSeed` | `20` | top-K similar artists pulled per seed |
+| `discover.albumsPerArtist` | `1` | album suggestions per surfaced artist |
+| `discover.minScore` | `0` | drop candidates below this aggregated score |
+| `discover.listenBrainzUrl` | public host | ListenBrainz Labs base URL override |
+| `discover.requested` | — | one-shot "Discover now" trigger (worker clears) |
+| `discover.result` | — | last-run status string surfaced in the UI |
+
+No credential key — the ListenBrainz Labs API is unauthenticated.
+
+## Web
+
+New page **`/discover`** (`page.tsx` server component + `discover-client.tsx`),
+linked in the nav in `layout.tsx`:
+
+- **"Discover now"** → `POST /api/discover/run` (sets `discover.requested`);
+  shows `discover.result`.
+- **Seed-search box** → `GET /api/discover/search?artist=<name>`: resolve the name
+  to an MBID (MB search), fetch ListenBrainz similar, return **unpersisted**
+  results inline; each is actionable via the same Follow/Want endpoints.
+- **Feed** → `GET /api/discover`: pending suggestions, artists and albums in two
+  sections, sorted by `score` desc, each showing *why* (`seedCount`, `sources`,
+  `score`) and its actions.
+
+Actions — **reuse existing slice-2 DB operations, no new grab logic:**
+
+- **Follow** (artist) → create `WatchedArtist` + browse its discography into
+  `monitored=false` `MonitoredRelease` rows (the same operation as the
+  `/api/artists` follow POST), then set the suggestion `status='followed'`.
+- **Want** (album) → create a `monitored=true` `MonitoredRelease` **directly from
+  the suggestion's already-stored fields** (`rgMbid`, `album`, types,
+  `firstReleaseDate`) — no redundant MusicBrainz search, since the sweep already
+  resolved them — equivalent to what the wanted list stores; set
+  `status='wanted'`.
+- **Dismiss** → set `status='dismissed'`.
+
+Endpoints (each with a co-located `.test.ts`, matching convention):
+`api/discover/route.ts` (feed), `api/discover/run/route.ts`,
+`api/discover/search/route.ts`, `api/discover/[id]/route.ts` (action).
+
+**Settings** — a "Discovery" section in `settings-form.tsx` mirroring the monitor
+fields: `enabled` + the tuning knobs above.
+
+## Error handling
+
+| Failure | Behavior |
+|---|---|
+| ListenBrainz unreachable during sweep | source `health()` false ⇒ that source contributes nothing this tick; logged; retried next sweep. Web seed-search returns an error surfaced in the UI. |
+| A single seed errors | logged and skipped; the rest of the sweep proceeds. |
+| Duplicate suggestion | `dedupeKey` unique + `ON CONFLICT`; re-sweeping never duplicates rows. |
+| Candidate already followed / in library | filtered out before insert (checked against `WatchedArtist` / `LibraryItem` / `MonitoredRelease`). |
+| Dismissed candidate reappears from source | conditional upsert leaves the dismissed row untouched; it stays out of the feed. |
+| Follow/Want race with existing row | reuse slice-2 operations, which already upsert idempotently. |
+| Discovery run throws | caught in the loop, `conn.rollback()`, logged; worker keeps running. |
+
+## Testing (against `lyra_test`, per the DB guard)
+
+- **Worker — `discovery.py`:** fake `SimilaritySource` + seeded Postgres. Assert
+  aggregation (score sum, seedCount, sources union), filtering vs `WatchedArtist`
+  / `LibraryItem` / `MonitoredRelease`, `minScore` cutoff, the **conditional
+  upsert never reviving a dismissed row**, album derivation through the
+  core-release predicate, and `lastDiscoveredAt` throttling.
+- **Worker — `_listenbrainz.py`:** thin adapter tested with faked HTTP against
+  recorded response shapes; `health()` true/false paths.
+- **Web:** route tests for the feed, each action (Follow/Want/Dismiss with mocked
+  DB), and seed-search (mocked MB + ListenBrainz). Reuse the `release-filter`
+  tests for the core-release predicate.
+
+## Notes / deferred
+
+- **Second similarity source (Last.fm) and tag-coherent re-weighting** are the
+  planned extensions behind `SimilaritySource`; adding either is a new source in
+  the registry + a `sources`-array boost, no core change.
+- **Album popularity ranking.** v1 picks the most-recent core release per artist.
+  A later refinement can rank an artist's release-groups by ListenBrainz
+  popular-recordings listen counts.
+- **Seed scope** is currently all `WatchedArtist` MBIDs. A "top artists only"
+  scoping (by library album count) is a possible knob if API load becomes an issue.
+- **Discovery is synchronous in the worker loop**, like the scan — fine for the
+  weekly cadence and bounded by `maxSeeds`, but it briefly blocks job-claim during
+  a sweep. Chunking mirrors the scan's deferred item.