Commit Graph

108 Commits

Author SHA1 Message Date
Jonathan 55cf88f66e fix(worker): canonicalize bonus-track filenames past the MB tracklist
A track beyond the MusicBrainz tracklist (a Qobuz bonus track) kept streamrip's
raw name (e.g. "13. John Mayer - St. Patrick's Day (Album Version).flac") while
the main tracklist got clean "## Title" names. plan_track_names now canonicalizes
such extras: strip the leading track-number prefix and a leading "<artist> - ",
yielding "13 St. Patrick's Day (Album Version).flac". Requires an explicit
delimiter after the number so titles like "99 Luftballons" are left alone; with
no tracklist resolved at all the on-disk stem is still kept.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-15 02:37:08 +02:00
Jonathan 0c5e73eb17 fix(worker): parallelize per-source searches in the match phase
Soulseek's slskd search polls up to ~2 min; searching adapters sequentially
added that on top of Qobuz's instant search. Run the per-adapter searches
concurrently via a ThreadPoolExecutor so total match time is the slowest
single source, not their sum. Only Qobuz drives streamrip's shared asyncio
loop and there's one Qobuz adapter, so no two threads use that loop at once.
Results collected in adapter order (map preserves order) for deterministic
candidate ordering; per-adapter failures still swallowed (down source = no
candidates).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-15 02:37:08 +02:00
Jonathan 0325467aba fix(worker): smooth Qobuz download progress via streamrip byte hook
streamrip reports progress only per concurrent track, so Job.downloadProgress
came solely from the file-count poller — 0% until every track landed, then a
jump to 100%. Patch streamrip.media.track.get_progress_callback to aggregate
per-track byte deltas over an estimated album total (avg started-track size ×
len(album.tracks)) into one smooth, generally-climbing 0..1 fraction routed
through the pipeline's on_progress. Capped at 0.99; the pipeline sets 1.0 on
completion. No signature changes — track count read from the resolved album.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-15 02:37:08 +02:00
Jonathan 4a768b7122 fix(discover): drop suggestions for non-artists (no core discography)
ListenBrainz surfaces producers/session players/band members (Max Martin,
Dominic Howard) that co-occur in listens but have no releases of their own —
they showed as bare rows with an empty page. _derive_albums now drops a
surfaced artist (deletes their pending suggestion) when their MB discography
has no core release-group (Album/Single/EP, no secondary type), catching both
empty discographies and soundtrack/production-only credits, while keeping real
singles/EP-only artists. On browse failure the suggestion is kept.

Test fake browser gains default_core so scoring tests keep surfacing candidates.
worker 249 tests.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-15 00:38:56 +02:00
Jonathan 9915bfbda2 feat(discover): full reset — clear + rebuild suggestions
POST /api/discover/reset sets discover.resetRequested; the worker, at the
start of the next sweep, clears all PENDING suggestions + contributions +
seed throttles and nulls WatchedArtist.lastDiscoveredAt (keeping
followed/wanted/dismissed decisions), then regenerates everything with current
logic. A 'Reset' button on /discover (2-step confirm) triggers it.

worker 247, web 190 tests.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-15 00:20:17 +02:00
Jonathan 288d55269d fix(discover): hide stale singles + replace album set on re-derive
- Client filters the row's album thumbnails to full albums only (hides
  singles/EPs left over from pre-albums-only sweeps).
- Worker: re-deriving an artist deletes its pending album suggestions that are
  no longer picks (stale singles / now-owned), keeping wanted/dismissed rows,
  so the suggestion set self-cleans instead of accumulating.

worker 246, web 189 tests.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-15 00:10:10 +02:00
Jonathan 194b4911c7 feat(discover): derive newest + most-popular album per artist
_derive_albums now also picks the artist's most-played album (Last.fm,
matched into the browsed discography by MBID then normalized title),
alongside the newest. Same release -> one row tagged newest,popular.
albumReason persisted; run_discovery threads an optional popularity provider.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-14 23:39:18 +02:00
Jonathan efdd592761 feat(discover): Last.fm album-popularity helper
LastfmAlbumPopularity.top_albums(conn, name) ranks an artist's albums by
Last.fm playcount (read-through ApiCache, 12h; errors -> []). registry
build_album_popularity returns it only when lastfm.api_key is set.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-14 23:36:41 +02:00
Jonathan e04c1c9795 feat(discover): also seed discovery from owned-album artists
Discovery seeded only from followed artists. Now it also seeds from artists
you own albums by (LibraryItem.artistMbid, captured since the MBID own-state
work) but don't follow — so your library shapes suggestions, not just your
follows.

- New DiscoverySeed throttle table (migration add_discovery_seed) mirroring
  WatchedArtist.lastDiscoveredAt, so each library-derived seed is swept once
  per interval (WatchedArtist seeds keep their own throttle).
- _seed_artists merges both seed sources least-recently-swept first, capped at
  chunk_size; run_discovery stamps the right throttle per seed and evicts
  contributions from seeds that are neither followed nor owned.
- Artists you already have (followed OR owned) are excluded from the suggestion
  candidates — discovery surfaces only genuinely new artists.
- discover.seedFromLibrary config (default true) + a Discovery settings toggle.

NOTE: existing LibraryItem rows have null artistMbid (populated on new
import/scan), so a re-scan is needed before old albums seed discovery.
worker 236, web 187 tests.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-14 22:39:52 +02:00
Jonathan 18a22db7fb feat(library): match own-state on artist MBID, not just name
Own-state ("in library") relied on an exact artist-NAME match, so an owned
album whose name differed from the MB canonical (punctuation/locale/feat.)
showed as not-owned even when followed. Capture the MusicBrainz artist MBID
on LibraryItem and prefer it for matching.

- New LibraryItem.artistMbid (migration add_library_artist_mbid). Populated at
  import (pipeline) + scan, both of which already hold the resolved target;
  the pipeline also now stores rgMbid (it previously didn't). COALESCE on
  conflict backfills MBIDs on re-scan without clobbering.
- /api/artists "in library, not followed" matches by artistMbid first, name
  case-insensitively as fallback (rows predating the column). Library route
  prefers the item's own artistMbid over the release join.

Last.fm/discover own-state still use name-match (they work) — a smaller
follow-up. worker 233, web 186 tests.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-14 21:55:24 +02:00
Jonathan 1c1b98967a feat(pipeline): reject downloads far shorter than the MB total duration
The verify step checked track COUNT but not playtime, so a truncated file or
a short preview substituted for a real track could import as "complete". Add
a total-duration check: sum the staged audio's playtime (mutagen) and fail
the job when it falls below 85% of MusicBrainz's expected total
(target.total_duration_s, already resolved). Generous tolerance so
edition/encoding differences don't false-positive; bonus tracks (over-long)
always pass.

The duration reader is injected (measure_duration param, default the real
mutagen-based reader) so the fail/pass paths are unit-tested; it returns None
for the offline fakes (no real audio), skipping the check exactly like the
staged-count guard. worker 233 tests.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-14 21:49:15 +02:00
Jonathan 1943fe2bdf feat(discover): max-normalize similarity scores across sources
_record_contributions now normalizes each source's top-N to [0,1] by its own
top score before summing across sources. ListenBrainz co-occurrence counts
(thousands) no longer drown Last.fm's 0–1 match scores, so a small-scale
source contributes real ranking signal instead of only adding candidates.

Discovery score assertions updated to the normalized scale (single-candidate
fakes normalize to 1.0); added a cross-source test showing a small-scale
source's candidate ranks equal to a large-scale one. min_score is now on the
[0, n_sources] scale (default 0.0 → unaffected). worker 231 tests.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-14 21:43:02 +02:00
Jonathan 8c8a34e320 feat(library): replace/upgrade an owned album on demand
Add Request.force (migration add_request_force): the pipeline's intake now
skips the "already in library" dedupe for a forced job, so an owned album is
re-downloaded. The import step still keeps the new copy only when it's higher
quality, so a forced upgrade can never downgrade what's on disk.

- Worker: _is_force_job → dedupe bypassed when Request.force.
- POST /api/library/[id]/upgrade finds the matching MonitoredRelease and
  enqueues a force request (guards against stacking on an in-flight job).
- Library route exposes monitoredReleaseId; the album modal shows a
  "Replace / upgrade" button when a release exists.

Worker + web tests added (pipeline force re-acquire; upgrade route).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-14 21:28:54 +02:00
Jonathan 855a0f15fe feat(library): ignore a release (do-not-monitor)
Add MonitoredRelease.ignored (migration add_release_ignored). The monitor's
enqueue_due skips ignored releases, and they drop off the Wanted list —
independent of `monitored`, so auto-monitor-future can't silently re-grab
an album the user chose to ignore.

- Worker enqueue_due: AND NOT mr.ignored.
- /api/releases/[id] PATCH now accepts `ignored`; /api/wanted filters it out;
  artist detail exposes `ignored`.
- UI: Ignore button on Wanted rows; discography shows an "Ignored" badge with
  an Ignore/Un-ignore toggle (monitor + search disabled while ignored).

Worker + web tests added.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-14 21:03:54 +02:00
Jonathan 6ca39859fa feat(library): surface albums the scan couldn't match
The library scan silently skipped album folders with no MusicBrainz match
(or an unreadable file). Persist them so they're visible and fixable.

- New UnmatchedAlbum model (migration add_unmatched_album), keyed by path.
- scan_chunk records a skipped album as unmatched (reason "no MusicBrainz
  match" or "unreadable: <err>"), clears the row when an album later
  resolves, and prunes stale rows (a prior scan's, or a removed folder)
  when a scan completes.
- GET /api/library/unmatched lists them; DELETE ?id= dismisses one.
- /library shows a "Couldn't match" section (path + reason + Dismiss).

Worker + web tests added.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-14 20:56:31 +02:00
Jonathan c9905937cc feat(discover): suggest full albums only (filter singles/EPs)
Discovery's "Suggested albums" surfaced singles and EPs because
_derive_albums filtered with is_core_release (Album/Single/EP). Add a
discovery-only _album_kind_ok gate that, when discover.albumsOnly (new
config, default true), requires primary_type == "Album" with no secondary
types. is_core_release is untouched — the monitor/scan still use it.

Exposed discover.albumsOnly in the discover config route + a Discovery
settings toggle. Worker + web tests added.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-14 20:37:28 +02:00
Jonathan 426b36775f fix(worker): reject a download with 0-byte/missing tracks (#16 partial)
Completeness only compared the adapter's self-reported track_count to expected,
so a silently-skipped track (0-byte placeholder — see the Qobuz 24K Magic bug)
imported as a "complete" album. Now the verify also counts the NON-EMPTY audio
actually staged and fails "incomplete download" when fewer than expected are
present (guarded with `staged and ...` so fake adapters that stage nothing keep
using the count path). A real broken download now goes needs_attention (or falls
through to another source) instead of importing a dead track.

Full per-track duration matching against MB recording lengths (the rest of #16)
remains a larger follow-up. worker 224 tests / 7-skip.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-14 13:36:11 +02:00
Jonathan bc42e546f0 fix(worker): raise http.client header limit so Qobuz tracks stop failing
Bruno Mars "24K Magic" track 1 repeatedly downloaded as a 0-byte file. The
worker log showed the real cause: streamrip's track download hit
HTTPException("got more than 100 headers") — Qobuz's CDN returns >100 response
headers for some tracks and Python's http.client rejects them at the default
_MAXHEADERS=100, so streamrip skips the track and leaves an empty placeholder.

- Raise http.client._MAXHEADERS to 1000 on import of the Qobuz adapter, so those
  responses parse and the track downloads.
- _count_staged_audio now skips 0-byte files (a failed-track placeholder is not a
  real track) — improves the download-progress estimate and stops counting dead
  tracks. (A verify-level "reject 0-byte tracks" safety net is a follow-up under
  #16 per-track duration matching — it needs the pipeline fakes to write real
  files, deferred to avoid disproportionate churn here.)

worker 223 tests / 7-skip.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-14 13:17:20 +02:00
Jonathan 9a95c33b7e fix(worker): a corrupt/unreadable album no longer aborts the whole scan
Live-testing #9 surfaced a pre-existing gap: one truncated file mutagen chokes
on ("file said 4 bytes, read 0 bytes") raised out of _process_album and aborted
the entire library scan (worklist abandoned). Now scan_chunk catches a per-album
error, rolls back, logs it, counts it as skipped, marks the item done, and
continues — so a single bad file can't block the rest of the library.

worker tests: new case asserts a corrupt album is skipped while a good one still
imports (and captures its on-disk trackNames).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-14 13:00:02 +02:00
Jonathan dcf0bb0696 feat: show real on-disk tracklist in the Library album modal (#9)
The Library modal showed MusicBrainz's canonical tracklist, hiding incomplete
downloads / edition or track-order mismatches, and showed nothing for albums
without an rgMbid. Now it shows the REAL files on disk.

- LibraryItem gains trackNames String[] (migration add_library_track_names):
  the album's on-disk "## Title.ext" audio filenames.
- Captured at import (pipeline._import lists the final album folder) and at scan
  (scan._record_owned now also refreshes trackNames on re-scan via ON CONFLICT
  DO UPDATE + xmax=0 to preserve the newly-imported flag). New shared
  library.list_audio_files() helper.
- /api/library exposes trackNames; the AlbumModal prefers on-disk tracks when
  present (parsed "## Title" → position/title, zero network, labeled "On disk ·
  N tracks"), falling back to the MusicBrainz listing otherwise. Items imported
  before this column show the MB fallback until re-scanned.

worker 221 tests / 7-skip, web 153, tsc + build clean.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-14 12:55:54 +02:00
Jonathan 746d61b264 feat(worker): reclaim in-flight jobs on startup (crash-resume)
A worker crash/restart mid-pipeline left its job stuck in matching/matched/
downloading/tagging with no auto-recovery — only the manual Retry (needs_attention
only) could rescue it. Now on startup the worker resets any such orphaned job
back to 'requested' so it gets re-claimed.

Safe at startup: this fresh worker owns none of those jobs, staging is already
cleared (clear_staging_root runs first), and the pipeline redoes intake→import
idempotently with an atomic import. Terminal jobs (imported/needs_attention) are
left untouched; claimedAt/error/downloadProgress are reset. Logged on startup.

worker 218 tests / 7-skip (in-flight→requested, terminal untouched, re-claimable).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-14 12:37:16 +02:00
Jonathan b1757b631c feat(worker): smooth download % from adapter byte-level progress
The Floor bar used a file-count poller that jumps 0→~99% for small/fast albums
(streamrip reports only 0/1). But yt-dlp and slskd already emit real byte-level
progress via the on_progress callback the pipeline was discarding
(lambda _pct: None).

- Thread a throttled on_progress → Job.downloadProgress (write on a >=1% move or
  every 0.5s; clamped 0..1; write failures swallowed). Called synchronously from
  adapter.download on the pipeline thread, so it reuses the main conn safely.
- The file-count poller stays as the FALLBACK: it skips writing while a
  `reports` flag is set (an adapter is driving real progress), and the flag is
  cleared per attempt so a fall-through to streamrip still gets the estimate.
- Smooth bars for YouTube + Soulseek now; Qobuz/streamrip keeps the file-count
  estimate until its per-track callback is hooked later.

worker 216 tests / 7-skip (on_progress write/throttle/clamp covered; existing
pipeline tests now exercise the fake's on_progress end-to-end).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-14 11:41:51 +02:00
Jonathan 034ae40084 feat(worker): share the MusicBrainz cache from scans/discovery
The worker's MB browser (discography + artist search, used by monitor sweeps,
library scans, and Last.fm discovery) re-fetched every time. Route it through
the same ApiCache table + logical keys the web app uses, so a discography one
process fetched warms the other, and repeat lookups stop hitting MB.

- apicache.py cached_api(): read-through, serve-stale-on-outage, never caches
  null, best-effort (a cache-DB error degrades to a live fetch, never breaks
  scan/sweep/discovery). Its own DEDICATED autocommit connection so it can't
  commit the worker's in-flight transaction, and autocommit avoids freezing
  Postgres now() (which would break TTL math).
- CachingMbBrowser decorates MusicBrainzBrowser, serializing to the SAME
  camelCase JSON shape the web writes (round-trip tested). registry.build_browser
  wraps it when a DSN is present; the Last.fm source's browser rides it too.
- Hourly prune drops ApiCache rows > 90d so the table stays bounded.
- worker 213 tests / 7-skip.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-14 11:17:51 +02:00
Jonathan 8cd5392ec3 fix(security): fail fast on a missing/placeholder/public LYRA_SECRET_KEY
.env.example shipped a valid, PUBLIC base64 key — if a user didn't replace it,
every credential got encrypted with a publicly-known key and everything still
"worked," so the footgun was silent.

- .env.example now ships an obvious placeholder (CHANGE_ME_generate_with_...)
  with a "you MUST replace this" note and the openssl hint.
- web + worker validate LYRA_SECRET_KEY at startup: missing / placeholder /
  wrong-length ⇒ loud FATAL banner and refuse to start (web via the Next.js
  instrumentation hook, worker via _require_secret_key before touching the DB).
  The old public example key ⇒ a loud WARNING but not fatal, since an existing
  install may have encrypted its creds under it (rotating needs re-entry).
- Validation lives in an edge-safe web/src/lib/secret-key.ts (no node:crypto)
  so the instrumentation hook bundles for both runtimes; crypto.ts re-exports.
- README emphasizes generating a fresh key + keeping it stable.

Verified live: worker exits 1 on placeholder/unset before any DB call; web
instrumentation throws and refuses to serve on the placeholder. web 138 tests,
worker 204/7-skip, tsc clean.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-14 10:19:46 +02:00
Jonathan f74fd5ab88 fix(worker): retrieve Soulseek downloads from slskd's dir into staging
SlskdClient enqueued the transfer and polled to Completed but never moved any
files into `dest` — slskd saves them to its OWN downloads dir, so staging stayed
empty and every Soulseek job failed "incomplete download". Qobuz/YouTube write
straight into staging and were unaffected.

- SlskdClient._retrieve() moves the finished files out of slskd's downloads dir
  into staging after completion, matching tolerantly by basename and preferring
  a parent-folder + size match (slskd's on-disk layout varies by version).
- download() returns the moved count as track_count so the pipeline's
  completeness check reflects what actually landed; raises loudly if nothing was
  found (misconfigured/unmounted downloads dir).
- Downloads dir resolves slskd.downloadsDir Config > SLSKD_DOWNLOADS_ROOT env >
  /slskd-downloads default; docker-compose mounts ${SLSKD_DOWNLOADS_DIR} there.
- .env.example documents SLSKD_DOWNLOADS_DIR; retrieval unit-tested offline.

slskd runs as an external daemon, so the shared dir must be a host path both
slskd and the worker mount — set once Lyra runs on the slskd host. Not yet
verified live end-to-end (no slskd access from this VM).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-14 09:59:16 +02:00
Jonathan 9d5e1ff58a feat: track download progress on Job (migration + worker poller + API)
Add Job.downloadProgress (0.0-1.0), derived from a worker background
poller that counts audio files landing in staging / expected track
count, since streamrip only reports 0/1. No UI yet (slice B).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-13 23:23:15 +02:00
Jonathan 53c0dcc256 fix(worker): run streamrip on one persistent event loop (fixes Qobuz download failures)
Each search/download called asyncio.run, creating and closing a fresh event loop.
streamrip's module-global download-concurrency Semaphore binds to the loop on first
use, so the 2nd+ download reused a Semaphore bound to a closed loop -> "Semaphore is
bound to a different event loop", tracks failed, album arrived short -> "incomplete
download" / needs_attention. Run all streamrip coroutines on one persistent loop.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-13 23:04:25 +02:00
Jonathan fdca692bb5 fix(worker): mark the chosen candidate at download-start, not after success
_mark_chosen ran only after a download succeeded, so throughout the 'downloading'
state no candidate was chosen — the Floor row's "source · format · tracks" line was
blank (only elapsed showed). Now mark the candidate exclusively at each download
attempt so `chosen` reflects the source in flight (and updates on fall-through).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-13 22:17:18 +02:00
Jonathan 146bf32f23 feat(worker): Last.fm artist.getSimilar as a second discovery SimilaritySource
Adds LastfmSource, resolving Last.fm's often-empty artist mbid via the
injected MB browser (name->mbid cached) so every emitted SimilarArtist.mbid
is a real MusicBrainz artist MBID. Registered in build_similarity_sources,
gated on lastfm.api_key so the ListenBrainz-only path is unchanged when
unset.
2026-07-13 19:23:18 +02:00
Jonathan 6f325d30b9 fix(worker): evict removed-seed contributions + clear scan worklist on abandon
Two row-leak bugs found in whole-branch review:

- discovery.run_discovery now prunes DiscoverySeedContribution rows whose
  seedMbid is no longer a WatchedArtist (unfollow/removal has no FK cascade)
  and recomputes the touched candidates' scores this sweep, instead of
  letting a dead seed's contribution inflate the aggregate forever.

- maybe_run_scan's exception handler around scan_chunk now calls
  clear_worklist and clears scan.id, matching the drain path. Previously
  an exception mid-chunk left the already-done rows plus the remaining
  worklist for that scan_id orphaned forever, since the next scan mints a
  fresh uuid. Also clear scan.id in the build_worklist failure handler for
  consistency.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-13 15:39:38 +02:00
Jonathan 0115b82867 feat(worker): persisted scan worklist (O(N) scan, robust to mid-scan changes)
Replace scan_chunk's O(N^2) tree re-walk-and-skip-to-cursor with a
persisted ScanWorkItem worklist: build_worklist walks the tree once at
scan start and inserts one row per album (idempotent via ON CONFLICT),
scan_chunk pops up to `limit` not-done rows by indexed cursor and marks
them done, and clear_worklist drops the scan's rows once drained.
scan_library is reimplemented on top of the worklist (one-shot scan_id,
drain, clear) with an unchanged signature/behavior. main.py's
maybe_run_scan now drives a generated scan.id through Config
(scan.id/scan.inProgress/scan.progress/scan.result) instead of a
scan.cursor string, so a mid-scan library change no longer risks
skipping or re-walking albums.
2026-07-13 15:26:44 +02:00
Jonathan de052a4ef7 feat(worker): per-seed discovery contributions so score sums across chunks/sweeps
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-13 15:18:30 +02:00
Jonathan bdd8fe72e9 feat: add DiscoverySeedContribution + ScanWorkItem tables 2026-07-13 15:14:58 +02:00
Jonathan a67c552f29 feat(worker): chunk the discovery sweep (one chunk per loop iteration)
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-13 00:57:00 +02:00
Jonathan 5351cd029d feat(worker): discovery chunk_size + seed-count in result (retire max_seeds)
Reconciled the pre-existing test_config_from_config_parses_and_defaults
test in test_discovery.py, which referenced max_seeds/maxSeeds and would
otherwise have broken.
2026-07-13 00:52:08 +02:00
Jonathan df7bf2fcf8 fix(worker): import_album recovers orphaned .old/.importing from an interrupted swap
Adds a crash-recovery preamble that restores `final` from `.old` when a
prior swap was interrupted after rename(final, old) but before
rename(tmp, final), and always drops stale `.old`/`.importing` orphans.

Also guards the swap: if nothing new was staged into tmp (empty/missing
staging) and `final` already exists, leave it untouched instead of
swapping in an empty copy. This is required beyond the literal preamble
because the pre-existing swap logic unconditionally replaced `final`
with `tmp` regardless of whether anything was actually imported, which
would otherwise destroy a just-recovered (or any existing) copy whenever
staging is empty.
2026-07-13 00:46:40 +02:00
Jonathan e825e3d7ad feat(worker): separate download staging volume (STAGING_DIR)
Staging is no longer forced under the library. The worker reads STAGING_ROOT
(container path, default /music/.staging) and stages per-job downloads there;
docker-compose mounts ${STAGING_DIR:-${MUSIC_DIR}/.staging} at /staging with
STAGING_ROOT=/staging. Set STAGING_DIR to a fast local disk when the library
(MUSIC_DIR) is a network share so temp download I/O stays off the share.

Safe cross-volume: import_album already assembles into a temp dir on the
library volume and swaps atomically there, so partial downloads never touch
the library and the final swap stays atomic. clear_staging_root now clears the
root's contents (mount-safe) rather than removing the root. Documented in
.env.example; new tests cover the separate-root path.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-12 23:44:27 +02:00
Jonathan 082365005c fix(worker): reconcile every job + backfill owned releases
Wanted releases pressed while the monitor is off never got reconciled, so
they lingered on the wanted list. Extract finish_job() and call it for every
finished job (not just when monitor.enabled) so the MonitoredRelease
transitions to fulfilled. Add fulfill_owned_releases(), run once at startup,
to backfill releases already owned at/above the quality cutoff (self-heals
items stuck from before this fix). Both idempotent + tested.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-12 15:08:24 +02:00
Jonathan 8bbc1db32f feat(worker): chunk the library scan + reconnect on DB loss
Two worker-loop robustness fixes:

- Scan chunking: maybe_run_scan advances the library scan by at most one
  scan_chunk (default scan.chunkSize=25 albums) per loop iteration, so a
  large library no longer blocks job-claim/monitor for minutes-to-hours.
  Progress persists in scan.inProgress/scan.cursor/scan.progress and is
  resumable; scan.result (unchanged format) is written on completion.

- DB reconnect: wrap the loop body in `except psycopg.OperationalError`
  (AdminShutdown subclasses it) to close the dead handle and reconnect via
  wait_for_db() in-process, instead of crashing and relying on the
  container restart policy. Verified via a backend-termination test.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-12 12:37:06 +02:00
Jonathan 62b60e3dae refactor(scan): extract scan_chunk from scan_library
scan_chunk resolves album folders whose (artist, album) sorts after a
cursor, up to a limit, returning (new_cursor, imported, skipped, done).
scan_library becomes the unbounded case (limit=None), so its behavior and
tests are unchanged. Idempotent: all writes are ON CONFLICT, so an
overlapping/repeated cursor is safe. Enables chunked scanning in the loop.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-12 12:36:54 +02:00
Jonathan 9ba587d136 fix(worker): run discovery at most once per loop iteration
The scheduled-tick branch and the discover.requested one-shot branch
both read the same stale per-iteration config snapshot, so an enabled+due
sweep coinciding with a pending request ran discovery twice back-to-back
(idempotent but wasteful). Extract maybe_run_discovery, which unifies both
triggers into a single run and is now unit-tested.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-12 10:42:48 +02:00
Jonathan 830de07997 fix(monitor): only auto-monitor core studio releases
autoMonitorFuture flagged any new release, so it could auto-monitor a
newly-released live album or compilation. Gate it on is_core_release so
auto-monitoring matches the studio-only default the UI shows.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-12 10:42:42 +02:00
Jonathan a15f4ae8bb feat(discovery): build_similarity_sources + worker sweep/trigger wiring
Adds registry.build_similarity_sources() (always constructs
ListenBrainzSource; reachability is a per-run health() concern, not a
startup gate) and wires discovery into the worker's main loop: a
scheduled sweep gated on discover.enabled + DISCOVER_TICK_SECONDS, and
a one-shot discover.requested trigger mirroring the existing
scan.requested pattern.

Also hardens run_discovery's health-check pass: a source whose
health() raises is now logged and skipped instead of aborting the
whole sweep (accepted finding from the Task-3 review).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-12 00:22:34 +02:00
Jonathan 6f504a7c04 feat(discovery): ListenBrainz similarity source
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-12 00:17:58 +02:00
Jonathan 0338d8e119 feat(discovery): derive core-album suggestions for surfaced artists
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-12 00:13:40 +02:00
Jonathan ccf3fb59ab feat(discovery): run_discovery aggregates similar artists into suggestions
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-12 00:07:09 +02:00
Jonathan 889e33355f feat(discovery): SimilaritySource protocol + FakeSimilaritySource
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-12 00:03:45 +02:00
Jonathan 0ddc9db611 feat(discovery): DiscoverySuggestion schema + WatchedArtist.lastDiscoveredAt
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-11 23:58:45 +02:00
Jonathan 4c1d3c639e fix: scan backfills release type metadata onto pre-existing rows
MonitoredRelease rows created before an artist's discography was
populated (e.g. an owned album recorded before _populate_discography
ran) never got primaryType/secondaryTypes backfilled, wrongly
excluding them from the studio-only core view. _populate_discography
now DO UPDATEs the MB type metadata on conflict instead of DO NOTHING,
leaving monitored/state/currentQualityClass/firstGrabbedAt untouched.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-11 18:12:37 +02:00
Jonathan f2dbef047c feat: WatchedArtist.showAllTypes toggle (default false)
Adds the per-artist release-type filter toggle: off (default) restricts
a discography view to core studio releases, on shows everything
(live, compilations, etc.).
2026-07-11 18:03:29 +02:00