Commit Graph

105 Commits

Author SHA1 Message Date
Jonathan 0325467aba fix(worker): smooth Qobuz download progress via streamrip byte hook
streamrip reports progress only per concurrent track, so Job.downloadProgress
came solely from the file-count poller — 0% until every track landed, then a
jump to 100%. Patch streamrip.media.track.get_progress_callback to aggregate
per-track byte deltas over an estimated album total (avg started-track size ×
len(album.tracks)) into one smooth, generally-climbing 0..1 fraction routed
through the pipeline's on_progress. Capped at 0.99; the pipeline sets 1.0 on
completion. No signature changes — track count read from the resolved album.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-15 02:37:08 +02:00
Jonathan f72d193ef7 chore(discover): raise_for_status on Last.fm fetches
A 4xx/5xx (rate limit / auth) now raises instead of parsing an error body;
callers already handle it (discovery per-source try/except; album fetch
returns []). Closes the last small deferred cleanup.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-15 00:50:35 +02:00
Jonathan 4a768b7122 fix(discover): drop suggestions for non-artists (no core discography)
ListenBrainz surfaces producers/session players/band members (Max Martin,
Dominic Howard) that co-occur in listens but have no releases of their own —
they showed as bare rows with an empty page. _derive_albums now drops a
surfaced artist (deletes their pending suggestion) when their MB discography
has no core release-group (Album/Single/EP, no secondary type), catching both
empty discographies and soundtrack/production-only credits, while keeping real
singles/EP-only artists. On browse failure the suggestion is kept.

Test fake browser gains default_core so scoring tests keep surfacing candidates.
worker 249 tests.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-15 00:38:56 +02:00
Jonathan 9915bfbda2 feat(discover): full reset — clear + rebuild suggestions
POST /api/discover/reset sets discover.resetRequested; the worker, at the
start of the next sweep, clears all PENDING suggestions + contributions +
seed throttles and nulls WatchedArtist.lastDiscoveredAt (keeping
followed/wanted/dismissed decisions), then regenerates everything with current
logic. A 'Reset' button on /discover (2-step confirm) triggers it.

worker 247, web 190 tests.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-15 00:20:17 +02:00
Jonathan 288d55269d fix(discover): hide stale singles + replace album set on re-derive
- Client filters the row's album thumbnails to full albums only (hides
  singles/EPs left over from pre-albums-only sweeps).
- Worker: re-deriving an artist deletes its pending album suggestions that are
  no longer picks (stale singles / now-owned), keeping wanted/dismissed rows,
  so the suggestion set self-cleans instead of accumulating.

worker 246, web 189 tests.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-15 00:10:10 +02:00
Jonathan e8fd4bb0a9 feat(discover): supply Last.fm popularity to the sweep
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-14 23:40:33 +02:00
Jonathan 194b4911c7 feat(discover): derive newest + most-popular album per artist
_derive_albums now also picks the artist's most-played album (Last.fm,
matched into the browsed discography by MBID then normalized title),
alongside the newest. Same release -> one row tagged newest,popular.
albumReason persisted; run_discovery threads an optional popularity provider.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-14 23:39:18 +02:00
Jonathan efdd592761 feat(discover): Last.fm album-popularity helper
LastfmAlbumPopularity.top_albums(conn, name) ranks an artist's albums by
Last.fm playcount (read-through ApiCache, 12h; errors -> []). registry
build_album_popularity returns it only when lastfm.api_key is set.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-14 23:36:41 +02:00
Jonathan e04c1c9795 feat(discover): also seed discovery from owned-album artists
Discovery seeded only from followed artists. Now it also seeds from artists
you own albums by (LibraryItem.artistMbid, captured since the MBID own-state
work) but don't follow — so your library shapes suggestions, not just your
follows.

- New DiscoverySeed throttle table (migration add_discovery_seed) mirroring
  WatchedArtist.lastDiscoveredAt, so each library-derived seed is swept once
  per interval (WatchedArtist seeds keep their own throttle).
- _seed_artists merges both seed sources least-recently-swept first, capped at
  chunk_size; run_discovery stamps the right throttle per seed and evicts
  contributions from seeds that are neither followed nor owned.
- Artists you already have (followed OR owned) are excluded from the suggestion
  candidates — discovery surfaces only genuinely new artists.
- discover.seedFromLibrary config (default true) + a Discovery settings toggle.

NOTE: existing LibraryItem rows have null artistMbid (populated on new
import/scan), so a re-scan is needed before old albums seed discovery.
worker 236, web 187 tests.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-14 22:39:52 +02:00
Jonathan 18a22db7fb feat(library): match own-state on artist MBID, not just name
Own-state ("in library") relied on an exact artist-NAME match, so an owned
album whose name differed from the MB canonical (punctuation/locale/feat.)
showed as not-owned even when followed. Capture the MusicBrainz artist MBID
on LibraryItem and prefer it for matching.

- New LibraryItem.artistMbid (migration add_library_artist_mbid). Populated at
  import (pipeline) + scan, both of which already hold the resolved target;
  the pipeline also now stores rgMbid (it previously didn't). COALESCE on
  conflict backfills MBIDs on re-scan without clobbering.
- /api/artists "in library, not followed" matches by artistMbid first, name
  case-insensitively as fallback (rows predating the column). Library route
  prefers the item's own artistMbid over the release join.

Last.fm/discover own-state still use name-match (they work) — a smaller
follow-up. worker 233, web 186 tests.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-14 21:55:24 +02:00
Jonathan 1c1b98967a feat(pipeline): reject downloads far shorter than the MB total duration
The verify step checked track COUNT but not playtime, so a truncated file or
a short preview substituted for a real track could import as "complete". Add
a total-duration check: sum the staged audio's playtime (mutagen) and fail
the job when it falls below 85% of MusicBrainz's expected total
(target.total_duration_s, already resolved). Generous tolerance so
edition/encoding differences don't false-positive; bonus tracks (over-long)
always pass.

The duration reader is injected (measure_duration param, default the real
mutagen-based reader) so the fail/pass paths are unit-tested; it returns None
for the offline fakes (no real audio), skipping the check exactly like the
staged-count guard. worker 233 tests.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-14 21:49:15 +02:00
Jonathan 1943fe2bdf feat(discover): max-normalize similarity scores across sources
_record_contributions now normalizes each source's top-N to [0,1] by its own
top score before summing across sources. ListenBrainz co-occurrence counts
(thousands) no longer drown Last.fm's 0–1 match scores, so a small-scale
source contributes real ranking signal instead of only adding candidates.

Discovery score assertions updated to the normalized scale (single-candidate
fakes normalize to 1.0); added a cross-source test showing a small-scale
source's candidate ranks equal to a large-scale one. min_score is now on the
[0, n_sources] scale (default 0.0 → unaffected). worker 231 tests.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-14 21:43:02 +02:00
Jonathan 8c8a34e320 feat(library): replace/upgrade an owned album on demand
Add Request.force (migration add_request_force): the pipeline's intake now
skips the "already in library" dedupe for a forced job, so an owned album is
re-downloaded. The import step still keeps the new copy only when it's higher
quality, so a forced upgrade can never downgrade what's on disk.

- Worker: _is_force_job → dedupe bypassed when Request.force.
- POST /api/library/[id]/upgrade finds the matching MonitoredRelease and
  enqueues a force request (guards against stacking on an in-flight job).
- Library route exposes monitoredReleaseId; the album modal shows a
  "Replace / upgrade" button when a release exists.

Worker + web tests added (pipeline force re-acquire; upgrade route).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-14 21:28:54 +02:00
Jonathan 855a0f15fe feat(library): ignore a release (do-not-monitor)
Add MonitoredRelease.ignored (migration add_release_ignored). The monitor's
enqueue_due skips ignored releases, and they drop off the Wanted list —
independent of `monitored`, so auto-monitor-future can't silently re-grab
an album the user chose to ignore.

- Worker enqueue_due: AND NOT mr.ignored.
- /api/releases/[id] PATCH now accepts `ignored`; /api/wanted filters it out;
  artist detail exposes `ignored`.
- UI: Ignore button on Wanted rows; discography shows an "Ignored" badge with
  an Ignore/Un-ignore toggle (monitor + search disabled while ignored).

Worker + web tests added.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-14 21:03:54 +02:00
Jonathan 6ca39859fa feat(library): surface albums the scan couldn't match
The library scan silently skipped album folders with no MusicBrainz match
(or an unreadable file). Persist them so they're visible and fixable.

- New UnmatchedAlbum model (migration add_unmatched_album), keyed by path.
- scan_chunk records a skipped album as unmatched (reason "no MusicBrainz
  match" or "unreadable: <err>"), clears the row when an album later
  resolves, and prunes stale rows (a prior scan's, or a removed folder)
  when a scan completes.
- GET /api/library/unmatched lists them; DELETE ?id= dismisses one.
- /library shows a "Couldn't match" section (path + reason + Dismiss).

Worker + web tests added.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-14 20:56:31 +02:00
Jonathan c9905937cc feat(discover): suggest full albums only (filter singles/EPs)
Discovery's "Suggested albums" surfaced singles and EPs because
_derive_albums filtered with is_core_release (Album/Single/EP). Add a
discovery-only _album_kind_ok gate that, when discover.albumsOnly (new
config, default true), requires primary_type == "Album" with no secondary
types. is_core_release is untouched — the monitor/scan still use it.

Exposed discover.albumsOnly in the discover config route + a Discovery
settings toggle. Worker + web tests added.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-14 20:37:28 +02:00
Jonathan 426b36775f fix(worker): reject a download with 0-byte/missing tracks (#16 partial)
Completeness only compared the adapter's self-reported track_count to expected,
so a silently-skipped track (0-byte placeholder — see the Qobuz 24K Magic bug)
imported as a "complete" album. Now the verify also counts the NON-EMPTY audio
actually staged and fails "incomplete download" when fewer than expected are
present (guarded with `staged and ...` so fake adapters that stage nothing keep
using the count path). A real broken download now goes needs_attention (or falls
through to another source) instead of importing a dead track.

Full per-track duration matching against MB recording lengths (the rest of #16)
remains a larger follow-up. worker 224 tests / 7-skip.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-14 13:36:11 +02:00
Jonathan eb51489ba9 feat(ops): web + worker container healthchecks (#20)
docker-compose only health-checked db. Add:
- web: an unauthenticated GET /api/health (excluded from the auth middleware) +
  a compose healthcheck that hits it (node global fetch — no curl/wget in image).
- worker: a background daemon thread touches /tmp/worker.alive every ~15s
  (independent of the main loop, so it stays fresh through a long download) +
  a compose healthcheck on the file's mtime (unhealthy if >90s stale).

Now `docker compose ps` reports real health for all services.
worker 223 tests, web 154, tsc + build + compose config clean.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-14 13:33:21 +02:00
Jonathan bc42e546f0 fix(worker): raise http.client header limit so Qobuz tracks stop failing
Bruno Mars "24K Magic" track 1 repeatedly downloaded as a 0-byte file. The
worker log showed the real cause: streamrip's track download hit
HTTPException("got more than 100 headers") — Qobuz's CDN returns >100 response
headers for some tracks and Python's http.client rejects them at the default
_MAXHEADERS=100, so streamrip skips the track and leaves an empty placeholder.

- Raise http.client._MAXHEADERS to 1000 on import of the Qobuz adapter, so those
  responses parse and the track downloads.
- _count_staged_audio now skips 0-byte files (a failed-track placeholder is not a
  real track) — improves the download-progress estimate and stops counting dead
  tracks. (A verify-level "reject 0-byte tracks" safety net is a follow-up under
  #16 per-track duration matching — it needs the pipeline fakes to write real
  files, deferred to avoid disproportionate churn here.)

worker 223 tests / 7-skip.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-14 13:17:20 +02:00
Jonathan 9a95c33b7e fix(worker): a corrupt/unreadable album no longer aborts the whole scan
Live-testing #9 surfaced a pre-existing gap: one truncated file mutagen chokes
on ("file said 4 bytes, read 0 bytes") raised out of _process_album and aborted
the entire library scan (worklist abandoned). Now scan_chunk catches a per-album
error, rolls back, logs it, counts it as skipped, marks the item done, and
continues — so a single bad file can't block the rest of the library.

worker tests: new case asserts a corrupt album is skipped while a good one still
imports (and captures its on-disk trackNames).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-14 13:00:02 +02:00
Jonathan dcf0bb0696 feat: show real on-disk tracklist in the Library album modal (#9)
The Library modal showed MusicBrainz's canonical tracklist, hiding incomplete
downloads / edition or track-order mismatches, and showed nothing for albums
without an rgMbid. Now it shows the REAL files on disk.

- LibraryItem gains trackNames String[] (migration add_library_track_names):
  the album's on-disk "## Title.ext" audio filenames.
- Captured at import (pipeline._import lists the final album folder) and at scan
  (scan._record_owned now also refreshes trackNames on re-scan via ON CONFLICT
  DO UPDATE + xmax=0 to preserve the newly-imported flag). New shared
  library.list_audio_files() helper.
- /api/library exposes trackNames; the AlbumModal prefers on-disk tracks when
  present (parsed "## Title" → position/title, zero network, labeled "On disk ·
  N tracks"), falling back to the MusicBrainz listing otherwise. Items imported
  before this column show the MB fallback until re-scanned.

worker 221 tests / 7-skip, web 153, tsc + build clean.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-14 12:55:54 +02:00
Jonathan 746d61b264 feat(worker): reclaim in-flight jobs on startup (crash-resume)
A worker crash/restart mid-pipeline left its job stuck in matching/matched/
downloading/tagging with no auto-recovery — only the manual Retry (needs_attention
only) could rescue it. Now on startup the worker resets any such orphaned job
back to 'requested' so it gets re-claimed.

Safe at startup: this fresh worker owns none of those jobs, staging is already
cleared (clear_staging_root runs first), and the pipeline redoes intake→import
idempotently with an atomic import. Terminal jobs (imported/needs_attention) are
left untouched; claimedAt/error/downloadProgress are reset. Logged on startup.

worker 218 tests / 7-skip (in-flight→requested, terminal untouched, re-claimable).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-14 12:37:16 +02:00
Jonathan b1757b631c feat(worker): smooth download % from adapter byte-level progress
The Floor bar used a file-count poller that jumps 0→~99% for small/fast albums
(streamrip reports only 0/1). But yt-dlp and slskd already emit real byte-level
progress via the on_progress callback the pipeline was discarding
(lambda _pct: None).

- Thread a throttled on_progress → Job.downloadProgress (write on a >=1% move or
  every 0.5s; clamped 0..1; write failures swallowed). Called synchronously from
  adapter.download on the pipeline thread, so it reuses the main conn safely.
- The file-count poller stays as the FALLBACK: it skips writing while a
  `reports` flag is set (an adapter is driving real progress), and the flag is
  cleared per attempt so a fall-through to streamrip still gets the estimate.
- Smooth bars for YouTube + Soulseek now; Qobuz/streamrip keeps the file-count
  estimate until its per-track callback is hooked later.

worker 216 tests / 7-skip (on_progress write/throttle/clamp covered; existing
pipeline tests now exercise the fake's on_progress end-to-end).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-14 11:41:51 +02:00
Jonathan 034ae40084 feat(worker): share the MusicBrainz cache from scans/discovery
The worker's MB browser (discography + artist search, used by monitor sweeps,
library scans, and Last.fm discovery) re-fetched every time. Route it through
the same ApiCache table + logical keys the web app uses, so a discography one
process fetched warms the other, and repeat lookups stop hitting MB.

- apicache.py cached_api(): read-through, serve-stale-on-outage, never caches
  null, best-effort (a cache-DB error degrades to a live fetch, never breaks
  scan/sweep/discovery). Its own DEDICATED autocommit connection so it can't
  commit the worker's in-flight transaction, and autocommit avoids freezing
  Postgres now() (which would break TTL math).
- CachingMbBrowser decorates MusicBrainzBrowser, serializing to the SAME
  camelCase JSON shape the web writes (round-trip tested). registry.build_browser
  wraps it when a DSN is present; the Last.fm source's browser rides it too.
- Hourly prune drops ApiCache rows > 90d so the table stays bounded.
- worker 213 tests / 7-skip.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-14 11:17:51 +02:00
Jonathan 8cd5392ec3 fix(security): fail fast on a missing/placeholder/public LYRA_SECRET_KEY
.env.example shipped a valid, PUBLIC base64 key — if a user didn't replace it,
every credential got encrypted with a publicly-known key and everything still
"worked," so the footgun was silent.

- .env.example now ships an obvious placeholder (CHANGE_ME_generate_with_...)
  with a "you MUST replace this" note and the openssl hint.
- web + worker validate LYRA_SECRET_KEY at startup: missing / placeholder /
  wrong-length ⇒ loud FATAL banner and refuse to start (web via the Next.js
  instrumentation hook, worker via _require_secret_key before touching the DB).
  The old public example key ⇒ a loud WARNING but not fatal, since an existing
  install may have encrypted its creds under it (rotating needs re-entry).
- Validation lives in an edge-safe web/src/lib/secret-key.ts (no node:crypto)
  so the instrumentation hook bundles for both runtimes; crypto.ts re-exports.
- README emphasizes generating a fresh key + keeping it stable.

Verified live: worker exits 1 on placeholder/unset before any DB call; web
instrumentation throws and refuses to serve on the placeholder. web 138 tests,
worker 204/7-skip, tsc clean.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-14 10:19:46 +02:00
Jonathan f74fd5ab88 fix(worker): retrieve Soulseek downloads from slskd's dir into staging
SlskdClient enqueued the transfer and polled to Completed but never moved any
files into `dest` — slskd saves them to its OWN downloads dir, so staging stayed
empty and every Soulseek job failed "incomplete download". Qobuz/YouTube write
straight into staging and were unaffected.

- SlskdClient._retrieve() moves the finished files out of slskd's downloads dir
  into staging after completion, matching tolerantly by basename and preferring
  a parent-folder + size match (slskd's on-disk layout varies by version).
- download() returns the moved count as track_count so the pipeline's
  completeness check reflects what actually landed; raises loudly if nothing was
  found (misconfigured/unmounted downloads dir).
- Downloads dir resolves slskd.downloadsDir Config > SLSKD_DOWNLOADS_ROOT env >
  /slskd-downloads default; docker-compose mounts ${SLSKD_DOWNLOADS_DIR} there.
- .env.example documents SLSKD_DOWNLOADS_DIR; retrieval unit-tested offline.

slskd runs as an external daemon, so the shared dir must be a host path both
slskd and the worker mount — set once Lyra runs on the slskd host. Not yet
verified live end-to-end (no slskd access from this VM).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-14 09:59:16 +02:00
Jonathan 9d5e1ff58a feat: track download progress on Job (migration + worker poller + API)
Add Job.downloadProgress (0.0-1.0), derived from a worker background
poller that counts audio files landing in staging / expected track
count, since streamrip only reports 0/1. No UI yet (slice B).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-13 23:23:15 +02:00
Jonathan 53c0dcc256 fix(worker): run streamrip on one persistent event loop (fixes Qobuz download failures)
Each search/download called asyncio.run, creating and closing a fresh event loop.
streamrip's module-global download-concurrency Semaphore binds to the loop on first
use, so the 2nd+ download reused a Semaphore bound to a closed loop -> "Semaphore is
bound to a different event loop", tracks failed, album arrived short -> "incomplete
download" / needs_attention. Run all streamrip coroutines on one persistent loop.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-13 23:04:25 +02:00
Jonathan fdca692bb5 fix(worker): mark the chosen candidate at download-start, not after success
_mark_chosen ran only after a download succeeded, so throughout the 'downloading'
state no candidate was chosen — the Floor row's "source · format · tracks" line was
blank (only elapsed showed). Now mark the candidate exclusively at each download
attempt so `chosen` reflects the source in flight (and updates on fall-through).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-13 22:17:18 +02:00
Jonathan 146bf32f23 feat(worker): Last.fm artist.getSimilar as a second discovery SimilaritySource
Adds LastfmSource, resolving Last.fm's often-empty artist mbid via the
injected MB browser (name->mbid cached) so every emitted SimilarArtist.mbid
is a real MusicBrainz artist MBID. Registered in build_similarity_sources,
gated on lastfm.api_key so the ListenBrainz-only path is unchanged when
unset.
2026-07-13 19:23:18 +02:00
Jonathan 6f325d30b9 fix(worker): evict removed-seed contributions + clear scan worklist on abandon
Two row-leak bugs found in whole-branch review:

- discovery.run_discovery now prunes DiscoverySeedContribution rows whose
  seedMbid is no longer a WatchedArtist (unfollow/removal has no FK cascade)
  and recomputes the touched candidates' scores this sweep, instead of
  letting a dead seed's contribution inflate the aggregate forever.

- maybe_run_scan's exception handler around scan_chunk now calls
  clear_worklist and clears scan.id, matching the drain path. Previously
  an exception mid-chunk left the already-done rows plus the remaining
  worklist for that scan_id orphaned forever, since the next scan mints a
  fresh uuid. Also clear scan.id in the build_worklist failure handler for
  consistency.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-13 15:39:38 +02:00
Jonathan 0115b82867 feat(worker): persisted scan worklist (O(N) scan, robust to mid-scan changes)
Replace scan_chunk's O(N^2) tree re-walk-and-skip-to-cursor with a
persisted ScanWorkItem worklist: build_worklist walks the tree once at
scan start and inserts one row per album (idempotent via ON CONFLICT),
scan_chunk pops up to `limit` not-done rows by indexed cursor and marks
them done, and clear_worklist drops the scan's rows once drained.
scan_library is reimplemented on top of the worklist (one-shot scan_id,
drain, clear) with an unchanged signature/behavior. main.py's
maybe_run_scan now drives a generated scan.id through Config
(scan.id/scan.inProgress/scan.progress/scan.result) instead of a
scan.cursor string, so a mid-scan library change no longer risks
skipping or re-walking albums.
2026-07-13 15:26:44 +02:00
Jonathan de052a4ef7 feat(worker): per-seed discovery contributions so score sums across chunks/sweeps
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-13 15:18:30 +02:00
Jonathan a67c552f29 feat(worker): chunk the discovery sweep (one chunk per loop iteration)
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-13 00:57:00 +02:00
Jonathan 5351cd029d feat(worker): discovery chunk_size + seed-count in result (retire max_seeds)
Reconciled the pre-existing test_config_from_config_parses_and_defaults
test in test_discovery.py, which referenced max_seeds/maxSeeds and would
otherwise have broken.
2026-07-13 00:52:08 +02:00
Jonathan df7bf2fcf8 fix(worker): import_album recovers orphaned .old/.importing from an interrupted swap
Adds a crash-recovery preamble that restores `final` from `.old` when a
prior swap was interrupted after rename(final, old) but before
rename(tmp, final), and always drops stale `.old`/`.importing` orphans.

Also guards the swap: if nothing new was staged into tmp (empty/missing
staging) and `final` already exists, leave it untouched instead of
swapping in an empty copy. This is required beyond the literal preamble
because the pre-existing swap logic unconditionally replaced `final`
with `tmp` regardless of whether anything was actually imported, which
would otherwise destroy a just-recovered (or any existing) copy whenever
staging is empty.
2026-07-13 00:46:40 +02:00
Jonathan e825e3d7ad feat(worker): separate download staging volume (STAGING_DIR)
Staging is no longer forced under the library. The worker reads STAGING_ROOT
(container path, default /music/.staging) and stages per-job downloads there;
docker-compose mounts ${STAGING_DIR:-${MUSIC_DIR}/.staging} at /staging with
STAGING_ROOT=/staging. Set STAGING_DIR to a fast local disk when the library
(MUSIC_DIR) is a network share so temp download I/O stays off the share.

Safe cross-volume: import_album already assembles into a temp dir on the
library volume and swaps atomically there, so partial downloads never touch
the library and the final swap stays atomic. clear_staging_root now clears the
root's contents (mount-safe) rather than removing the root. Documented in
.env.example; new tests cover the separate-root path.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-12 23:44:27 +02:00
Jonathan 606ea82b70 feat: real worker status, covers on Recently Pressed, library counter
- Worker stamps a throttled worker.heartbeat; GET /api/worker reports online
  from its updatedAt recency (DB-time); masthead shows a live listening/offline
  dot instead of a static decoration.
- /api/requests enriched with rgMbid; Recently Pressed shows cover thumbs.
- Library album count moved from the filter row into a section header.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-12 17:58:23 +02:00
Jonathan 082365005c fix(worker): reconcile every job + backfill owned releases
Wanted releases pressed while the monitor is off never got reconciled, so
they lingered on the wanted list. Extract finish_job() and call it for every
finished job (not just when monitor.enabled) so the MonitoredRelease
transitions to fulfilled. Add fulfill_owned_releases(), run once at startup,
to backfill releases already owned at/above the quality cutoff (self-heals
items stuck from before this fix). Both idempotent + tested.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-12 15:08:24 +02:00
Jonathan 8bbc1db32f feat(worker): chunk the library scan + reconnect on DB loss
Two worker-loop robustness fixes:

- Scan chunking: maybe_run_scan advances the library scan by at most one
  scan_chunk (default scan.chunkSize=25 albums) per loop iteration, so a
  large library no longer blocks job-claim/monitor for minutes-to-hours.
  Progress persists in scan.inProgress/scan.cursor/scan.progress and is
  resumable; scan.result (unchanged format) is written on completion.

- DB reconnect: wrap the loop body in `except psycopg.OperationalError`
  (AdminShutdown subclasses it) to close the dead handle and reconnect via
  wait_for_db() in-process, instead of crashing and relying on the
  container restart policy. Verified via a backend-termination test.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-12 12:37:06 +02:00
Jonathan 62b60e3dae refactor(scan): extract scan_chunk from scan_library
scan_chunk resolves album folders whose (artist, album) sorts after a
cursor, up to a limit, returning (new_cursor, imported, skipped, done).
scan_library becomes the unbounded case (limit=None), so its behavior and
tests are unchanged. Idempotent: all writes are ON CONFLICT, so an
overlapping/repeated cursor is safe. Enables chunked scanning in the loop.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-12 12:36:54 +02:00
Jonathan 9ba587d136 fix(worker): run discovery at most once per loop iteration
The scheduled-tick branch and the discover.requested one-shot branch
both read the same stale per-iteration config snapshot, so an enabled+due
sweep coinciding with a pending request ran discovery twice back-to-back
(idempotent but wasteful). Extract maybe_run_discovery, which unifies both
triggers into a single run and is now unit-tested.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-12 10:42:48 +02:00
Jonathan 830de07997 fix(monitor): only auto-monitor core studio releases
autoMonitorFuture flagged any new release, so it could auto-monitor a
newly-released live album or compilation. Gate it on is_core_release so
auto-monitoring matches the studio-only default the UI shows.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-12 10:42:42 +02:00
Jonathan d2c581d8e7 fix(discovery): use a valid ListenBrainz similar-artists algorithm
The placeholder algorithm string (contribution_5_threshold_15_limit_50) is not a
permitted enum member of the LB Labs endpoint — every lookup 400'd, making discovery
inert. Verified against the live API 2026-07-12: the days_7500/contribution_3/
threshold_10/limit_100 variant returns rows shaped {artist_mbid,name,comment,score},
which the existing defensive parser already handles. Fixed in both the worker adapter
and the web seed-search client (kept in sync).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-12 00:54:33 +02:00
Jonathan a15f4ae8bb feat(discovery): build_similarity_sources + worker sweep/trigger wiring
Adds registry.build_similarity_sources() (always constructs
ListenBrainzSource; reachability is a per-run health() concern, not a
startup gate) and wires discovery into the worker's main loop: a
scheduled sweep gated on discover.enabled + DISCOVER_TICK_SECONDS, and
a one-shot discover.requested trigger mirroring the existing
scan.requested pattern.

Also hardens run_discovery's health-check pass: a source whose
health() raises is now logged and skipped instead of aborting the
whole sweep (accepted finding from the Task-3 review).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-12 00:22:34 +02:00
Jonathan 6f504a7c04 feat(discovery): ListenBrainz similarity source
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-12 00:17:58 +02:00
Jonathan 0338d8e119 feat(discovery): derive core-album suggestions for surfaced artists
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-12 00:13:40 +02:00
Jonathan ccf3fb59ab feat(discovery): run_discovery aggregates similar artists into suggestions
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-12 00:07:09 +02:00
Jonathan 889e33355f feat(discovery): SimilaritySource protocol + FakeSimilaritySource
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-12 00:03:45 +02:00
Jonathan 4c1d3c639e fix: scan backfills release type metadata onto pre-existing rows
MonitoredRelease rows created before an artist's discography was
populated (e.g. an owned album recorded before _populate_discography
ran) never got primaryType/secondaryTypes backfilled, wrongly
excluding them from the studio-only core view. _populate_discography
now DO UPDATEs the MB type metadata on conflict instead of DO NOTHING,
leaving monitored/state/currentQualityClass/firstGrabbedAt untouched.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-11 18:12:37 +02:00