Live-testing #9 surfaced a pre-existing gap: one truncated file mutagen chokes
on ("file said 4 bytes, read 0 bytes") raised out of _process_album and aborted
the entire library scan (worklist abandoned). Now scan_chunk catches a per-album
error, rolls back, logs it, counts it as skipped, marks the item done, and
continues — so a single bad file can't block the rest of the library.
worker tests: new case asserts a corrupt album is skipped while a good one still
imports (and captures its on-disk trackNames).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The Library modal showed MusicBrainz's canonical tracklist, hiding incomplete
downloads / edition or track-order mismatches, and showed nothing for albums
without an rgMbid. Now it shows the REAL files on disk.
- LibraryItem gains trackNames String[] (migration add_library_track_names):
the album's on-disk "## Title.ext" audio filenames.
- Captured at import (pipeline._import lists the final album folder) and at scan
(scan._record_owned now also refreshes trackNames on re-scan via ON CONFLICT
DO UPDATE + xmax=0 to preserve the newly-imported flag). New shared
library.list_audio_files() helper.
- /api/library exposes trackNames; the AlbumModal prefers on-disk tracks when
present (parsed "## Title" → position/title, zero network, labeled "On disk ·
N tracks"), falling back to the MusicBrainz listing otherwise. Items imported
before this column show the MB fallback until re-scanned.
worker 221 tests / 7-skip, web 153, tsc + build clean.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
A worker crash/restart mid-pipeline left its job stuck in matching/matched/
downloading/tagging with no auto-recovery — only the manual Retry (needs_attention
only) could rescue it. Now on startup the worker resets any such orphaned job
back to 'requested' so it gets re-claimed.
Safe at startup: this fresh worker owns none of those jobs, staging is already
cleared (clear_staging_root runs first), and the pipeline redoes intake→import
idempotently with an atomic import. Terminal jobs (imported/needs_attention) are
left untouched; claimedAt/error/downloadProgress are reset. Logged on startup.
worker 218 tests / 7-skip (in-flight→requested, terminal untouched, re-claimable).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The Floor bar used a file-count poller that jumps 0→~99% for small/fast albums
(streamrip reports only 0/1). But yt-dlp and slskd already emit real byte-level
progress via the on_progress callback the pipeline was discarding
(lambda _pct: None).
- Thread a throttled on_progress → Job.downloadProgress (write on a >=1% move or
every 0.5s; clamped 0..1; write failures swallowed). Called synchronously from
adapter.download on the pipeline thread, so it reuses the main conn safely.
- The file-count poller stays as the FALLBACK: it skips writing while a
`reports` flag is set (an adapter is driving real progress), and the flag is
cleared per attempt so a fall-through to streamrip still gets the estimate.
- Smooth bars for YouTube + Soulseek now; Qobuz/streamrip keeps the file-count
estimate until its per-track callback is hooked later.
worker 216 tests / 7-skip (on_progress write/throttle/clamp covered; existing
pipeline tests now exercise the fake's on_progress end-to-end).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The worker's MB browser (discography + artist search, used by monitor sweeps,
library scans, and Last.fm discovery) re-fetched every time. Route it through
the same ApiCache table + logical keys the web app uses, so a discography one
process fetched warms the other, and repeat lookups stop hitting MB.
- apicache.py cached_api(): read-through, serve-stale-on-outage, never caches
null, best-effort (a cache-DB error degrades to a live fetch, never breaks
scan/sweep/discovery). Its own DEDICATED autocommit connection so it can't
commit the worker's in-flight transaction, and autocommit avoids freezing
Postgres now() (which would break TTL math).
- CachingMbBrowser decorates MusicBrainzBrowser, serializing to the SAME
camelCase JSON shape the web writes (round-trip tested). registry.build_browser
wraps it when a DSN is present; the Last.fm source's browser rides it too.
- Hourly prune drops ApiCache rows > 90d so the table stays bounded.
- worker 213 tests / 7-skip.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
.env.example shipped a valid, PUBLIC base64 key — if a user didn't replace it,
every credential got encrypted with a publicly-known key and everything still
"worked," so the footgun was silent.
- .env.example now ships an obvious placeholder (CHANGE_ME_generate_with_...)
with a "you MUST replace this" note and the openssl hint.
- web + worker validate LYRA_SECRET_KEY at startup: missing / placeholder /
wrong-length ⇒ loud FATAL banner and refuse to start (web via the Next.js
instrumentation hook, worker via _require_secret_key before touching the DB).
The old public example key ⇒ a loud WARNING but not fatal, since an existing
install may have encrypted its creds under it (rotating needs re-entry).
- Validation lives in an edge-safe web/src/lib/secret-key.ts (no node:crypto)
so the instrumentation hook bundles for both runtimes; crypto.ts re-exports.
- README emphasizes generating a fresh key + keeping it stable.
Verified live: worker exits 1 on placeholder/unset before any DB call; web
instrumentation throws and refuses to serve on the placeholder. web 138 tests,
worker 204/7-skip, tsc clean.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
SlskdClient enqueued the transfer and polled to Completed but never moved any
files into `dest` — slskd saves them to its OWN downloads dir, so staging stayed
empty and every Soulseek job failed "incomplete download". Qobuz/YouTube write
straight into staging and were unaffected.
- SlskdClient._retrieve() moves the finished files out of slskd's downloads dir
into staging after completion, matching tolerantly by basename and preferring
a parent-folder + size match (slskd's on-disk layout varies by version).
- download() returns the moved count as track_count so the pipeline's
completeness check reflects what actually landed; raises loudly if nothing was
found (misconfigured/unmounted downloads dir).
- Downloads dir resolves slskd.downloadsDir Config > SLSKD_DOWNLOADS_ROOT env >
/slskd-downloads default; docker-compose mounts ${SLSKD_DOWNLOADS_DIR} there.
- .env.example documents SLSKD_DOWNLOADS_DIR; retrieval unit-tested offline.
slskd runs as an external daemon, so the shared dir must be a host path both
slskd and the worker mount — set once Lyra runs on the slskd host. Not yet
verified live end-to-end (no slskd access from this VM).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Add Job.downloadProgress (0.0-1.0), derived from a worker background
poller that counts audio files landing in staging / expected track
count, since streamrip only reports 0/1. No UI yet (slice B).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Each search/download called asyncio.run, creating and closing a fresh event loop.
streamrip's module-global download-concurrency Semaphore binds to the loop on first
use, so the 2nd+ download reused a Semaphore bound to a closed loop -> "Semaphore is
bound to a different event loop", tracks failed, album arrived short -> "incomplete
download" / needs_attention. Run all streamrip coroutines on one persistent loop.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
_mark_chosen ran only after a download succeeded, so throughout the 'downloading'
state no candidate was chosen — the Floor row's "source · format · tracks" line was
blank (only elapsed showed). Now mark the candidate exclusively at each download
attempt so `chosen` reflects the source in flight (and updates on fall-through).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Adds LastfmSource, resolving Last.fm's often-empty artist mbid via the
injected MB browser (name->mbid cached) so every emitted SimilarArtist.mbid
is a real MusicBrainz artist MBID. Registered in build_similarity_sources,
gated on lastfm.api_key so the ListenBrainz-only path is unchanged when
unset.
Two row-leak bugs found in whole-branch review:
- discovery.run_discovery now prunes DiscoverySeedContribution rows whose
seedMbid is no longer a WatchedArtist (unfollow/removal has no FK cascade)
and recomputes the touched candidates' scores this sweep, instead of
letting a dead seed's contribution inflate the aggregate forever.
- maybe_run_scan's exception handler around scan_chunk now calls
clear_worklist and clears scan.id, matching the drain path. Previously
an exception mid-chunk left the already-done rows plus the remaining
worklist for that scan_id orphaned forever, since the next scan mints a
fresh uuid. Also clear scan.id in the build_worklist failure handler for
consistency.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Replace scan_chunk's O(N^2) tree re-walk-and-skip-to-cursor with a
persisted ScanWorkItem worklist: build_worklist walks the tree once at
scan start and inserts one row per album (idempotent via ON CONFLICT),
scan_chunk pops up to `limit` not-done rows by indexed cursor and marks
them done, and clear_worklist drops the scan's rows once drained.
scan_library is reimplemented on top of the worklist (one-shot scan_id,
drain, clear) with an unchanged signature/behavior. main.py's
maybe_run_scan now drives a generated scan.id through Config
(scan.id/scan.inProgress/scan.progress/scan.result) instead of a
scan.cursor string, so a mid-scan library change no longer risks
skipping or re-walking albums.
Reconciled the pre-existing test_config_from_config_parses_and_defaults
test in test_discovery.py, which referenced max_seeds/maxSeeds and would
otherwise have broken.
Adds a crash-recovery preamble that restores `final` from `.old` when a
prior swap was interrupted after rename(final, old) but before
rename(tmp, final), and always drops stale `.old`/`.importing` orphans.
Also guards the swap: if nothing new was staged into tmp (empty/missing
staging) and `final` already exists, leave it untouched instead of
swapping in an empty copy. This is required beyond the literal preamble
because the pre-existing swap logic unconditionally replaced `final`
with `tmp` regardless of whether anything was actually imported, which
would otherwise destroy a just-recovered (or any existing) copy whenever
staging is empty.
Staging is no longer forced under the library. The worker reads STAGING_ROOT
(container path, default /music/.staging) and stages per-job downloads there;
docker-compose mounts ${STAGING_DIR:-${MUSIC_DIR}/.staging} at /staging with
STAGING_ROOT=/staging. Set STAGING_DIR to a fast local disk when the library
(MUSIC_DIR) is a network share so temp download I/O stays off the share.
Safe cross-volume: import_album already assembles into a temp dir on the
library volume and swaps atomically there, so partial downloads never touch
the library and the final swap stays atomic. clear_staging_root now clears the
root's contents (mount-safe) rather than removing the root. Documented in
.env.example; new tests cover the separate-root path.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- Worker stamps a throttled worker.heartbeat; GET /api/worker reports online
from its updatedAt recency (DB-time); masthead shows a live listening/offline
dot instead of a static decoration.
- /api/requests enriched with rgMbid; Recently Pressed shows cover thumbs.
- Library album count moved from the filter row into a section header.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Wanted releases pressed while the monitor is off never got reconciled, so
they lingered on the wanted list. Extract finish_job() and call it for every
finished job (not just when monitor.enabled) so the MonitoredRelease
transitions to fulfilled. Add fulfill_owned_releases(), run once at startup,
to backfill releases already owned at/above the quality cutoff (self-heals
items stuck from before this fix). Both idempotent + tested.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Two worker-loop robustness fixes:
- Scan chunking: maybe_run_scan advances the library scan by at most one
scan_chunk (default scan.chunkSize=25 albums) per loop iteration, so a
large library no longer blocks job-claim/monitor for minutes-to-hours.
Progress persists in scan.inProgress/scan.cursor/scan.progress and is
resumable; scan.result (unchanged format) is written on completion.
- DB reconnect: wrap the loop body in `except psycopg.OperationalError`
(AdminShutdown subclasses it) to close the dead handle and reconnect via
wait_for_db() in-process, instead of crashing and relying on the
container restart policy. Verified via a backend-termination test.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
scan_chunk resolves album folders whose (artist, album) sorts after a
cursor, up to a limit, returning (new_cursor, imported, skipped, done).
scan_library becomes the unbounded case (limit=None), so its behavior and
tests are unchanged. Idempotent: all writes are ON CONFLICT, so an
overlapping/repeated cursor is safe. Enables chunked scanning in the loop.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The scheduled-tick branch and the discover.requested one-shot branch
both read the same stale per-iteration config snapshot, so an enabled+due
sweep coinciding with a pending request ran discovery twice back-to-back
(idempotent but wasteful). Extract maybe_run_discovery, which unifies both
triggers into a single run and is now unit-tested.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
autoMonitorFuture flagged any new release, so it could auto-monitor a
newly-released live album or compilation. Gate it on is_core_release so
auto-monitoring matches the studio-only default the UI shows.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The placeholder algorithm string (contribution_5_threshold_15_limit_50) is not a
permitted enum member of the LB Labs endpoint — every lookup 400'd, making discovery
inert. Verified against the live API 2026-07-12: the days_7500/contribution_3/
threshold_10/limit_100 variant returns rows shaped {artist_mbid,name,comment,score},
which the existing defensive parser already handles. Fixed in both the worker adapter
and the web seed-search client (kept in sync).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Adds registry.build_similarity_sources() (always constructs
ListenBrainzSource; reachability is a per-run health() concern, not a
startup gate) and wires discovery into the worker's main loop: a
scheduled sweep gated on discover.enabled + DISCOVER_TICK_SECONDS, and
a one-shot discover.requested trigger mirroring the existing
scan.requested pattern.
Also hardens run_discovery's health-check pass: a source whose
health() raises is now logged and skipped instead of aborting the
whole sweep (accepted finding from the Task-3 review).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
MonitoredRelease rows created before an artist's discography was
populated (e.g. an owned album recorded before _populate_discography
ran) never got primaryType/secondaryTypes backfilled, wrongly
excluding them from the studio-only core view. _populate_discography
now DO UPDATEs the MB type metadata on conflict instead of DO NOTHING,
leaving monitored/state/currentQualityClass/firstGrabbedAt untouched.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
A followed artist previously showed only the one owned album ("of 1")
because scan_library recorded just the resolved release. Now scan
browses each artist's full MusicBrainz release-group list once per
run and stores it as unmonitored MonitoredRelease rows, matching
web-follow behavior. The owned album keeps its monitored/fulfilled
state via ON CONFLICT DO NOTHING.
Probe each album exactly once and thread the ProbeResult through for
tags, quality, and format instead of hardcoding LibraryItem.format to
"FLAC" and re-probing on the tag-fallback path. Also skip album-level
directories starting with "." (previously only artist-level dirs were
skipped).
Adds the AudioProbe seam (probe.py, _probe.py MutagenProbe with a lazy
mutagen import, FakeAudioProbe) and scan_library, which walks /music,
resolves each album against MusicBrainz (falling back to embedded tags
when the folder name doesn't resolve), and idempotently records matched
albums as have (LibraryItem), monitored (MonitoredRelease, fulfilled),
and followed (WatchedArtist, autoMonitorFuture=false).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Rewrite run_pipeline's download->import section to download into a per-job
staging dir (cleared before each candidate), verify completeness there,
promote only a complete album into the library via import_album, tag the
final dir, then DB-import. The staging dir is always removed in a
try/finally, so a partial/failed download never touches or corrupts the
library, and a retry can no longer leave orphan files behind.
Also point test_upgrade_e2e.py's run_pipeline call at dest_root="/tmp/lib"
(matching every other pipeline test) since import_album now unconditionally
creates the final dir, which requires a writable dest_root.
The monitor's quality-upgrade path was inert: run_pipeline's intake
short-circuit treated any existing LibraryItem as "done", so a
monitor-driven upgrade job for a below-cutoff copy jumped straight to
imported without downloading. Intake now checks whether the job is
monitor-driven (Request.monitoredReleaseId IS NOT NULL) and, if so,
only short-circuits when the existing copy already meets the quality
cutoff; plain manual requests keep exact slice-1 behavior. Also drops
an unused `field` import in browser.py.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- worker had no IPv6 route -> Qobuz CDN (dual-stack) failed ~half the tracks
with 'Network is unreachable'; disable IPv6 in the container (IPv4 only).
- StreamripClient counted metadata tracks (always full) -> a partial download
falsely imported; now count the actual files written (short count -> needs_attention).
- streamrip writes into a nested 'Artist - Album [..]/' folder the tagger never saw;
flatten audio files up into the album dir so tagging/organization runs.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Qobuz's email/password API login returns 401 even with valid credentials
(a known Qobuz limitation, unaffected by streamrip version). Add settings
fields for a Qobuz user ID + auth token (token stored encrypted); when both
are present StreamripClient uses use_auth_token=True, else falls back to
email/password.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
streamrip's email/password login expects the MD5 digest, not plaintext
(caused 'Invalid credentials'). Also wrap login() so streamrip's exception —
which embeds the email/password/app_id — can never reach the worker logs;
re-raise a clean credential-free RuntimeError instead.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Jonathan