Reconciled the pre-existing test_config_from_config_parses_and_defaults
test in test_discovery.py, which referenced max_seeds/maxSeeds and would
otherwise have broken.
Adds a crash-recovery preamble that restores `final` from `.old` when a
prior swap was interrupted after rename(final, old) but before
rename(tmp, final), and always drops stale `.old`/`.importing` orphans.
Also guards the swap: if nothing new was staged into tmp (empty/missing
staging) and `final` already exists, leave it untouched instead of
swapping in an empty copy. This is required beyond the literal preamble
because the pre-existing swap logic unconditionally replaced `final`
with `tmp` regardless of whether anything was actually imported, which
would otherwise destroy a just-recovered (or any existing) copy whenever
staging is empty.
Staging is no longer forced under the library. The worker reads STAGING_ROOT
(container path, default /music/.staging) and stages per-job downloads there;
docker-compose mounts ${STAGING_DIR:-${MUSIC_DIR}/.staging} at /staging with
STAGING_ROOT=/staging. Set STAGING_DIR to a fast local disk when the library
(MUSIC_DIR) is a network share so temp download I/O stays off the share.
Safe cross-volume: import_album already assembles into a temp dir on the
library volume and swaps atomically there, so partial downloads never touch
the library and the final swap stays atomic. clear_staging_root now clears the
root's contents (mount-safe) rather than removing the root. Documented in
.env.example; new tests cover the separate-root path.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Wanted releases pressed while the monitor is off never got reconciled, so
they lingered on the wanted list. Extract finish_job() and call it for every
finished job (not just when monitor.enabled) so the MonitoredRelease
transitions to fulfilled. Add fulfill_owned_releases(), run once at startup,
to backfill releases already owned at/above the quality cutoff (self-heals
items stuck from before this fix). Both idempotent + tested.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Two worker-loop robustness fixes:
- Scan chunking: maybe_run_scan advances the library scan by at most one
scan_chunk (default scan.chunkSize=25 albums) per loop iteration, so a
large library no longer blocks job-claim/monitor for minutes-to-hours.
Progress persists in scan.inProgress/scan.cursor/scan.progress and is
resumable; scan.result (unchanged format) is written on completion.
- DB reconnect: wrap the loop body in `except psycopg.OperationalError`
(AdminShutdown subclasses it) to close the dead handle and reconnect via
wait_for_db() in-process, instead of crashing and relying on the
container restart policy. Verified via a backend-termination test.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
scan_chunk resolves album folders whose (artist, album) sorts after a
cursor, up to a limit, returning (new_cursor, imported, skipped, done).
scan_library becomes the unbounded case (limit=None), so its behavior and
tests are unchanged. Idempotent: all writes are ON CONFLICT, so an
overlapping/repeated cursor is safe. Enables chunked scanning in the loop.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The scheduled-tick branch and the discover.requested one-shot branch
both read the same stale per-iteration config snapshot, so an enabled+due
sweep coinciding with a pending request ran discovery twice back-to-back
(idempotent but wasteful). Extract maybe_run_discovery, which unifies both
triggers into a single run and is now unit-tested.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
autoMonitorFuture flagged any new release, so it could auto-monitor a
newly-released live album or compilation. Gate it on is_core_release so
auto-monitoring matches the studio-only default the UI shows.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Adds registry.build_similarity_sources() (always constructs
ListenBrainzSource; reachability is a per-run health() concern, not a
startup gate) and wires discovery into the worker's main loop: a
scheduled sweep gated on discover.enabled + DISCOVER_TICK_SECONDS, and
a one-shot discover.requested trigger mirroring the existing
scan.requested pattern.
Also hardens run_discovery's health-check pass: a source whose
health() raises is now logged and skipped instead of aborting the
whole sweep (accepted finding from the Task-3 review).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
MonitoredRelease rows created before an artist's discography was
populated (e.g. an owned album recorded before _populate_discography
ran) never got primaryType/secondaryTypes backfilled, wrongly
excluding them from the studio-only core view. _populate_discography
now DO UPDATEs the MB type metadata on conflict instead of DO NOTHING,
leaving monitored/state/currentQualityClass/firstGrabbedAt untouched.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Adds the per-artist release-type filter toggle: off (default) restricts
a discography view to core studio releases, on shows everything
(live, compilations, etc.).
A followed artist previously showed only the one owned album ("of 1")
because scan_library recorded just the resolved release. Now scan
browses each artist's full MusicBrainz release-group list once per
run and stores it as unmonitored MonitoredRelease rows, matching
web-follow behavior. The owned album keeps its monitored/fulfilled
state via ON CONFLICT DO NOTHING.
Probe each album exactly once and thread the ProbeResult through for
tags, quality, and format instead of hardcoding LibraryItem.format to
"FLAC" and re-probing on the tag-fallback path. Also skip album-level
directories starting with "." (previously only artist-level dirs were
skipped).
Adds the AudioProbe seam (probe.py, _probe.py MutagenProbe with a lazy
mutagen import, FakeAudioProbe) and scan_library, which walks /music,
resolves each album against MusicBrainz (falling back to embedded tags
when the folder name doesn't resolve), and idempotently records matched
albums as have (LibraryItem), monitored (MonitoredRelease, fulfilled),
and followed (WatchedArtist, autoMonitorFuture=false).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Test suites were running DELETE FROM.../deleteMany() against whatever
DATABASE_URL pointed to, which was the live app database, wiping user
credentials and library. Add a guard in both the worker conftest and
web test setup that raises before any cleanup unless the target DB
name ends in '_test'. Default DATABASE_URL to lyra_test when unset in
the worker, and make `npm test` target lyra_test explicitly.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Rewrite run_pipeline's download->import section to download into a per-job
staging dir (cleared before each candidate), verify completeness there,
promote only a complete album into the library via import_album, tag the
final dir, then DB-import. The staging dir is always removed in a
try/finally, so a partial/failed download never touches or corrupts the
library, and a retry can no longer leave orphan files behind.
Also point test_upgrade_e2e.py's run_pipeline call at dest_root="/tmp/lib"
(matching every other pipeline test) since import_album now unconditionally
creates the final dir, which requires a writable dest_root.
The monitor's quality-upgrade path was inert: run_pipeline's intake
short-circuit treated any existing LibraryItem as "done", so a
monitor-driven upgrade job for a below-cutoff copy jumped straight to
imported without downloading. Intake now checks whether the job is
monitor-driven (Request.monitoredReleaseId IS NOT NULL) and, if so,
only short-circuits when the existing copy already meets the quality
cutoff; plain manual requests keep exact slice-1 behavior. Also drops
an unused `field` import in browser.py.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- worker had no IPv6 route -> Qobuz CDN (dual-stack) failed ~half the tracks
with 'Network is unreachable'; disable IPv6 in the container (IPv4 only).
- StreamripClient counted metadata tracks (always full) -> a partial download
falsely imported; now count the actual files written (short count -> needs_attention).
- streamrip writes into a nested 'Artist - Album [..]/' folder the tagger never saw;
flatten audio files up into the album dir so tagging/organization runs.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Qobuz's email/password API login returns 401 even with valid credentials
(a known Qobuz limitation, unaffected by streamrip version). Add settings
fields for a Qobuz user ID + auth token (token stored encrypted); when both
are present StreamripClient uses use_auth_token=True, else falls back to
email/password.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
streamrip's email/password login expects the MD5 digest, not plaintext
(caused 'Invalid credentials'). Also wrap login() so streamrip's exception —
which embeds the email/password/app_id — can never reach the worker logs;
re-raise a clean credential-free RuntimeError instead.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Wrap each adapter's search() call in the match stage in try/except so
a down source (dead slskd, Qobuz auth error, etc.) contributes no
candidates instead of crashing run_pipeline and killing the worker.
Jonathan