A near-stalled Soulseek peer drives the EWMA speed toward zero, so
_eta_seconds returned int(remaining/speed) values far past int4 max. Writing
that to Job.downloadEtaSeconds (an integer column) raised "integer out of
range", which aborted the SHARED pipeline connection's transaction. The
on_progress except clause logged but never rolled back, so every subsequent
query cascaded "current transaction is aborted" and the whole job failed and
requeued (9 such failures / 24h observed in prod).
- Cap _eta_seconds at ~100h (359999s), well inside int4.
- Roll back the shared conn in on_progress's except so a failed progress
write can never poison the pipeline txn, honoring the existing docstring
promise that "a progress write must never fail the download".
- Regression test for the near-stall cap.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Confidence scoring read the artist from the immediate folder as "Artist - Album", so
common FLAC rip layouts — artist in a parent folder ("Rihanna\(2009) Rated R\…") or
buried ("2009 - Rihanna - Rated R") — parsed with no/garbage artist and scored below
the 0.7 gate. Real FLACs got filtered out and a clean-named MP3 (0.88) won, so Lyra
downloaded lossy when lossless was available (Rihanna "Rated R": 8 FLACs all <0.5).
_parse_search_responses now takes the requested artist and, when it appears anywhere
in the candidate's path (via _path_matches), sets it as the candidate artist so
confidence reflects reality. Especially helps blocklisted-artist fallbacks, whose
album-only results come from varied folder structures. 337 tests pass.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Peer selection ranked (free-slot, advertised-speed, short-queue), ignoring file
size. But Soulseek FLAC rips of one album vary ~2x in size (a 24-bit/bloated rip vs
a standard CD rip) and Lyra scores them the SAME quality class (the slskd search
exposes no bit-depth, so every FLAC is class 2). So a peer advertising high speed
but serving 40MB/track files would out-rank a smaller standard rip that actually
finishes sooner — then blow past the 60-min per-peer backstop, time out, fall
through, and restart at 0% (Linkin Park "Hybrid Theory" looping for hours).
Rank by estimated transfer time instead: album bytes ÷ advertised speed, after the
free-slot check. Prefers whichever peer FINISHES first — fast peers and/or smaller
standard-edition rips — with no quality loss (all class 2). Equal-size case still
reduces to speed order, so existing ranking behaviour is preserved.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Show a live "~Xm left" countdown on the download progress bar. The slskd download
loop already polls byte transfer every 3s; measure the actual throughput (EWMA-
smoothed bytes/sec), compute seconds-remaining from the album's total bytes, and
surface it. New nullable Job.downloadEtaSeconds (migration; web entrypoint runs
prisma migrate deploy), threaded through the on_progress callback (optional 2nd arg,
so other adapters/callers are unaffected — they report no ETA). API exposes it;
queue.tsx renders etaLabel() after the track count. Null when unknown (no sample yet
or a source that doesn't report bytes). Worker 330 tests, web 214 tests, both green;
verified live-rendered as "42% · 6/14 tracks · ~4m left".
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The Soulseek server silently drops searches containing certain blocklisted terms
(major-label artists / album titles — Adele, Rihanna, Beyoncé, "Lemonade", …),
returning 0 responses ("TimedOut"). The blocked term poisons the whole query even
though the OTHER field usually isn't blocked and the content is on the network
(e.g. searching "Beyoncé …" → 0, but "I Am… Sasha Fierce" alone → 250 with 136
real Beyoncé folders).
When the primary "{artist} {album}" search returns nothing, retry with each field
alone (normalized to fold accents/punctuation) and keep only results whose file
path still matches the dropped field (_keep_matching), so a same-titled album by a
different artist is never grabbed. Recovers blocklisted albums with a distinctive
title; safely yields nothing when the title is too generic (e.g. Adele "21") rather
than mis-grabbing. Fallback only fires on a 0-response primary, so the happy path is
unchanged.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Draining the queue via Soulseek was stalling: slow peers (~100KB/s) never finished a
full FLAC album within the fixed 10-min timeout, got cancelled at ~89%, and the
fall-through restarted from scratch on the next peer — while single-song folders named
like the album were also being tried.
- Stall detection: abandon a peer with no byte progress for ~90s (queued/dead) instead
of waiting the full timeout; a peer that IS progressing keeps its slot up to a 60-min
backstop, so a slow-but-working transfer can finish.
- Rank search candidates by peer: free upload slot, then speed, then shortest queue.
- Drop candidates with far fewer tracks than the release (single-song folders) up front,
so no download attempt is wasted on them.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Two compounding causes of 'many downloads of one album from different sources' in
slskd: (1) the slskd client raised on timeout/error WITHOUT cancelling the transfer
it enqueued, so as the pipeline fell through to other peers each left a transfer
running; (2) the incomplete-download fall-through was uncapped, grinding through
every candidate of a popular album (~200 Soulseek peers), amplified now that Qobuz
pacing pushes most jobs to Soulseek.
- SlskdClient.download now cancels its enqueued transfers (DELETE) on any abandon.
- The download loop caps at _MAX_DOWNLOAD_ATTEMPTS (6) candidates.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
streamrip reports progress only per concurrent track, so Job.downloadProgress
came solely from the file-count poller — 0% until every track landed, then a
jump to 100%. Patch streamrip.media.track.get_progress_callback to aggregate
per-track byte deltas over an estimated album total (avg started-track size ×
len(album.tracks)) into one smooth, generally-climbing 0..1 fraction routed
through the pipeline's on_progress. Capped at 0.99; the pipeline sets 1.0 on
completion. No signature changes — track count read from the resolved album.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
ListenBrainz surfaces producers/session players/band members (Max Martin,
Dominic Howard) that co-occur in listens but have no releases of their own —
they showed as bare rows with an empty page. _derive_albums now drops a
surfaced artist (deletes their pending suggestion) when their MB discography
has no core release-group (Album/Single/EP, no secondary type), catching both
empty discographies and soundtrack/production-only credits, while keeping real
singles/EP-only artists. On browse failure the suggestion is kept.
Test fake browser gains default_core so scoring tests keep surfacing candidates.
worker 249 tests.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Bruno Mars "24K Magic" track 1 repeatedly downloaded as a 0-byte file. The
worker log showed the real cause: streamrip's track download hit
HTTPException("got more than 100 headers") — Qobuz's CDN returns >100 response
headers for some tracks and Python's http.client rejects them at the default
_MAXHEADERS=100, so streamrip skips the track and leaves an empty placeholder.
- Raise http.client._MAXHEADERS to 1000 on import of the Qobuz adapter, so those
responses parse and the track downloads.
- _count_staged_audio now skips 0-byte files (a failed-track placeholder is not a
real track) — improves the download-progress estimate and stops counting dead
tracks. (A verify-level "reject 0-byte tracks" safety net is a follow-up under
#16 per-track duration matching — it needs the pipeline fakes to write real
files, deferred to avoid disproportionate churn here.)
worker 223 tests / 7-skip.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
SlskdClient enqueued the transfer and polled to Completed but never moved any
files into `dest` — slskd saves them to its OWN downloads dir, so staging stayed
empty and every Soulseek job failed "incomplete download". Qobuz/YouTube write
straight into staging and were unaffected.
- SlskdClient._retrieve() moves the finished files out of slskd's downloads dir
into staging after completion, matching tolerantly by basename and preferring
a parent-folder + size match (slskd's on-disk layout varies by version).
- download() returns the moved count as track_count so the pipeline's
completeness check reflects what actually landed; raises loudly if nothing was
found (misconfigured/unmounted downloads dir).
- Downloads dir resolves slskd.downloadsDir Config > SLSKD_DOWNLOADS_ROOT env >
/slskd-downloads default; docker-compose mounts ${SLSKD_DOWNLOADS_DIR} there.
- .env.example documents SLSKD_DOWNLOADS_DIR; retrieval unit-tested offline.
slskd runs as an external daemon, so the shared dir must be a host path both
slskd and the worker mount — set once Lyra runs on the slskd host. Not yet
verified live end-to-end (no slskd access from this VM).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Each search/download called asyncio.run, creating and closing a fresh event loop.
streamrip's module-global download-concurrency Semaphore binds to the loop on first
use, so the 2nd+ download reused a Semaphore bound to a closed loop -> "Semaphore is
bound to a different event loop", tracks failed, album arrived short -> "incomplete
download" / needs_attention. Run all streamrip coroutines on one persistent loop.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Adds the AudioProbe seam (probe.py, _probe.py MutagenProbe with a lazy
mutagen import, FakeAudioProbe) and scan_library, which walks /music,
resolves each album against MusicBrainz (falling back to embedded tags
when the folder name doesn't resolve), and idempotently records matched
albums as have (LibraryItem), monitored (MonitoredRelease, fulfilled),
and followed (WatchedArtist, autoMonitorFuture=false).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- worker had no IPv6 route -> Qobuz CDN (dual-stack) failed ~half the tracks
with 'Network is unreachable'; disable IPv6 in the container (IPv4 only).
- StreamripClient counted metadata tracks (always full) -> a partial download
falsely imported; now count the actual files written (short count -> needs_attention).
- streamrip writes into a nested 'Artist - Album [..]/' folder the tagger never saw;
flatten audio files up into the album dir so tagging/organization runs.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Qobuz's email/password API login returns 401 even with valid credentials
(a known Qobuz limitation, unaffected by streamrip version). Add settings
fields for a Qobuz user ID + auth token (token stored encrypted); when both
are present StreamripClient uses use_auth_token=True, else falls back to
email/password.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
streamrip's email/password login expects the MD5 digest, not plaintext
(caused 'Invalid credentials'). Also wrap login() so streamrip's exception —
which embeds the email/password/app_id — can never reach the worker logs;
re-raise a clean credential-free RuntimeError instead.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Jonathan