Jonathan/Lyra
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Compare commits

base: Jonathan/Lyra:a94ba97eee08b0e591dd0c851d6d26f64642ebab
Branches Tags
Jonathan/Lyra:main Jonathan/Lyra:fix/slskd-resume-download Jonathan/Lyra:fix/slskd-large-album-backstop Jonathan/Lyra:worktree-fix-eta-overflow
...
compare: Jonathan/Lyra:main
Branches Tags
Jonathan/Lyra:fix/slskd-resume-download Jonathan/Lyra:fix/slskd-large-album-backstop Jonathan/Lyra:main Jonathan/Lyra:worktree-fix-eta-overflow

80 Commits
a94ba97eee ... main

Author SHA1 Message Date
Jonathan 395944f3e1 fix(worker): scale slskd download backstop to album size (stop abandoning big albums) 
The absolute download backstop was a flat _MAX_XFER_POLLS=1200 (~60min). A large
album (e.g. Drake "Scorpion", 521MB / 25 FLACs) from a slow-but-healthy serial
peer (~400KB/s, serving one file at a time) needs ~60-70min, so it was cut off
at 60min and — because an abandoned transfer is cancelled and re-enqueued from
scratch — re-downloaded from zero every attempt, never finishing and eventually
going needs_attention at the attempt cap.

Replace the flat cap with _max_polls(total_bytes): budget the backstop at a
conservative floor throughput (~64KB/s), clamped to [20min, 2h]. Scorpion now
gets the full 2h ceiling instead of 60min. The 90s stall timeout is unchanged,
so a truly dead/queued peer is still abandoned fast.

Follow-up (not in this change): resume across retries (skip files slskd already
completed) so an abandoned large download doesn't restart from zero.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
 2026-07-21 23:27:44 +02:00
Jonathan 10430037fe fix(worker): cap slskd ETA + rollback on progress-write failure (stop txn-abort job churn) 
A near-stalled Soulseek peer drives the EWMA speed toward zero, so
_eta_seconds returned int(remaining/speed) values far past int4 max. Writing
that to Job.downloadEtaSeconds (an integer column) raised "integer out of
range", which aborted the SHARED pipeline connection's transaction. The
on_progress except clause logged but never rolled back, so every subsequent
query cascaded "current transaction is aborted" and the whole job failed and
requeued (9 such failures / 24h observed in prod).

- Cap _eta_seconds at ~100h (359999s), well inside int4.
- Roll back the shared conn in on_progress's except so a failed progress
  write can never poison the pipeline txn, honoring the existing docstring
  promise that "a progress write must never fail the download".
- Regression test for the near-stall cap.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
 2026-07-21 20:42:13 +02:00
Jonathan 01657d4e1b fix(web): order Recently Pressed by press time + cap at 10 
Recently Pressed was derived from the request list (ordered by Request.createdAt), so
an album requested long ago but imported just now sorted to the bottom — freshly
pressed albums never appeared near the top ("new albums don't get added"). Order by
press time (job.updatedAt ≈ import time) instead, and show only the 10 most recent.
The "Pressed" stat tile and the "N in library" note still reflect the true total; the
per-row date label now shows when it was pressed, not requested.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
 2026-07-20 23:17:54 +02:00
Jonathan 0ad1fd4cd5 fix(worker): trust the requested artist when it's in the file path (stop MP3-over-FLAC) 
Confidence scoring read the artist from the immediate folder as "Artist - Album", so
common FLAC rip layouts — artist in a parent folder ("Rihanna\(2009) Rated R\…") or
buried ("2009 - Rihanna - Rated R") — parsed with no/garbage artist and scored below
the 0.7 gate. Real FLACs got filtered out and a clean-named MP3 (0.88) won, so Lyra
downloaded lossy when lossless was available (Rihanna "Rated R": 8 FLACs all <0.5).

_parse_search_responses now takes the requested artist and, when it appears anywhere
in the candidate's path (via _path_matches), sets it as the candidate artist so
confidence reflects reality. Especially helps blocklisted-artist fallbacks, whose
album-only results come from varied folder structures. 337 tests pass.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
 2026-07-20 23:05:38 +02:00
Jonathan 7ed8ca2848 fix(worker): prefer the original release-group over same-titled reissues 
The real root cause of "Hybrid Theory" re-downloading forever: MB lists the 2000
original and a 2023 reissue as SEPARATE same-titled Album release-groups. They tie on
(artist, title, is_album), so _best_release_group kept whichever MB relevance-ordered
first — the 2023 reissue (13 tracks / ~52 min). Its inflated tracklist+runtime then
failed the completeness/duration checks against the standard 12-track album sources
deliver, looping endlessly. Add an earliest-first-release-date tie-break so the
original group wins. (Complements _pick_release, which handles reissue releases
*within* a group.) 335 tests pass.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
 2026-07-20 22:32:25 +02:00
Jonathan 5f59283bf0 fix(worker): resolve MB to the original edition + loosen duration gate 
Root cause of albums re-downloading forever despite completing (Linkin Park "Hybrid
Theory"): the MB resolver read the tracklist from an arbitrary releases[0], often a
later reissue/deluxe. HT resolved to a 2023 13-track reissue (~52 min); a complete
2000 12-track download (~38 min) then failed _download_problem's duration gate
(2267s < 2877s×0.85) and fell through peer after peer, never importing.

- _pick_release: choose the ORIGINAL standard edition (Official, earliest date)
  instead of releases[0], so completeness/duration are judged against the edition
  sources actually deliver.
- _DURATION_MIN_RATIO 0.85 -> 0.65: a source legitimately delivering a shorter
  edition than MB's release shouldn't be rejected; only genuinely truncated/preview
  downloads (a fraction of expected runtime) should be.

Together with the est-time peer ranking, this stops the download→reject→re-download
loop. 334 tests pass.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
 2026-07-20 22:27:44 +02:00
Jonathan d2a86932e0 fix(worker): rank Soulseek peers by estimated transfer time, not raw speed 
Peer selection ranked (free-slot, advertised-speed, short-queue), ignoring file
size. But Soulseek FLAC rips of one album vary ~2x in size (a 24-bit/bloated rip vs
a standard CD rip) and Lyra scores them the SAME quality class (the slskd search
exposes no bit-depth, so every FLAC is class 2). So a peer advertising high speed
but serving 40MB/track files would out-rank a smaller standard rip that actually
finishes sooner — then blow past the 60-min per-peer backstop, time out, fall
through, and restart at 0% (Linkin Park "Hybrid Theory" looping for hours).

Rank by estimated transfer time instead: album bytes ÷ advertised speed, after the
free-slot check. Prefers whichever peer FINISHES first — fast peers and/or smaller
standard-edition rips — with no quality loss (all class 2). Equal-size case still
reduces to speed order, so existing ranking behaviour is preserved.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
 2026-07-20 22:17:44 +02:00
Jonathan 73f1ae6e2f feat(web): order the In-queue tab next-up-first with a "Next up" marker 
The queue tab showed newest-first (API returns createdAt desc) but the worker claims
the oldest queued job next (claim_next: ORDER BY createdAt ASC) — so the item that ran
next was at the BOTTOM. Sort the queue bucket ascending so it matches processing
order, and badge the first claimable row (oldest, not individually paused) as "Next up".

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
 2026-07-20 20:56:42 +02:00
Jonathan c49cee9a20 feat: download ETA on the press bar from measured Soulseek throughput 
Show a live "~Xm left" countdown on the download progress bar. The slskd download
loop already polls byte transfer every 3s; measure the actual throughput (EWMA-
smoothed bytes/sec), compute seconds-remaining from the album's total bytes, and
surface it. New nullable Job.downloadEtaSeconds (migration; web entrypoint runs
prisma migrate deploy), threaded through the on_progress callback (optional 2nd arg,
so other adapters/callers are unaffected — they report no ETA). API exposes it;
queue.tsx renders etaLabel() after the track count. Null when unknown (no sample yet
or a source that doesn't report bytes). Worker 330 tests, web 214 tests, both green;
verified live-rendered as "42% · 6/14 tracks · ~4m left".

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
 2026-07-20 20:16:58 +02:00
Jonathan db8b43b0e8 feat(web): tabbed On-the-Press view + kebab menu for press actions 
The press page interleaved active downloads, queued items, and errors in one long
list — hard to see what's actually pressing. Split the non-imported rows into three
tabs (Active / In queue / Errors) with live counts; the default tab follows the work
(Active if anything's running, else Queue, else Errors) until the user picks one.
Move the press-level actions (Pause/Resume, Clear queue, Stop now) off the toolbar
into a vertical "⋮" kebab menu on the right of the header — Clear queue disables when
the queue is empty, Stop now only shows while downloading.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
 2026-07-20 19:41:11 +02:00
Jonathan 434cd2b22c feat(worker): recover blocklisted Soulseek searches via album/artist-only fallback 
The Soulseek server silently drops searches containing certain blocklisted terms
(major-label artists / album titles — Adele, Rihanna, Beyoncé, "Lemonade", …),
returning 0 responses ("TimedOut"). The blocked term poisons the whole query even
though the OTHER field usually isn't blocked and the content is on the network
(e.g. searching "Beyoncé …" → 0, but "I Am… Sasha Fierce" alone → 250 with 136
real Beyoncé folders).

When the primary "{artist} {album}" search returns nothing, retry with each field
alone (normalized to fold accents/punctuation) and keep only results whose file
path still matches the dropped field (_keep_matching), so a same-titled album by a
different artist is never grabbed. Recovers blocklisted albums with a distinctive
title; safely yields nothing when the title is too generic (e.g. Adele "21") rather
than mis-grabbing. Fallback only fires on a 0-response primary, so the happy path is
unchanged.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
 2026-07-20 19:06:07 +02:00
Jonathan d7ea29a97a fix(worker): requeue jobs that fail on an unhandled pipeline exception 
An unhandled exception in run_pipeline (e.g. a MusicBrainz TLS drop surfacing as
SSL: UNEXPECTED_EOF, after the MB retries are exhausted) left the job stranded in
its mid-pipeline 'matching' state: the except handler logged "pipeline failed" and
rolled back, but never reset the job. Since claim_next only picks 'requested', the
job was orphaned — stuck showing "Searching sources…" on the press until the next
worker restart's reclaim_stuck_jobs happened to sweep it up.

Add requeue_or_fail: on an unhandled failure, requeue to 'requested' for a bounded
retry (transient network blips clear on retry), then fall to 'needs_attention' at
the attempt cap (5) so a persistently-broken job stays visible instead of looping.
Mirrors reclaim_stuck_jobs (requeue) and pipeline._fail (needs_attention), and
resets Request.status to match.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
 2026-07-20 18:40:41 +02:00
Jonathan e8483ee22b fix(worker): match library ownership by rgMbid + retire stale failed attempts 
Two related monitor bugs surfaced by owned albums that never leave the press:

1. Ownership was matched on (artist, album) strings, but MusicBrainz canonicalizes
   the artist on import ("Kanye West"->"Ye", "Ne-Yo"->"Ne‐Yo", "The Goo Goo Dolls"
   ->"Goo Goo Dolls"). The strings no longer matched, so reconcile/enqueue/backfill
   never recognized the album as owned — it stayed "wanted" and re-pressed every
   retry interval. Match on the release-group MBID when the library row has one
   (falling back to artist+album for scanned rows without an MBID) across
   _in_library_at_cutoff, fulfill_owned_releases, and reconcile.

2. Each failed attempt left a needs_attention Request behind, so a hard-to-match
   album piled up phantom "needs attention" cards on the press even after a later
   attempt imported it (e.g. Maroon 5 "V": 3 stale cards + 1 success). Retire prior
   failed attempts when a release is fulfilled or re-attempted (_retire_failed_attempts,
   monitor-owned requests only — manual requests have no monitoredReleaseId).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
 2026-07-20 17:04:31 +02:00
Jonathan ca5bbfa9da fix(worker): retry transient MusicBrainz failures with backoff 
The matching stage makes up to three musicbrainzngs calls per job, none of
which retried. A dropped TLS connection (surfacing as NetworkError,
"<urlopen error [SSL: UNEXPECTED_EOF_WHILE_READING]>") or a 503 rate-limit
(ResponseError) failed the entire pipeline for the job, which then re-queued
and re-ran from scratch — ~36% of jobs churned this way over a 6h window.

Wrap the three MB calls in _with_retry (4 attempts, 1s/2s/4s exponential
backoff), catching only the two transient musicbrainzngs error types so real
bugs still propagate. musicbrainzngs stays lazily imported to keep module
import offline.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
 2026-07-20 16:45:37 +02:00
Jonathan 675d964bbe fix(worker): smarter Soulseek peer selection + stall/progress-aware transfers 
Draining the queue via Soulseek was stalling: slow peers (~100KB/s) never finished a
full FLAC album within the fixed 10-min timeout, got cancelled at ~89%, and the
fall-through restarted from scratch on the next peer — while single-song folders named
like the album were also being tried.

- Stall detection: abandon a peer with no byte progress for ~90s (queued/dead) instead
  of waiting the full timeout; a peer that IS progressing keeps its slot up to a 60-min
  backstop, so a slow-but-working transfer can finish.
- Rank search candidates by peer: free upload slot, then speed, then shortest queue.
- Drop candidates with far fewer tracks than the release (single-song folders) up front,
  so no download attempt is wasted on them.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
 2026-07-15 19:54:14 +02:00
Jonathan 57f97e9838 feat(worker): defer Qobuz upgrades when it's gated instead of re-grabbing same quality 
With qualityCutoff=3 (want hi-res), a Soulseek-grabbed CD copy stays below cutoff and
is re-attempted for upgrade. Without a guard, when Qobuz is gated by pacing those
attempts would (a) re-search every owned album each monitor cycle and (b) re-download
a same-quality Soulseek copy that keep-if-better discards — pure churn.

Add an upgrade guard: a >= CD-lossless copy can only be improved by hi-res Qobuz, so
when Qobuz is gated, skip the job WITHOUT searching (pre-search skip); and as a safety
net, skip the download when the best-ranked candidate can't beat the existing copy.
Force jobs always proceed. Extracted the shared keep-existing path into a helper.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
 2026-07-15 19:15:43 +02:00
Jonathan 1fc61a0f5a fix(worker): cancel abandoned slskd transfers + cap download fall-through 
Two compounding causes of 'many downloads of one album from different sources' in
slskd: (1) the slskd client raised on timeout/error WITHOUT cancelling the transfer
it enqueued, so as the pipeline fell through to other peers each left a transfer
running; (2) the incomplete-download fall-through was uncapped, grinding through
every candidate of a popular album (~200 Soulseek peers), amplified now that Qobuz
pacing pushes most jobs to Soulseek.

- SlskdClient.download now cancels its enqueued transfers (DELETE) on any abandon.
- The download loop caps at _MAX_DOWNLOAD_ATTEMPTS (6) candidates.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
 2026-07-15 19:03:05 +02:00
Jonathan 67f374c470 feat: Qobuz pacing budget gate (human-like volume + cadence) 
Avoid re-triggering a USER_BLOCKED account by capping Qobuz to a human-looking
volume and cadence. A per-album gate (worker/lyra_worker/qobuz_gate.py) allows
Qobuz only within active hours, under today's cap (a warm-up ramp of a steady
cap), and past a randomized spacing since the last Qobuz download; the gap spreads
the day's budget across active hours with jitter. When the gate is closed the
pipeline drops Qobuz candidates so the job falls through to another source (and
monitored albums upgrade to Qobuz later); a Qobuz-only setup is never gated.
Config keys read live each loop; counters reset at the day boundary via the DB
clock. Tunable in Settings -> Qobuz -> Pacing (new /api/qobuz/pacing route).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
 2026-07-15 18:45:56 +02:00
Jonathan d47ed12e8e feat(worker): fall through to the next source on an incomplete download 
Completeness/duration verification was applied only to the first candidate that
downloaded successfully; if that copy was incomplete or too short the whole job
failed, even when a later-ranked source had a complete copy (e.g. John Mayer 'The
Search for Everything' — Qobuz reliably fails one hi-res track, so it never fell
back to the deluxe edition or Soulseek).

Extract the checks into a pure _download_problem() and run it INSIDE the download
loop: a candidate that downloads but fails verification now falls through to the
next-ranked source, and the job only fails (needs_attention) when every candidate
is incomplete — reporting the last shortfall reason.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
 2026-07-15 13:41:04 +02:00
Jonathan b0eec41823 fix(worker): judge download completeness by the source's edition, not MB's 
The completeness + duration guards used MusicBrainz's target.track_count /
total_duration_s, but MB picks an arbitrary releases[0] that is often a
deluxe/expanded edition with more tracks (and runtime) than the standard album a
source legitimately delivers. So complete standard-edition downloads were falsely
rejected — The Script 'No Sound Without Silence' (11 vs MB 12) as 'incomplete
download', Skillet 'Unleashed' (12 vs MB's 20-track deluxe) as 'too short'.

Now completeness is judged against the chosen source's own track count
(winner.track_count): a truncated download (fewer files than the source promised)
still fails, and the duration expectation is scaled to the delivered edition's
size. A separate 'implausibly small vs MB' guard (source has <= half MB's tracks)
still rejects a lone-track 'full album' video. Genuine partials (e.g. John Mayer,
a Qobuz track failing mid-download) still correctly fail.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
 2026-07-15 13:22:01 +02:00
Jonathan 108114ffa3 fix(web): replace stray null byte with space in Recently-Pressed dedupe key 
A literal NUL had crept into the `${r.artist} ${r.album}` dedupe key on the
Floor, making git treat queue.tsx as a binary file (unreviewable diffs). Restore
the intended space; behavior is unchanged (the key stays unique per album).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
 2026-07-15 12:44:22 +02:00
Jonathan 6451cf3a3c feat(web): Floor press + per-item controls (pause/resume/clear/stop/remove) 
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
 2026-07-15 12:33:44 +02:00
Jonathan 927266d042 fix(mb): rank release groups by credited artist, not title alone 
The worker MB resolver picked release groups on (title_ratio, is_album)
and never scored the artist — even though resolve() knows the followed
artist. For a generic title, same-titled release groups by *different*
artists tie on title, and the is_album tiebreak (added in 06e7f91 for
Off the Wall) then actively preferred a foreign Album over the correct
release. Concretely, following Lake Street Dive's "Fun Machine" EP
resolved to an unrelated band "Bastards of Melody"'s same-titled Album,
which then propagated as the downloaded/imported artist.

Make credited-artist similarity the primary sort key, above title and
above is_album. It's a soft signal, never a hard filter, so credited-
name variations (feat., punctuation) still resolve; is_album now only
breaks ties within the same artist+title, preserving the Off the Wall
fix. Verified live: resolve("Lake Street Dive", "Fun Machine") now
returns the 6-track LSD EP.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
 2026-07-15 12:32:12 +02:00
Jonathan c6945bd89e feat(web): inter-job download delay setting (Settings → Monitor) 
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
 2026-07-15 12:24:42 +02:00
Jonathan 3f624c3395 feat(web): DELETE /api/requests/[id] to remove a queued item 
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
 2026-07-15 12:20:16 +02:00
Jonathan ecec0b6073 test(pause-api): harden "reject in-flight" test with durability assertion 
Strengthen test to prove the job's paused state remains unmodified when
pause/unpause is rejected for a downloading job. Tests now seeds paused=true,
attempts paused=false, verifies 400, and fetches from DB to assert paused is
still true — catching regressions that would mutate before the guard check.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
 2026-07-15 12:18:03 +02:00
Jonathan b78867435a feat(web): per-item pause route for queued jobs 
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
 2026-07-15 12:14:45 +02:00
Jonathan 3d30e7dc8b feat(web): /api/floor pause/resume/clear/stop controls 
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
 2026-07-15 12:09:45 +02:00
Jonathan 8c95ff735b feat(worker): pause gate, inter-job delay, and force-stop watcher in main loop 
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
 2026-07-15 12:06:40 +02:00
Jonathan bff590a00f feat(worker): claim_next skips per-item paused jobs 
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
 2026-07-15 12:01:38 +02:00
Jonathan 20869bdc13 feat(worker): floor pause/delay/kill-switch helpers 
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
 2026-07-15 11:57:34 +02:00
Jonathan 9c2495825f feat(db): add Job.paused column for per-item press pause 
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
 2026-07-15 11:54:19 +02:00
Jonathan 878eac066e docs: implementation plan for Press controls + inter-job delay 
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
 2026-07-15 11:50:54 +02:00
Jonathan 603c62ef87 docs: design spec for Press controls + inter-job delay 
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
 2026-07-15 11:43:09 +02:00
Jonathan 06e7f91897 fix(mb): resolve same-named albums to the Album, not the Single 
MusicBrainz search returns release-groups tied on title similarity in an
arbitrary order; `max(groups, key=_ratio)` kept the first, so Michael
Jackson's "Off the Wall" resolved to the 2-track *single* release-group
(Off the Wall / Get on the Floor) instead of the 10-track album. Because
plan_track_names labels files purely by position, that short tracklist got
mapped onto the full album — track 2 became "Get on the Floor" instead of
"Rock with You", and the real "Off the Wall"/"Get on the Floor" files kept
their streamrip names, yielding duplicate titles.

Extract selection into a pure `_best_release_group()` that breaks title-
similarity ties toward primary-type == "Album". Similarity still dominates;
album preference only settles ties. Live-verified: Off the Wall now resolves
to the 10-track album with track 2 = "Rock With You".

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
 2026-07-15 10:49:26 +02:00
Jonathan 0443f10a3b feat(lastfm): disambiguation picker for same-named MB artists 
Last.fm's artist MBID can point at the wrong same-named band (it mapped
"Looking Glass" to an Australian stoner band, not the US pop group), and the
/lastfm rows trusted it verbatim — so opening/following grabbed the wrong
artist and the grabbed album never showed on the artist page. Now open/follow
resolve via MB search: resolveArtistChoice keeps candidates whose name matches
exactly — 1 match is used directly (even over a disagreeing Last.fm MBID), 2+
surface an ArtistPicker showing each MB disambiguation ("70's US pop group,
track 'Brandy'" vs "Australian stoner rock") with MB's top hit flagged
Suggested. Core decision logic unit-tested; no worker/schema change.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
 2026-07-15 03:43:54 +02:00
Jonathan a8ea3441fc feat(floor): show "preparing…" bar during the pre-download window 
A Qobuz login/metadata/artwork phase (and a Soulseek peer-queue or YouTube
info-fetch) can sit at 0% for a few minutes before byte progress flows, which
read as a stuck bar. While a downloading job's progress is still 0, render the
indeterminate "preparing…" bar (same style as the searching/finishing phases);
it flips to the determinate % bar on the first reported byte and never back
(the progress aggregator only increases from 0).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
 2026-07-15 03:43:54 +02:00
Jonathan ccaaa41fec docs: slim README for self-hosting, move advanced topics to docs/DEPLOYMENT.md 
Restructure the docs ahead of sharing the project for others to self-host. The
README is now a focused get-it-running path (overview, architecture, prereqs,
quick start, operational notes, security, backup/restore). Moved the fuller
feature tour, the maintainer pre-built-image release flow, and the Gluetun VPN
guide into a new docs/DEPLOYMENT.md, cross-linked both ways. The pre-built-image
section is now labelled a maintainer process with a LYRA_REGISTRY override so a
self-hoster can point at their own registry (or just build from source).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
 2026-07-15 02:57:45 +02:00
Jonathan b1f6408da6 feat(lastfm): inline Follow / Want buttons on the browse rows 
Previously the /lastfm rows only showed a status chip; following an artist or
wanting an album meant opening the modal first. Add an inline Follow button to
each Artists-tab row and an inline Want button to each Albums-tab row, reusing
the same endpoints as the modals (resolve MB id/release-group as needed, then
POST /api/artists or /api/discover/want). Optimistic per-row state flips the
button to the Following/Wanted chip without a reload; the button is disabled
while the request is in flight.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
 2026-07-15 02:39:13 +02:00
Jonathan 55cf88f66e fix(worker): canonicalize bonus-track filenames past the MB tracklist 
A track beyond the MusicBrainz tracklist (a Qobuz bonus track) kept streamrip's
raw name (e.g. "13. John Mayer - St. Patrick's Day (Album Version).flac") while
the main tracklist got clean "## Title" names. plan_track_names now canonicalizes
such extras: strip the leading track-number prefix and a leading "<artist> - ",
yielding "13 St. Patrick's Day (Album Version).flac". Requires an explicit
delimiter after the number so titles like "99 Luftballons" are left alone; with
no tracklist resolved at all the on-disk stem is still kept.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
 2026-07-15 02:37:08 +02:00
Jonathan 0c5e73eb17 fix(worker): parallelize per-source searches in the match phase 
Soulseek's slskd search polls up to ~2 min; searching adapters sequentially
added that on top of Qobuz's instant search. Run the per-adapter searches
concurrently via a ThreadPoolExecutor so total match time is the slowest
single source, not their sum. Only Qobuz drives streamrip's shared asyncio
loop and there's one Qobuz adapter, so no two threads use that loop at once.
Results collected in adapter order (map preserves order) for deterministic
candidate ordering; per-adapter failures still swallowed (down source = no
candidates).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
 2026-07-15 02:37:08 +02:00
Jonathan 0325467aba fix(worker): smooth Qobuz download progress via streamrip byte hook 
streamrip reports progress only per concurrent track, so Job.downloadProgress
came solely from the file-count poller — 0% until every track landed, then a
jump to 100%. Patch streamrip.media.track.get_progress_callback to aggregate
per-track byte deltas over an estimated album total (avg started-track size ×
len(album.tracks)) into one smooth, generally-climbing 0..1 fraction routed
through the pipeline's on_progress. Capped at 0.99; the pipeline sets 1.0 on
completion. No signature changes — track count read from the resolved album.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
 2026-07-15 02:37:08 +02:00
Jonathan c7bab8e997 docs: handoff plan for 3 post-deploy UX fixes 
Qobuz progress smoothing, parallel source searches, bonus-track filename
canonicalization. Lyra is live on the VM; these are worker-only polish.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
 2026-07-15 02:24:56 +02:00
Jonathan 53e7f0b7f2 build: script to publish web/worker images to the Gitea registry 
Adds scripts/build-push.sh (build + push lyra-web/lyra-worker :latest and
:<sha> to git.jger.nl) + a README 'Deploying to a server (pre-built images)'
section. The server pulls these via the vm-download ops repo (docker/lyra).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
 2026-07-15 01:06:46 +02:00
Jonathan f72d193ef7 chore(discover): raise_for_status on Last.fm fetches 
A 4xx/5xx (rate limit / auth) now raises instead of parsing an error body;
callers already handle it (discovery per-source try/except; album fetch
returns []). Closes the last small deferred cleanup.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
 2026-07-15 00:50:35 +02:00
Jonathan 4a768b7122 fix(discover): drop suggestions for non-artists (no core discography) 
ListenBrainz surfaces producers/session players/band members (Max Martin,
Dominic Howard) that co-occur in listens but have no releases of their own —
they showed as bare rows with an empty page. _derive_albums now drops a
surfaced artist (deletes their pending suggestion) when their MB discography
has no core release-group (Album/Single/EP, no secondary type), catching both
empty discographies and soundtrack/production-only credits, while keeping real
singles/EP-only artists. On browse failure the suggestion is kept.

Test fake browser gains default_core so scoring tests keep surfacing candidates.
worker 249 tests.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
 2026-07-15 00:38:56 +02:00
Jonathan 9915bfbda2 feat(discover): full reset — clear + rebuild suggestions 
POST /api/discover/reset sets discover.resetRequested; the worker, at the
start of the next sweep, clears all PENDING suggestions + contributions +
seed throttles and nulls WatchedArtist.lastDiscoveredAt (keeping
followed/wanted/dismissed decisions), then regenerates everything with current
logic. A 'Reset' button on /discover (2-step confirm) triggers it.

worker 247, web 190 tests.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
 2026-07-15 00:20:17 +02:00
Jonathan 7e27654b5a chore: remove accidentally-committed screenshot 
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
 2026-07-15 00:12:36 +02:00
Jonathan 288d55269d fix(discover): hide stale singles + replace album set on re-derive 
- Client filters the row's album thumbnails to full albums only (hides
  singles/EPs left over from pre-albums-only sweeps).
- Worker: re-deriving an artist deletes its pending album suggestions that are
  no longer picks (stale singles / now-owned), keeping wanted/dismissed rows,
  so the suggestion set self-cleans instead of accumulating.

worker 246, web 189 tests.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
 2026-07-15 00:10:10 +02:00
Jonathan 48d5f287b2 style(discover): compact stacked album items in each row 
Each suggested album is now a small horizontal item (30px cover + title/badge
on one line, like Recently Pressed but smaller) stacked vertically, so both
albums fit under each other without enlarging the row.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
 2026-07-15 00:00:37 +02:00
Jonathan 947c86bc8f feat(discover): unified artist rows with inline album thumbnails 
/discover is now one list of artist rows (no Artists|Albums tabs). Each row
shows the artist (name links to the full page — ArtistModal dropped from
Discover), the 'Similar to' reason, and its newest/most-played album
thumbnails with badges. Album-click opens the AlbumModal (cover + tracklist +
Want). Find-similar results link to the full page too.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
 2026-07-14 23:46:14 +02:00
Jonathan a8c1600111 fix(discover): dismissing an artist also dismisses its albums 
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
 2026-07-14 23:43:30 +02:00
Jonathan 202fc0db3c feat(discover): group suggestions into artist rows 
/api/discover now returns rows[] grouped by artist, each carrying its album
suggestions (with albumReason) + the seed names, ordered by score. Album
suggestions whose artist has no pending artist suggestion still yield a row.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
 2026-07-14 23:41:56 +02:00
Jonathan e8fd4bb0a9 feat(discover): supply Last.fm popularity to the sweep 
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
 2026-07-14 23:40:33 +02:00
Jonathan 194b4911c7 feat(discover): derive newest + most-popular album per artist 
_derive_albums now also picks the artist's most-played album (Last.fm,
matched into the browsed discography by MBID then normalized title),
alongside the newest. Same release -> one row tagged newest,popular.
albumReason persisted; run_discovery threads an optional popularity provider.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
 2026-07-14 23:39:18 +02:00
Jonathan efdd592761 feat(discover): Last.fm album-popularity helper 
LastfmAlbumPopularity.top_albums(conn, name) ranks an artist's albums by
Last.fm playcount (read-through ApiCache, 12h; errors -> []). registry
build_album_popularity returns it only when lastfm.api_key is set.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
 2026-07-14 23:36:41 +02:00
Jonathan a09093f798 feat(discover): add DiscoverySuggestion.albumReason 
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
 2026-07-14 23:34:58 +02:00
Jonathan e74d1d9c7b docs: implementation plan for Discover unified rows 
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
 2026-07-14 23:33:04 +02:00
Jonathan 9dd235c2bc docs: spec for Discover unified rows + newest/most-popular albums 
Design for collapsing the Artists|Albums tabs into one artist-centric row,
surfacing each similar artist's newest AND most-popular (Last.fm) album inline,
album-click → AlbumModal tracklist, artist name → full page (drop the Discover
ArtistModal). Approved design; implementation plan next.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
 2026-07-14 23:29:23 +02:00
Jonathan e04c1c9795 feat(discover): also seed discovery from owned-album artists 
Discovery seeded only from followed artists. Now it also seeds from artists
you own albums by (LibraryItem.artistMbid, captured since the MBID own-state
work) but don't follow — so your library shapes suggestions, not just your
follows.

- New DiscoverySeed throttle table (migration add_discovery_seed) mirroring
  WatchedArtist.lastDiscoveredAt, so each library-derived seed is swept once
  per interval (WatchedArtist seeds keep their own throttle).
- _seed_artists merges both seed sources least-recently-swept first, capped at
  chunk_size; run_discovery stamps the right throttle per seed and evicts
  contributions from seeds that are neither followed nor owned.
- Artists you already have (followed OR owned) are excluded from the suggestion
  candidates — discovery surfaces only genuinely new artists.
- discover.seedFromLibrary config (default true) + a Discovery settings toggle.

NOTE: existing LibraryItem rows have null artistMbid (populated on new
import/scan), so a re-scan is needed before old albums seed discovery.
worker 236, web 187 tests.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
 2026-07-14 22:39:52 +02:00
Jonathan a47d37a787 feat(discover): show why each item is suggested 
Each suggestion now carries the followed artists (seeds) that surfaced it,
from DiscoverySeedContribution joined to the current watched roster (an
unfollowed seed drops out), highest-scoring seed first. /discover renders
"Similar to Arctic Monkeys, Coldplay +2" under each artist and album row —
both kinds key off the candidate's artistMbid, so an album inherits its
artist's seeds. web 187 tests.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
 2026-07-14 22:07:05 +02:00
Jonathan 18a22db7fb feat(library): match own-state on artist MBID, not just name 
Own-state ("in library") relied on an exact artist-NAME match, so an owned
album whose name differed from the MB canonical (punctuation/locale/feat.)
showed as not-owned even when followed. Capture the MusicBrainz artist MBID
on LibraryItem and prefer it for matching.

- New LibraryItem.artistMbid (migration add_library_artist_mbid). Populated at
  import (pipeline) + scan, both of which already hold the resolved target;
  the pipeline also now stores rgMbid (it previously didn't). COALESCE on
  conflict backfills MBIDs on re-scan without clobbering.
- /api/artists "in library, not followed" matches by artistMbid first, name
  case-insensitively as fallback (rows predating the column). Library route
  prefers the item's own artistMbid over the release join.

Last.fm/discover own-state still use name-match (they work) — a smaller
follow-up. worker 233, web 186 tests.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
 2026-07-14 21:55:24 +02:00
Jonathan 1c1b98967a feat(pipeline): reject downloads far shorter than the MB total duration 
The verify step checked track COUNT but not playtime, so a truncated file or
a short preview substituted for a real track could import as "complete". Add
a total-duration check: sum the staged audio's playtime (mutagen) and fail
the job when it falls below 85% of MusicBrainz's expected total
(target.total_duration_s, already resolved). Generous tolerance so
edition/encoding differences don't false-positive; bonus tracks (over-long)
always pass.

The duration reader is injected (measure_duration param, default the real
mutagen-based reader) so the fail/pass paths are unit-tested; it returns None
for the offline fakes (no real audio), skipping the check exactly like the
staged-count guard. worker 233 tests.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
 2026-07-14 21:49:15 +02:00
Jonathan 1943fe2bdf feat(discover): max-normalize similarity scores across sources 
_record_contributions now normalizes each source's top-N to [0,1] by its own
top score before summing across sources. ListenBrainz co-occurrence counts
(thousands) no longer drown Last.fm's 0–1 match scores, so a small-scale
source contributes real ranking signal instead of only adding candidates.

Discovery score assertions updated to the normalized scale (single-candidate
fakes normalize to 1.0); added a cross-source test showing a small-scale
source's candidate ranks equal to a large-scale one. min_score is now on the
[0, n_sources] scale (default 0.0 → unaffected). worker 231 tests.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
 2026-07-14 21:43:02 +02:00
Jonathan 0f64e3c83a feat(library): bulk operations (delete / ignore) 
Add a Select mode on /library: album cards become multi-select, with a bulk
bar to Delete selected (2-step confirm) or Ignore selected. POST
/api/library/bulk { action, ids } applies the op per id and reports ok/failed
counts.

Extracted the delete + ignore logic into lib/library-ops.ts
(removeLibraryItem, ignoreLibraryItem); the single DELETE route now reuses
removeLibraryItem. web 185 tests.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
 2026-07-14 21:37:01 +02:00
Jonathan 8c8a34e320 feat(library): replace/upgrade an owned album on demand 
Add Request.force (migration add_request_force): the pipeline's intake now
skips the "already in library" dedupe for a forced job, so an owned album is
re-downloaded. The import step still keeps the new copy only when it's higher
quality, so a forced upgrade can never downgrade what's on disk.

- Worker: _is_force_job → dedupe bypassed when Request.force.
- POST /api/library/[id]/upgrade finds the matching MonitoredRelease and
  enqueues a force request (guards against stacking on an in-flight job).
- Library route exposes monitoredReleaseId; the album modal shows a
  "Replace / upgrade" button when a release exists.

Worker + web tests added (pipeline force re-acquire; upgrade route).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
 2026-07-14 21:28:54 +02:00
Jonathan 7719b2a0d8 feat(library): on-demand integrity check 
GET /api/library/integrity stats every album folder on disk (via the /music
mount) and flags: missing folders, folders with no audio, zero-byte/
unreadable (truncated) files, and on-disk track count drift vs the names
captured at import/scan. Metadata-only (readdir + stat), so it runs only when
the user clicks "Check now" from a new Integrity section on /library; each
flagged album opens its modal to fix or delete. SectionHeader.note widened to
ReactNode to host the button. web 178 tests.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
 2026-07-14 21:20:39 +02:00
Jonathan 6453cfd27b feat(library): surface possible duplicate albums 
GET /api/library/duplicates groups library items that look like the same
release held more than once — sharing a MusicBrainz release-group, or the
same artist + edition-normalized title (stripEditionQualifiers), unioned so
transitive matches collapse into one group. The (artist,album) unique
constraint rules out exact dupes, so these near-dupes are what's worth
flagging.

/library shows a "Possible duplicates" section; each entry opens the album
modal to edit or delete. web 175 tests.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
 2026-07-14 21:16:03 +02:00
Jonathan f78c88df3f feat(library): edit/fix album metadata 
Correct an owned album's artist/album/year from the Library modal ("Edit
metadata"). PATCH /api/library/[id] renames the folder to the canonical
Artist/Album (Year) scheme on the /music mount, updates the LibraryItem,
and re-resolves the MusicBrainz release-group (cover art) when the
artist/album changed (best-effort — keeps the old rgMbid on an MB outage).

- New shared lib/library-path.ts (safeName, albumDir, yearFromPath); the
  manual-import route now uses it so edits and imports name folders
  identically.
- Library GET falls back to the folder's "(YYYY)" for the year when there's
  no matching release, so edited/manual years show in the grid.
- Does NOT rewrite embedded file tags (a worker/mutagen job) — Lyra reads
  from the folder + DB. Guards paths inside the library root; 409 on folder
  or (artist,album) collision.

web 172 tests.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
 2026-07-14 21:12:02 +02:00
Jonathan 855a0f15fe feat(library): ignore a release (do-not-monitor) 
Add MonitoredRelease.ignored (migration add_release_ignored). The monitor's
enqueue_due skips ignored releases, and they drop off the Wanted list —
independent of `monitored`, so auto-monitor-future can't silently re-grab
an album the user chose to ignore.

- Worker enqueue_due: AND NOT mr.ignored.
- /api/releases/[id] PATCH now accepts `ignored`; /api/wanted filters it out;
  artist detail exposes `ignored`.
- UI: Ignore button on Wanted rows; discography shows an "Ignored" badge with
  an Ignore/Un-ignore toggle (monitor + search disabled while ignored).

Worker + web tests added.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
 2026-07-14 21:03:54 +02:00
Jonathan 6ca39859fa feat(library): surface albums the scan couldn't match 
The library scan silently skipped album folders with no MusicBrainz match
(or an unreadable file). Persist them so they're visible and fixable.

- New UnmatchedAlbum model (migration add_unmatched_album), keyed by path.
- scan_chunk records a skipped album as unmatched (reason "no MusicBrainz
  match" or "unreadable: <err>"), clears the row when an album later
  resolves, and prunes stale rows (a prior scan's, or a removed folder)
  when a scan completes.
- GET /api/library/unmatched lists them; DELETE ?id= dismisses one.
- /library shows a "Couldn't match" section (path + reason + Dismiss).

Worker + web tests added.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
 2026-07-14 20:56:31 +02:00
Jonathan 9e5d5d3af4 feat(library): sort control on the pressings grid 
Add a Sort dropdown to /library next to the filter: Recently added
(default, importedAt desc — the prior order), Release date (album year,
newest first, unknown-year last), Album A–Z, Artist A–Z. Client-side sort
of the already-fetched albums; importedAt was already on the route, just
declared on the client type now.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
 2026-07-14 20:49:43 +02:00
Jonathan 2c3d6e77d5 feat(artists): sort control on the roster 
Add a Sort dropdown to /artists next to the filter: Recently followed
(default, createdAt desc — the prior order), Name A–Z, Most in library,
Most monitored, Most releases. Client-side sort of the already-fetched
roster; the filter box and "In library, not followed" section are
unchanged. Exposes WatchedArtist.createdAt on /api/artists for the sort.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
 2026-07-14 20:46:24 +02:00
Jonathan 72e9c2d47d feat(discover): Artists|Albums tabs on /discover 
Suggested artists and Suggested albums were stacked sections; make them
tabs like /lastfm (.tabs bar + count badge). Discover-now/freshness and
Find-similar stay above the tabs.

The Albums tab filters to full-length albums (isFullAlbum) — a client-side
stopgap that hides any pre-D1 singles/EP suggestions still in the DB (new
sweeps already filter server-side via discover.albumsOnly).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
 2026-07-14 20:42:33 +02:00
Jonathan c9905937cc feat(discover): suggest full albums only (filter singles/EPs) 
Discovery's "Suggested albums" surfaced singles and EPs because
_derive_albums filtered with is_core_release (Album/Single/EP). Add a
discovery-only _album_kind_ok gate that, when discover.albumsOnly (new
config, default true), requires primary_type == "Album" with no secondary
types. is_core_release is untouched — the monitor/scan still use it.

Exposed discover.albumsOnly in the discover config route + a Discovery
settings toggle. Worker + web tests added.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
 2026-07-14 20:37:28 +02:00
Jonathan 7957c621cf docs: how to run Lyra behind a Gluetun VPN container 
Adds a "Running behind a VPN (Gluetun)" README section: the whole-stack
network_mode: service:gluetun setup (DATABASE_URL @db -> @localhost, ports moved
to gluetun, FIREWALL_INPUT_PORTS + FIREWALL_OUTBOUND_SUBNETS), a compose sketch,
and caveats (Qobuz geo-sensitivity, the IPv6 sysctl not applying but harmless, MB
per-IP rate limit covered by the cache, bind mounts unaffected).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
 2026-07-14 20:29:39 +02:00
Jonathan 0147975b14 feat(web): MB-resolve manual imports for instant cover art 
A manually-imported album had no release-group MBID, so it showed the ◉
placeholder until a Scan matched it. Now the import resolves the release group
(searchReleaseGroup, via the shared MB cache) and stores it on the new
LibraryItem.rgMbid column (migration add_library_rg_mbid). The library route
prefers LibraryItem.rgMbid, falling back to the MonitoredRelease join for
scanned items — so a manual album gets cover art immediately. Best-effort: a
no-match / MB outage just leaves rgMbid null.

web 164 tests, tsc + build clean.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
 2026-07-14 20:05:59 +02:00
Jonathan d52517ef39 feat(web): cache cover art + dedupe Recently Pressed 
- Cover art now loads through GET /api/cover/[rgMbid], which fetches from the
  Cover Art Archive once, caches the image on disk (LYRA_COVER_CACHE, default
  /tmp/lyra-covers), and serves it with a 30-day Cache-Control — so the browser
  stops re-fetching it on every Library / Recently-Pressed render. Missing covers
  get a negative marker (re-checked daily). CoverArt points at the endpoint.
- Recently Pressed dedupes by artist+album (re-requesting/re-downloading an album
  left multiple completed Requests showing the same album repeatedly); newest of
  each is kept.

web 164 tests, tsc + build clean.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
 2026-07-14 20:03:36 +02:00
Jonathan 36a34181c9 fix(web): exclude large-upload route from auth middleware (10MB body cap) 
The real cause of the failed 584MB album upload: Next.js caps the request body
at 10MB for any route that passes through middleware, and the import route went
through the auth gate. Exclude /api/library/import from the middleware matcher so
the body streams uncapped to busboy, and authenticate it inline instead (new
lib/auth.isAuthed reads + verifies the session cookie from the request).

web 161 tests, tsc + build clean.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
 2026-07-14 14:41:06 +02:00
Jonathan ce628ca245 fix(web): stream large album uploads to disk (busboy) instead of buffering 
Manual import used request.formData(), which buffers the whole body and caps out
around ~15MB — a real FLAC album is hundreds of MB (Hollywood's Bleeding is
584MB), so it failed with "expected multipart form data". Now the multipart body
is STREAMED with busboy: each file pipes straight to a hidden .import-<uuid>
staging dir on the library filesystem, then the dir is atomically renamed into
place. No full-body buffering, no size cap. Cleaned up on any failure; guards
(400 missing fields/audio, 409 exists) unchanged. web 161 tests.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
 2026-07-14 14:37:25 +02:00
123 changed files with 8195 additions and 601 deletions
Expand all files Collapse all files
README.md
+78 -84
View File
@@ -7,83 +7,54 @@ sources of differing quality: **Qobuz** (lossless/hi-res), **Soulseek** (P2P), a
**ListenBrainz** and **Last.fm** (Spotify's recommendation/related-artists APIs were
retired for new apps, so discovery is MetaBrainz-native and MBID-clean — no Spotify).
## Status
It runs on your own server as a small `docker compose` stack. This README covers getting
it running; see **[docs/DEPLOYMENT.md](docs/DEPLOYMENT.md)** for the full feature tour,
publishing pre-built images, and running behind a VPN.
Built in three slices:
## What it does
1. **Acquisition engine** — ✅ **done**. Resolve a request against MusicBrainz,
match/rank across Qobuz/Soulseek/YouTube, download the best source into an
isolated staging dir, verify completeness, tag (mutagen), and atomically promote
the clean album into `Artist/Album (Year)/## Title.ext`.
2. **Library manager** — ✅ **done**. Watched artists (live MB search + follow),
per-release monitor toggles + auto-monitor-future, a wanted list with retry +
quality-upgrade window, and a worker "monitor" that discovers new releases and
auto-grabs through the acquisition pipeline. Plus a **library scan** that ingests
existing on-disk albums as have + monitored + followed, and a per-artist
studio-only / show-all release-type filter.
3. **Discovery** — ✅ **done**. Recommendation-driven finding of new music via
**ListenBrainz** similar-artists and **Last.fm** `artist.getSimilar` (both
MBID-native), behind a pluggable `SimilaritySource` interface — Last.fm registers
as a second source automatically once its API key is set, and cross-source scores
are summed per candidate. A background sweep (and a "Discover now" button)
aggregates artists similar to the ones you follow into a manual review feed of
suggested **artists** and **albums** at `/discover`; **Follow** and **Want**
reuse the slice-2 follow / wanted flows, **Dismiss** is permanent. Plus a
web-side seed-search box. Every suggestion and seed-search result opens a
read-only **artist preview** (`/discover/artist/[mbid]`) — discography,
per-album tracklists, and "in library" / "monitored" badges — where you can
Follow or Want before committing. No Spotify (its recommendation APIs are dead
for new apps).
- **Acquisition** — resolve a request against MusicBrainz, match/rank across
Qobuz/Soulseek/YouTube, download the best source, verify completeness, tag, and organize
into `Artist/Album (Year)/## Title.ext`.
- **Library manager** — follow artists (live MB search), per-release monitor toggles, a
wanted list with retry + quality-upgrade, a background monitor that auto-grabs new
releases, and a scan that ingests albums already on disk.
- **Discovery** (`/discover`) — recommendation-driven suggestions from ListenBrainz +
Last.fm similar-artists, with Follow / Want / Dismiss and a read-only artist preview.
- **Browse & watch** — Last.fm top artists/albums (`/lastfm`, with inline Follow/Want), a
cover-art **Library** grid (`/library`), and **The Floor** (home): a live queue of
in-flight acquisitions with three-phase progress and a Retry button.
Alongside the three slices:
- **Last.fm browse** (`/lastfm`) — your Last.fm top artists and albums (by period),
with a "hide items already in my library" toggle; each artist opens a modal of your
most-played songs and albums, and every album is one click to **Want**.
- **Library** (`/library`) — a cover-art grid of everything on disk, filterable, each
album opening a tracklist modal.
- **The Floor** (home) — a live queue of in-flight acquisitions with three-phase
progress (**Searching → Downloading → Finishing**), a live download percentage, a
per-job "pressing ticket" (source · format · tracks · elapsed), and a **Retry**
button on anything that needs attention.
The whole UI runs a hand-rolled "Pressing Plant" design system (warm-paper theme, self-hosted
Fraunces serif for prose vs. system mono for machine data, light/dark aware).
**Operational notes:** the monitor and the discovery sweep are both **off by
default** — set `monitor.enabled=true` (or Settings → Monitor, + optional `monitor.*`
tuning) to activate auto-grab/upgrade, and `discover.enabled=true` (or
Settings → Discovery, + optional `discover.*` tuning) to activate the scheduled
discovery sweep. The `/discover` "Discover now" button and seed-search work with the
sweep off. Credentials (Qobuz, Soulseek, Last.fm) are entered in Settings and persist
across `docker compose up -d --build` rebuilds (don't use `down -v`; secrets are
encrypted at rest with `LYRA_SECRET_KEY`). Tests run against a separate `lyra_test`
database (a guard blocks the live DB). Deferred hardening + the full roadmap live in
the implementation-plan "Notes" sections under
[`docs/superpowers/plans/`](docs/superpowers/plans/).
Full detail on each of these is in [docs/DEPLOYMENT.md → Features in depth](docs/DEPLOYMENT.md#features-in-depth).
## Architecture
The `docker compose` stack is three services on a home server:
The `docker compose` stack is a few services on one host:
- **db** — Postgres, the shared source of truth.
- **db** — Postgres, the shared source of truth (all durable state lives here).
- **web** — the Next.js app: UI + API (search, request, queue, progress, settings).
- **worker** — the Python worker: all source integration (streamrip, yt-dlp, slskd
client, mutagen; MusicBrainz + ListenBrainz + Last.fm for metadata and discovery),
running the six-stage acquisition pipeline plus the background monitor and discovery
sweeps.
- **worker** — the Python worker: all source integration (streamrip, yt-dlp, slskd client,
mutagen; MusicBrainz + ListenBrainz + Last.fm for metadata and discovery), running the
acquisition pipeline plus the background monitor and discovery sweeps.
- **db-backup** — a sidecar that `pg_dump`s the database on a schedule (see [Backup &
restore](#backup--restore)).
Soulseek support talks to an **external slskd** daemon (an off-the-shelf headless
Soulseek client) — the worker reaches it via the `slskd.url` + `slskd.api_key`
credentials set in Settings, so run slskd separately rather than as part of this stack.
Soulseek support talks to an **external slskd** daemon (an off-the-shelf headless Soulseek
client) — the worker reaches it via the `slskd.url` + `slskd.api_key` credentials set in
Settings, so run slskd separately rather than as part of this stack.
A shared `/music` volume is the library, with a separate `/staging` volume for
in-progress downloads (point `STAGING_DIR` at a fast local disk when `MUSIC_DIR` is a
network share).
A shared `/music` volume is the library, with a separate `/staging` volume for in-progress
downloads (point `STAGING_DIR` at a fast local disk when `MUSIC_DIR` is a network share).
See [`docs/superpowers/specs/`](docs/superpowers/specs/) for the full design.
## Prerequisites
## Running it
- **Docker** + **Docker Compose**.
- A directory for the music library (set `MUSIC_DIR`).
- Optional, entered in **Settings** after first run: a **Qobuz** account (lossless/hi-res),
a running **slskd** daemon (Soulseek), and a **Last.fm** API key (browse + discovery).
None are required to start — Lyra runs with whatever you configure.
## Quick start
```sh
cp .env.example .env # then edit: set LYRA_SECRET_KEY (openssl rand -base64 32) and MUSIC_DIR
@@ -97,10 +68,26 @@ the placeholder. Once set, keep it stable: changing it makes already-stored cred
undecryptable (you'd re-enter them in Settings). Optionally set `LYRA_PASSWORD` too (see
[Security](#security)).
The web entrypoint runs `prisma migrate deploy` on every rebuild, so schema changes
reach the live DB automatically. Enter Qobuz / Soulseek / Last.fm credentials in
**Settings** (they persist across rebuilds), then follow an artist or request an album
to feed The Floor.
The web entrypoint runs `prisma migrate deploy` on every rebuild, so schema changes reach
the live DB automatically. Enter Qobuz / Soulseek / Last.fm credentials in **Settings** (they
persist across rebuilds), then follow an artist or request an album to feed The Floor.
`.env.example` documents every setting (`MUSIC_DIR`, `STAGING_DIR`, `SLSKD_DOWNLOADS_DIR`,
backup knobs). Note in particular: point `SLSKD_DOWNLOADS_DIR` at slskd's own downloads
directory (Lyra must run on the same host as slskd for Soulseek delivery to work), and point
`STAGING_DIR` at a fast local disk if `MUSIC_DIR` is a network share.
## Operational notes
- The **monitor** and the **discovery sweep** are both **off by default** — set
`monitor.enabled=true` (or Settings → Monitor, + optional `monitor.*` tuning) to activate
auto-grab/upgrade, and `discover.enabled=true` (or Settings → Discovery, + optional
`discover.*` tuning) to activate the scheduled discovery sweep. The `/discover` "Discover
now" button and seed-search work with the sweep off.
- **Credentials persist** across `docker compose up -d --build` rebuilds — don't use
`down -v` (it wipes the Postgres volume); secrets are encrypted at rest with
`LYRA_SECRET_KEY`.
- **Tests** run against a separate `lyra_test` database (a guard blocks the live DB).
## Managing the library
@@ -110,6 +97,24 @@ it — re-Want it from the artist page if you change your mind). This is why the
also bind-mounts `MUSIC_DIR` at `/music` (read-write) — deletion is guarded to only ever
remove folders strictly inside the library root.
## Security
Lyra has **no per-user accounts**. Two layers protect it:
- **Shared-password gate.** Set `LYRA_PASSWORD` in `.env` to require a single shared
password for the entire UI and API. A request without a valid session cookie is
redirected to `/login` (or gets `401` for `/api/*`). The cookie is httpOnly and carries
an HMAC of the password keyed by `LYRA_SECRET_KEY`, so it can't be forged. **If
`LYRA_PASSWORD` is unset, the app is completely open** — anyone who can reach the host
can browse the library, trigger downloads, and read/overwrite the stored credentials in
Settings. Set it for any deployment that isn't on a fully trusted, isolated network.
- **Network.** The shared password is deliberately minimal (single password, bearer
cookie, plain HTTP on the LAN). For anything internet-facing, also put Lyra behind a
reverse proxy with TLS and/or a VPN or auth gateway — don't rely on the shared password
alone. See [docs/DEPLOYMENT.md](docs/DEPLOYMENT.md#running-behind-a-vpn-gluetun) for a
Gluetun VPN setup.
## Backup & restore
**All** durable state — library metadata, monitored/wanted lists, discovery data,
@@ -140,19 +145,8 @@ docker compose start web worker
is safe. To restore onto a brand-new host, bring up just `db` first
(`docker compose up -d db`), restore, then `up -d --build` the rest.
## Security
## More
Lyra has **no per-user accounts**. Two layers protect it:
- **Shared-password gate.** Set `LYRA_PASSWORD` in `.env` to require a single shared
password for the entire UI and API. A request without a valid session cookie is
redirected to `/login` (or gets `401` for `/api/*`). The cookie is httpOnly and carries
an HMAC of the password keyed by `LYRA_SECRET_KEY`, so it can't be forged. **If
`LYRA_PASSWORD` is unset, the app is completely open** — anyone who can reach the host
can browse the library, trigger downloads, and read/overwrite the stored credentials in
Settings. Set it for any deployment that isn't on a fully trusted, isolated network.
- **Network.** The shared password is deliberately minimal (single password, bearer
cookie, plain HTTP on the LAN). For anything internet-facing, also put Lyra behind a
reverse proxy with TLS and/or a VPN or auth gateway — don't rely on the shared password
alone.
- **[docs/DEPLOYMENT.md](docs/DEPLOYMENT.md)** — full feature tour, publishing pre-built
images to a registry, and running the whole stack behind a Gluetun VPN.
- **[docs/superpowers/specs/](docs/superpowers/specs/)** — the full design specs.
docs/DEPLOYMENT.md
+177
View File
@@ -0,0 +1,177 @@
# Lyra — deployment & advanced guide
Topics beyond the [README](../README.md) quick-start: the full feature tour, publishing
pre-built images to a registry, and running the whole stack behind a VPN. For a basic
self-hosted setup you only need the README — start there.
- [Features in depth](#features-in-depth)
- [Publishing pre-built images](#publishing-pre-built-images)
- [Running behind a VPN (Gluetun)](#running-behind-a-vpn-gluetun)
---
## Features in depth
Lyra was built in three slices, all complete:
1. **Acquisition engine.** Resolve a request against MusicBrainz, match/rank across
Qobuz/Soulseek/YouTube, download the best source into an isolated staging dir, verify
completeness, tag (mutagen), and atomically promote the clean album into
`Artist/Album (Year)/## Title.ext`.
2. **Library manager.** Watched artists (live MB search + follow), per-release monitor
toggles + auto-monitor-future, a wanted list with retry + quality-upgrade window, and a
worker "monitor" that discovers new releases and auto-grabs through the acquisition
pipeline. Plus a **library scan** that ingests existing on-disk albums as
have + monitored + followed, and a per-artist studio-only / show-all release-type filter.
3. **Discovery.** Recommendation-driven finding of new music via **ListenBrainz**
similar-artists and **Last.fm** `artist.getSimilar` (both MBID-native), behind a
pluggable `SimilaritySource` interface — Last.fm registers as a second source
automatically once its API key is set, and cross-source scores are summed per candidate.
A background sweep (and a "Discover now" button) aggregates artists similar to the ones
you follow into a manual review feed of suggested **artists** and **albums** at
`/discover`; **Follow** and **Want** reuse the slice-2 follow / wanted flows, **Dismiss**
is permanent. Plus a web-side seed-search box. Every suggestion and seed-search result
opens a read-only **artist preview** (`/discover/artist/[mbid]`) — discography, per-album
tracklists, and "in library" / "monitored" badges — where you can Follow or Want before
committing. No Spotify (its recommendation APIs are dead for new apps).
Alongside the three slices:
- **Last.fm browse** (`/lastfm`) — your Last.fm top artists and albums (by period), with a
"hide items already in my library" toggle and inline **Follow** / **Want** buttons on each
row; each artist also opens a modal of your most-played songs and albums.
- **Library** (`/library`) — a cover-art grid of everything on disk, filterable, each album
opening a tracklist modal.
- **The Floor** (home) — a live queue of in-flight acquisitions with three-phase progress
(**Searching → Downloading → Finishing**), a live download percentage, a per-job "pressing
ticket" (source · format · tracks · elapsed), and a **Retry** button on anything that needs
attention.
The whole UI runs a hand-rolled "Pressing Plant" design system (warm-paper theme, self-hosted
Fraunces serif for prose vs. system mono for machine data, light/dark aware).
Deferred hardening and the full roadmap live in the implementation-plan "Notes" sections under
[`docs/superpowers/plans/`](superpowers/plans/); the full design specs are under
[`docs/superpowers/specs/`](superpowers/specs/).
---
## Publishing pre-built images
> **Maintainer release process** — this is how the project maintainer publishes images to a
> private registry. If you're self-hosting your own copy, ignore this and just build from
> source with `docker compose up -d --build` (see the README). You'd only need this if you
> run your own registry and want servers to **pull** rather than build.
The `web` and `worker` images can be published to a container registry and consumed by a
compose file that pulls instead of builds. The maintainer publishes to a Gitea container
registry consumed by a compose file in the
[`vm-download`](https://git.jger.nl/Jonathan/vm-download) ops repo (`docker/lyra/`).
Publish a new release from this repo (amd64):
```sh
docker login <your-registry> # once — registry user + a token with package:write
./scripts/build-push.sh # builds + pushes lyra-web / lyra-worker (:latest and :<sha>)
```
`scripts/build-push.sh` defaults the registry namespace to `git.jger.nl/jonathan`; override it
with `LYRA_REGISTRY=<your-registry>/<namespace>`.
Then on the server:
```sh
cd /opt/git/vm-download && git pull
cd docker/lyra && docker compose pull && docker compose up -d
```
`db` (Postgres) and `db-backup` use the stock `postgres:17` image, so only the two custom
images are built here.
---
## Running behind a VPN (Gluetun)
Lyra runs fine with its traffic routed through a [Gluetun](https://github.com/qdm12/gluetun)
VPN container — useful for hiding the download traffic (Soulseek / Qobuz / YouTube). Nothing
in Lyra is VPN-incompatible; the only thing that changes is container networking, because a
service using `network_mode: "service:gluetun"` gives up its own network stack and shares
Gluetun's. Two consequences:
- Containers that share Gluetun's namespace reach each other over **`localhost`**, not by
Docker service name — so `DATABASE_URL` changes from `@db:5432` to **`@localhost:5432`**.
- **Published ports move to the `gluetun` container** (the individual services can no longer
publish their own).
The simplest, most consistent setup is to put the **whole stack** behind Gluetun. Sketch
(merge into your existing Gluetun compose; adjust the VPN provider block to yours):
```yaml
services:
gluetun:
image: qmcgaw/gluetun
cap_add: [NET_ADMIN]
environment:
# ... your VPN provider / credentials ...
FIREWALL_INPUT_PORTS: "3000" # let your browser reach the web UI
FIREWALL_OUTBOUND_SUBNETS: "192.168.0.0/16" # your LAN — so the worker can reach slskd
ports:
- "8770:3000" # Lyra web (published on gluetun, not on `web`)
# - "5432:5432" # only if you want DB access from the LAN
web:
build: ./web
network_mode: "service:gluetun" # no `ports:` / `networks:` here
environment:
DATABASE_URL: postgresql://lyra:lyra@localhost:5432/lyra # `db` -> `localhost`
LYRA_SECRET_KEY: ${LYRA_SECRET_KEY}
LYRA_PASSWORD: ${LYRA_PASSWORD:-}
HOSTNAME: 0.0.0.0
LYRA_LIBRARY_ROOT: /music
volumes:
- ${MUSIC_DIR:-./music}:/music
depends_on:
gluetun: { condition: service_healthy }
db: { condition: service_healthy }
worker:
build: ./worker
network_mode: "service:gluetun"
environment:
DATABASE_URL: postgresql://lyra:lyra@localhost:5432/lyra
LYRA_SECRET_KEY: ${LYRA_SECRET_KEY}
STAGING_ROOT: /staging
SLSKD_DOWNLOADS_ROOT: /slskd-downloads
volumes:
- ${MUSIC_DIR:-./music}:/music
- ${STAGING_DIR:-${MUSIC_DIR:-./music}/.staging}:/staging
- ${SLSKD_DOWNLOADS_DIR:-./slskd-downloads}:/slskd-downloads
depends_on:
gluetun: { condition: service_healthy }
db: { condition: service_healthy }
db:
image: postgres:17
network_mode: "service:gluetun" # shares the namespace so `localhost:5432` resolves
environment:
POSTGRES_USER: ${POSTGRES_USER}
POSTGRES_PASSWORD: ${POSTGRES_PASSWORD}
POSTGRES_DB: ${POSTGRES_DB}
volumes:
- postgres-data:/var/lib/postgresql/data
```
(Give `db-backup` the same `network_mode` and `PGHOST: localhost`.) The `HOSTNAME=0.0.0.0`
and the built-in healthchecks keep working, since everything is in one namespace.
Caveats worth knowing:
- **Qobuz is geo-sensitive** — its catalog/availability varies by country, so pick a VPN
exit in a region where your Qobuz account works, or some downloads will fail to match.
- The worker's `net.ipv6.conf.all.disable_ipv6` sysctl **can't apply behind Gluetun** (a
shared netns can't set sysctls) — harmless, because Gluetun blocks IPv6 anyway, which is
exactly what that sysctl worked around (Qobuz's CDN resolving to an unroutable IPv6).
- MusicBrainz rate-limits per IP; a shared VPN exit is marginally more likely to hit the
1 req/s ceiling, but the built-in response cache makes that a non-issue in normal use.
- Bind mounts (`/music`, `/staging`, `/slskd-downloads`), the Postgres volume, and backups
are filesystem, not network — **completely unaffected** by the VPN.
docs/superpowers/plans/2026-07-14-discover-unified-rows.md
+247
View File
@@ -0,0 +1,247 @@
# Discover Unified Rows + Newest/Most-Popular Albums — Implementation Plan
> **For agentic workers:** REQUIRED SUB-SKILL: superpowers:subagent-driven-development or superpowers:executing-plans. Steps use checkbox (`- [ ]`) syntax.
**Goal:** Collapse the Discover Artists|Albums tabs into one artist-centric row that shows each similar artist's newest AND most-popular album inline (album-click → tracklist), with the artist name linking to the full page.
**Architecture:** Worker derives two albums per surfaced artist (newest = existing; most-popular = Last.fm `artist.getTopAlbums` matched into the browsed MB discography), tagging each `DiscoverySuggestion.albumReason`. The `/api/discover` route groups pending suggestions by artist into rows carrying `albums[]` + `seeds[]`. The client renders one row per artist with album thumbnails.
**Tech Stack:** Python 3 + psycopg (worker), Next.js 15 App Router + Prisma + vitest (web), Playwright (verify).
## Global Constraints
- Popularity source is Last.fm `artist.getTopAlbums`; **graceful fallback to newest-only** when no `lastfm.api_key` or no match. No new required config.
- `albumReason` values are exactly `"newest"`, `"popular"`, `"newest,popular"`; badge labels `Newest` / `Most played` / both; `null` → no badge.
- Never break the sweep: any Last.fm/popularity error → treat as no popular pick.
- Deploy is `docker compose up -d --build web worker` (never `down -v`). Tests: worker `lyra_test`, web `lyra_test`.
- Reverts the D2 tab bar (intentional).
---
### Task 1: `albumReason` column + migration
**Files:**
- Modify: `web/prisma/schema.prisma` (DiscoverySuggestion model)
- Create: `web/prisma/migrations/<ts>_add_discovery_album_reason/migration.sql` (via prisma)
- [ ] **Step 1:** Add `albumReason String?` to `model DiscoverySuggestion` with a comment (nullable, backward compatible; null → no badge).
- [ ] **Step 2:** `cd web && DATABASE_URL=postgresql://lyra:lyra@localhost:5432/lyra_test npx prisma migrate dev --name add_discovery_album_reason`. Expected: "Applying migration … add_discovery_album_reason".
- [ ] **Step 3:** Commit (`feat(discover): add DiscoverySuggestion.albumReason`).
---
### Task 2: worker Last.fm album-popularity helper
**Files:**
- Create: `worker/lyra_worker/similarity/_lastfm_albums.py`
- Modify: `worker/lyra_worker/registry.py` (add `build_album_popularity`)
- Test: `worker/tests/test_lastfm_albums.py`
**Interfaces:**
- Produces: `LastfmAlbumPopularity(api_key, base_url=_BASE, timeout=15.0)` with
`top_albums(conn, artist_name) -> list[[name:str, mbid:str|None, playcount:int]]`
(playcount desc; cached in ApiCache key `lfm-artist-top-albums:{norm}`, 12h). Errors → `[]`.
- Produces: `build_album_popularity(config, dsn) -> LastfmAlbumPopularity | None`
(None when `lastfm.api_key` unset).
- [ ] **Step 1: failing test** — parse a canned `artist.gettopalbums` JSON body into
`[[name, mbid, playcount], …]` ordered by playcount desc; a `dict` (single album) is
wrapped; missing mbid → None; a body with `error` → `[]`. Inject the parse via a helper
`_parse_top_albums(data)` so no network is needed. Also assert `build_album_popularity({}, None) is None`.
- [ ] **Step 2:** run → FAIL (module missing).
- [ ] **Step 3: implement.**
```python
# _lastfm_albums.py
import requests
from lyra_worker.apicache import cached_api
_BASE = "https://ws.audioscrobbler.com/2.0/"
def _parse_top_albums(data) -> list[list]:
if isinstance(data.get("error"), (int, float)):
return []
rows = data.get("topalbums", {}).get("album", [])
if isinstance(rows, dict):
rows = [rows]
out = []
for r in rows:
name = r.get("name") or ""
if not name:
continue
out.append([name, r.get("mbid") or None, int(r.get("playcount") or 0)])
out.sort(key=lambda a: a[2], reverse=True)
return out
class LastfmAlbumPopularity:
def __init__(self, api_key, base_url=_BASE, timeout: float = 15.0, limit: int = 25):
self._key, self._base, self._timeout, self._limit = api_key, base_url, timeout, limit
def _fetch(self, artist_name: str) -> list[list]:
try:
res = requests.get(self._base, params={
"method": "artist.gettopalbums", "artist": artist_name,
"api_key": self._key, "format": "json", "limit": str(self._limit),
"autocorrect": "1",
}, timeout=self._timeout)
return _parse_top_albums(res.json())
except Exception:
return []
def top_albums(self, conn, artist_name: str) -> list[list]:
norm = (artist_name or "").strip().lower()
if not norm:
return []
return cached_api(conn, f"lfm-artist-top-albums:{norm}", 43200,
lambda: self._fetch(artist_name))
```
```python
# registry.py — add
from lyra_worker.similarity._lastfm_albums import LastfmAlbumPopularity
def build_album_popularity(config: dict, dsn: str | None = None):
key = config.get("lastfm.api_key")
return LastfmAlbumPopularity(api_key=key) if key else None
```
- [ ] **Step 4:** run → PASS.
- [ ] **Step 5:** commit (`feat(discover): Last.fm album-popularity helper`).
---
### Task 3: `_derive_albums` picks newest + most-popular
**Files:**
- Modify: `worker/lyra_worker/discovery.py` (`_derive_albums`, `_upsert_album`, `run_discovery`)
- Test: `worker/tests/test_discovery_albums.py`
**Interfaces:**
- Consumes: `popularity.top_albums(conn, name)` from Task 2 (or None).
- Produces: `_derive_albums(conn, browser, surfaced, cfg, popularity=None)`; `run_discovery(conn, sources, browser, cfg, popularity=None)`; `_upsert_album(conn, artist, rg, reason)`.
- [ ] **Step 1: failing tests** (inject a fake popularity — no live Last.fm):
- newest + most-popular → two album rows, `albumReason` `"newest"` / `"popular"`.
- popular pick == newest rg → one row, `albumReason == "newest,popular"`.
- `popularity=None` → newest-only, `albumReason == "newest"`.
- popular match resolves by title when Last.fm mbid absent; skips owned/non-qualifying and falls to next-highest playcount; no match → newest-only.
```python
class FakePopularity:
def __init__(self, data): self._d = data # {artist_name: [[name, mbid, plays], …]}
def top_albums(self, conn, name): return self._d.get(name, [])
```
- [ ] **Step 2:** run → FAIL.
- [ ] **Step 3: implement.** In `_derive_albums`, after building `core` (qualifying un-owned, newest-first):
```python
def _norm(s): return (s or "").strip().lower()
# newest picks (existing)
picks = {} # rg_mbid -> reason
for rg in core[: cfg.albums_per_artist]:
picks[rg.rg_mbid] = "newest"
# most-popular pick via Last.fm, matched into `core`
if popularity is not None and core:
by_mbid = {g.rg_mbid: g for g in core}
by_title = {_norm(g.title): g for g in core}
for name, mbid, _plays in popularity.top_albums(conn, artist["name"]):
g = (by_mbid.get(mbid) if mbid else None) or by_title.get(_norm(name))
if g is None:
continue
picks[g.rg_mbid] = "newest,popular" if picks.get(g.rg_mbid) == "newest" else "popular"
break
for rg in core:
if rg.rg_mbid in picks:
albums += _upsert_album(conn, artist, rg, picks[rg.rg_mbid])
have.add(rg.rg_mbid)
```
Add `reason` param to `_upsert_album` → include `"albumReason"` in the INSERT column list +
`ON CONFLICT DO UPDATE SET … "albumReason" = EXCLUDED."albumReason"`. Thread `popularity`
through `run_discovery` → `_derive_albums`.
- [ ] **Step 4:** run → PASS; then full worker suite green.
- [ ] **Step 5:** commit (`feat(discover): derive newest + most-popular album per artist`).
---
### Task 4: wire popularity into the worker loop
**Files:**
- Modify: `worker/lyra_worker/main.py` (`_run_discovery` builds + passes popularity)
- Test: existing `test_discovery_trigger.py` still green (popularity defaults None there).
- [ ] **Step 1:** In `main.py`, import `build_album_popularity`; in `_run_discovery` build it from `get_config(conn)` and pass to `run_discovery(conn, sources, browser, cfg, popularity=pop)`.
- [ ] **Step 2:** run worker suite → PASS.
- [ ] **Step 3:** commit (`feat(discover): supply Last.fm popularity to the sweep`).
---
### Task 5: `/api/discover` groups by artist
**Files:**
- Modify: `web/src/app/api/discover/route.ts`
- Test: `web/src/app/api/discover/route.test.ts`
**Interfaces:**
- Produces: `GET` returns `{ rows: [{ id, artistMbid, artistName, score, seedCount, sources, seeds, albums: [{ id, rgMbid, album, primaryType, secondaryTypes, firstReleaseDate, albumReason }] }], search? }`. Rows ordered by score desc.
- [ ] **Step 1: failing test** — seed pending artist + its two album suggestions (`albumReason` newest/popular) sharing `artistMbid`; assert one row with `albums.length === 2`, ordered; seeds present; an album whose artist has no pending artist-suggestion still yields a row.
- [ ] **Step 2:** run → FAIL.
- [ ] **Step 3: implement.** Fetch all pending suggestions (as today) + the seeds map (as today). Group by `artistMbid`: artist-kind rows seed the row header (id/score/seedCount/sources); album-kind rows push into `albums[]` (include `albumReason`). For an `artistMbid` with only album rows, synthesize a header from the album's `artistName`/`artistMbid` (no `id`, score = max album score). Sort rows by score desc; sort each `albums[]` by `firstReleaseDate` desc. Return `{ rows }`.
- [ ] **Step 4:** run → PASS.
- [ ] **Step 5:** commit (`feat(discover): group suggestions into artist rows`).
---
### Task 6: dismiss cascade
**Files:**
- Modify: `web/src/app/api/discover/[id]/route.ts`
- Test: `web/src/app/api/discover/[id]/route.test.ts`
- [ ] **Step 1: failing test** — dismissing an artist suggestion also marks its pending album suggestions (same `artistMbid`) dismissed; a dismissed album is not returned by `/api/discover`.
- [ ] **Step 2:** run → FAIL.
- [ ] **Step 3: implement.** In the dismiss branch, after dismissing the artist suggestion, `updateMany({ where: { kind: "album", artistMbid, status: "pending" }, data: { status: "dismissed" } })`. Follow/Want unchanged.
- [ ] **Step 4:** run → PASS.
- [ ] **Step 5:** commit (`fix(discover): dismissing an artist also dismisses its albums`).
---
### Task 7: unified-row client (drop tabs + ArtistModal)
**Files:**
- Modify: `web/src/app/discover/discover-client.tsx`
- Modify: `web/src/app/design.css` (row + thumb strip + badge styles)
- [ ] **Step 1:** Replace `Feed`/`tab` state with `rows` from `/api/discover` (`{ rows }`). Remove the tab bar, `ArtistModal` import + `artistModal` state + its JSX + the `isFullAlbum`/albums-tab code.
- [ ] **Step 2:** Render one `<li class="list-row disco-row">` per row:
- artist block: `<a href={`/discover/artist/${row.artistMbid}`}>{row.artistName}</a>`, `score`, `similarTo(row.seeds)` (keep existing helper).
- album strip: `row.albums.map` → a `<button class="disco-thumb" onClick={() => setAlbumModal(album with artistMbid=row.artistMbid)}>`: `<CoverArt>` + title + badge (`badgeFor(albumReason)`) + a `Want` button (`act(album.id, "want")`).
- row actions: `Follow` (`act(row.id, "follow")` when `row.id`, else follow-by-mbid via `POST /api/artists`), `Dismiss` (`act(row.id, "dismiss")`, hidden when no `row.id`).
- `badgeFor(r)`: `newest`→`Newest`, `popular`→`Most played`, `newest,popular`→`Newest · Most played`, null→none.
- [ ] **Step 3:** keep `AlbumModal` (thumb click) + `find-similar` search block (its result artist links become `<a href>` to the full page, no modal). Keep Discover-now/freshness header.
- [ ] **Step 4:** CSS: `.disco-row` flex (artist block | thumb strip | actions), `.disco-thumb` (small cover + title + `.badge`), wraps under artist on narrow widths.
- [ ] **Step 5:** `npx tsc --noEmit` + `npm test` + `npm run build` → all green.
- [ ] **Step 6:** commit (`feat(discover): unified artist rows with inline album thumbnails`).
---
### Task 8: deploy + live verify
- [ ] **Step 1:** `docker compose up -d --build web worker`; confirm migration applied + all healthy.
- [ ] **Step 2:** Playwright on `/discover`: rows render; artist name navigates to `/discover/artist/[mbid]`; album thumbnail opens `AlbumModal` with a tracklist; badges show.
- [ ] **Step 3:** Trigger a library **Scan** (populates artistMbid) then **Discover now**; confirm `albumReason` populates (`Most played` badges appear) once a fresh sweep runs. (If a full sweep is impractical to force cheaply, rely on the injected-data Playwright check + unit tests, and note it.)
- [ ] **Step 4:** Update `MEMORY.md` / project-state with the shipped feature + commit range.
## Self-Review
- **Spec coverage:** Part A → Tasks 2–4; Part B → Task 5; Part C dismiss → Task 6, client/modal → Task 7; schema → Task 1; tests throughout; live verify → Task 8. ✓
- **Placeholders:** none — code shown for each implementing step. ✓
- **Type consistency:** `top_albums(conn, name) -> list[[name, mbid|None, plays]]` used identically in Tasks 2–3; `_upsert_album(…, reason)` and `_derive_albums(…, popularity=None)` consistent; route emits `rows[]` consumed by Task 7. ✓
docs/superpowers/plans/2026-07-15-post-deploy-ux-fixes.md
+96
View File
@@ -0,0 +1,96 @@
# Post-deploy UX fixes (3 items) — handoff plan
> **Context:** Lyra is DEPLOYED + LIVE on the media VM (ssh alias `download` = jonathan@10.10.0.103,
> hostname `servarr`, x86_64). Pre-built images from `git.jger.nl/jonathan/lyra-{web,worker}`, whole
> stack behind its own gluetun VPN, deploy repo `Jonathan/vm-download` → `docker/lyra/`. Library was
> wiped to a blank slate; user rebuilds via Last.fm. Two test presses succeeded end-to-end (John Mayer
> Continuum q3 hi-res, Room for Squares q2) — pipeline is solid. These 3 are UX polish the user hit
> in real use. See MEMORY.md → project-state.md "VM DEPLOYMENT" for full deploy details.
**Workflow for EACH fix:** implement in `worker/` → run worker tests (`cd worker && source .venv/bin/activate
&& python -m pytest`) → **on the DEV box** `./scripts/build-push.sh` (needs `docker login git.jger.nl`
already done this session; a fresh session may need re-login — a **real TTY**, not the `!`-prefix) →
on VM `ssh download 'cd /opt/git/vm-download/docker/lyra && docker compose pull && docker compose up -d worker'`
→ verify with a test press (queue via SQL, see below) and watch `Job.downloadProgress`/on-disk result.
Commit per fix; branch `feat/post-deploy-ux`, ff-merge to main, push.
**Queue a test press on the VM** (mimics the Wanted "Search now"): pick a `MonitoredRelease` id from
`SELECT id,"artistName",album FROM "MonitoredRelease" WHERE monitored AND NOT ignored AND state<>'fulfilled'`,
then insert a Request+Job (use a `/tmp/*.sql` file + `docker cp` + `psql -f` to avoid ssh quoting hell):
`INSERT INTO "Request"(id,artist,album,"monitoredReleaseId",status,"createdAt") SELECT gen_random_uuid()::text,"artistName",album,id,'pending',now() FROM "MonitoredRelease" WHERE id='<mrid>' RETURNING id` → capture into a Job insert `(id,"requestId",state,"currentStage",attempts,"downloadProgress","createdAt","updatedAt") VALUES (gen_random_uuid()::text,rid,'requested','intake',0,0,now(),now())`.
---
## Fix 1 — Smooth Qobuz download progress (byte-level)
**Problem:** the Floor progress bar sits at 0% for the whole Qobuz download then jumps to 100%.
Qobuz/streamrip's per-track byte callback is NOT hooked, so `Job.downloadProgress` comes only from the
file-count poller (`pipeline._poll_download_progress`) — and streamrip downloads tracks concurrently, so
0 files are "complete" until they all land at once. yt-dlp + slskd already report bytes via `on_progress`
(smooth); only Qobuz doesn't.
**Files:** `worker/lyra_worker/adapters/_streamrip.py` (the Qobuz download), `worker/lyra_worker/pipeline.py`
(threads `on_progress` → `_make_on_progress` → `Job.downloadProgress`; poller is the fallback when the
per-attempt `reports` Event is set).
**Approach:** hook streamrip's progress. streamrip drives progress via a callback/console; investigate its
API (it uses a `rip.utils`/`Downloadable` byte-progress hook or a `tqdm` callback). Aggregate downloaded
bytes / total bytes summed across the album's concurrent tracks → a single fraction → call the pipeline's
`on_progress(frac)` (which sets `reports` and writes throttled `Job.downloadProgress`). Keep the file-count
poller as the fallback (adapters that don't report). Match the throttle already in `_make_on_progress`
(≥1% move or every 0.5s).
**Test:** hard to unit-test streamrip internals — add a unit test that a fake byte-callback aggregates to a
monotonic 0→1 fraction, plus **live-verify** with a test press: `Job.downloadProgress` climbs mid-download
instead of 0→100. (Known caveat if streamrip exposes no usable byte hook: fall back to a smoother
file-count cadence, or accept the jump — document the finding.)
## Fix 2 — Parallelize the source searches (Soulseek adds ~2 min)
**Problem:** every acquisition spends up to ~2 min in the Soulseek search. `SlskdClient.search_album` polls
`_SEARCH_POLLS=40 × _POLL_SECONDS=3s = 120s` (early-exits on `isComplete`, but slskd searches genuinely run
long). The pipeline searches adapters SEQUENTIALLY (`for adapter in adapters: adapter.search(target)`), so
Soulseek's 2 min is added on top of Qobuz's ~instant search.
**Files:** `worker/lyra_worker/pipeline.py` (the match phase, `for adapter in adapters` loop);
optionally `worker/lyra_worker/adapters/_slskd.py` (`_SEARCH_POLLS`).
**Approach:** run the per-adapter searches **concurrently** (e.g. `ThreadPoolExecutor`, one thread per
adapter), collect all candidates when the last returns → total match wall-clock = slowest single search,
not the sum. So Qobuz returns instantly and only Soulseek's ~2 min gates. Keep the existing per-adapter
try/except (a failing source contributes nothing). **Thread-safety caveat:** streamrip uses a module-level
persistent asyncio loop (`_streamrip._run`) — verify concurrent Qobuz search from a worker thread is safe
(it runs coroutines on that shared loop; may need a lock or to keep Qobuz on the calling thread while
Soulseek/YouTube go to threads). Optionally also drop `_SEARCH_POLLS` to ~20 (60s) with the early-exit.
**Test:** a pipeline test asserting all adapters' candidates are collected (order-independent) when searched
concurrently; existing match/ranking tests stay green. Live-verify a press: match phase finishes in ~the
Soulseek time, not Soulseek+Qobuz.
## Fix 3 — Canonicalize bonus-track filenames on import
**Problem:** tracks BEYOND the MusicBrainz tracklist keep streamrip's raw name. Real example: Room for
Squares imported 12 clean `## Title.flac` + one `13. John Mayer - St. Patrick's Day (Album Version).flac`
(MB lists 12; Qobuz had a 13th bonus). The main tracklist gets `## Title` but extras don't.
**Files:** `worker/lyra_worker/library.py` (`import_album` — how it names/moves audio into
`Artist/Album (Year)/## Title.ext`), and/or the tagger `worker/lyra_worker/_mutagen.py`.
**Approach:** find where files are renamed to `## Title.ext`. For a track with no MB position/title (an
extra), derive a clean name from its own embedded tags (track number + title) or by stripping the
`^\d+\.?\s*(Artist\s*-\s*)?` prefix from the streamrip filename → `## Title.ext`. Ensure the position keeps
counting past the MB list (13, 14, …) and the on-disk `trackNames` capture stays consistent.
**Test:** unit-test the filename-canonicalizer on inputs like `13. John Mayer - St. Patrick's Day (Album
Version).flac` → `13 St. Patrick's Day (Album Version).flac`; live-verify by re-pressing an album with a
bonus track (or `docker compose exec worker` a rename dry-run). NOTE: the already-imported Room for Squares
track 13 can be fixed by re-pressing (force upgrade) or a one-off rename — mention to the user.
---
## Self-review / notes
- All three are worker-only → only the `worker` image rebuilds/redeploys (`docker compose up -d worker`).
- Don't disrupt the live stack: the VM is in use; deploy the worker, verify one press, done.
- If context/time is tight, do them in priority order: **2 (search speed)** and **3 (filenames)** are
self-contained and low-risk; **1 (streamrip progress)** is the deepest (streamrip internals) — timebox it
and fall back to documenting if streamrip has no clean byte hook.
docs/superpowers/plans/2026-07-15-press-controls.md
+1189
View File
File diff suppressed because it is too large Load Diff
docs/superpowers/specs/2026-07-14-discover-unified-rows-design.md
+156
View File
@@ -0,0 +1,156 @@
# Discover redesign: unified artist rows + newest/most-popular albums
**Date:** 2026-07-14
**Status:** Approved (design), pending implementation plan
## Problem
`/discover` splits suggested artists and suggested albums into two tabs, but album
suggestions are not independent — each one is just "a notable album by a suggested similar
artist" (derived in `_derive_albums` from the surfaced artists). The two-tab split fragments a
single idea. Two further gaps:
- Only the **newest** un-owned album is surfaced per similar artist; the album the artist is
best known for (their **most popular**) is invisible unless it happens to be newest.
- Seeing an album's tracklist requires an extra modal hop, and the artist modal on Discover is
a lightweight duplicate of the richer full artist page.
## Goals
1. Keep album suggestions artist-mediated (no album-level similarity data exists in our
stack), but surface **two** albums per similar artist: **newest** and **most popular**.
2. Collapse the Artists|Albums tabs into **one unified list** — one row per suggested artist,
carrying that artist's album thumbnails inline.
3. On Discover, replace the artist modal with a direct link to the full artist page; keep
album-click → the existing `AlbumModal` (cover + tracklist + Want).
Non-goals: album-to-album similarity (no data source); collection-completion suggestions;
changing the Last.fm page's artist modal (it keeps its personal most-played view).
## Data-source decision: popularity
- **Chosen:** Last.fm `artist.getTopAlbums` — returns an artist's albums ranked by playcount,
each with a name and (usually) an MBID. The user already has Last.fm configured; no new
config. Confirmed live: the method exists (error 10 "invalid key" with a bogus key, not
error 3 "unknown method").
- **Rejected:** ListenBrainz `/1/popularity/release-group` — MBID-native and cleaner in
principle, but now requires a ListenBrainz auth token (anti-scraper), which is new config
friction. MusicBrainz exposes no popularity signal at all.
- **Fallback:** without Last.fm creds, or when no top album matches a qualifying un-owned
release group, the artist yields **newest-only** — byte-for-byte today's behavior.
## Design
### Part A — Album derivation (worker)
`_derive_albums(conn, browser, surfaced, cfg, popularity=None)` in
`worker/lyra_worker/discovery.py`.
For each surfaced artist:
1. Browse the artist's release groups from MB (already done) → filter to qualifying,
un-owned albums (`_album_kind_ok` + not in `_existing_rg_mbids`) — unchanged.
2. **Newest:** the newest `cfg.albums_per_artist` (default 1) by `first_release_date` desc —
unchanged, tagged `reason="newest"`.
3. **Most popular:** if `popularity` is available, fetch the artist's Last.fm top albums, and
pick the highest-playcount album that maps to a qualifying un-owned release group in the
browsed set — match by MBID first (when Last.fm supplies one that is in the browsed set),
else by normalized title (`stripEditionQualifiers`-equivalent, lowercased/trimmed). Tag
`reason="popular"`.
4. If the popular pick is the same release group as a newest pick, do not insert a second
row — the existing row's reason becomes `"newest,popular"`.
5. Upsert each as a `DiscoverySuggestion` (kind `album`) as today, additionally writing
`albumReason`.
**Popularity helper (injected, testable — mirrors `#16`'s `measure_duration`).** An object
with `top_albums(artist_name) -> list[(name, mbid_or_None, playcount)]`, ordered by playcount
desc. Built only when `lastfm.api_key` is set (worker decrypts via `crypto.py`); the fetch is
wrapped in `cached_api` (`ApiCache`, ~12h TTL, key `lfm-artist-top-albums:{norm}`). Errors are
swallowed → treated as "no popular pick" (newest-only). `run_discovery` constructs it from
config and passes it to `_derive_albums`; tests inject a fake or `None`.
**Cost:** ≤1 Last.fm call per surfaced artist per sweep, cached; sweeps are infrequent and
chunked. Only surfaced artists (post `min_score`) are queried.
### Part B — `/api/discover` groups suggestions by artist
`web/src/app/api/discover/route.ts` returns **rows** instead of two flat arrays. Build rows
from the union of pending artist-suggestions and the distinct artists of pending
album-suggestions (so an album is never orphaned), keyed by `artistMbid`:
```
rows: [{
id, // the artist suggestion id (for Follow/Dismiss), when present
artistMbid, artistName,
score, seedCount, sources,
seeds: string[], // "why suggested" (existing DiscoverySeedContribution join)
albums: [{
id, rgMbid, album, primaryType, secondaryTypes, firstReleaseDate,
albumReason, // "newest" | "popular" | "newest,popular"
}]
}]
```
Rows are ordered by `score` desc (artist suggestion score; album-only rows use the album's
score). The `find-similar` search flow is unchanged (separate endpoint, no albums).
### Part C — dismiss cascade
`POST /api/discover/[id]` (dismiss): when the target is an **artist** suggestion, also mark
its pending **album** suggestions (same `artistMbid`) dismissed, so the unified row disappears
cleanly rather than leaving stray albums. Follow/Want are unchanged.
### Part D — `/discover` client (unified rows, no tabs, no modal)
`web/src/app/discover/discover-client.tsx`:
- Remove the Artists|Albums tab bar and the `tab` state; render one `<ul>` of artist rows.
- Remove `ArtistModal` import + usage entirely. The artist **name links** to
`/discover/artist/{artistMbid}` (an `<a href>`; matches the existing full-page route).
- Each row renders its `albums[]` as small `CoverArt` thumbnails, each with:
- a **badge** from `albumReason`: `"newest"`→`Newest`, `"popular"`→`Most played`,
`"newest,popular"`→both; `null`→no badge;
- a **Want** action (`act(album.id, "want")`);
- **click → `AlbumModal`** (kept) with the album's `rgMbid`/title/artist → cover + tracklist
+ Want.
- Row actions stay: **Follow** / **Dismiss** (on `row.id`, the artist suggestion). If a row
has no artist suggestion id (album-only, e.g. artist dismissed earlier), Follow resolves by
MBID via `POST /api/artists`; Dismiss is hidden for that (degenerate) case.
- `find-similar` results keep their existing rendering, minus `ArtistModal` — their artist
links also go to the full page (consistency; `ArtistModal` is being removed from this file).
- New CSS for the row layout (artist block + horizontal album-thumb strip + actions),
responsive: the thumb strip wraps/stacks under the artist on narrow viewports.
### Schema / migration
`DiscoverySuggestion.albumReason String?` — nullable, backward compatible. Existing album
rows read as `null` (client treats null as no badge) until the next sweep re-derives them.
Migration `add_discovery_album_reason`.
## Testing
**Worker (`test_discovery_albums.py`):**
- newest + most-popular produce two rows with correct `albumReason`;
- dedupe when newest == popular → one row, `reason="newest,popular"`;
- no popularity helper → newest-only (unchanged), `reason="newest"`;
- popular pick skips owned / non-qualifying release groups and falls to the next-highest
playcount; no match → newest-only;
- injected fake popularity (no live Last.fm).
**Web:**
- `/api/discover` groups albums under their artist row; row carries `albums[]` + `seeds[]` +
`albumReason`; album-only artist still yields a row.
- dismiss cascade: dismissing an artist marks its album suggestions dismissed.
- existing route tests updated to the grouped shape.
**Live verify (Playwright):** unified rows render; artist name navigates to the full page;
album thumbnail opens `AlbumModal` with a tracklist; badges show; a fresh "Discover now" sweep
populates `albumReason` (needs a re-scan first so owned-artist MBIDs exist — pre-existing
caveat from `#18`).
## Risks / notes
- Last.fm top-album title↔MB release-group matching can miss (edition wording, localized
titles); acceptable — a miss just means newest-only for that artist. Normalized-title
matching + MBID-first keeps misses rare.
- Adds a Last.fm dependency to the album step of the sweep; fully optional + cached, and the
artist step already uses Last.fm as a similarity source.
- Reverts the D2 tab bar shipped earlier the same day; intentional per this design.
docs/superpowers/specs/2026-07-15-press-controls-design.md
+179
View File
@@ -0,0 +1,179 @@
# Press controls + inter-job delay — design
Date: 2026-07-15
## Problem
The monitor can enqueue a large backlog of acquisition jobs at once. The worker drains
them serially with **no pacing** (only a 2s idle poll when the queue is empty) and offers
**no runtime control** over the queue — the only per-job action on the Floor is Retry (on
`needs_attention`). The user wants:
1. A **configurable delay between downloads** to spread sustained Qobuz load over time.
2. **Per-item controls** on the Floor ("the Press"): pause and remove a queued item.
3. **Whole-press controls**: pause/resume, clear the queue, and a force-stop of the
currently-downloading album.
## Constraints (from the existing code)
- The worker is a **single, strictly-serial process** (`worker/lyra_worker/main.py:256-298`).
One job runs to completion in a blocking `run_pipeline` call before the loop does anything
else.
- **Config is read fresh every loop iteration** (`main.py:262`, `get_config`), so any Config
change takes effect within ~2s with no restart.
- An **in-flight download cannot be cleanly interrupted** — `run_pipeline` and the adapter
download are synchronous/blocking with only an `on_progress` callback, no cancel token. The
only interruption is process death, after which `reclaim_stuck_jobs` (`claim.py:8-24`)
requeues the in-flight job at startup and `clear_staging_root` (`main.py:233`) wipes the
partial download.
- `claim_next` (`claim.py:27-57`) claims the oldest `state='requested'` job.
- Deleting a `Request` cascades to its `Job` and `Candidate` rows (`schema.prisma:51,66`); the
linked `MonitoredRelease` is `SetNull`, not cascaded.
- `enqueue_due` (`monitor.py:115-147`) re-enqueues a monitored release once it has no live
(non-terminal) job and its `lastSearchedAt` is older than `retryIntervalHours` (default 6).
So a removed/cleared monitored item reappears later on the normal cadence unless `ignored`.
- The web container runs `prisma migrate deploy` on startup (`web/docker-entrypoint.sh`), so a
new migration auto-applies on deploy.
## Decisions (confirmed with the user)
- **Active item:** add a global **force-kill** ("Stop now") that aborts the in-flight download
via worker process-restart. Per-item controls act only on queued items.
- **Remove / Clear semantics:** **cancel-for-now**. Manual one-off requests are gone for good;
monitored/wanted releases stay wanted and may reappear on the monitor's normal cadence.
Permanent skip remains the existing per-release **Ignore**.
- **Pause scope:** **downloads only**. Pausing halts claiming/downloading; the monitor keeps
finding and enqueueing releases (they pile up in the queue) and Resume drains them.
## Data model
- **New column** `Job.paused Boolean @default(false)` — a Prisma migration
(`add_job_paused`). Auto-applies on deploy.
- **Config keys** (key/value `Config` rows, no migration):
- `floor.paused` — `"true"`/`"false"`; global press pause.
- `floor.jobDelaySeconds` — integer string, default `"0"`; delay between downloads.
- `floor.killRequested` — `"true"`/`"false"`; one-shot force-kill signal, cleared by the
worker on consumption.
## Worker changes
`worker/lyra_worker/main.py` and `worker/lyra_worker/claim.py`.
1. **Inter-job delay.** After `finish_job` (`main.py:297`), read `floor.jobDelaySeconds` from
the already-fresh `config` and sleep that many seconds before the next loop iteration.
Sleep in small (~2s) chunks so the loop stays responsive; the force-kill watcher runs on a
separate thread and works regardless. Parse with the existing `_int`-style guard (invalid →
0). A value of 0 preserves today's back-to-back behavior.
2. **Global pause.** Immediately before `claim_next` (`main.py:286`), if `floor.paused` is
truthy (`_TRUE` idiom), skip claiming: `time.sleep(IDLE_SLEEP); continue`. Because the loop
is serial, any in-flight job has already finished before this check — pause stops the
*next* download. Monitor sweep / scan / discovery (above the claim in the loop body)
continue to run, so the queue keeps filling while paused.
3. **Per-item pause.** `claim_next` `WHERE` gains `AND NOT "paused"`. A paused queued job is
skipped by the claim but remains `state='requested'`, so it is not "lost" and resumes
claiming as soon as `paused` is cleared. `reclaim_stuck_jobs` is unaffected (it only touches
in-flight states).
4. **Force-kill watcher.** A daemon thread started in `run_forever` (modeled on
`_liveness_heartbeat`, `main.py:216-226`) polls `floor.killRequested` on its own DB
connection every ~1.5s. When set, it clears the flag (commit) and calls `os._exit(0)`. The
container restart policy brings the worker back; startup `clear_staging_root` +
`reclaim_stuck_jobs` wipe the partial download and requeue the aborted job. The "Stop now"
action also sets `floor.paused=true`, so the worker returns **idle** rather than instantly
re-downloading the aborted album; the user resumes when ready.
## Infrastructure (ops repo)
The prod worker's Docker restart policy is currently **`no`** (`docker inspect` →
`RestartPolicy=no`), so process death — from force-kill *or any crash* — leaves it down. The
force-kill design requires `restart: unless-stopped` on the `worker` service in the ops repo
(`/opt/git/vm-download/docker/lyra/compose.yaml`). The in-repo `docker-compose.yml` already has
this (`docker-compose.yml:53`); this change brings prod in line and is good hygiene
independent of this feature. This is a one-line change in a separate repo and must be applied
before the force-kill feature is usable in prod.
## API (web) — follows existing `retry` / `library DELETE` patterns
- **`GET /api/floor`** → `{ paused: boolean, jobDelaySeconds: number }` (reads the Config
rows). Used by the Floor to render the paused banner and control states.
- **`POST /api/floor`** with `{ action }`:
- `"pause"` / `"resume"` → upsert `floor.paused`.
- `"clear"` → delete every `Request` whose `Job.state = 'requested'` (queued, not yet
claimed). Cascade removes the Jobs and Candidates. The in-flight and terminal jobs are
untouched. Returns the number cleared.
- `"stop"` → upsert `floor.paused='true'` and `floor.killRequested='true'`.
- **`POST /api/requests/[id]/pause`** with `{ paused: boolean }` → set `Job.paused`. Guard:
the job must be `state='requested'` (else `400`, matching the existing retry route), since
pausing an in-flight job has no effect.
- **`DELETE /api/requests/[id]`** → delete the `Request` (cascade Job + Candidates). Guard:
only when `Job.state = 'requested'`, so the active download is never touched (that is the
job of `Stop now`). Mirrors `DELETE /api/library/[id]`.
## Web UI
`web/src/app/queue.tsx` and `web/src/app/_ui/job-row.tsx`; styling via existing `.btn`
variants in `design.css`.
- **Press header** (above the queue list):
- `Pause the press` / `Resume` toggle (from `GET /api/floor`).
- `Clear queue` — confirm dialog showing the queued count; calls `POST /api/floor
{action:"clear"}`.
- `Stop now ⚠` — rendered only when a job is actively downloading (derivable from the
existing `/api/requests` data: any job in `matching`/`matched`/`downloading`/`tagging`).
Confirm dialog ("aborts the current download and pauses the press"); calls `POST /api/floor
{action:"stop"}`.
- A "Press paused — N queued" banner/chip when `paused`.
- **Per-item controls** (queued rows only — `job.state === 'requested'`), placed in the row
`note` slot as `.btn sm ghost`:
- `Pause` / `Resume` → `POST /api/requests/[id]/pause`.
- `Remove` → `DELETE /api/requests/[id]`.
- A per-item paused row shows a subtle "paused" chip and the `Resume` button.
- Active-download and `needs_attention` rows keep their current rendering (Retry unchanged).
- Polling cadence stays 2s; `GET /api/floor` is fetched alongside the existing three calls in
`refresh()`.
## Settings
Add a numeric field **"Delay between downloads (seconds)"** (default 0, min 0) to the settings
form (`web/src/app/settings/settings-form.tsx`), persisted to `floor.jobDelaySeconds` via the
existing settings config-write path (the same mechanism used for the other numeric/monitor
settings). Include short helptext explaining it paces sustained Qobuz load.
## Testing
- **Worker** (pytest, `worker/tests/`):
- `claim_next` skips `paused=true` queued jobs and still claims the next unpaused one.
- Global pause: the loop skips claiming when `floor.paused` is set (unit-test the claim-gate
helper).
- Delay: the delay value is read and applied (unit-test the parse/gate helper; keep the
sleep injectable/mockable).
- Kill watcher: when `floor.killRequested` is set, the watcher clears the flag and triggers
exit (mock `os._exit`, assert flag cleared + exit called).
- **Web** (route tests, mirroring `requests/[id]/retry/route.test.ts` and
`library/bulk/route.test.ts`):
- `POST /api/floor` for each action; `clear` deletes only `requested` jobs; `stop` sets both
flags.
- `POST /api/requests/[id]/pause` toggles `Job.paused`; rejects non-`requested`.
- `DELETE /api/requests/[id]` removes a queued Request; refuses an in-flight/terminal job.
- `GET /api/floor` returns the Config-derived state.
## Edge cases
- **Remove/Clear of monitored items** reappear later by design (cancel-for-now). No extra
handling; the per-release Ignore is the permanent path.
- **Kill with nothing downloading** still restarts the worker (brief blip). Minimized by only
showing `Stop now` while a download is active.
- **Delay during pause:** the delay sleeps only *between* jobs; pause is checked before each
claim, so a paused press never starts a delayed job.
- **Race:** clicking per-item Pause/Remove on a job that was just claimed — the guards
(`state='requested'`) reject it; the UI only renders those buttons on queued rows, and the
2s refresh reconciles.
## Out of scope
- Resuming a partially-downloaded album (no download-resume support; force-killed jobs
re-download from scratch).
- Reordering the queue / per-item priority.
- Any change to the per-release Ignore or the wanted list UI.
docs/superpowers/specs/2026-07-15-qobuz-pacing-design.md
+83
View File
@@ -0,0 +1,83 @@
# Qobuz pacing ("budget gate") — design
Date: 2026-07-15
## Problem
Bulk hi-res downloading got the Qobuz account blocked (`403 USER_BLOCKED`). We need to keep
using Qobuz for hi-res but at a volume and cadence that looks like a human listener, so the
account isn't flagged again — while a large wanted-list still drains promptly via other sources.
## Approach: a per-job Qobuz eligibility gate
One gate decides, **per album**, whether Qobuz is an eligible source *right now*. If the gate is
closed, Qobuz candidates are dropped before ranking, so the worker falls through to the best
non-Qobuz source (Soulseek/YouTube) and keeps moving — no idling. Monitored albums grabbed that
way are upgraded to Qobuz hi-res later, automatically, via the existing quality-upgrade window,
once Qobuz has budget again.
**Qobuz is eligible for a job only if ALL hold:**
1. **Active hours** — current hour ∈ [activeStart, activeEnd) (a daytime window; start < end).
2. **Under today's cap** — Qobuz downloads counted today < today's cap.
3. **Spacing elapsed** — now ≥ the randomized "next allowed" time set after the last Qobuz
download.
After each successful Qobuz download the worker bumps today's counter and sets the next-allowed
time. The counter resets at the day boundary (tracked by a stored date).
**Today's cap = warm-up ramp of the steady cap**, by account age (days since `warmupStartDate`):
| Account age (days) | Cap |
|---|---|
| 0–2 | min(5, steady) |
| 3–6 | min(10, steady) |
| 7–13 | min(20, steady) |
| 14+ | steady |
If `warmupStartDate` is unset, no ramp (steady cap applies).
**Spacing / jitter (combined):** the gap to the next Qobuz download is randomized around
`activeWindowSeconds / todayCap` (so the day's budget spreads evenly across active hours), ±30%,
with a hard floor (≥ 60 s). This *is* the jitter — never a fixed metronome — and it prevents
front-loading the whole daily cap into the first hour.
## Configuration (Config keys; editable in Settings → Qobuz → Pacing)
Settings:
- `qobuz.pacing.enabled` (default `true`)
- `qobuz.pacing.activeStartHour` (default `8`)
- `qobuz.pacing.activeEndHour` (default `23`)
- `qobuz.pacing.dailyCap` (steady cap, default `40`)
- `qobuz.pacing.warmupStartDate` (ISO date; set to the new account's start day)
Worker-managed state (not user-edited):
- `qobuz.pacing.countDay` (date the counter is for)
- `qobuz.pacing.countToday` (int)
- `qobuz.pacing.nextAllowedAt` (epoch seconds)
Current hour / day come from the **DB clock** (`now()`), avoiding container-timezone ambiguity.
Config is read fresh each worker loop, so changes take effect with no restart.
## Worker integration
- New module `worker/lyra_worker/qobuz_gate.py`: a `QobuzPacing` config dataclass
(`from_config`), a pure `today_cap(...)`, a pure `is_allowed(...)` decision, and thin
DB helpers `gate_open(conn, now)` (reads state + decides) and `record_download(conn, pacing,
now)` (bumps counter + sets next-allowed).
- `pipeline.py`: after searching sources, if `gate_open` is false AND at least one non-Qobuz
candidate exists, drop Qobuz candidates before ranking (fall-through). If Qobuz is the *only*
source available, keep it (a Qobuz-only setup isn't gated — holding would strand the album).
- After a successful download where `winner.source == "qobuz"`, call `record_download`.
## Testing
Pure-logic unit tests for `is_allowed` (each gate condition), `today_cap` (ramp boundaries), and
the spacing computation; pipeline tests that a gated Qobuz candidate is dropped in favour of a
non-Qobuz source, that an open gate keeps Qobuz, and that a successful Qobuz download records the
counter/next-allowed.
## Out of scope
- Holding/parking an album for later Qobuz when Qobuz is the only source (fails/last-source as
today). Multi-source setups fall through, which is the common case.
- Timezone-aware active-hours windows that wrap past midnight (start < end only).
scripts/build-push.sh
Executable
+29
View File
@@ -0,0 +1,29 @@
#!/usr/bin/env bash
# Build the Lyra web + worker images and push them to the Gitea container registry, so the VM
# can pull pre-built images instead of building from source.
#
# Usage (from the Lyra repo root):
# docker login git.jger.nl # once — Gitea username + a token with package:write
# ./scripts/build-push.sh # builds + pushes :latest and :<short-sha>
#
# Override the registry namespace with LYRA_REGISTRY (default git.jger.nl/jonathan).
set -euo pipefail
REGISTRY="${LYRA_REGISTRY:-git.jger.nl/jonathan}"
here="$(cd "$(dirname "$0")/.." && pwd)"
sha="$(git -C "$here" rev-parse --short HEAD)"
for svc in web worker; do
img="$REGISTRY/lyra-$svc"
echo "==> building $img ($sha)"
docker build -t "$img:latest" -t "$img:$sha" "$here/$svc"
done
for svc in web worker; do
img="$REGISTRY/lyra-$svc"
echo "==> pushing $img:latest and $img:$sha"
docker push "$img:latest"
docker push "$img:$sha"
done
echo "Done. Pulled on the VM with: docker compose pull && docker compose up -d"
web/package-lock.json
Generated
+31 -2
View File
@@ -7,8 +7,11 @@
"": {
"name": "lyra-web",
"version": "0.1.0",
"hasInstallScript": true,
"dependencies": {
"@prisma/client": "6.1.0",
"@types/busboy": "^1.5.4",
"busboy": "^1.6.0",
"next": "15.5.20",
"react": "19.0.0",
"react-dom": "19.0.0"
@@ -1457,6 +1460,15 @@
"tslib": "^2.8.0"
}
},
"node_modules/@types/busboy": {
"version": "1.5.4",
"resolved": "https://registry.npmjs.org/@types/busboy/-/busboy-1.5.4.tgz",
"integrity": "sha512-kG7WrUuAKK0NoyxfQHsVE6j1m01s6kMma64E+OZenQABMQyTJop1DumUWcLwAQ2JzpefU7PDYoRDKl8uZosFjw==",
"license": "MIT",
"dependencies": {
"@types/node": "*"
}
},
"node_modules/@types/estree": {
"version": "1.0.9",
"resolved": "https://registry.npmjs.org/@types/estree/-/estree-1.0.9.tgz",
@@ -1468,7 +1480,6 @@
"version": "22.10.2",
"resolved": "https://registry.npmjs.org/@types/node/-/node-22.10.2.tgz",
"integrity": "sha512-Xxr6BBRCAOQixvonOye19wnzyDiUtTeqldOOmj3CkeblonbccA12PFwlufvRdrpjXxqnmUaeiU5EOA+7s5diUQ==",
"dev": true,
"license": "MIT",
"dependencies": {
"undici-types": "~6.20.0"
@@ -1643,6 +1654,17 @@
"node": ">=12"
}
},
"node_modules/busboy": {
"version": "1.6.0",
"resolved": "https://registry.npmjs.org/busboy/-/busboy-1.6.0.tgz",
"integrity": "sha512-8SFQbg/0hQ9xy3UNTB0YEnsNBbWfhf7RtnzpL7TkBiTBRfrQ9Fxcnz7VJsleJpyp6rVLvXiuORqjlHi5q+PYuA==",
"dependencies": {
"streamsearch": "^1.1.0"
},
"engines": {
"node": ">=10.16.0"
}
},
"node_modules/cac": {
"version": "6.7.14",
"resolved": "https://registry.npmjs.org/cac/-/cac-6.7.14.tgz",
@@ -2157,6 +2179,14 @@
"dev": true,
"license": "MIT"
},
"node_modules/streamsearch": {
"version": "1.1.0",
"resolved": "https://registry.npmjs.org/streamsearch/-/streamsearch-1.1.0.tgz",
"integrity": "sha512-Mcc5wHehp9aXz1ax6bZUyY5afg9u2rv5cqQI3mRrYkGC8rW2hM02jWuwjtL++LS5qinSyhj2QfLyNsuc+VsExg==",
"engines": {
"node": ">=10.0.0"
}
},
"node_modules/styled-jsx": {
"version": "5.1.6",
"resolved": "https://registry.npmjs.org/styled-jsx/-/styled-jsx-5.1.6.tgz",
@@ -2248,7 +2278,6 @@
"version": "6.20.0",
"resolved": "https://registry.npmjs.org/undici-types/-/undici-types-6.20.0.tgz",
"integrity": "sha512-Ny6QZ2Nju20vw1SRHe3d9jVu6gJ+4e3+MMpqu7pqE5HT6WsTSlce++GQmK5UXS8mzV8DSYHrQH+Xrf2jVcuKNg==",
"dev": true,
"license": "MIT"
},
"node_modules/vite": {
web/package.json
+5 -3
View File
@@ -12,17 +12,19 @@
"db:dev": "prisma migrate dev"
},
"dependencies": {
"@prisma/client": "6.1.0",
"@types/busboy": "^1.5.4",
"busboy": "^1.6.0",
"next": "15.5.20",
"react": "19.0.0",
"react-dom": "19.0.0",
"@prisma/client": "6.1.0"
"react-dom": "19.0.0"
},
"devDependencies": {
"typescript": "5.7.2",
"@types/node": "22.10.2",
"@types/react": "19.0.1",
"@types/react-dom": "19.0.1",
"prisma": "6.1.0",
"typescript": "5.7.2",
"vitest": "2.1.8"
}
}
web/prisma/migrations/20260714180402_add_library_rg_mbid/migration.sql
+2
View File
@@ -0,0 +1,2 @@
-- AlterTable
ALTER TABLE "LibraryItem" ADD COLUMN "rgMbid" TEXT;
web/prisma/migrations/20260714185225_add_unmatched_album/migration.sql
+15
View File
@@ -0,0 +1,15 @@
-- CreateTable
CREATE TABLE "UnmatchedAlbum" (
"id" TEXT NOT NULL,
"artist" TEXT NOT NULL,
"album" TEXT NOT NULL,
"path" TEXT NOT NULL,
"reason" TEXT NOT NULL,
"scanId" TEXT NOT NULL,
"scannedAt" TIMESTAMP(3) NOT NULL DEFAULT CURRENT_TIMESTAMP,
CONSTRAINT "UnmatchedAlbum_pkey" PRIMARY KEY ("id")
);
-- CreateIndex
CREATE UNIQUE INDEX "UnmatchedAlbum_path_key" ON "UnmatchedAlbum"("path");
web/prisma/migrations/20260714190047_add_release_ignored/migration.sql
+2
View File
@@ -0,0 +1,2 @@
-- AlterTable
ALTER TABLE "MonitoredRelease" ADD COLUMN "ignored" BOOLEAN NOT NULL DEFAULT false;
web/prisma/migrations/20260714192517_add_request_force/migration.sql
+2
View File
@@ -0,0 +1,2 @@
-- AlterTable
ALTER TABLE "Request" ADD COLUMN "force" BOOLEAN NOT NULL DEFAULT false;
web/prisma/migrations/20260714195144_add_library_artist_mbid/migration.sql
+2
View File
@@ -0,0 +1,2 @@
-- AlterTable
ALTER TABLE "LibraryItem" ADD COLUMN "artistMbid" TEXT;
web/prisma/migrations/20260714203518_add_discovery_seed/migration.sql
+7
View File
@@ -0,0 +1,7 @@
-- CreateTable
CREATE TABLE "DiscoverySeed" (
"mbid" TEXT NOT NULL,
"lastDiscoveredAt" TIMESTAMP(3),
CONSTRAINT "DiscoverySeed_pkey" PRIMARY KEY ("mbid")
);
web/prisma/migrations/20260714213448_add_discovery_album_reason/migration.sql
+2
View File
@@ -0,0 +1,2 @@
-- AlterTable
ALTER TABLE "DiscoverySuggestion" ADD COLUMN "albumReason" TEXT;
web/prisma/migrations/20260715000000_add_job_paused/migration.sql
+2
View File
@@ -0,0 +1,2 @@
-- AlterTable
ALTER TABLE "Job" ADD COLUMN "paused" BOOLEAN NOT NULL DEFAULT false;
web/prisma/migrations/20260720200000_add_job_download_eta/migration.sql
+2
View File
@@ -0,0 +1,2 @@
-- Estimated seconds remaining for the active download (measured throughput). Nullable.
ALTER TABLE "Job" ADD COLUMN "downloadEtaSeconds" INTEGER;
web/prisma/schema.prisma
+46
View File
@@ -35,6 +35,10 @@ model Request {
album String
status RequestStatus @default(pending)
createdAt DateTime @default(now())
// Force re-acquisition: skip the "already in library" dedupe so an owned album is
// re-downloaded on demand. The import step still keeps the copy only if it's higher
// quality, so a forced upgrade never downgrades. Set by the Library "Replace / upgrade".
force Boolean @default(false)
job Job?
libraryItem LibraryItem?
@@ -54,6 +58,10 @@ model Job {
createdAt DateTime @default(now())
updatedAt DateTime @updatedAt
downloadProgress Float @default(0)
// Estimated seconds until the current download finishes, from measured throughput (slskd byte
// transfer rate). Null when unknown (no speed sample yet, or a source that doesn't report bytes).
downloadEtaSeconds Int?
paused Boolean @default(false)
candidates Candidate[]
}
@@ -82,6 +90,13 @@ model LibraryItem {
format String
qualityClass Int
importedAt DateTime @default(now())
// MusicBrainz release-group MBID when known (resolved at manual import) — gives the album
// cover art without depending on a matching MonitoredRelease row.
rgMbid String?
// MusicBrainz ARTIST MBID when known (captured at import + scan). Own-state ("in library")
// prefers this over the fragile exact artist-name match; null rows fall back to name-match
// (re-scan to populate).
artistMbid String?
// The album's on-disk audio filenames ("## Title.ext"), captured at import + scan, so the
// Library modal can show the REAL tracks on disk (revealing incomplete/edition mismatches)
// instead of MusicBrainz's canonical listing. Empty for items imported before this existed
@@ -123,6 +138,10 @@ model MonitoredRelease {
secondaryTypes String[]
firstReleaseDate String?
monitored Boolean @default(false)
// User marked this release "do not monitor" — the monitor skips it (never enqueued as
// wanted/upgrade) and it drops off the Wanted list, independent of `monitored`, so
// auto-monitor-future can't silently re-grab it. Reversible from the discography page.
ignored Boolean @default(false)
state MonitoredReleaseState @default(wanted)
currentQualityClass Int?
firstGrabbedAt DateTime?
@@ -156,6 +175,10 @@ model DiscoverySuggestion {
score Float
seedCount Int
sources String[]
// Why this album is suggested (kind='album' only): "newest" (the artist's newest un-owned
// release), "popular" (their most-played per Last.fm), or "newest,popular" when they're the
// same release. Null for artist suggestions / rows from before this column existed.
albumReason String?
status SuggestionStatus @default(pending)
dedupeKey String @unique
createdAt DateTime @default(now())
@@ -176,6 +199,14 @@ model DiscoverySeedContribution {
@@index([seedMbid])
}
// Throttle registry for discovery seeds that are NOT WatchedArtist rows — i.e. artists you own
// albums by (LibraryItem.artistMbid) but don't follow. Mirrors WatchedArtist.lastDiscoveredAt so
// each library-derived seed is swept once per interval, not every sweep.
model DiscoverySeed {
mbid String @id
lastDiscoveredAt DateTime?
}
model ScanWorkItem {
id String @id @default(cuid())
scanId String
@@ -188,6 +219,21 @@ model ScanWorkItem {
@@index([scanId, done, artist, album])
}
// Album folders on disk that the library scan could not resolve to a MusicBrainz release
// (no match, or an unreadable/corrupt file). Surfaced on /library so the user can fix the
// metadata or add them manually instead of them being silently skipped. Keyed by path; a
// row is removed when the album later matches, and stale rows (a prior scan's, or a folder
// since removed) are pruned when a scan completes.
model UnmatchedAlbum {
id String @id @default(cuid())
artist String
album String
path String @unique
reason String
scanId String
scannedAt DateTime @default(now())
}
// Generic read-through cache for external API responses shared by web + worker (keyed on
// the LOGICAL operation, not the request URL, so both processes share entries). TTL is
// applied at read time from fetchedAt in code (no expiresAt column → tunable, no migration).
web/src/app/_ui/artist-picker.tsx
+49
View File
@@ -0,0 +1,49 @@
"use client";
import { Modal } from "./modal";
import type { ArtistHit } from "@/lib/artist-resolve";
/** Disambiguation picker: several MusicBrainz artists share a name (e.g. two bands both called
* "Looking Glass"), so let the user choose the right one by its MB disambiguation text instead
* of silently trusting Last.fm's — sometimes wrong — MBID. Candidates arrive in MB relevance
* order, so the first is flagged "Suggested". */
export function ArtistPicker({
open,
name,
candidates,
onPick,
onClose,
}: {
open: boolean;
name: string;
candidates: ArtistHit[];
onPick: (mbid: string, name: string) => void;
onClose: () => void;
}) {
return (
<Modal open={open} onClose={onClose} title="Which artist?">
<p className="muted-note">
More than one artist is named “{name}”. Pick the right one.
</p>
<ul className="list">
{candidates.map((c, i) => (
<li key={c.mbid} className="list-row">
<div className="main">
<div className="rtitle">
<button className="linkish" onClick={() => onPick(c.mbid, c.name)}>
{c.name}
</button>
</div>
<div className="rmeta">
<span>{c.disambiguation || "no description on MusicBrainz"}</span>
</div>
</div>
<div className="actions">
{i === 0 ? <span className="chip working">Suggested</span> : null}
</div>
</li>
))}
</ul>
</Modal>
);
}
web/src/app/_ui/cover-art.tsx
+4 -4
View File
@@ -2,9 +2,9 @@
import { useState } from "react";
/** Album art from the Cover Art Archive (keyed by release-group MBID), loaded straight from
* their CDN by the browser — no backend, no MB API rate limit. Falls back to a placeholder
* when there's no MBID or no art. */
/** Album art keyed by release-group MBID, served through /api/cover (which caches it on disk +
* sends a long Cache-Control, so it isn't re-fetched from the Cover Art Archive on every
* Library / Recently-Pressed render). Falls back to a placeholder when there's no MBID or art. */
export function CoverArt({
rgMbid,
alt,
@@ -29,7 +29,7 @@ export function CoverArt({
// eslint-disable-next-line @next/next/no-img-element
<img
className={cls}
src={`https://coverartarchive.org/release-group/${rgMbid}/front-${size}`}
src={`/api/cover/${rgMbid}?size=${size}`}
alt={alt}
loading="lazy"
onError={() => setFailed(true)}
web/src/app/_ui/job-row.tsx
+3
View File
@@ -14,6 +14,7 @@ export function JobRow({
album,
kind,
label,
marker,
meta,
note,
bar,
@@ -22,6 +23,7 @@ export function JobRow({
album: string;
kind: Kind;
label: string;
marker?: string;
meta?: ReactNode;
note?: ReactNode;
bar?: ReactNode;
@@ -31,6 +33,7 @@ export function JobRow({
<div className="stripe" />
<div>
<h3 className="title">
{marker ? <span className="nextup">{marker}</span> : null}
{album} <span className="artist">· {artist}</span>
</h3>
{meta ? <div className="meta">{meta}</div> : null}
web/src/app/_ui/kebab-menu.tsx
+47
View File
@@ -0,0 +1,47 @@
"use client";
import { useEffect, useRef, useState, type ReactNode } from "react";
/** A vertical "⋮" trigger that opens a small popup menu. Closes on outside click, Escape, or any
* click inside the popup (so menu items don't each need to manage it). Children are the items —
* use <button className="menu-item">…</button>. */
export function KebabMenu({ label = "Actions", children }: { label?: string; children: ReactNode }) {
const [open, setOpen] = useState(false);
const ref = useRef<HTMLDivElement>(null);
useEffect(() => {
if (!open) return;
function onDoc(e: MouseEvent) {
if (ref.current && !ref.current.contains(e.target as Node)) setOpen(false);
}
function onKey(e: KeyboardEvent) {
if (e.key === "Escape") setOpen(false);
}
document.addEventListener("mousedown", onDoc);
document.addEventListener("keydown", onKey);
return () => {
document.removeEventListener("mousedown", onDoc);
document.removeEventListener("keydown", onKey);
};
}, [open]);
return (
<div className="menu" ref={ref}>
<button
type="button"
className="menu-trigger"
aria-haspopup="menu"
aria-expanded={open}
aria-label={label}
onClick={() => setOpen((o) => !o)}
>
⋮
</button>
{open ? (
<div className="menu-pop" role="menu" onClick={() => setOpen(false)}>
{children}
</div>
) : null}
</div>
);
}
web/src/app/_ui/section-header.tsx
+3 -1
View File
@@ -1,4 +1,6 @@
export function SectionHeader({ title, note }: { title: string; note?: string }) {
import type { ReactNode } from "react";
export function SectionHeader({ title, note }: { title: string; note?: ReactNode }) {
return (
<div className="dept">
<h2>{title}</h2>
web/src/app/_ui/status.test.ts
+16 -1
View File
@@ -1,5 +1,20 @@
import { describe, it, expect } from "vitest";
import { describeJob, timeAgo } from "./status";
import { describeJob, etaLabel, timeAgo } from "./status";
describe("etaLabel", () => {
it("formats seconds, minutes, and hours", () => {
expect(etaLabel(45)).toBe("~45s left");
expect(etaLabel(240)).toBe("~4m left");
expect(etaLabel(3900)).toBe("~1h 5m left");
});
it("returns null for missing or non-positive input", () => {
expect(etaLabel(null)).toBeNull();
expect(etaLabel(undefined)).toBeNull();
expect(etaLabel(0)).toBeNull();
expect(etaLabel(-5)).toBeNull();
});
});
describe("describeJob", () => {
it("maps pipeline states to literal labels + severity kind", () => {
web/src/app/_ui/status.ts
+10
View File
@@ -27,6 +27,16 @@ export function timeAgo(value: string | Date | null | undefined, now: number): s
return `${Math.floor(h / 24)}d`;
}
/** Compact download ETA like "~4m left", "~45s left", "~1h 5m left". Null/≤0 → no label. */
export function etaLabel(seconds: number | null | undefined): string | null {
if (seconds == null || seconds <= 0) return null;
if (seconds < 60) return `~${seconds}s left`;
const m = Math.round(seconds / 60);
if (m < 60) return `~${m}m left`;
const h = Math.floor(m / 60);
return `~${h}h ${m % 60}m left`;
}
export function describeJob(state: string, stage: string): { label: string; kind: Kind } {
void stage; // stage no longer drives the label; kept for call-site compatibility
return (
web/src/app/api/artists/[id]/route.ts
+1
View File
@@ -27,6 +27,7 @@ export async function GET(request: Request, { params }: { params: Promise<{ id:
secondaryTypes: r.secondaryTypes,
firstReleaseDate: r.firstReleaseDate,
monitored: r.monitored,
ignored: r.ignored,
state: r.state,
currentQualityClass: r.currentQualityClass,
})),
web/src/app/api/artists/route.test.ts
+10
View File
@@ -98,6 +98,16 @@ describe("artists collection API", () => {
expect(body.ownedNotFollowed).not.toContain("followed artist"); // case-insensitive match
});
it("GET matches owned artists to the watched roster by MBID despite a name mismatch", async () => {
// followed under the canonical MB name, but the library row's name differs (feat./locale)
await prisma.watchedArtist.create({ data: { mbid: "mb-beyonce", name: "Beyoncé" } });
await prisma.libraryItem.create({
data: { artist: "Beyonce", album: "Lemonade", artistMbid: "mb-beyonce", path: "/l", source: "scan", format: "FLAC", qualityClass: 2 },
});
const body = await (await GET(listReq())).json();
expect(body.ownedNotFollowed).not.toContain("Beyonce"); // matched by MBID → not "unfollowed"
});
it("GET ?mbid= returns the follow state for a single artist", async () => {
await prisma.watchedArtist.create({ data: { mbid: "followed-1", name: "Followed" } });
expect((await (await GET(listReq("?mbid=followed-1"))).json()).followed).toBe(true);
web/src/app/api/artists/route.ts
+13 -7
View File
@@ -20,16 +20,21 @@ export async function GET(request: Request) {
},
},
}),
prisma.libraryItem.findMany({ select: { artist: true }, distinct: ["artist"] }),
prisma.libraryItem.findMany({ select: { artist: true, artistMbid: true } }),
]);
// Artists you own music by (LibraryItem) but don't follow yet — name-matched
// case-insensitively against the watched roster (the have-via-scan name-match caveat).
// Artists you own music by (LibraryItem) but don't follow yet. Match on the MusicBrainz
// ARTIST MBID first (robust to name punctuation/locale/feat. variants), falling back to a
// case-insensitive name match for library rows that predate the artistMbid column.
const followedMbids = new Set(artists.map((a) => a.mbid));
const followedNames = new Set(artists.map((a) => a.name.trim().toLowerCase()));
const ownedNotFollowed = libraryArtists
.map((r) => r.artist)
.filter((name) => name.trim() && !followedNames.has(name.trim().toLowerCase()))
.sort((a, b) => a.localeCompare(b));
const isFollowed = (r: { artist: string; artistMbid: string | null }) =>
(r.artistMbid && followedMbids.has(r.artistMbid)) || followedNames.has(r.artist.trim().toLowerCase());
const ownedNotFollowed = [
...new Set(
libraryArtists.filter((r) => r.artist.trim() && !isFollowed(r)).map((r) => r.artist),
),
].sort((a, b) => a.localeCompare(b));
return Response.json({
artists: artists.map((a) => {
@@ -38,6 +43,7 @@ export async function GET(request: Request) {
id: a.id,
mbid: a.mbid,
name: a.name,
createdAt: a.createdAt,
autoMonitorFuture: a.autoMonitorFuture,
showAllTypes: a.showAllTypes,
releaseCount: visible.length,
web/src/app/api/cover/[rgMbid]/route.test.ts
+52
View File
@@ -0,0 +1,52 @@
import { describe, it, expect, beforeEach, afterEach, vi } from "vitest";
import { mkdtempSync, rmSync, existsSync } from "node:fs";
import { tmpdir } from "node:os";
import { join } from "node:path";
import { GET } from "./route";
let cache: string;
const original = process.env.LYRA_COVER_CACHE;
const RG = "735bcee6-6be6-3957-af15-d21ba2bd355b";
beforeEach(() => {
cache = mkdtempSync(join(tmpdir(), "lyra-cov-"));
process.env.LYRA_COVER_CACHE = cache;
});
afterEach(() => {
vi.unstubAllGlobals();
if (original === undefined) delete process.env.LYRA_COVER_CACHE;
else process.env.LYRA_COVER_CACHE = original;
rmSync(cache, { recursive: true, force: true });
});
const req = (rg: string) => new Request(`http://x/api/cover/${rg}?size=250`);
const params = (rg: string) => ({ params: Promise.resolve({ rgMbid: rg }) });
describe("GET /api/cover/[rgMbid]", () => {
it("400s a non-uuid", async () => {
expect((await GET(req("not-a-uuid"), params("not-a-uuid"))).status).toBe(400);
});
it("fetches from CAA once, then serves from the disk cache", async () => {
const fetchMock = vi.fn(async () => new Response(new Uint8Array([1, 2, 3]), { status: 200 }));
vi.stubGlobal("fetch", fetchMock);
const r1 = await GET(req(RG), params(RG));
expect(r1.status).toBe(200);
expect(r1.headers.get("cache-control")).toContain("max-age");
expect(existsSync(join(cache, `${RG}-250.jpg`))).toBe(true);
const r2 = await GET(req(RG), params(RG));
expect(r2.status).toBe(200);
expect(fetchMock).toHaveBeenCalledTimes(1); // second served from cache
});
it("caches a miss (404) so it isn't re-fetched", async () => {
const fetchMock = vi.fn(async () => new Response(null, { status: 404 }));
vi.stubGlobal("fetch", fetchMock);
expect((await GET(req(RG), params(RG))).status).toBe(404);
expect((await GET(req(RG), params(RG))).status).toBe(404);
expect(fetchMock).toHaveBeenCalledTimes(1); // negative cache hit on the second
});
});
web/src/app/api/cover/[rgMbid]/route.ts
+45
View File
@@ -0,0 +1,45 @@
import { existsSync } from "node:fs";
import { readFile, writeFile, mkdir } from "node:fs/promises";
import { join } from "node:path";
// Cover art proxy + cache. The browser used to hit the Cover Art Archive CDN directly on every
// Library / Recently-Pressed render; now it hits this, which caches the image on disk and serves
// it with a long Cache-Control so the browser caches it too. A negative marker caps re-fetching
// covers that don't exist.
const CACHEABLE = "public, max-age=2592000"; // 30 days
const NEG_CACHE = "public, max-age=86400"; // re-check a missing cover daily
const UUID = /^[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}$/i;
export async function GET(request: Request, { params }: { params: Promise<{ rgMbid: string }> }) {
const { rgMbid } = await params;
if (!UUID.test(rgMbid)) return new Response(null, { status: 400 });
const size = new URL(request.url).searchParams.get("size") === "500" ? "500" : "250";
const CACHE = process.env.LYRA_COVER_CACHE ?? "/tmp/lyra-covers";
const key = `${rgMbid}-${size}`;
const file = join(CACHE, `${key}.jpg`);
const miss = join(CACHE, `${key}.miss`);
try {
if (existsSync(file)) {
return new Response(await readFile(file), {
headers: { "Content-Type": "image/jpeg", "Cache-Control": CACHEABLE },
});
}
if (existsSync(miss)) {
return new Response(null, { status: 404, headers: { "Cache-Control": NEG_CACHE } });
}
const res = await fetch(`https://coverartarchive.org/release-group/${rgMbid}/front-${size}`, {
headers: { "User-Agent": "Lyra/0.1 ( https://git.jger.nl/Jonathan/Lyra )" },
});
await mkdir(CACHE, { recursive: true });
if (!res.ok) {
await writeFile(miss, "").catch(() => {});
return new Response(null, { status: 404, headers: { "Cache-Control": NEG_CACHE } });
}
const buf = Buffer.from(await res.arrayBuffer());
await writeFile(file, buf).catch(() => {});
return new Response(buf, { headers: { "Content-Type": "image/jpeg", "Cache-Control": CACHEABLE } });
} catch {
return new Response(null, { status: 502 });
}
}
web/src/app/api/discover/[id]/route.test.ts
+14
View File
@@ -63,6 +63,20 @@ describe("discover action API", () => {
expect((await prisma.discoverySuggestion.findUnique({ where: { id: s.id } }))!.status).toBe("followed");
});
it("dismissing an artist also dismisses its pending album suggestions", async () => {
const a = await prisma.discoverySuggestion.create({
data: { kind: "artist", artistMbid: "shared", artistName: "Cand", score: 1, seedCount: 1,
sources: ["listenbrainz"], secondaryTypes: [], dedupeKey: "artist:shared:" },
});
const alb = await prisma.discoverySuggestion.create({
data: { kind: "album", artistMbid: "shared", artistName: "Cand", rgMbid: "rgS", album: "Rec",
albumReason: "newest", score: 1, seedCount: 1, sources: ["listenbrainz"],
secondaryTypes: [], dedupeKey: "album:shared:rgS" },
});
await post(a.id, { action: "dismiss" });
expect((await prisma.discoverySuggestion.findUnique({ where: { id: alb.id } }))!.status).toBe("dismissed");
});
it("want upserts a monitored release from the suggestion and marks wanted", async () => {
const s = await album("rgW");
const res = await post(s.id, { action: "want" });
web/src/app/api/discover/[id]/route.ts
+8
View File
@@ -19,6 +19,14 @@ export async function POST(request: Request, { params }: { params: Promise<{ id:
if (action === "dismiss") {
await prisma.discoverySuggestion.update({ where: { id }, data: { status: "dismissed" } });
// Dismissing an artist also dismisses its album suggestions, so the whole unified row
// disappears cleanly instead of leaving the artist's albums stranded.
if (s.kind === "artist") {
await prisma.discoverySuggestion.updateMany({
where: { kind: "album", artistMbid: s.artistMbid, status: "pending" },
data: { status: "dismissed" },
});
}
return Response.json({ id, status: "dismissed" });
}
web/src/app/api/discover/config/route.ts
+2
View File
@@ -5,6 +5,8 @@ const DEFAULTS: Record<string, string> = {
"discover.intervalHours": "168",
"discover.similarPerSeed": "20",
"discover.albumsPerArtist": "1",
"discover.albumsOnly": "true",
"discover.seedFromLibrary": "true",
"discover.minScore": "0",
"discover.chunkSize": "5",
"discover.listenBrainzUrl": "",
web/src/app/api/discover/reset/route.test.ts
+17
View File
@@ -0,0 +1,17 @@
import { describe, it, expect } from "vitest";
import { prisma } from "@/lib/db";
import { POST } from "./route";
describe("POST /api/discover/reset", () => {
it("sets the reset + requested flags so the worker rebuilds on the next sweep", async () => {
const res = await POST();
expect(res.status).toBe(202);
expect((await res.json()).reset).toBe(true);
const cfg = Object.fromEntries(
(await prisma.config.findMany({ where: { key: { in: ["discover.resetRequested", "discover.requested"] } } }))
.map((c) => [c.key, c.value]),
);
expect(cfg["discover.resetRequested"]).toBe("true");
expect(cfg["discover.requested"]).toBe("true");
});
});
web/src/app/api/discover/reset/route.ts
+17
View File
@@ -0,0 +1,17 @@
import { prisma } from "@/lib/db";
// POST /api/discover/reset — request a full rebuild: the worker clears the pending
// suggestions, contributions, and seed throttles at the start of the next sweep, then
// regenerates everything with the current logic. Kicks a sweep immediately.
export async function POST() {
await Promise.all(
["discover.resetRequested", "discover.requested"].map((key) =>
prisma.config.upsert({
where: { key },
create: { key, value: "true", secret: false },
update: { value: "true" },
}),
),
);
return Response.json({ reset: true }, { status: 202 });
}
web/src/app/api/discover/route.test.ts
+47 -8
View File
@@ -3,22 +3,61 @@ import { prisma } from "@/lib/db";
import { GET } from "./route";
describe("discover feed API", () => {
it("returns pending artists and albums split, highest score first, excluding non-pending", async () => {
it("groups pending suggestions into artist rows with their albums, highest score first", async () => {
await prisma.discoverySuggestion.createMany({
data: [
{ kind: "artist", artistMbid: "a1", artistName: "Low", score: 0.2, seedCount: 1,
sources: ["listenbrainz"], secondaryTypes: [], dedupeKey: "artist:a1:" },
{ kind: "artist", artistMbid: "a2", artistName: "High", score: 0.9, seedCount: 2,
sources: ["listenbrainz"], secondaryTypes: [], dedupeKey: "artist:a2:" },
{ kind: "album", artistMbid: "a3", artistName: "Band", rgMbid: "rg1", album: "Rec",
score: 0.5, seedCount: 1, sources: ["listenbrainz"], secondaryTypes: [], dedupeKey: "album:a3:rg1" },
{ kind: "artist", artistMbid: "a4", artistName: "Gone", score: 5, seedCount: 1,
sources: ["listenbrainz"], secondaryTypes: [], status: "dismissed", dedupeKey: "artist:a4:" },
{ kind: "album", artistMbid: "a2", artistName: "High", rgMbid: "rg-new", album: "Latest",
firstReleaseDate: "2022-01-01", albumReason: "newest", score: 0.9, seedCount: 2,
sources: ["listenbrainz"], secondaryTypes: [], dedupeKey: "album:a2:rg-new" },
{ kind: "album", artistMbid: "a2", artistName: "High", rgMbid: "rg-hit", album: "Big Hit",
firstReleaseDate: "2018-01-01", albumReason: "popular", score: 0.9, seedCount: 2,
sources: ["listenbrainz"], secondaryTypes: [], dedupeKey: "album:a2:rg-hit" },
{ kind: "artist", artistMbid: "a3", artistName: "Gone", score: 5, seedCount: 1,
sources: ["listenbrainz"], secondaryTypes: [], status: "dismissed", dedupeKey: "artist:a3:" },
],
});
const body = await (await GET()).json();
expect(body.artists.map((a: any) => a.artistName)).toEqual(["High", "Low"]); // dismissed excluded
expect(body.albums.map((a: any) => a.album)).toEqual(["Rec"]);
expect(body.artists[0]).toMatchObject({ artistMbid: "a2", seedCount: 2, sources: ["listenbrainz"] });
expect(body.rows.map((r: { artistName: string }) => r.artistName)).toEqual(["High", "Low"]); // dismissed excluded, score desc
const high = body.rows[0];
expect(high).toMatchObject({ id: expect.any(String), artistMbid: "a2", seedCount: 2 });
// albums sorted newest-first, each carrying its reason
expect(high.albums.map((a: { album: string; albumReason: string }) => [a.album, a.albumReason]))
.toEqual([["Latest", "newest"], ["Big Hit", "popular"]]);
expect(body.rows[1].albums).toEqual([]); // "Low" has no album suggestions
});
it("yields a row for an album whose artist has no pending artist suggestion", async () => {
await prisma.discoverySuggestion.create({
data: { kind: "album", artistMbid: "orphan", artistName: "Orphan Band", rgMbid: "rgX",
album: "Stray", albumReason: "newest", score: 0.4, seedCount: 1, sources: ["lastfm"],
secondaryTypes: [], dedupeKey: "album:orphan:rgX" },
});
const body = await (await GET()).json();
expect(body.rows).toHaveLength(1);
expect(body.rows[0]).toMatchObject({ id: null, artistName: "Orphan Band" });
expect(body.rows[0].albums[0].album).toBe("Stray");
});
it("annotates rows with the followed-artist seeds that surfaced them", async () => {
await prisma.watchedArtist.createMany({
data: [{ mbid: "seed-am", name: "Arctic Monkeys" }, { mbid: "seed-cp", name: "Coldplay" }],
});
await prisma.discoverySuggestion.create({
data: { kind: "artist", artistMbid: "cand1", artistName: "The Strokes", score: 0.9, seedCount: 2,
sources: ["listenbrainz"], secondaryTypes: [], dedupeKey: "artist:cand1:" },
});
await prisma.discoverySeedContribution.createMany({
data: [
{ candidateMbid: "cand1", candidateName: "The Strokes", seedMbid: "seed-am", score: 0.6, sources: ["listenbrainz"] },
{ candidateMbid: "cand1", candidateName: "The Strokes", seedMbid: "seed-cp", score: 0.3, sources: ["listenbrainz"] },
{ candidateMbid: "cand1", candidateName: "The Strokes", seedMbid: "gone", score: 0.9, sources: ["listenbrainz"] },
],
});
const body = await (await GET()).json();
expect(body.rows[0].seeds).toEqual(["Arctic Monkeys", "Coldplay"]); // score desc, unfollowed dropped
});
});
web/src/app/api/discover/route.ts
+83 -9
View File
@@ -1,25 +1,99 @@
import { prisma } from "@/lib/db";
// One Discover row = a suggested artist plus that artist's suggested albums (newest /
// most-played), since album suggestions are always derived from a surfaced artist. Pending
// suggestions are grouped by artistMbid so the UI shows a single artist-centric row.
type Row = {
id: string | null; // the artist suggestion id (Follow/Dismiss); null if only albums exist
artistMbid: string;
artistName: string;
score: number;
seedCount: number;
sources: string[];
seeds: string[];
albums: {
id: string;
rgMbid: string | null;
album: string | null;
primaryType: string | null;
secondaryTypes: string[];
firstReleaseDate: string | null;
albumReason: string | null;
}[];
};
export async function GET() {
const rows = await prisma.discoverySuggestion.findMany({
where: { status: "pending" },
orderBy: [{ score: "desc" }, { createdAt: "asc" }],
});
const map = (r: (typeof rows)[number]) => ({
id: r.id,
// "Why suggested": the followed artists (seeds) that surfaced each candidate, from the
// per-seed contributions joined to the current watched roster (an unfollowed seed drops out).
const candidateMbids = [...new Set(rows.map((r) => r.artistMbid))];
const contribs = candidateMbids.length
? await prisma.discoverySeedContribution.findMany({
where: { candidateMbid: { in: candidateMbids } },
orderBy: { score: "desc" },
})
: [];
const seedNames = new Map(
(await prisma.watchedArtist.findMany({ select: { mbid: true, name: true } })).map((w) => [w.mbid, w.name]),
);
const seedsByCandidate = new Map<string, string[]>();
for (const c of contribs) {
const name = seedNames.get(c.seedMbid);
if (!name) continue;
const list = seedsByCandidate.get(c.candidateMbid) ?? [];
if (!list.includes(name)) list.push(name);
seedsByCandidate.set(c.candidateMbid, list);
}
// Group by artist. An artist suggestion sets the row header; album suggestions fill albums[].
// An album whose artist has no pending artist suggestion still yields a header (never orphaned).
const byArtist = new Map<string, Row>();
const ensure = (r: (typeof rows)[number]): Row => {
let row = byArtist.get(r.artistMbid);
if (!row) {
row = {
id: null,
artistMbid: r.artistMbid,
artistName: r.artistName,
score: r.score,
seedCount: r.seedCount,
sources: r.sources,
seeds: seedsByCandidate.get(r.artistMbid) ?? [],
albums: [],
};
byArtist.set(r.artistMbid, row);
}
return row;
};
for (const r of rows) {
const row = ensure(r);
if (r.kind === "artist") {
row.id = r.id;
row.score = r.score;
row.seedCount = r.seedCount;
row.sources = r.sources;
} else {
row.albums.push({
id: r.id,
rgMbid: r.rgMbid,
album: r.album,
primaryType: r.primaryType,
secondaryTypes: r.secondaryTypes,
firstReleaseDate: r.firstReleaseDate,
score: r.score,
seedCount: r.seedCount,
sources: r.sources,
});
return Response.json({
artists: rows.filter((r) => r.kind === "artist").map(map),
albums: rows.filter((r) => r.kind === "album").map(map),
albumReason: r.albumReason,
});
row.score = Math.max(row.score, r.score); // album-only row scores by its best album
}
}
for (const row of byArtist.values()) {
row.albums.sort((a, b) => (b.firstReleaseDate ?? "").localeCompare(a.firstReleaseDate ?? ""));
}
const out = [...byArtist.values()].sort((a, b) => b.score - a.score);
return Response.json({ rows: out });
}
web/src/app/api/floor/route.test.ts
+63
View File
@@ -0,0 +1,63 @@
import { describe, it, expect } from "vitest";
import { GET, POST } from "./route";
import { prisma } from "@/lib/db";
function post(action: string) {
return new Request("http://localhost/api/floor", {
method: "POST",
headers: { "content-type": "application/json" },
body: JSON.stringify({ action }),
});
}
async function flag(key: string) {
return (await prisma.config.findUnique({ where: { key } }))?.value ?? null;
}
async function queuedRequest(album: string, state = "requested") {
return prisma.request.create({
data: { artist: "A", album, status: "pending", job: { create: { state: state as never } } },
include: { job: true },
});
}
describe("floor API", () => {
it("GET reports paused=false by default", async () => {
const res = await GET();
expect(await res.json()).toEqual({ paused: false });
});
it("pause and resume toggle floor.paused", async () => {
await POST(post("pause"));
expect(await flag("floor.paused")).toBe("true");
const paused = await (await GET()).json();
expect(paused).toEqual({ paused: true });
await POST(post("resume"));
expect(await flag("floor.paused")).toBe("false");
});
it("stop sets both paused and killRequested", async () => {
const res = await POST(post("stop"));
expect(res.status).toBe(200);
expect(await flag("floor.paused")).toBe("true");
expect(await flag("floor.killRequested")).toBe("true");
});
it("clear deletes only queued (requested) requests", async () => {
const queued = await queuedRequest("Queued");
const active = await queuedRequest("Downloading", "downloading");
const res = await POST(post("clear"));
const body = await res.json();
expect(body.cleared).toBe(1);
expect(await prisma.request.findUnique({ where: { id: queued.id } })).toBeNull();
expect(await prisma.request.findUnique({ where: { id: active.id } })).not.toBeNull();
});
it("rejects an unknown action", async () => {
const res = await POST(post("nope"));
expect(res.status).toBe(400);
});
});
web/src/app/api/floor/route.ts
+49
View File
@@ -0,0 +1,49 @@
import { prisma } from "@/lib/db";
async function getFlag(key: string): Promise<boolean> {
const row = await prisma.config.findUnique({ where: { key } });
return row?.value === "true";
}
async function setFlag(key: string, value: boolean): Promise<void> {
const v = value ? "true" : "false";
await prisma.config.upsert({
where: { key },
create: { key, value: v, secret: false },
update: { value: v },
});
}
export async function GET() {
return Response.json({ paused: await getFlag("floor.paused") });
}
export async function POST(request: Request) {
let body: { action?: string };
try {
body = (await request.json()) as { action?: string };
} catch {
return Response.json({ error: "invalid JSON" }, { status: 400 });
}
switch (body.action) {
case "pause":
await setFlag("floor.paused", true);
return Response.json({ ok: true, paused: true });
case "resume":
await setFlag("floor.paused", false);
return Response.json({ ok: true, paused: false });
case "stop":
await setFlag("floor.paused", true);
await setFlag("floor.killRequested", true);
return Response.json({ ok: true, paused: true, stopped: true });
case "clear": {
const { count } = await prisma.request.deleteMany({
where: { job: { is: { state: "requested" } } },
});
return Response.json({ ok: true, cleared: count });
}
default:
return Response.json({ error: "unknown action" }, { status: 400 });
}
}
web/src/app/api/library/[id]/route.test.ts
+64 -2
View File
@@ -1,9 +1,13 @@
import { describe, it, expect, beforeEach, afterEach } from "vitest";
import { describe, it, expect, beforeEach, afterEach, vi } from "vitest";
import { mkdtempSync, writeFileSync, existsSync, mkdirSync, rmSync } from "node:fs";
import { tmpdir } from "node:os";
import { join } from "node:path";
import { prisma } from "@/lib/db";
import { DELETE } from "./route";
import { DELETE, PATCH } from "./route";
vi.mock("@/lib/musicbrainz", () => ({ searchReleaseGroup: vi.fn() }));
import { searchReleaseGroup } from "@/lib/musicbrainz";
const mockSearch = vi.mocked(searchReleaseGroup);
let root: string;
const original = process.env.LYRA_LIBRARY_ROOT;
@@ -59,3 +63,61 @@ describe("DELETE /api/library/[id]", () => {
expect(await prisma.libraryItem.findUnique({ where: { id: item.id } })).not.toBeNull();
});
});
function patchReq(body: unknown) {
return new Request("http://x", { method: "PATCH", headers: { "content-type": "application/json" }, body: JSON.stringify(body) });
}
describe("PATCH /api/library/[id]", () => {
beforeEach(() => mockSearch.mockReset());
it("renames the folder, updates the record, and re-resolves rgMbid on an artist/album change", async () => {
const dir = join(root, "Old Artist", "Old Album (2019)");
mkdirSync(dir, { recursive: true });
writeFileSync(join(dir, "01 Track.flac"), "audio");
const item = await prisma.libraryItem.create({
data: { artist: "Old Artist", album: "Old Album", path: dir, source: "scan", format: "FLAC", qualityClass: 3, rgMbid: "old-rg" },
});
mockSearch.mockResolvedValue({ rgMbid: "new-rg", title: "New Album", artistMbid: "x", artistName: "New Artist" } as never);
const res = await PATCH(patchReq({ artist: "New Artist", album: "New Album", year: "2020" }), req(item.id));
expect(res.status).toBe(200);
const moved = join(root, "New Artist", "New Album (2020)");
expect(existsSync(moved)).toBe(true);
expect(existsSync(dir)).toBe(false);
const row = (await prisma.libraryItem.findUnique({ where: { id: item.id } }))!;
expect(row).toMatchObject({ artist: "New Artist", album: "New Album", path: moved, rgMbid: "new-rg" });
});
it("changing only the year renames without calling MusicBrainz", async () => {
const dir = join(root, "A", "B (2000)");
mkdirSync(dir, { recursive: true });
const item = await prisma.libraryItem.create({
data: { artist: "A", album: "B", path: dir, source: "scan", format: "FLAC", qualityClass: 2, rgMbid: "keep" },
});
const res = await PATCH(patchReq({ year: "2001" }), req(item.id));
expect(res.status).toBe(200);
expect(mockSearch).not.toHaveBeenCalled();
expect(existsSync(join(root, "A", "B (2001)"))).toBe(true);
expect((await prisma.libraryItem.findUnique({ where: { id: item.id } }))!.rgMbid).toBe("keep");
});
it("409s when the target folder already exists", async () => {
mkdirSync(join(root, "A", "B (2000)"), { recursive: true });
mkdirSync(join(root, "A", "C (2000)"), { recursive: true });
const item = await prisma.libraryItem.create({
data: { artist: "A", album: "B", path: join(root, "A", "B (2000)"), source: "scan", format: "FLAC", qualityClass: 2 },
});
const res = await PATCH(patchReq({ album: "C" }), req(item.id));
expect(res.status).toBe(409);
});
it("validates input and 404s an unknown id", async () => {
const item = await prisma.libraryItem.create({
data: { artist: "A", album: "B", path: join(root, "A", "B"), source: "scan", format: "FLAC", qualityClass: 1 },
});
expect((await PATCH(patchReq({ year: "20" }), req(item.id))).status).toBe(400); // bad year
expect((await PATCH(patchReq({ artist: "" }), req(item.id))).status).toBe(400); // empty artist
expect((await PATCH(patchReq({ album: "Z" }), req("nope"))).status).toBe(404);
});
});
web/src/app/api/library/[id]/route.ts
+103 -31
View File
@@ -1,39 +1,111 @@
import { rm, rmdir } from "node:fs/promises";
import { existsSync } from "node:fs";
import { mkdir, rename, rmdir } from "node:fs/promises";
import { dirname, resolve, sep } from "node:path";
import { Prisma } from "@prisma/client";
import { prisma } from "@/lib/db";
import { albumDir, yearFromPath } from "@/lib/library-path";
import { searchReleaseGroup } from "@/lib/musicbrainz";
import { removeLibraryItem } from "@/lib/library-ops";
function insideRoot(root: string, target: string): boolean {
return target !== root && target.startsWith(root + sep);
}
// PATCH /api/library/[id] — fix an owned album's metadata: rename its folder to the canonical
// Artist/Album (Year) scheme on disk, update the LibraryItem, and re-resolve the MusicBrainz
// release-group (for cover art) when the artist/album changed. Does NOT rewrite the audio
// files' embedded tags — that's a worker (mutagen) job; Lyra reads from the folder + DB.
export async function PATCH(request: Request, { params }: { params: Promise<{ id: string }> }) {
const { id } = await params;
let body: unknown;
try {
body = await request.json();
} catch {
return Response.json({ error: "invalid JSON" }, { status: 400 });
}
const { artist, album, year } = (body ?? {}) as { artist?: unknown; album?: unknown; year?: unknown };
const strOrUndef = (v: unknown) => (v === undefined ? undefined : typeof v === "string" ? v.trim() : null);
const nextArtist = strOrUndef(artist);
const nextAlbum = strOrUndef(album);
const nextYear = strOrUndef(year); // "" clears the year; a 4-digit string sets it
if (nextArtist === null || nextAlbum === null || nextYear === null) {
return Response.json({ error: "artist, album, year must be strings" }, { status: 400 });
}
if (nextArtist === undefined && nextAlbum === undefined && nextYear === undefined) {
return Response.json({ error: "nothing to update" }, { status: 400 });
}
if (nextArtist === "" || nextAlbum === "") {
return Response.json({ error: "artist and album cannot be empty" }, { status: 400 });
}
if (nextYear && !/^\d{4}$/.test(nextYear)) {
return Response.json({ error: "year must be a 4-digit year or blank" }, { status: 400 });
}
const item = await prisma.libraryItem.findUnique({ where: { id } });
if (!item) return Response.json({ error: "not found" }, { status: 404 });
const root = resolve(process.env.LYRA_LIBRARY_ROOT ?? "/music");
const oldPath = resolve(item.path);
if (!insideRoot(root, oldPath)) {
return Response.json({ error: "refusing to touch a path outside the library" }, { status: 400 });
}
const finalArtist = nextArtist ?? item.artist;
const finalAlbum = nextAlbum ?? item.album;
// year isn't stored on LibraryItem; it lives only in the folder name. Keep the current
// folder's year unless the caller passed one (empty string clears it).
const finalYear = nextYear === undefined ? (yearFromPath(item.path) ?? undefined) : nextYear || undefined;
const newPath = albumDir(root, finalArtist, finalAlbum, finalYear);
if (newPath !== oldPath) {
if (!insideRoot(root, newPath)) {
return Response.json({ error: "refusing to write outside the library" }, { status: 400 });
}
if (existsSync(newPath)) {
return Response.json({ error: "a folder for that artist/album/year already exists" }, { status: 409 });
}
try {
await mkdir(dirname(newPath), { recursive: true });
await rename(oldPath, newPath);
const oldParent = dirname(oldPath);
if (oldParent !== root && oldParent !== dirname(newPath)) await rmdir(oldParent).catch(() => {});
} catch {
return Response.json({ error: "failed to rename the album folder" }, { status: 500 });
}
}
// Re-resolve the release-group when the artist/album changed (year alone doesn't change it).
// Best-effort: on an MB outage, keep the existing rgMbid rather than dropping the cover art.
let rgMbid = item.rgMbid;
if (nextArtist !== undefined || nextAlbum !== undefined) {
try {
rgMbid = (await searchReleaseGroup(finalArtist, finalAlbum))?.rgMbid ?? null;
} catch {
/* keep existing rgMbid */
}
}
try {
const updated = await prisma.libraryItem.update({
where: { id },
data: { artist: finalArtist, album: finalAlbum, path: newPath, rgMbid },
});
return Response.json({ id: updated.id, artist: updated.artist, album: updated.album, path: updated.path, rgMbid: updated.rgMbid });
} catch (e) {
if (e instanceof Prisma.PrismaClientKnownRequestError && e.code === "P2002") {
// moved the folder already; the (artist, album) row collides with another library item
return Response.json({ error: "another library item already has that artist/album" }, { status: 409 });
}
throw e;
}
}
// DELETE /api/library/[id] — remove an owned album: delete its folder on disk, drop the
// LibraryItem, and stop monitoring the matching release so the monitor doesn't re-grab it.
// DELETE /api/library/[id] — remove an owned album: delete its folder on disk, drop the
// LibraryItem, and stop monitoring the matching release so the monitor doesn't re-grab it.
export async function DELETE(_request: Request, { params }: { params: Promise<{ id: string }> }) {
const { id } = await params;
const item = await prisma.libraryItem.findUnique({ where: { id } });
if (!item) return Response.json({ error: "not found" }, { status: 404 });
// Guard: only ever delete a folder strictly inside the library root.
const root = resolve(process.env.LYRA_LIBRARY_ROOT ?? "/music");
const target = resolve(item.path);
if (target === root || !target.startsWith(root + sep)) {
return Response.json({ error: "refusing to delete a path outside the library" }, { status: 400 });
}
try {
await rm(target, { recursive: true, force: true });
// prune the now-possibly-empty artist folder (best-effort; rmdir fails if it has other albums)
const parent = dirname(target);
if (parent !== root) await rmdir(parent).catch(() => {});
} catch {
return Response.json({ error: "failed to delete files on disk" }, { status: 500 });
}
// Drop the record + stop monitoring so auto-monitor won't immediately re-download it. The
// release stays in the artist's discography (dormant) and can be re-wanted later.
await prisma.$transaction([
prisma.libraryItem.delete({ where: { id } }),
prisma.monitoredRelease.updateMany({
where: { artistName: item.artist, album: item.album },
data: { monitored: false, currentQualityClass: null },
}),
]);
return Response.json({ ok: true });
const r = await removeLibraryItem(id);
return r.ok ? Response.json({ ok: true }) : Response.json({ error: r.error }, { status: r.status });
}
web/src/app/api/library/[id]/upgrade/route.test.ts
+54
View File
@@ -0,0 +1,54 @@
import { describe, it, expect } from "vitest";
import { prisma } from "@/lib/db";
import { POST } from "./route";
const ctx = (id: string) => ({ params: Promise.resolve({ id }) });
async function owned(artist: string, album: string, rgMbid: string | null) {
return prisma.libraryItem.create({
data: { artist, album, path: "/m", source: "scan", format: "FLAC", qualityClass: 2, rgMbid },
});
}
describe("POST /api/library/[id]/upgrade", () => {
it("enqueues a forced re-acquire request against the matching release", async () => {
await prisma.monitoredRelease.create({
data: { artistMbid: "a1", artistName: "Adele", rgMbid: "rg-25", album: "25", secondaryTypes: [],
monitored: true, state: "fulfilled", currentQualityClass: 2 },
});
const item = await owned("Adele", "25", "rg-25");
const res = await POST(new Request("http://x", { method: "POST" }), ctx(item.id));
expect(res.status).toBe(202);
expect((await res.json()).enqueued).toBe(true);
const reqRow = await prisma.request.findFirst({ where: { album: "25" }, include: { job: true } });
expect(reqRow?.force).toBe(true);
expect(reqRow?.monitoredReleaseId).toBeTruthy();
expect(reqRow?.job).toBeTruthy();
});
it("400s when there is no matching release", async () => {
const item = await owned("Nobody", "Untracked", null);
const res = await POST(new Request("http://x", { method: "POST" }), ctx(item.id));
expect(res.status).toBe(400);
});
it("does not stack a second job while one is in flight", async () => {
const mr = await prisma.monitoredRelease.create({
data: { artistMbid: "a1", artistName: "Adele", rgMbid: "rg-21", album: "21", secondaryTypes: [], monitored: true, state: "wanted" },
});
const item = await owned("Adele", "21", "rg-21");
await prisma.request.create({
data: { artist: "Adele", album: "21", monitoredReleaseId: mr.id, job: { create: { state: "downloading" } } },
});
const res = await POST(new Request("http://x", { method: "POST" }), ctx(item.id));
expect(res.status).toBe(202);
expect((await res.json()).enqueued).toBe(false);
});
it("404s an unknown library item", async () => {
const res = await POST(new Request("http://x", { method: "POST" }), ctx("nope"));
expect(res.status).toBe(404);
});
});
web/src/app/api/library/[id]/upgrade/route.ts
+41
View File
@@ -0,0 +1,41 @@
import { prisma } from "@/lib/db";
// POST /api/library/[id]/upgrade — force a re-acquire of an owned album: enqueue a job that
// skips the "already in library" dedupe (Request.force). The pipeline downloads the best
// available copy and the import step keeps it only if it's higher quality, so this can upgrade
// but never downgrades. Requires a matching MonitoredRelease (scanned/monitored albums have
// one) to carry the acquisition context.
export async function POST(_request: Request, { params }: { params: Promise<{ id: string }> }) {
const { id } = await params;
const item = await prisma.libraryItem.findUnique({ where: { id } });
if (!item) return Response.json({ error: "not found" }, { status: 404 });
// Prefer the release-group match; fall back to an artist+album match.
const release =
(item.rgMbid ? await prisma.monitoredRelease.findUnique({ where: { rgMbid: item.rgMbid } }) : null) ??
(await prisma.monitoredRelease.findFirst({ where: { artistName: item.artist, album: item.album } }));
if (!release) {
return Response.json(
{ error: "no release info for this album — follow the artist or re-scan to enable upgrades" },
{ status: 400 },
);
}
// Don't stack a second job on a release that's already in flight.
const activeJob = await prisma.job.findFirst({
where: { request: { monitoredReleaseId: release.id }, state: { notIn: ["imported", "needs_attention"] } },
});
if (activeJob) return Response.json({ enqueued: false }, { status: 202 });
await prisma.request.create({
data: {
artist: release.artistName,
album: release.album,
monitoredReleaseId: release.id,
force: true,
job: { create: {} },
},
});
await prisma.monitoredRelease.update({ where: { id: release.id }, data: { lastSearchedAt: new Date() } });
return Response.json({ enqueued: true }, { status: 202 });
}
web/src/app/api/library/bulk/route.test.ts
+54
View File
@@ -0,0 +1,54 @@
import { describe, it, expect, beforeEach, afterEach } from "vitest";
import { mkdtempSync, mkdirSync, writeFileSync, existsSync, rmSync } from "node:fs";
import { tmpdir } from "node:os";
import { join } from "node:path";
import { prisma } from "@/lib/db";
import { POST } from "./route";
let root: string;
const original = process.env.LYRA_LIBRARY_ROOT;
beforeEach(() => {
root = mkdtempSync(join(tmpdir(), "lyra-bulk-"));
process.env.LYRA_LIBRARY_ROOT = root;
});
afterEach(() => {
if (original === undefined) delete process.env.LYRA_LIBRARY_ROOT;
else process.env.LYRA_LIBRARY_ROOT = original;
rmSync(root, { recursive: true, force: true });
});
function post(body: unknown) {
return POST(new Request("http://x", { method: "POST", headers: { "content-type": "application/json" }, body: JSON.stringify(body) }));
}
describe("POST /api/library/bulk", () => {
it("deletes several albums and their folders", async () => {
const ids: string[] = [];
for (const [artist, album] of [["A", "One"], ["B", "Two"]]) {
const dir = join(root, artist, album);
mkdirSync(dir, { recursive: true });
writeFileSync(join(dir, "01.flac"), "x");
ids.push((await prisma.libraryItem.create({ data: { artist, album, path: dir, source: "scan", format: "FLAC", qualityClass: 2 } })).id);
}
const res = await post({ action: "delete", ids });
expect(res.status).toBe(200);
expect(await res.json()).toMatchObject({ action: "delete", ok: 2, failed: [] });
expect(await prisma.libraryItem.count()).toBe(0);
expect(existsSync(join(root, "A"))).toBe(false);
});
it("ignores the matching releases for the selected albums", async () => {
const mr = await prisma.monitoredRelease.create({
data: { artistMbid: "a1", artistName: "Adele", rgMbid: "rg-21", album: "21", secondaryTypes: [], monitored: true, state: "wanted" },
});
const li = await prisma.libraryItem.create({ data: { artist: "Adele", album: "21", path: join(root, "Adele", "21"), source: "scan", format: "FLAC", qualityClass: 2, rgMbid: "rg-21" } });
const res = await post({ action: "ignore", ids: [li.id] });
expect((await res.json()).ok).toBe(1);
expect((await prisma.monitoredRelease.findUnique({ where: { id: mr.id } }))!.ignored).toBe(true);
});
it("rejects a bad action or empty ids", async () => {
expect((await post({ action: "nuke", ids: ["x"] })).status).toBe(400);
expect((await post({ action: "delete", ids: [] })).status).toBe(400);
});
});
web/src/app/api/library/bulk/route.ts
+36
View File
@@ -0,0 +1,36 @@
import { removeLibraryItem, ignoreLibraryItem, type OpResult } from "@/lib/library-ops";
// POST /api/library/bulk — apply an action to several owned albums at once.
// Body: { action: "delete" | "ignore", ids: string[] }. Returns per-id success counts.
const ACTIONS: Record<string, (id: string) => Promise<OpResult>> = {
delete: removeLibraryItem,
ignore: ignoreLibraryItem,
};
export async function POST(request: Request) {
let body: unknown;
try {
body = await request.json();
} catch {
return Response.json({ error: "invalid JSON" }, { status: 400 });
}
const { action, ids } = (body ?? {}) as { action?: unknown; ids?: unknown };
if (typeof action !== "string" || !(action in ACTIONS)) {
return Response.json({ error: "action must be 'delete' or 'ignore'" }, { status: 400 });
}
if (!Array.isArray(ids) || ids.length === 0 || !ids.every((x) => typeof x === "string")) {
return Response.json({ error: "ids must be a non-empty string array" }, { status: 400 });
}
const op = ACTIONS[action];
// Run sequentially — deletes touch the filesystem and prune shared artist folders; serial
// keeps that predictable and avoids hammering the mount.
let ok = 0;
const failed: string[] = [];
for (const id of ids as string[]) {
const r = await op(id);
if (r.ok) ok++;
else failed.push(id);
}
return Response.json({ action, ok, failed });
}
web/src/app/api/library/duplicates/route.test.ts
+35
View File
@@ -0,0 +1,35 @@
import { describe, it, expect } from "vitest";
import { prisma } from "@/lib/db";
import { GET } from "./route";
async function li(artist: string, album: string, rgMbid: string | null, path = "/m") {
return prisma.libraryItem.create({
data: { artist, album, path, source: "scan", format: "FLAC", qualityClass: 2, rgMbid },
});
}
describe("library duplicates API", () => {
it("groups items sharing a release-group", async () => {
await li("Adele", "21", "rg-21");
await li("Adele", "21 (Deluxe)", "rg-21"); // same rg → duplicate
await li("Adele", "25", "rg-25"); // unrelated
const { duplicates } = await (await GET()).json();
expect(duplicates).toHaveLength(1);
expect(duplicates[0].map((d: { album: string }) => d.album).sort()).toEqual(["21", "21 (Deluxe)"]);
});
it("groups items with the same edition-normalized title even without a shared rgMbid", async () => {
await li("Nirvana", "Nevermind", null);
await li("Nirvana", "Nevermind (Deluxe Edition)", "rg-x");
const { duplicates } = await (await GET()).json();
expect(duplicates).toHaveLength(1);
expect(duplicates[0]).toHaveLength(2);
});
it("returns nothing when there are no duplicates", async () => {
await li("A", "One", "rg-1");
await li("B", "Two", "rg-2");
const { duplicates } = await (await GET()).json();
expect(duplicates).toEqual([]);
});
});
web/src/app/api/library/duplicates/route.ts
+61
View File
@@ -0,0 +1,61 @@
import { prisma } from "@/lib/db";
import { stripEditionQualifiers } from "@/lib/musicbrainz";
import { yearFromPath } from "@/lib/library-path";
// GET /api/library/duplicates — groups of library items that look like the same album held
// more than once: sharing a MusicBrainz release-group, or the same artist + edition-normalized
// title (e.g. "Album" vs "Album (Deluxe Edition)"). The (artist, album) unique constraint
// rules out exact dupes, so these are the near-dupes worth surfacing.
function normTitleKey(artist: string, album: string): string {
return `${artist.trim().toLowerCase()}|${stripEditionQualifiers(album).trim().toLowerCase()}`;
}
export async function GET() {
const items = await prisma.libraryItem.findMany({ orderBy: [{ artist: "asc" }, { album: "asc" }] });
// Union-find: connect items that share a non-null rgMbid or the same normalized title key.
const parent = new Map<string, string>();
const find = (x: string): string => {
let r = x;
while (parent.get(r) !== r) r = parent.get(r)!;
while (parent.get(x) !== r) { const n = parent.get(x)!; parent.set(x, r); x = n; }
return r;
};
const union = (a: string, b: string) => { parent.set(find(a), find(b)); };
for (const it of items) parent.set(it.id, it.id);
const byRg = new Map<string, string>();
const byTitle = new Map<string, string>();
for (const it of items) {
if (it.rgMbid) {
const seen = byRg.get(it.rgMbid);
if (seen) union(seen, it.id); else byRg.set(it.rgMbid, it.id);
}
const tk = normTitleKey(it.artist, it.album);
const seenT = byTitle.get(tk);
if (seenT) union(seenT, it.id); else byTitle.set(tk, it.id);
}
const groups = new Map<string, typeof items>();
for (const it of items) {
const root = find(it.id);
(groups.get(root) ?? groups.set(root, []).get(root)!).push(it);
}
const duplicates = [...groups.values()]
.filter((g) => g.length > 1)
.map((g) =>
g.map((it) => ({
id: it.id,
artist: it.artist,
album: it.album,
format: it.format,
qualityClass: it.qualityClass,
rgMbid: it.rgMbid,
year: yearFromPath(it.path),
path: it.path,
})),
);
return Response.json({ duplicates });
}
web/src/app/api/library/import/route.test.ts
+2 -1
View File
@@ -1,4 +1,5 @@
import { describe, it, expect, beforeEach, afterEach } from "vitest";
import { describe, it, expect, beforeEach, afterEach, vi } from "vitest";
vi.mock("@/lib/musicbrainz", () => ({ searchReleaseGroup: vi.fn(async () => null) }));
import { mkdtempSync, existsSync, readFileSync, readdirSync, rmSync } from "node:fs";
import { tmpdir } from "node:os";
import { join } from "node:path";
web/src/app/api/library/import/route.ts
+91 -49
View File
@@ -1,73 +1,114 @@
import { mkdir, rename, writeFile, rm } from "node:fs/promises";
import { existsSync } from "node:fs";
import { resolve, join, extname } from "node:path";
import { createWriteStream, existsSync } from "node:fs";
import { mkdir, rename, rm } from "node:fs/promises";
import { resolve, join, dirname, extname } from "node:path";
import { Readable } from "node:stream";
import { pipeline } from "node:stream/promises";
import { randomUUID } from "node:crypto";
import Busboy from "busboy";
import { prisma } from "@/lib/db";
import { isAuthed } from "@/lib/auth";
import { searchReleaseGroup } from "@/lib/musicbrainz";
import { safeName, albumDir } from "@/lib/library-path";
// Manually import an album (a ripped CD / files already on the PC): upload the audio (+ an
// optional cover) with artist/album/year, and Lyra writes it into the library. The web
// container bind-mounts /music, so it can write directly. A later library Scan will
// MB-resolve it (cover art, canonical types) and re-probe quality.
// container bind-mounts /music, so it can write directly. A later library Scan MB-resolves it
// (cover art, canonical types) and re-probes quality.
//
// The body is STREAMED to disk with busboy rather than buffered via request.formData(), which
// caps out around ~15MB — real FLAC albums are hundreds of MB.
const AUDIO_EXT = new Set([".flac", ".mp3", ".m4a", ".opus", ".ogg", ".aac", ".wav"]);
const LOSSLESS_EXT = new Set([".flac", ".wav"]);
// filesystem-illegal path chars → "_" (matches the worker's safe_name; control chars can't
// reach here through a form field, so the range is omitted).
const ILLEGAL = /[<>:"/\\|?*]/g;
function safeName(s: string): string {
const cleaned = s.replace(ILLEGAL, "_").trim().replace(/^\.+|\.+$/g, "").trim();
return cleaned || "Unknown";
}
export async function POST(request: Request) {
let form: FormData;
try {
form = await request.formData();
} catch {
// This route is excluded from the auth middleware (to avoid the 10MB body cap), so it must
// authenticate itself.
if (!(await isAuthed(request))) {
return Response.json({ error: "unauthorized" }, { status: 401 });
}
const ct = request.headers.get("content-type") ?? "";
if (!ct.includes("multipart/form-data") || !request.body) {
return Response.json({ error: "expected multipart form data" }, { status: 400 });
}
const artist = String(form.get("artist") ?? "").trim();
const album = String(form.get("album") ?? "").trim();
const year = String(form.get("year") ?? "").trim();
if (!artist || !album) return Response.json({ error: "artist and album are required" }, { status: 400 });
if (year && !/^\d{4}$/.test(year)) return Response.json({ error: "year must be 4 digits" }, { status: 400 });
const files = form.getAll("files").filter((f): f is File => f instanceof File);
const audio = files
.filter((f) => AUDIO_EXT.has(extname(f.name).toLowerCase()))
.sort((a, b) => a.name.localeCompare(b.name));
if (audio.length === 0) return Response.json({ error: "no audio files provided" }, { status: 400 });
const cover = files.find((f) => /\.(jpe?g|png)$/i.test(f.name));
const root = resolve(process.env.LYRA_LIBRARY_ROOT ?? "/music");
const albumFolder = year ? `${safeName(album)} (${year})` : safeName(album);
const finalDir = join(root, safeName(artist), albumFolder);
// stage under a hidden dir on the library filesystem so the final move is an atomic rename
// (the scan skips dot-dirs). Cleaned up on any failure.
const staging = join(root, `.import-${randomUUID()}`);
await mkdir(staging, { recursive: true });
if (existsSync(finalDir)) {
return Response.json({ error: "a folder for that album already exists on disk" }, { status: 409 });
const fields: Record<string, string> = {};
const audioNames: string[] = [];
let hasCover = false;
try {
await new Promise<void>((res, rej) => {
const bb = Busboy({ headers: { "content-type": ct } });
const writes: Promise<void>[] = [];
bb.on("field", (name, val) => {
fields[name] = val;
});
bb.on("file", (_name, stream, info) => {
const ext = extname(info.filename).toLowerCase();
if (AUDIO_EXT.has(ext)) {
const safe = safeName(info.filename);
audioNames.push(safe);
writes.push(pipeline(stream, createWriteStream(join(staging, safe))));
} else if (/\.(jpe?g|png)$/i.test(info.filename) && !hasCover) {
hasCover = true;
writes.push(pipeline(stream, createWriteStream(join(staging, "cover.jpg"))));
} else {
stream.resume(); // discard anything that isn't audio or a cover
}
if (await prisma.libraryItem.findFirst({ where: { artist, album } })) {
});
bb.on("error", rej);
bb.on("close", () => Promise.all(writes).then(() => res(), rej));
Readable.fromWeb(request.body as import("node:stream/web").ReadableStream).pipe(bb);
});
} catch {
await rm(staging, { recursive: true, force: true });
return Response.json({ error: "failed to write the upload to disk" }, { status: 500 });
}
const artist = (fields.artist ?? "").trim();
const album = (fields.album ?? "").trim();
const year = (fields.year ?? "").trim();
const audio = audioNames.sort((a, b) => a.localeCompare(b));
const bad =
!artist || !album
? "artist and album are required"
: audio.length === 0
? "no audio files provided"
: year && !/^\d{4}$/.test(year)
? "year must be 4 digits"
: null;
if (bad) {
await rm(staging, { recursive: true, force: true });
return Response.json({ error: bad }, { status: 400 });
}
const finalDir = albumDir(root, artist, album, year || undefined);
if (existsSync(finalDir) || (await prisma.libraryItem.findFirst({ where: { artist, album } }))) {
await rm(staging, { recursive: true, force: true });
return Response.json({ error: "this album is already in the library" }, { status: 409 });
}
// Assemble in a sibling temp dir then rename into place, so a failed/partial upload never
// leaves a half-written album in the library.
const tmpDir = finalDir + ".importing";
await rm(tmpDir, { recursive: true, force: true });
await mkdir(tmpDir, { recursive: true });
await mkdir(dirname(finalDir), { recursive: true });
await rename(staging, finalDir);
// Resolve the release group so the album gets cover art immediately (best-effort — a
// no-match just leaves rgMbid null; a later Scan can still match it). Uses the shared MB cache.
let rgMbid: string | null = null;
try {
for (const f of audio) {
await writeFile(join(tmpDir, safeName(f.name)), Buffer.from(await f.arrayBuffer()));
}
if (cover) await writeFile(join(tmpDir, "cover.jpg"), Buffer.from(await cover.arrayBuffer()));
await rename(tmpDir, finalDir);
rgMbid = (await searchReleaseGroup(artist, album))?.rgMbid ?? null;
} catch {
await rm(tmpDir, { recursive: true, force: true });
return Response.json({ error: "failed to write the album to disk" }, { status: 500 });
/* MB unavailable — carry on without cover art */
}
const firstExt = extname(audio[0].name).toLowerCase();
const firstExt = extname(audio[0]).toLowerCase();
await prisma.libraryItem.create({
data: {
artist,
@@ -78,7 +119,8 @@ export async function POST(request: Request) {
// rough class from the extension (2 = CD-lossless, 1 = lossy); a later scan re-probes and
// can upgrade a hi-res file to class 3.
qualityClass: LOSSLESS_EXT.has(firstExt) ? 2 : 1,
trackNames: audio.map((f) => safeName(f.name)),
rgMbid,
trackNames: audio,
},
});
web/src/app/api/library/integrity/route.test.ts
+63
View File
@@ -0,0 +1,63 @@
import { describe, it, expect, beforeEach, afterEach } from "vitest";
import { mkdtempSync, mkdirSync, writeFileSync, rmSync } from "node:fs";
import { tmpdir } from "node:os";
import { join } from "node:path";
import { prisma } from "@/lib/db";
import { GET } from "./route";
let root: string;
const original = process.env.LYRA_LIBRARY_ROOT;
beforeEach(() => {
root = mkdtempSync(join(tmpdir(), "lyra-integ-"));
process.env.LYRA_LIBRARY_ROOT = root;
});
afterEach(() => {
if (original === undefined) delete process.env.LYRA_LIBRARY_ROOT;
else process.env.LYRA_LIBRARY_ROOT = original;
rmSync(root, { recursive: true, force: true });
});
async function item(artist: string, album: string, dir: string, trackNames: string[] = []) {
return prisma.libraryItem.create({
data: { artist, album, path: dir, source: "scan", format: "FLAC", qualityClass: 2, trackNames },
});
}
describe("library integrity API", () => {
it("flags a healthy album with no issues", async () => {
const dir = join(root, "A", "Good (2020)");
mkdirSync(dir, { recursive: true });
writeFileSync(join(dir, "01 One.flac"), "xxxx");
writeFileSync(join(dir, "02 Two.flac"), "yyyy");
await item("A", "Good", dir, ["01 One.flac", "02 Two.flac"]);
const { checked, issues } = await (await GET()).json();
expect(checked).toBe(1);
expect(issues).toEqual([]);
});
it("flags a missing folder, empty files, and a track-count drift", async () => {
await item("Gone", "Missing", join(root, "Gone", "Missing"));
const zdir = join(root, "Z", "Zero (2021)");
mkdirSync(zdir, { recursive: true });
writeFileSync(join(zdir, "01 A.flac"), ""); // 0 bytes → truncated
writeFileSync(join(zdir, "02 B.flac"), "ok");
await item("Z", "Zero", zdir, ["01 A.flac", "02 B.flac", "03 C.flac"]); // expected 3, on disk 2
const { issues } = await (await GET()).json();
const byAlbum = Object.fromEntries(issues.map((i: { album: string; issues: string[] }) => [i.album, i.issues]));
expect(byAlbum["Missing"]).toEqual(["folder is missing on disk"]);
expect(byAlbum["Zero"]).toContain("1 empty/unreadable file");
expect(byAlbum["Zero"]).toContain("2 of 3 tracks on disk");
});
it("flags a folder with no audio", async () => {
const dir = join(root, "N", "NoAudio");
mkdirSync(dir, { recursive: true });
writeFileSync(join(dir, "cover.jpg"), "img");
await item("N", "NoAudio", dir);
const { issues } = await (await GET()).json();
expect(issues[0].issues).toEqual(["no audio files in the folder"]);
});
});
web/src/app/api/library/integrity/route.ts
+51
View File
@@ -0,0 +1,51 @@
import { readdir, stat } from "node:fs/promises";
import { join, resolve, sep, extname } from "node:path";
import { prisma } from "@/lib/db";
// GET /api/library/integrity — on-demand disk check of every owned album folder (via the
// /music mount). Flags folders that are missing, hold no audio, contain zero-byte (truncated)
// files, or whose on-disk audio count drifted from what was captured at import/scan. Cheap
// (metadata only: readdir + stat), so it runs only when the user asks.
const AUDIO_EXT = new Set([".flac", ".mp3", ".m4a", ".opus", ".ogg", ".aac", ".wav"]);
export async function GET() {
const items = await prisma.libraryItem.findMany({ orderBy: [{ artist: "asc" }, { album: "asc" }] });
const root = resolve(process.env.LYRA_LIBRARY_ROOT ?? "/music");
const results = await Promise.all(
items.map(async (li) => {
const dir = resolve(li.path);
const issues: string[] = [];
if (dir !== root && !dir.startsWith(root + sep)) {
return { id: li.id, artist: li.artist, album: li.album, issues: ["path is outside the library root"] };
}
let names: string[];
try {
names = await readdir(dir);
} catch {
return { id: li.id, artist: li.artist, album: li.album, issues: ["folder is missing on disk"] };
}
const audio = names.filter((n) => AUDIO_EXT.has(extname(n).toLowerCase()));
if (audio.length === 0) {
issues.push("no audio files in the folder");
} else {
let empty = 0;
for (const n of audio) {
try {
if ((await stat(join(dir, n))).size === 0) empty++;
} catch {
empty++; // unreadable counts as broken
}
}
if (empty > 0) issues.push(`${empty} empty/unreadable file${empty === 1 ? "" : "s"}`);
const expected = li.trackNames.length;
if (expected > 0 && audio.length !== expected) {
issues.push(`${audio.length} of ${expected} tracks on disk`);
}
}
return { id: li.id, artist: li.artist, album: li.album, issues };
}),
);
return Response.json({ checked: items.length, issues: results.filter((r) => r.issues.length > 0) });
}
web/src/app/api/library/route.ts
BIN
View File
Binary file not shown.
web/src/app/api/library/unmatched/route.test.ts
+33
View File
@@ -0,0 +1,33 @@
import { describe, it, expect } from "vitest";
import { prisma } from "@/lib/db";
import { GET, DELETE } from "./route";
describe("library unmatched API", () => {
it("lists unmatched albums ordered by artist/album", async () => {
await prisma.unmatchedAlbum.createMany({
data: [
{ artist: "Zed", album: "Late", path: "/music/Zed/Late", reason: "no MusicBrainz match", scanId: "s1" },
{ artist: "Ann", album: "Early", path: "/music/Ann/Early", reason: "unreadable: boom", scanId: "s1" },
],
});
const res = await GET();
expect(res.status).toBe(200);
const { unmatched } = await res.json();
expect(unmatched.map((u: { artist: string }) => u.artist)).toEqual(["Ann", "Zed"]);
expect(unmatched[0]).toMatchObject({ album: "Early", reason: "unreadable: boom" });
});
it("dismisses one entry by id", async () => {
const row = await prisma.unmatchedAlbum.create({
data: { artist: "X", album: "Y", path: "/music/X/Y", reason: "no MusicBrainz match", scanId: "s1" },
});
const res = await DELETE(new Request(`http://t/api/library/unmatched?id=${row.id}`, { method: "DELETE" }));
expect(res.status).toBe(200);
expect(await prisma.unmatchedAlbum.count()).toBe(0);
});
it("404s dismissing an unknown id", async () => {
const res = await DELETE(new Request("http://t/api/library/unmatched?id=nope", { method: "DELETE" }));
expect(res.status).toBe(404);
});
});
web/src/app/api/library/unmatched/route.ts
+27
View File
@@ -0,0 +1,27 @@
import { prisma } from "@/lib/db";
// GET /api/library/unmatched — album folders the scan couldn't resolve to a MusicBrainz
// release (no match, or an unreadable file). Surfaced on /library so they aren't silently lost.
export async function GET() {
const rows = await prisma.unmatchedAlbum.findMany({ orderBy: [{ artist: "asc" }, { album: "asc" }] });
return Response.json({
unmatched: rows.map((r) => ({
id: r.id,
artist: r.artist,
album: r.album,
path: r.path,
reason: r.reason,
scannedAt: r.scannedAt,
})),
});
}
// DELETE /api/library/unmatched?id= — dismiss one entry (the user has handled it, or doesn't
// care). Does not touch files on disk; a later re-scan re-surfaces it if still unresolved.
export async function DELETE(request: Request) {
const id = new URL(request.url).searchParams.get("id");
if (!id) return Response.json({ error: "id is required" }, { status: 400 });
const deleted = await prisma.unmatchedAlbum.deleteMany({ where: { id } });
if (deleted.count === 0) return Response.json({ error: "not found" }, { status: 404 });
return Response.json({ ok: true });
}
web/src/app/api/monitor/config/route.test.ts
+20
View File
@@ -34,3 +34,23 @@ describe("monitor config API", () => {
expect(await prisma.config.count()).toBe(before);
});
});
describe("monitor config — floor.jobDelaySeconds", () => {
it("defaults jobDelaySeconds to 0 and round-trips a new value", async () => {
const before = await (await GET()).json();
expect(before["floor.jobDelaySeconds"]).toBe("0");
await PATCH(
new Request("http://localhost/api/monitor/config", {
method: "PATCH",
headers: { "content-type": "application/json" },
body: JSON.stringify({ "floor.jobDelaySeconds": "45" }),
}),
);
const stored = await prisma.config.findUnique({ where: { key: "floor.jobDelaySeconds" } });
expect(stored?.value).toBe("45");
const after = await (await GET()).json();
expect(after["floor.jobDelaySeconds"]).toBe("45");
});
});
web/src/app/api/monitor/config/route.ts
+1
View File
@@ -6,6 +6,7 @@ const DEFAULTS: Record<string, string> = {
"monitor.retryIntervalHours": "6",
"monitor.qualityCutoff": "2",
"monitor.upgradeWindowDays": "14",
"floor.jobDelaySeconds": "0",
};
export async function GET() {
web/src/app/api/qobuz/pacing/route.test.ts
+37
View File
@@ -0,0 +1,37 @@
import { describe, it, expect } from "vitest";
import { GET, PATCH } from "./route";
import { prisma } from "@/lib/db";
function patch(body: Record<string, string>) {
return new Request("http://localhost/api/qobuz/pacing", {
method: "PATCH",
headers: { "content-type": "application/json" },
body: JSON.stringify(body),
});
}
describe("qobuz pacing config API", () => {
it("returns defaults when nothing is stored", async () => {
const cfg = await (await GET()).json();
expect(cfg["qobuz.pacing.enabled"]).toBe("true");
expect(cfg["qobuz.pacing.activeStartHour"]).toBe("8");
expect(cfg["qobuz.pacing.activeEndHour"]).toBe("23");
expect(cfg["qobuz.pacing.dailyCap"]).toBe("40");
expect(cfg["qobuz.pacing.warmupStartDate"]).toBe("");
});
it("round-trips allow-listed settings and persists them", async () => {
await PATCH(patch({ "qobuz.pacing.dailyCap": "25", "qobuz.pacing.activeStartHour": "9" }));
const stored = await prisma.config.findUnique({ where: { key: "qobuz.pacing.dailyCap" } });
expect(stored?.value).toBe("25");
const cfg = await (await GET()).json();
expect(cfg["qobuz.pacing.dailyCap"]).toBe("25");
expect(cfg["qobuz.pacing.activeStartHour"]).toBe("9");
});
it("ignores keys not in the allow-list (e.g. worker-managed counters)", async () => {
await PATCH(patch({ "qobuz.pacing.countToday": "999", "qobuz.pacing.dailyCap": "30" }));
expect(await prisma.config.findUnique({ where: { key: "qobuz.pacing.countToday" } })).toBeNull();
expect((await prisma.config.findUnique({ where: { key: "qobuz.pacing.dailyCap" } }))?.value).toBe("30");
});
});
web/src/app/api/qobuz/pacing/route.ts
+39
View File
@@ -0,0 +1,39 @@
import { prisma } from "@/lib/db";
// Qobuz pacing ("budget gate") settings. The worker reads these Config keys live; state keys
// (countDay/countToday/nextAllowedAt) are worker-managed and intentionally NOT editable here.
const DEFAULTS: Record<string, string> = {
"qobuz.pacing.enabled": "true",
"qobuz.pacing.activeStartHour": "8",
"qobuz.pacing.activeEndHour": "23",
"qobuz.pacing.dailyCap": "40",
"qobuz.pacing.warmupStartDate": "",
};
export async function GET() {
const rows = await prisma.config.findMany({ where: { key: { in: Object.keys(DEFAULTS) } } });
const stored = Object.fromEntries(rows.map((r) => [r.key, r.value]));
return Response.json(
Object.fromEntries(Object.entries(DEFAULTS).map(([k, d]) => [k, stored[k] ?? d])),
);
}
export async function PATCH(request: Request) {
let body: unknown;
try {
body = await request.json();
} catch {
return Response.json({ error: "invalid JSON" }, { status: 400 });
}
const entries = Object.entries((body ?? {}) as Record<string, unknown>).filter(([k]) =>
Object.prototype.hasOwnProperty.call(DEFAULTS, k),
);
for (const [key, value] of entries) {
await prisma.config.upsert({
where: { key },
create: { key, value: String(value), secret: false },
update: { value: String(value) },
});
}
return Response.json({ updated: entries.length });
}
web/src/app/api/releases/[id]/route.test.ts
+11 -2
View File
@@ -22,9 +22,18 @@ describe("PATCH /api/releases/[id]", () => {
expect((await prisma.monitoredRelease.findUnique({ where: { id: r.id } }))!.monitored).toBe(true);
});
it("400s a non-boolean and 404s an unknown release", async () => {
it("toggles ignored independently of monitored", async () => {
const r = await seedRelease(true);
const res = await PATCH(patchReq({ ignored: true }), ctx(r.id));
expect(res.status).toBe(200);
const row = (await prisma.monitoredRelease.findUnique({ where: { id: r.id } }))!;
expect(row.ignored).toBe(true);
expect(row.monitored).toBe(true); // ignore doesn't clear monitored
});
it("400s when neither monitored nor ignored is a boolean, 404s an unknown release", async () => {
const r = await seedRelease();
expect((await PATCH(patchReq({ monitored: 1 }), ctx(r.id))).status).toBe(400);
expect((await PATCH(patchReq({ monitored: true }), ctx("nope"))).status).toBe(404);
expect((await PATCH(patchReq({ ignored: true }), ctx("nope"))).status).toBe(404);
});
});
web/src/app/api/releases/[id]/route.ts
+8 -5
View File
@@ -9,13 +9,16 @@ export async function PATCH(request: Request, { params }: { params: Promise<{ id
} catch {
return Response.json({ error: "invalid JSON" }, { status: 400 });
}
const { monitored } = (body ?? {}) as { monitored?: unknown };
if (typeof monitored !== "boolean") {
return Response.json({ error: "monitored (boolean) is required" }, { status: 400 });
const { monitored, ignored } = (body ?? {}) as { monitored?: unknown; ignored?: unknown };
if (typeof monitored !== "boolean" && typeof ignored !== "boolean") {
return Response.json({ error: "monitored or ignored (boolean) is required" }, { status: 400 });
}
const data: { monitored?: boolean; ignored?: boolean } = {};
if (typeof monitored === "boolean") data.monitored = monitored;
if (typeof ignored === "boolean") data.ignored = ignored;
try {
const updated = await prisma.monitoredRelease.update({ where: { id }, data: { monitored } });
return Response.json({ id: updated.id, monitored: updated.monitored });
const updated = await prisma.monitoredRelease.update({ where: { id }, data });
return Response.json({ id: updated.id, monitored: updated.monitored, ignored: updated.ignored });
} catch (e) {
if (e instanceof Prisma.PrismaClientKnownRequestError && e.code === "P2025") {
return Response.json({ error: "not found" }, { status: 404 });
web/src/app/api/requests/[id]/pause/route.test.ts
+49
View File
@@ -0,0 +1,49 @@
import { describe, it, expect } from "vitest";
import { POST } from "./route";
import { prisma } from "@/lib/db";
import type { Prisma } from "@prisma/client";
function req(id: string, paused: boolean) {
return new Request(`http://localhost/api/requests/${id}/pause`, {
method: "POST",
headers: { "content-type": "application/json" },
body: JSON.stringify({ paused }),
});
}
async function seed(jobData: Prisma.JobCreateWithoutRequestInput) {
return prisma.request.create({
data: { artist: "A", album: "B", status: "pending", job: { create: jobData } },
include: { job: true },
});
}
describe("per-item pause API", () => {
it("pauses a queued job", async () => {
const created = await seed({ state: "requested" });
const res = await POST(req(created.id, true), { params: Promise.resolve({ id: created.id }) });
expect(res.status).toBe(200);
const job = await prisma.job.findUnique({ where: { id: created.job!.id } });
expect(job?.paused).toBe(true);
});
it("resumes a paused job", async () => {
const created = await seed({ state: "requested", paused: true });
await POST(req(created.id, false), { params: Promise.resolve({ id: created.id }) });
const job = await prisma.job.findUnique({ where: { id: created.job!.id } });
expect(job?.paused).toBe(false);
});
it("rejects pausing an in-flight job", async () => {
const created = await seed({ state: "downloading", paused: true });
const res = await POST(req(created.id, false), { params: Promise.resolve({ id: created.id }) });
expect(res.status).toBe(400);
const job = await prisma.job.findUnique({ where: { id: created.job!.id } });
expect(job?.paused).toBe(true);
});
it("404s for a missing request", async () => {
const res = await POST(req("nope", true), { params: Promise.resolve({ id: "nope" }) });
expect(res.status).toBe(404);
});
});
web/src/app/api/requests/[id]/pause/route.ts
+26
View File
@@ -0,0 +1,26 @@
import { prisma } from "@/lib/db";
export async function POST(request: Request, { params }: { params: Promise<{ id: string }> }) {
const { id } = await params;
let body: { paused?: unknown };
try {
body = (await request.json()) as { paused?: unknown };
} catch {
return Response.json({ error: "invalid JSON" }, { status: 400 });
}
if (typeof body.paused !== "boolean") {
return Response.json({ error: "paused must be a boolean" }, { status: 400 });
}
const req = await prisma.request.findUnique({ where: { id }, include: { job: true } });
if (!req || !req.job) {
return Response.json({ error: "not found" }, { status: 404 });
}
if (req.job.state !== "requested") {
return Response.json({ error: "only queued jobs can be paused" }, { status: 400 });
}
await prisma.job.update({ where: { id: req.job.id }, data: { paused: body.paused } });
return Response.json({ ok: true, paused: body.paused });
}
web/src/app/api/requests/[id]/route.test.ts
+37
View File
@@ -0,0 +1,37 @@
import { describe, it, expect } from "vitest";
import { DELETE } from "./route";
import { prisma } from "@/lib/db";
import type { Prisma } from "@prisma/client";
function del(id: string) {
return new Request(`http://localhost/api/requests/${id}`, { method: "DELETE" });
}
async function seed(jobData: Prisma.JobCreateWithoutRequestInput) {
return prisma.request.create({
data: { artist: "A", album: "B", status: "pending", job: { create: jobData } },
include: { job: true },
});
}
describe("remove request API", () => {
it("deletes a queued request and cascades its job", async () => {
const created = await seed({ state: "requested" });
const res = await DELETE(del(created.id), { params: Promise.resolve({ id: created.id }) });
expect(res.status).toBe(200);
expect(await prisma.request.findUnique({ where: { id: created.id } })).toBeNull();
expect(await prisma.job.findUnique({ where: { id: created.job!.id } })).toBeNull();
});
it("refuses to remove an in-flight job", async () => {
const created = await seed({ state: "downloading" });
const res = await DELETE(del(created.id), { params: Promise.resolve({ id: created.id }) });
expect(res.status).toBe(400);
expect(await prisma.request.findUnique({ where: { id: created.id } })).not.toBeNull();
});
it("404s for a missing request", async () => {
const res = await DELETE(del("nope"), { params: Promise.resolve({ id: "nope" }) });
expect(res.status).toBe(404);
});
});
web/src/app/api/requests/[id]/route.ts
+16
View File
@@ -0,0 +1,16 @@
import { prisma } from "@/lib/db";
export async function DELETE(_request: Request, { params }: { params: Promise<{ id: string }> }) {
const { id } = await params;
const req = await prisma.request.findUnique({ where: { id }, include: { job: true } });
if (!req) {
return Response.json({ error: "not found" }, { status: 404 });
}
if (req.job && req.job.state !== "requested") {
return Response.json({ error: "only queued items can be removed" }, { status: 400 });
}
await prisma.request.delete({ where: { id } }); // cascades Job + Candidates
return Response.json({ ok: true });
}
web/src/app/api/requests/route.ts
+2
View File
@@ -64,6 +64,8 @@ export async function GET() {
claimedAt: r.job.claimedAt,
updatedAt: r.job.updatedAt,
downloadProgress: r.job.downloadProgress,
downloadEtaSeconds: r.job.downloadEtaSeconds,
paused: r.job.paused,
candidateCount: cands.length,
chosen: chosen ? { source: chosen.source, format: chosen.format, trackCount: chosen.trackCount } : null,
};
web/src/app/api/wanted/route.ts
+1 -1
View File
@@ -4,7 +4,7 @@ import { searchReleaseGroup } from "@/lib/musicbrainz";
export async function GET() {
const items = await prisma.monitoredRelease.findMany({
where: { monitored: true, state: { not: "fulfilled" } },
where: { monitored: true, ignored: false, state: { not: "fulfilled" } },
orderBy: [{ lastSearchedAt: { sort: "asc", nulls: "first" } }, { createdAt: "asc" }],
});
return Response.json({
web/src/app/artists/[id]/discography-client.tsx
+16 -2
View File
@@ -15,12 +15,14 @@ type Release = {
secondaryTypes: string[];
firstReleaseDate: string | null;
monitored: boolean;
ignored: boolean;
state: string;
currentQualityClass: number | null;
};
type Detail = { id: string; mbid: string; name: string; autoMonitorFuture: boolean; showAllTypes: boolean; releases: Release[] };
function badge(r: Release): { label: string; cls: string } {
if (r.ignored) return { label: "Ignored", cls: "chip" };
if (r.currentQualityClass !== null) return { label: `Have q${r.currentQualityClass}`, cls: "chip done" };
if (r.monitored) return { label: r.state === "wanted" ? "Wanted" : r.state, cls: "chip working" };
return { label: "Dormant", cls: "chip" };
@@ -53,6 +55,15 @@ export function DiscographyClient({ id }: { id: string }) {
toast(res?.ok ? `Searching for ${r.album}` : `Couldn't search for ${r.album}`);
refresh();
}
async function toggleIgnore(r: Release) {
const res = await fetch(`/api/releases/${r.id}`, {
method: "PATCH",
headers: { "content-type": "application/json" },
body: JSON.stringify({ ignored: !r.ignored }),
}).catch(() => null);
if (!res?.ok) toast(`Couldn't update ${r.album}`);
refresh();
}
const counts = useMemo(() => {
const c: Record<string, number> = {};
@@ -108,12 +119,15 @@ export function DiscographyClient({ id }: { id: string }) {
<div className="actions">
<span className={b.cls}>{b.label}</span>
<label className="toggle">
<input type="checkbox" aria-label={`monitor ${r.album}`} checked={r.monitored} onChange={() => toggleMonitor(r)} />
<input type="checkbox" aria-label={`monitor ${r.album}`} checked={r.monitored} onChange={() => toggleMonitor(r)} disabled={r.ignored} />
monitor
</label>
<button className="btn sm ghost" onClick={() => searchNow(r)}>
<button className="btn sm ghost" onClick={() => searchNow(r)} disabled={r.ignored}>
Search now
</button>
<button className="btn sm ghost" onClick={() => toggleIgnore(r)}>
{r.ignored ? "Un-ignore" : "Ignore"}
</button>
</div>
</li>
);
web/src/app/artists/artists-client.tsx
+23 -2
View File
@@ -10,6 +10,7 @@ type Artist = {
id: string;
mbid: string;
name: string;
createdAt: string;
autoMonitorFuture: boolean;
releaseCount: number;
monitoredCount: number;
@@ -17,11 +18,21 @@ type Artist = {
};
type Hit = { mbid: string; name: string; disambiguation: string };
type SortKey = "followed" | "name" | "have" | "monitored" | "releases";
const SORTS: { key: SortKey; label: string; cmp: (a: Artist, b: Artist) => number }[] = [
{ key: "followed", label: "Recently followed", cmp: (a, b) => b.createdAt.localeCompare(a.createdAt) },
{ key: "name", label: "Name A–Z", cmp: (a, b) => a.name.localeCompare(b.name) },
{ key: "have", label: "Most in library", cmp: (a, b) => b.haveCount - a.haveCount || a.name.localeCompare(b.name) },
{ key: "monitored", label: "Most monitored", cmp: (a, b) => b.monitoredCount - a.monitoredCount || a.name.localeCompare(b.name) },
{ key: "releases", label: "Most releases", cmp: (a, b) => b.releaseCount - a.releaseCount || a.name.localeCompare(b.name) },
];
export function ArtistsClient() {
const [artists, setArtists] = useState<Artist[]>([]);
const [ownedNotFollowed, setOwnedNotFollowed] = useState<string[]>([]);
const [justFollowed, setJustFollowed] = useState<Set<string>>(new Set());
const [filter, setFilter] = useState("");
const [sort, setSort] = useState<SortKey>("followed");
const [showAdd, setShowAdd] = useState(false);
const [query, setQuery] = useState("");
const [hits, setHits] = useState<Hit[]>([]);
@@ -113,8 +124,10 @@ export function ArtistsClient() {
const followedMbids = new Set(artists.map((a) => a.mbid));
const shown = useMemo(() => {
const n = filter.trim().toLowerCase();
return n ? artists.filter((a) => a.name.toLowerCase().includes(n)) : artists;
}, [artists, filter]);
const filtered = n ? artists.filter((a) => a.name.toLowerCase().includes(n)) : artists;
const cmp = (SORTS.find((s) => s.key === sort) ?? SORTS[0]).cmp;
return [...filtered].sort(cmp);
}, [artists, filter, sort]);
const ownedShown = useMemo(
() => ownedNotFollowed.filter((name) => !justFollowed.has(name)),
[ownedNotFollowed, justFollowed],
@@ -137,6 +150,14 @@ export function ArtistsClient() {
onChange={(e) => setFilter(e.target.value)}
/>
</label>
<label className="field">
<span>Sort</span>
<select aria-label="sort artists" value={sort} onChange={(e) => setSort(e.target.value as SortKey)}>
{SORTS.map((s) => (
<option key={s.key} value={s.key}>{s.label}</option>
))}
</select>
</label>
</div>
<SectionHeader
web/src/app/design.css
+58
View File
@@ -245,6 +245,9 @@ nav.contents .sep { flex: 1; }
.btn.ghost { border-color: var(--rule-2); color: var(--graphite); }
.btn.ghost:hover { background: transparent; border-color: var(--ink); color: var(--ink); }
.press-controls { display: flex; gap: 0.5rem; align-items: center; margin: 0 0 1rem; flex-wrap: wrap; }
.item-controls { display: inline-flex; gap: 0.5rem; align-items: center; }
.toggle {
font-family: var(--mono); font-size: 0.64rem; letter-spacing: 0.1em; text-transform: uppercase;
color: var(--graphite); display: inline-flex; align-items: center; gap: 7px; cursor: pointer;
@@ -284,6 +287,28 @@ nav.contents .sep { flex: 1; }
font-family: var(--mono); font-size: 0.64rem; letter-spacing: 0.12em; text-transform: uppercase; color: var(--accent);
}
.save-row { display: flex; align-items: center; gap: 14px; margin-top: 8px; }
.edit-meta { display: flex; flex-direction: column; gap: 8px; width: 100%; }
.rmeta.why { font-style: italic; color: var(--graphite); margin-top: 2px; }
/* Discover unified row: artist block | compact album list (stacked) | actions.
Each album is a small horizontal item (cover + title/badge on one line) so two stack
vertically without making the row taller than the artist block. */
.disco-row { align-items: center; flex-wrap: wrap; gap: 14px 20px; }
.disco-row .main { flex: 1 1 200px; }
.disco-albums { display: flex; flex-direction: column; gap: 6px; flex: 0 1 300px; min-width: 220px; }
.disco-album { display: flex; align-items: center; gap: 8px; }
.disco-thumb { display: flex; align-items: center; gap: 8px; flex: 1; min-width: 0; background: none; border: 0; padding: 0; cursor: pointer; text-align: left; }
.disco-thumb .da-cover { width: 30px; height: 30px; flex-shrink: 0; }
.da-text { display: flex; flex-direction: column; min-width: 0; line-height: 1.15; }
.disco-album .da-title { font-size: 12px; color: var(--ink); white-space: nowrap; overflow: hidden; text-overflow: ellipsis; }
.disco-album .badge { font-family: var(--mono); font-size: 9px; text-transform: uppercase; letter-spacing: 0.04em; color: var(--graphite); }
.disco-album .btn.sm.want { padding: 1px 8px; flex-shrink: 0; }
.sel-tools { display: inline-flex; align-items: center; gap: 10px; }
.bulk-bar { display: flex; align-items: center; gap: 12px; flex-wrap: wrap; margin: 4px 0 12px; padding: 8px 12px; border: 1px solid var(--rule); background: var(--paper-2); }
.album-card { position: relative; }
.album-card.selected { outline: 2px solid var(--accent); outline-offset: 2px; }
.sel-check { position: absolute; top: 6px; left: 6px; z-index: 1; width: 22px; height: 22px; display: flex; align-items: center; justify-content: center; background: var(--accent); color: var(--paper); font-weight: 700; border: 1px solid var(--rule); }
.album-card:not(.selected) .sel-check { background: var(--paper-2); color: transparent; }
.settings-section { margin-bottom: 30px; }
.help { font-size: 0.8rem; color: var(--graphite); line-height: 1.5; margin: 4px 0 14px; max-width: 62ch; }
.help b { color: var(--ink); font-weight: 600; }
@@ -361,6 +386,39 @@ nav.contents .sep { flex: 1; }
.tab:hover { color: var(--ink); }
.tab.active { color: var(--ink); border-bottom-color: var(--accent); }
.tab .n { color: var(--rule-2); margin-left: 6px; }
.tab.has-errors .n { color: var(--alert); }
/* "Next up" badge on the first queued row (the item the worker claims next) */
.nextup {
display: inline-block; vertical-align: middle; margin-right: 9px; transform: translateY(-2px);
font-family: var(--mono); font-size: 0.56rem; letter-spacing: 0.13em; text-transform: uppercase;
color: var(--accent-ink); background: var(--accent); padding: 2px 7px; border-radius: 4px;
}
/* ── Kebab menu (⋮ popup for press actions) ───────────── */
.dept.has-actions { align-items: center; }
.dept.has-actions .fill { transform: none; }
.menu { position: relative; display: inline-flex; }
.menu-trigger {
font-size: 1.15rem; line-height: 1; color: var(--graphite); background: transparent;
border: 1px solid var(--rule); border-radius: 6px; width: 32px; height: 30px; cursor: pointer;
display: inline-flex; align-items: center; justify-content: center;
}
.menu-trigger:hover { color: var(--ink); border-color: var(--rule-2); }
.menu-pop {
position: absolute; top: calc(100% + 6px); right: 0; z-index: 30; min-width: 190px;
background: var(--paper-2); border: 1px solid var(--rule-2); border-radius: 9px;
box-shadow: 0 10px 30px rgba(0, 0, 0, 0.22); padding: 5px;
display: flex; flex-direction: column; gap: 1px;
}
.menu-item {
font-family: var(--mono); font-size: 0.68rem; letter-spacing: 0.1em; text-transform: uppercase;
text-align: left; background: transparent; border: 0; color: var(--ink);
padding: 10px 11px; border-radius: 6px; cursor: pointer; white-space: nowrap;
}
.menu-item:hover { background: var(--rule); }
.menu-item:disabled { color: var(--rule-2); cursor: default; background: transparent; }
.menu-item.danger { color: var(--alert); }
/* ── Discography row (cover thumb + click-to-modal) ───── */
.disco-open {
web/src/app/discover/discover-client.tsx
+143 -74
View File
@@ -4,52 +4,81 @@ import { useCallback, useEffect, useState } from "react";
import { PageHead } from "../_ui/page-head";
import { timeAgo } from "../_ui/status";
import { SectionHeader } from "../_ui/section-header";
import { ArtistModal } from "../_ui/artist-modal";
import { AlbumModal } from "../_ui/album-modal";
import { CoverArt } from "../_ui/cover-art";
import { toast } from "../_ui/toast";
type Suggestion = {
type Album = {
id: string;
artistMbid: string;
artistName: string;
rgMbid: string | null;
album: string | null;
primaryType: string | null;
secondaryTypes: string[];
firstReleaseDate: string | null;
albumReason: string | null;
};
type Row = {
id: string | null;
artistMbid: string;
artistName: string;
score: number;
seedCount: number;
sources: string[];
seeds: string[];
albums: Album[];
};
/** "Similar to Radiohead, Coldplay +2" — the followed artists that surfaced this suggestion. */
function similarTo(seeds: string[]): string | null {
if (!seeds.length) return null;
const shown = seeds.slice(0, 2).join(", ");
const extra = seeds.length - 2;
return `Similar to ${shown}${extra > 0 ? ` +${extra} more` : ""}`;
}
/** Full-length albums only — hide singles/EPs and secondary-type releases (live/comp) that
* pre-date the albums-only derivation and still linger in the DB. */
function isFullAlbum(a: Album): boolean {
return (a.primaryType ?? "Album") === "Album" && (a.secondaryTypes?.length ?? 0) === 0;
}
/** Human badge for why an album is surfaced. */
function badgeFor(reason: string | null): string | null {
if (reason === "newest") return "Newest";
if (reason === "popular") return "Most played";
if (reason === "newest,popular") return "Newest · Most played";
return null;
}
/** "just now" / "3h ago" / "2d ago" — timeAgo has no suffix, so add one (except "just now"). */
function agoLabel(value: string): string {
const a = timeAgo(value, Date.now());
return !a || a === "just now" ? a || "recently" : `${a} ago`;
}
type Feed = { artists: Suggestion[]; albums: Suggestion[] };
type SearchResult = {
seed: { mbid: string; name: string } | null;
similar: { mbid: string; name: string; score: number }[];
};
export function DiscoverClient() {
const [feed, setFeed] = useState<Feed>({ artists: [], albums: [] });
const [rows, setRows] = useState<Row[]>([]);
const [result, setResult] = useState<string | null>(null);
const [lastRunAt, setLastRunAt] = useState<string | null>(null);
const [running, setRunning] = useState(false);
const [query, setQuery] = useState("");
const [search, setSearch] = useState<SearchResult | null>(null);
const [busy, setBusy] = useState(false);
const [confirmReset, setConfirmReset] = useState(false);
const [followedSimilar, setFollowedSimilar] = useState<Set<string>>(new Set());
const [artistModal, setArtistModal] = useState<{ mbid: string; name: string } | null>(null);
const [albumModal, setAlbumModal] = useState<Suggestion | null>(null);
const [albumModal, setAlbumModal] = useState<{ a: Album; artistName: string; artistMbid: string } | null>(null);
const loadFeed = useCallback(async () => {
setFeed(await (await fetch("/api/discover")).json());
const loadRows = useCallback(async () => {
setRows((await (await fetch("/api/discover")).json()).rows ?? []);
}, []);
useEffect(() => {
loadFeed();
loadRows();
fetch("/api/discover/run")
.then((r) => r.json())
.then((s: { result: string | null; requested: boolean; inProgress: boolean; lastRunAt: string | null }) => {
@@ -57,7 +86,7 @@ export function DiscoverClient() {
setLastRunAt(s.lastRunAt);
setRunning(s.requested || s.inProgress);
});
}, [loadFeed]);
}, [loadRows]);
useEffect(() => {
if (!running) return;
@@ -66,19 +95,30 @@ export function DiscoverClient() {
setResult(s.result);
setLastRunAt(s.lastRunAt);
if (!s.requested && !s.inProgress) {
// the sweep finished (neither queued nor in progress) — refresh the feed
setRunning(false);
loadFeed();
loadRows();
}
}, 2000);
return () => clearInterval(id);
}, [running, loadFeed]);
}, [running, loadRows]);
async function discoverNow() {
await fetch("/api/discover/run", { method: "POST" });
setRunning(true);
}
async function resetDiscover() {
setConfirmReset(false);
const res = await fetch("/api/discover/reset", { method: "POST" }).catch(() => null);
if (res && (res.ok || res.status === 202)) {
setRows([]); // clears immediately; the rebuild repopulates
setRunning(true);
toast("Rebuilding suggestions from scratch…");
} else {
toast("Couldn't reset");
}
}
async function runSearch(e: React.FormEvent) {
e.preventDefault();
if (!query.trim()) return;
@@ -90,18 +130,26 @@ export function DiscoverClient() {
}
}
async function followSimilar(a: { mbid: string; name: string }) {
async function followByMbid(mbid: string, name: string) {
const res = await fetch("/api/artists", {
method: "POST",
headers: { "content-type": "application/json" },
body: JSON.stringify({ mbid: a.mbid, name: a.name }),
});
if (res.ok || res.status === 409) {
body: JSON.stringify({ mbid, name }),
}).catch(() => null);
return !!res && (res.ok || res.status === 409);
}
async function followSimilar(a: { mbid: string; name: string }) {
if (await followByMbid(a.mbid, a.name)) {
setFollowedSimilar((s) => new Set(s).add(a.mbid));
toast(`Following ${a.name}`);
} else {
toast(`Couldn't follow ${a.name}`);
}
}
// Suggestion actions key off a suggestion id. A row with no artist-suggestion id (album-only,
// e.g. the artist was dismissed earlier) falls back to a plain follow-by-MBID.
async function act(id: string, action: "follow" | "want" | "dismiss") {
const res = await fetch(`/api/discover/${id}`, {
method: "POST",
@@ -113,7 +161,17 @@ export function DiscoverClient() {
return;
}
toast(action === "follow" ? "Following" : action === "want" ? "Added to wanted" : "Dismissed");
loadFeed();
loadRows();
}
async function followRow(row: Row) {
if (row.id) return act(row.id, "follow");
if (await followByMbid(row.artistMbid, row.artistName)) {
toast(`Following ${row.artistName}`);
loadRows();
} else {
toast(`Couldn't follow ${row.artistName}`);
}
}
return (
@@ -124,6 +182,17 @@ export function DiscoverClient() {
<button className="btn accent" onClick={discoverNow} disabled={running}>
{running ? "Discovering…" : "Discover now"}
</button>
{confirmReset ? (
<>
<span className="rmeta">Clear all suggestions and rebuild from scratch?</span>
<button className="btn sm accent" onClick={resetDiscover}>Reset</button>
<button className="btn sm ghost" onClick={() => setConfirmReset(false)}>Cancel</button>
</>
) : (
<>
<button className="btn sm ghost" onClick={() => setConfirmReset(true)} disabled={running}>
Reset
</button>
<span className="rmeta">
{running
? "Sweeping your followed artists for new suggestions…"
@@ -131,6 +200,8 @@ export function DiscoverClient() {
? `Last discovered ${agoLabel(lastRunAt)}${result ? ` · ${result}` : ""}`
: "Not yet run — press Discover now to sweep your followed artists"}
</span>
</>
)}
</div>
<SectionHeader title="Find similar" />
@@ -150,9 +221,7 @@ export function DiscoverClient() {
<li key={s.mbid} className="list-row">
<div className="main">
<div className="rtitle">
<button className="linkish" onClick={() => setArtistModal({ mbid: s.mbid, name: s.name })}>
{s.name}
</button>
<a href={`/discover/artist/${s.mbid}?name=${encodeURIComponent(s.name)}`}>{s.name}</a>
</div>
<div className="rmeta">
<span className="score">similarity {s.score.toFixed(1)}</span>
@@ -172,85 +241,85 @@ export function DiscoverClient() {
</ul>
) : null}
<SectionHeader title="Suggested artists" note={`${feed.artists.length}`} />
{feed.artists.length === 0 ? (
<SectionHeader title="Suggested artists" note={`${rows.length}`} />
{rows.length === 0 ? (
<p className="muted-note">No suggestions yet — run Discover, or follow a few artists to seed it.</p>
) : (
<ul className="list">
{feed.artists.map((s) => (
<li key={s.id} className="list-row">
{rows.map((row) => {
const albums = row.albums.filter(isFullAlbum);
return (
<li key={row.artistMbid} className="list-row disco-row">
<div className="main">
<div className="rtitle">
<button className="linkish" onClick={() => setArtistModal({ mbid: s.artistMbid, name: s.artistName })}>
{s.artistName}
</button>
<a href={`/discover/artist/${row.artistMbid}?name=${encodeURIComponent(row.artistName)}`}>
{row.artistName}
</a>
</div>
<div className="rmeta">
<span className="score">score {s.score.toFixed(2)}</span>
<span className="score">score {row.score.toFixed(2)}</span>
<span className="dot">·</span>
<span>{s.seedCount} seed{s.seedCount === 1 ? "" : "s"}</span>
<span className="dot">·</span>
<span>{s.sources.join(", ")}</span>
<span>{row.sources.join(", ")}</span>
</div>
{similarTo(row.seeds) ? <div className="rmeta why">{similarTo(row.seeds)}</div> : null}
</div>
{albums.length > 0 ? (
<div className="disco-albums">
{albums.map((a) => (
<div key={a.id} className="disco-album">
<button
className="disco-thumb"
onClick={() => setAlbumModal({ a, artistName: row.artistName, artistMbid: row.artistMbid })}
aria-label={`Open ${a.album ?? ""}`}
>
<CoverArt rgMbid={a.rgMbid} alt={a.album ?? ""} className="da-cover" />
<span className="da-text">
<span className="da-title">{a.album}</span>
{badgeFor(a.albumReason) ? <span className="badge">{badgeFor(a.albumReason)}</span> : null}
</span>
</button>
<button className="btn sm ghost want" onClick={() => act(a.id, "want")}>
Want
</button>
</div>
))}
</div>
) : null}
<div className="actions">
<button className="btn sm" onClick={() => act(s.id, "follow")}>
<button className="btn sm" onClick={() => followRow(row)}>
Follow
</button>
<button className="btn sm ghost" onClick={() => act(s.id, "dismiss")}>
{row.id ? (
<button className="btn sm ghost" onClick={() => act(row.id!, "dismiss")}>
Dismiss
</button>
) : null}
</div>
</li>
))}
);
})}
</ul>
)}
<SectionHeader title="Suggested albums" note={`${feed.albums.length}`} />
{feed.albums.length === 0 ? (
<p className="muted-note">No album suggestions yet.</p>
) : (
<ul className="list">
{feed.albums.map((s) => (
<li key={s.id} className="list-row">
<div className="main">
<button className="disco-open" onClick={() => setAlbumModal(s)} aria-label={`Open ${s.album ?? ""}`}>
<CoverArt rgMbid={s.rgMbid} alt={s.album ?? ""} className="thumb" />
<span>
<span className="rtitle">
{s.album} <span className="artist">· {s.artistName}</span>
</span>
<span className="rmeta">
<span className="score">score {s.score.toFixed(2)}</span>
</span>
</span>
</button>
</div>
<div className="actions">
<button className="btn sm" onClick={() => act(s.id, "want")}>
Want
</button>
<button className="btn sm ghost" onClick={() => act(s.id, "dismiss")}>
Dismiss
</button>
</div>
</li>
))}
</ul>
)}
{artistModal ? (
<ArtistModal open onClose={() => setArtistModal(null)} mbid={artistModal.mbid} initialName={artistModal.name} />
) : null}
{albumModal ? (
<AlbumModal
open
onClose={() => setAlbumModal(null)}
album={{ rgMbid: albumModal.rgMbid, album: albumModal.album ?? "", artist: albumModal.artistName, year: null, type: null, artistMbid: albumModal.artistMbid }}
album={{
rgMbid: albumModal.a.rgMbid,
album: albumModal.a.album ?? "",
artist: albumModal.artistName,
year: albumModal.a.firstReleaseDate?.slice(0, 4) ?? null,
type: albumModal.a.primaryType,
artistMbid: albumModal.artistMbid,
}}
actions={
<button
className="btn sm"
onClick={() => {
act(albumModal.id, "want");
act(albumModal.a.id, "want");
setAlbumModal(null);
}}
>
web/src/app/lastfm/lastfm-client.tsx
BIN
View File
Binary file not shown.
web/src/app/library/library-client.tsx
+332 -6
View File
@@ -14,22 +14,115 @@ type Album = {
album: string;
format: string;
qualityClass: number;
importedAt: string;
rgMbid: string | null;
year: string | null;
primaryType: string | null;
artistMbid: string | null;
monitoredReleaseId: string | null;
trackNames: string[];
};
type SortKey = "added" | "release" | "album" | "artist";
const SORTS: { key: SortKey; label: string; cmp: (a: Album, b: Album) => number }[] = [
{ key: "added", label: "Recently added", cmp: (a, b) => b.importedAt.localeCompare(a.importedAt) },
// Release date, newest first; albums with an unknown year sort to the bottom.
{ key: "release", label: "Release date", cmp: (a, b) => (b.year ?? "0000").localeCompare(a.year ?? "0000") || a.album.localeCompare(b.album) },
{ key: "album", label: "Album A–Z", cmp: (a, b) => a.album.localeCompare(b.album) },
{ key: "artist", label: "Artist A–Z", cmp: (a, b) => a.artist.localeCompare(b.artist) || (a.year ?? "").localeCompare(b.year ?? "") },
];
type Unmatched = { id: string; artist: string; album: string; path: string; reason: string };
type DupItem = { id: string; artist: string; album: string; format: string; qualityClass: number; year: string | null };
type IntegrityIssue = { id: string; artist: string; album: string; issues: string[] };
export function LibraryClient() {
const [albums, setAlbums] = useState<Album[]>([]);
const [unmatched, setUnmatched] = useState<Unmatched[]>([]);
const [duplicates, setDuplicates] = useState<DupItem[][]>([]);
const [integrity, setIntegrity] = useState<{ checked: number; issues: IntegrityIssue[] } | null>(null);
const [checking, setChecking] = useState(false);
const [selectMode, setSelectMode] = useState(false);
const [selected, setSelected] = useState<Set<string>>(new Set());
const [bulkConfirmDelete, setBulkConfirmDelete] = useState(false);
function toggleSelect(id: string) {
setSelected((s) => {
const n = new Set(s);
if (n.has(id)) n.delete(id); else n.add(id);
return n;
});
}
function exitSelect() {
setSelectMode(false);
setSelected(new Set());
setBulkConfirmDelete(false);
}
async function bulk(action: "delete" | "ignore") {
const ids = [...selected];
if (ids.length === 0) return;
const res = await fetch("/api/library/bulk", {
method: "POST",
headers: { "content-type": "application/json" },
body: JSON.stringify({ action, ids }),
}).catch(() => null);
if (res?.ok) {
const body = await res.json();
toast(
`${action === "delete" ? "Deleted" : "Ignoring"} ${body.ok} album${body.ok === 1 ? "" : "s"}` +
(body.failed?.length ? ` · ${body.failed.length} failed` : ""),
);
exitSelect();
refresh();
} else {
toast(`Bulk ${action} failed`);
}
}
const [q, setQ] = useState("");
const [sort, setSort] = useState<SortKey>("added");
const [open, setOpen] = useState<Album | null>(null);
const [confirmDelete, setConfirmDelete] = useState(false);
const [editing, setEditing] = useState(false);
const [edit, setEdit] = useState({ artist: "", album: "", year: "" });
const [saving, setSaving] = useState(false);
function show(a: Album | null) {
setOpen(a);
setConfirmDelete(false);
setEditing(false);
}
function startEdit() {
if (!open) return;
setEdit({ artist: open.artist, album: open.album, year: open.year ?? "" });
setEditing(true);
}
async function saveEdit() {
if (!open) return;
if (!edit.artist.trim() || !edit.album.trim()) {
toast("Artist and album are required");
return;
}
setSaving(true);
try {
const res = await fetch(`/api/library/${open.id}`, {
method: "PATCH",
headers: { "content-type": "application/json" },
body: JSON.stringify({ artist: edit.artist.trim(), album: edit.album.trim(), year: edit.year.trim() }),
}).catch(() => null);
if (res?.ok) {
toast(`Updated ${edit.album.trim()}`);
setEditing(false);
show(null);
refresh();
} else {
toast((res && (await res.json().catch(() => null))?.error) || "Couldn't update album");
}
} finally {
setSaving(false);
}
}
async function remove() {
@@ -49,16 +142,60 @@ export function LibraryClient() {
fetch("/api/library")
.then((r) => r.json())
.then((d) => setAlbums(d.albums ?? []));
fetch("/api/library/unmatched")
.then((r) => r.json())
.then((d) => setUnmatched(d.unmatched ?? []));
fetch("/api/library/duplicates")
.then((r) => r.json())
.then((d) => setDuplicates(d.duplicates ?? []));
}
useEffect(() => {
refresh();
}, []);
async function upgrade() {
if (!open) return;
const res = await fetch(`/api/library/${open.id}/upgrade`, { method: "POST" }).catch(() => null);
if (res && (res.status === 202 || res.ok)) {
const body = await res.json().catch(() => null);
toast(body?.enqueued === false ? `Already searching for ${open.album}` : `Re-acquiring ${open.album} — keeps it only if better`);
show(null);
} else {
toast((res && (await res.json().catch(() => null))?.error) || "Couldn't start the upgrade");
}
}
async function checkIntegrity() {
setChecking(true);
try {
const res = await fetch("/api/library/integrity").catch(() => null);
if (res?.ok) {
setIntegrity(await res.json());
} else {
toast("Couldn't run the integrity check");
}
} finally {
setChecking(false);
}
}
async function dismissUnmatched(u: Unmatched) {
const res = await fetch(`/api/library/unmatched?id=${u.id}`, { method: "DELETE" }).catch(() => null);
if (res?.ok) {
setUnmatched((list) => list.filter((x) => x.id !== u.id));
} else {
toast("Couldn't dismiss");
}
}
const shown = useMemo(() => {
const needle = q.trim().toLowerCase();
if (!needle) return albums;
return albums.filter((a) => `${a.artist} ${a.album}`.toLowerCase().includes(needle));
}, [albums, q]);
const filtered = needle
? albums.filter((a) => `${a.artist} ${a.album}`.toLowerCase().includes(needle))
: albums;
const cmp = (SORTS.find((s) => s.key === sort) ?? SORTS[0]).cmp;
return [...filtered].sort(cmp);
}, [albums, q, sort]);
return (
<div>
@@ -70,6 +207,14 @@ export function LibraryClient() {
<span>Filter</span>
<input aria-label="filter library" placeholder="Artist or album…" value={q} onChange={(e) => setQ(e.target.value)} />
</label>
<label className="field">
<span>Sort</span>
<select aria-label="sort library" value={sort} onChange={(e) => setSort(e.target.value as SortKey)}>
{SORTS.map((s) => (
<option key={s.key} value={s.key}>{s.label}</option>
))}
</select>
</label>
</form>
{albums.length === 0 ? (
@@ -77,13 +222,46 @@ export function LibraryClient() {
) : (
<SectionHeader
title="Pressings"
note={q ? `${shown.length} of ${albums.length} albums` : `${albums.length} albums`}
note={
<span className="sel-tools">
<span>{q ? `${shown.length} of ${albums.length}` : `${albums.length} albums`}</span>
<button className="btn sm ghost" onClick={() => (selectMode ? exitSelect() : setSelectMode(true))}>
{selectMode ? "Cancel" : "Select"}
</button>
</span>
}
/>
)}
{selectMode ? (
<div className="bulk-bar">
<span>{selected.size} selected</span>
{bulkConfirmDelete ? (
<>
<span className="rmeta">Delete {selected.size} album{selected.size === 1 ? "" : "s"} and their files?</span>
<button className="btn sm accent" onClick={() => bulk("delete")} disabled={selected.size === 0}>Delete</button>
<button className="btn sm ghost" onClick={() => setBulkConfirmDelete(false)}>Cancel</button>
</>
) : (
<>
<button className="btn sm ghost" onClick={() => setBulkConfirmDelete(true)} disabled={selected.size === 0}>Delete selected</button>
<button className="btn sm ghost" onClick={() => bulk("ignore")} disabled={selected.size === 0}>Ignore selected</button>
</>
)}
</div>
) : null}
{albums.length > 0 ? (
<div className="album-grid">
{shown.map((a) => (
<button key={a.id} className="album-card" onClick={() => show(a)} aria-label={`Open ${a.album}`}>
<button
key={a.id}
className={`album-card${selectMode && selected.has(a.id) ? " selected" : ""}`}
onClick={() => (selectMode ? toggleSelect(a.id) : show(a))}
aria-label={selectMode ? `Select ${a.album}` : `Open ${a.album}`}
aria-pressed={selectMode ? selected.has(a.id) : undefined}
>
{selectMode ? <span className="sel-check" aria-hidden>{selected.has(a.id) ? "✓" : ""}</span> : null}
<CoverArt rgMbid={a.rgMbid} alt={a.album} size={250} />
<div className="ac-title">{a.album}</div>
<div className="ac-artist">{a.artist}</div>
@@ -97,6 +275,120 @@ export function LibraryClient() {
</div>
) : null}
{albums.length > 0 ? (
<>
<SectionHeader
title="Integrity"
note={
<button className="btn sm ghost" onClick={checkIntegrity} disabled={checking}>
{checking ? "Checking…" : "Check now"}
</button>
}
/>
{integrity ? (
integrity.issues.length === 0 ? (
<p className="muted-note">Checked {integrity.checked} albums — no problems found.</p>
) : (
<ul className="list">
{integrity.issues.map((it) => {
const album = albums.find((a) => a.id === it.id);
return (
<li key={it.id} className="list-row">
<div className="main">
{album ? (
<button className="linkish rtitle" onClick={() => show(album)}>
{it.artist} — {it.album}
</button>
) : (
<div className="rtitle">{it.artist} — {it.album}</div>
)}
<div className="rmeta">
<span className="score">{it.issues.join(" · ")}</span>
</div>
</div>
</li>
);
})}
</ul>
)
) : (
<p className="muted-note">
Check every album folder on disk for missing folders, empty/truncated files, and
track-count drift.
</p>
)}
</>
) : null}
{duplicates.length > 0 ? (
<>
<SectionHeader title="Possible duplicates" note={`${duplicates.length}`} />
<p className="muted-note">
Albums that look like the same release held more than once (same MusicBrainz release,
or the same title ignoring edition wording). Open one to edit or delete it.
</p>
{duplicates.map((group, gi) => (
<ul className="list" key={gi} style={{ marginBottom: 10 }}>
{group.map((d) => {
const album = albums.find((a) => a.id === d.id);
return (
<li key={d.id} className="list-row">
<div className="main">
{album ? (
<button className="linkish rtitle" onClick={() => show(album)}>
{d.artist} — {d.album}
</button>
) : (
<div className="rtitle">{d.artist} — {d.album}</div>
)}
<div className="rmeta">
<span className="score">
{d.format}
{d.qualityClass >= 3 ? " · hi-res" : ""}
{d.year ? ` · ${d.year}` : ""}
</span>
</div>
</div>
</li>
);
})}
</ul>
))}
</>
) : null}
{unmatched.length > 0 ? (
<>
<SectionHeader title="Couldn’t match" note={`${unmatched.length}`} />
<p className="muted-note">
Album folders on disk the scan couldn’t resolve to a MusicBrainz release. Fix the
folder name (Artist / Album (Year)) or re-add them manually, then re-scan. Dismiss to
hide an entry.
</p>
<ul className="list">
{unmatched.map((u) => (
<li key={u.id} className="list-row">
<div className="main">
<div className="rtitle">
{u.artist ? `${u.artist} — ` : ""}
{u.album}
</div>
<div className="rmeta">
<span className="dim">{u.path}</span> <span className="dot">·</span>{" "}
<span className="score">{u.reason}</span>
</div>
</div>
<div className="actions">
<button className="btn sm ghost" onClick={() => dismissUnmatched(u)}>
Dismiss
</button>
</div>
</li>
))}
</ul>
</>
) : null}
{open ? (
<AlbumModal
open
@@ -104,7 +396,31 @@ export function LibraryClient() {
album={{ rgMbid: open.rgMbid, album: open.album, artist: open.artist, year: open.year, type: open.primaryType, artistMbid: open.artistMbid, trackNames: open.trackNames }}
status={<span className="chip done">In library · {open.format}</span>}
actions={
confirmDelete ? (
editing ? (
<div className="edit-meta">
<label className="field">
<span>Artist</span>
<input aria-label="edit artist" value={edit.artist} onChange={(e) => setEdit((s) => ({ ...s, artist: e.target.value }))} />
</label>
<label className="field">
<span>Album</span>
<input aria-label="edit album" value={edit.album} onChange={(e) => setEdit((s) => ({ ...s, album: e.target.value }))} />
</label>
<label className="field">
<span>Year</span>
<input aria-label="edit year" placeholder="YYYY" value={edit.year} onChange={(e) => setEdit((s) => ({ ...s, year: e.target.value }))} />
</label>
<div className="save-row">
<button className="btn sm accent" onClick={saveEdit} disabled={saving}>
{saving ? "Saving…" : "Save"}
</button>
<button className="btn sm ghost" onClick={() => setEditing(false)} disabled={saving}>
Cancel
</button>
</div>
<p className="muted-note">Renames the folder on disk and re-resolves cover art. Embedded file tags are left as-is.</p>
</div>
) : confirmDelete ? (
<>
<span className="rmeta">Delete this album and its files from disk?</span>
<button className="btn sm accent" onClick={remove}>
@@ -115,9 +431,19 @@ export function LibraryClient() {
</button>
</>
) : (
<>
{open.monitoredReleaseId ? (
<button className="btn sm ghost" onClick={upgrade}>
Replace / upgrade
</button>
) : null}
<button className="btn sm ghost" onClick={startEdit}>
Edit metadata
</button>
<button className="btn sm ghost" onClick={() => setConfirmDelete(true)}>
Delete album
</button>
</>
)
}
/>
web/src/app/queue.tsx
+225 -48
View File
@@ -1,14 +1,34 @@
"use client";
import { useEffect, useState } from "react";
import { describeJob, timeAgo } from "./_ui/status";
import { describeJob, etaLabel, timeAgo } from "./_ui/status";
import { StatTiles } from "./_ui/stat-tiles";
import { SectionHeader } from "./_ui/section-header";
import { JobRow } from "./_ui/job-row";
import { ProgressBar } from "./_ui/progress-bar";
import { PressedList } from "./_ui/pressed-list";
import { KebabMenu } from "./_ui/kebab-menu";
import { toast } from "./_ui/toast";
type Tab = "active" | "queue" | "errors";
// Split the on-the-press rows into three honest buckets so active presses aren't buried between
// queued items and errors. Active = being worked (searching/downloading/finishing), Queue =
// waiting to be claimed, Errors = needs attention.
function categoryOf(state: string, stage: string): Tab {
const kind = describeJob(state, stage).kind;
if (kind === "attention") return "errors";
if (state === "requested") return "queue";
return "active";
}
const TAB_LABEL: Record<Tab, string> = { active: "Active", queue: "In queue", errors: "Errors" };
const TAB_EMPTY: Record<Tab, string> = {
active: "Nothing pressing right now.",
queue: "The queue is empty.",
errors: "No errors — everything's flowing.",
};
type Row = {
id: string;
artist: string;
@@ -26,6 +46,8 @@ type Row = {
candidateCount: number;
chosen: { source: string; format: string; trackCount: number } | null;
downloadProgress: number;
downloadEtaSeconds: number | null;
paused: boolean;
} | null;
};
@@ -41,13 +63,16 @@ function jobOf(r: Row) {
candidateCount: 0,
chosen: null,
downloadProgress: 0,
downloadEtaSeconds: null,
paused: false,
}
);
}
function pressedMark(r: Row): string {
if (!r.createdAt) return "Lyra";
const d = new Date(r.createdAt);
const when = r.job?.updatedAt ?? r.createdAt; // when it was pressed, not when it was requested
if (!when) return "Lyra";
const d = new Date(when);
return `Lyra · ${d.toLocaleDateString("en-GB", { day: "2-digit", month: "short" }).toUpperCase()}`;
}
@@ -57,16 +82,22 @@ export function Queue() {
const [watching, setWatching] = useState(0);
const [artist, setArtist] = useState("");
const [album, setAlbum] = useState("");
const [paused, setPaused] = useState(false);
// null = follow the smart default (Active if anything's active, else Queue, else Errors); once
// the user picks a tab, their choice sticks.
const [tab, setTab] = useState<Tab | null>(null);
async function refresh() {
const [reqRes, wantRes, artRes] = await Promise.all([
const [reqRes, wantRes, artRes, floorRes] = await Promise.all([
fetch("/api/requests"),
fetch("/api/wanted"),
fetch("/api/artists"),
fetch("/api/floor"),
]);
setRows((await reqRes.json()).requests ?? []);
setWanted(((await wantRes.json()).wanted ?? []).length);
setWatching(((await artRes.json()).artists ?? []).length);
setPaused((await floorRes.json()).paused ?? false);
}
useEffect(() => {
@@ -85,6 +116,55 @@ export function Queue() {
refresh();
}
async function floorAction(action: string, note: string) {
const res = await fetch("/api/floor", {
method: "POST",
headers: { "content-type": "application/json" },
body: JSON.stringify({ action }),
}).catch(() => null);
if (!res?.ok) {
toast("Couldn't update the press");
return;
}
toast(note);
refresh();
}
async function clearPress() {
const queued = rows.filter((r) => jobOf(r).state === "requested").length;
if (queued === 0) return;
if (!confirm(`Clear ${queued} queued item${queued === 1 ? "" : "s"} from the press?`)) return;
await floorAction("clear", "Cleared the queue");
}
async function stopPress() {
if (!confirm("Stop the current download and pause the press? The worker restarts and the aborted album re-downloads when you resume.")) return;
await floorAction("stop", "Stopping the press…");
}
async function pauseItem(id: string, next: boolean) {
const res = await fetch(`/api/requests/${id}/pause`, {
method: "POST",
headers: { "content-type": "application/json" },
body: JSON.stringify({ paused: next }),
}).catch(() => null);
if (!res?.ok) {
toast("Couldn't update the item");
return;
}
refresh();
}
async function removeItem(id: string) {
const res = await fetch(`/api/requests/${id}`, { method: "DELETE" }).catch(() => null);
if (!res?.ok) {
toast("Couldn't remove the item");
return;
}
toast("Removed from the press");
refresh();
}
async function submit(e: React.FormEvent) {
e.preventDefault();
if (!artist.trim() || !album.trim()) return;
@@ -105,42 +185,44 @@ export function Queue() {
}
const active = rows.filter((r) => jobOf(r).state !== "imported");
const pressed = rows.filter((r) => jobOf(r).state === "imported");
const buckets: Record<Tab, Row[]> = { active: [], queue: [], errors: [] };
for (const r of active) {
const j = jobOf(r);
buckets[categoryOf(j.state, j.currentStage)].push(r);
}
// The API returns rows newest-first, but the worker claims the OLDEST queued job next
// (claim_next: ORDER BY createdAt ASC). Order the queue tab to match, so the item that runs
// next sits at the top instead of the bottom.
const ts = (r: Row) => (r.createdAt ? new Date(r.createdAt).getTime() : 0);
buckets.queue.sort((a, b) => ts(a) - ts(b));
// "Next up" = the first queued item the worker will actually claim (oldest, not individually
// paused) — mark it so the ordering reads clearly.
const nextUpId = buckets.queue.find((r) => !jobOf(r).paused)?.id ?? null;
const queuedCount = buckets.queue.length;
const downloading = buckets.active.length > 0;
// Smart default: land on the tab that has something worth looking at.
const defaultTab: Tab = buckets.active.length ? "active" : queuedCount ? "queue" : "errors";
const activeTab: Tab = tab ?? defaultTab;
const shown = buckets[activeTab];
return (
<div>
<StatTiles
items={[
{ k: "On the press", v: active.length },
{ k: "Pressed", v: pressed.length, accent: true },
{ k: "Wanted", v: wanted },
{ k: "Watching", v: watching, unit: "artists" },
]}
/>
// "Recently Pressed": imported albums ordered by when they were PRESSED (job.updatedAt ≈ import
// time), newest first — an album requested long ago but imported just now is recently pressed
// even though its Request is old (rows arrive by request createdAt, which buried fresh imports).
// Dedupe by artist+album (re-downloads leave several completed Requests; keep the most recent).
const seenPressed = new Set<string>();
const pressedAll = rows
.filter((r) => jobOf(r).state === "imported")
.sort((a, b) => new Date(jobOf(b).updatedAt ?? 0).getTime() - new Date(jobOf(a).updatedAt ?? 0).getTime())
.filter((r) => {
const key = `${r.artist} ${r.album}`.toLowerCase();
if (seenPressed.has(key)) return false;
seenPressed.add(key);
return true;
});
const pressed = pressedAll.slice(0, 10); // show only the 10 most recently pressed
<SectionHeader title="On the Press" note={`${active.length} running`} />
<form className="request-form" onSubmit={submit}>
<label className="field">
<span>Artist</span>
<input aria-label="artist" placeholder="Artist" value={artist} onChange={(e) => setArtist(e.target.value)} />
</label>
<label className="field">
<span>Album</span>
<input aria-label="album" placeholder="Album" value={album} onChange={(e) => setAlbum(e.target.value)} />
</label>
<button type="submit" className="btn accent">
Request
</button>
</form>
{active.length === 0 ? (
<p className="empty">The press is quiet. Queue a release above, or send one over from Wanted or Discover.</p>
) : (
<section className="floor">
{(() => {
const now = Date.now();
return active.map((r) => {
function renderRow(r: Row) {
const j = jobOf(r);
const d = describeJob(j.state, j.currentStage);
const attention = d.kind === "attention";
@@ -171,6 +253,24 @@ export function Queue() {
Retry
</button>
</>
) : j.state === "requested" ? (
<span className="item-controls">
{j.paused ? (
<>
<span className="chip">Paused</span>{" "}
<button className="btn sm ghost" onClick={() => pauseItem(r.id, false)}>
Resume
</button>
</>
) : (
<button className="btn sm ghost" onClick={() => pauseItem(r.id, true)}>
Pause
</button>
)}{" "}
<button className="btn sm ghost" onClick={() => removeItem(r.id)}>
Remove
</button>
</span>
) : undefined;
let bar: React.ReactNode;
@@ -178,15 +278,18 @@ export function Queue() {
bar = <ProgressBar kind="working" indeterminate caption="searching" />;
} else if (j.state === "downloading") {
const pct = Math.round((j.downloadProgress || 0) * 100);
if (pct <= 0) {
// Bytes haven't started flowing yet — Qobuz login/metadata/artwork, a Soulseek peer
// queue, or a YouTube info-fetch. Show an indeterminate bar instead of a stuck-looking
// 0% (it flips to the % bar on the first byte).
bar = <ProgressBar kind="working" indeterminate caption="preparing…" />;
} else {
const total = j.chosen?.trackCount ?? 0;
const done = total ? Math.round((pct / 100) * total) : 0;
bar = (
<ProgressBar
kind="working"
percent={pct}
caption={total ? `${pct}% · ${done}/${total} tracks` : `${pct}%`}
/>
);
const base = total ? `${pct}% · ${done}/${total} tracks` : `${pct}%`;
const eta = etaLabel(j.downloadEtaSeconds);
bar = <ProgressBar kind="working" percent={pct} caption={eta ? `${base} · ${eta}` : base} />;
}
} else if (j.state === "tagging") {
bar = <ProgressBar kind="verify" indeterminate caption="finishing" />;
} else {
@@ -200,19 +303,93 @@ export function Queue() {
album={r.album}
kind={d.kind}
label={d.label}
marker={r.id === nextUpId ? "Next up" : undefined}
meta={meta}
note={note}
bar={bar}
/>
);
});
})()}
</section>
}
return (
<div>
<StatTiles
items={[
{ k: "On the press", v: active.length },
{ k: "Pressed", v: pressedAll.length, accent: true },
{ k: "Wanted", v: wanted },
{ k: "Watching", v: watching, unit: "artists" },
]}
/>
<div className="dept has-actions">
<h2>On the Press</h2>
<span className="fill" />
{paused ? <span className="count">paused</span> : null}
<KebabMenu label="Press actions">
{paused ? (
<button className="menu-item" onClick={() => floorAction("resume", "Press resumed")}>
Resume the press
</button>
) : (
<button className="menu-item" onClick={() => floorAction("pause", "Press paused")}>
Pause the press
</button>
)}
<button className="menu-item" onClick={clearPress} disabled={queuedCount === 0}>
Clear queue
</button>
{downloading ? (
<button className="menu-item danger" onClick={stopPress}>
Stop now
</button>
) : null}
</KebabMenu>
</div>
<form className="request-form" onSubmit={submit}>
<label className="field">
<span>Artist</span>
<input aria-label="artist" placeholder="Artist" value={artist} onChange={(e) => setArtist(e.target.value)} />
</label>
<label className="field">
<span>Album</span>
<input aria-label="album" placeholder="Album" value={album} onChange={(e) => setAlbum(e.target.value)} />
</label>
<button type="submit" className="btn accent">
Request
</button>
</form>
{active.length === 0 ? (
<p className="empty">The press is quiet. Queue a release above, or send one over from Wanted or Discover.</p>
) : (
<>
<div className="tabs" role="tablist">
{(["active", "queue", "errors"] as Tab[]).map((t) => (
<button
key={t}
role="tab"
aria-selected={activeTab === t}
className={`tab${activeTab === t ? " active" : ""}${t === "errors" && buckets.errors.length ? " has-errors" : ""}`}
onClick={() => setTab(t)}
>
{TAB_LABEL[t]}
<span className="n">{buckets[t].length}</span>
</button>
))}
</div>
{shown.length === 0 ? (
<p className="empty">{TAB_EMPTY[activeTab]}</p>
) : (
<section className="floor">{shown.map(renderRow)}</section>
)}
</>
)}
{pressed.length > 0 ? (
<>
<SectionHeader title="Recently Pressed" note={`${pressed.length} in library`} />
<SectionHeader title="Recently Pressed" note={`last ${pressed.length} · ${pressedAll.length} in library`} />
<PressedList
items={pressed.map((r) => ({ id: r.id, artist: r.artist, album: r.album, matrix: pressedMark(r), rgMbid: r.rgMbid ?? null }))}
/>
web/src/app/settings/settings-form.tsx
+93
View File
@@ -31,6 +31,7 @@ const MONITOR_FIELDS: [string, string][] = [
["monitor.retryIntervalHours", "Retry interval (hours)"],
["monitor.qualityCutoff", "Quality cutoff"],
["monitor.upgradeWindowDays", "Upgrade window (days)"],
["floor.jobDelaySeconds", "Delay between downloads (seconds)"],
];
export function SettingsForm() {
@@ -53,6 +54,8 @@ export function SettingsForm() {
const [discoverSaved, setDiscoverSaved] = useState(false);
const [monitorCfg, setMonitorCfg] = useState<Record<string, string>>({});
const [monitorSaved, setMonitorSaved] = useState(false);
const [pacingCfg, setPacingCfg] = useState<Record<string, string>>({});
const [pacingSaved, setPacingSaved] = useState(false);
useEffect(() => {
fetch("/api/scan")
@@ -105,6 +108,12 @@ export function SettingsForm() {
.then((c: Record<string, string>) => setMonitorCfg(c));
}, []);
useEffect(() => {
fetch("/api/qobuz/pacing")
.then((r) => r.json())
.then((c: Record<string, string>) => setPacingCfg(c));
}, []);
async function saveCredentials(e: React.FormEvent) {
e.preventDefault();
await fetch("/api/config", {
@@ -159,6 +168,21 @@ export function SettingsForm() {
setTimeout(() => setMonitorSaved(false), 1500);
}
function setPacing(key: string, value: string) {
setPacingCfg((c) => ({ ...c, [key]: value }));
}
async function savePacing(e: React.FormEvent) {
e.preventDefault();
await fetch("/api/qobuz/pacing", {
method: "PATCH",
headers: { "content-type": "application/json" },
body: JSON.stringify(pacingCfg),
});
setPacingSaved(true);
setTimeout(() => setPacingSaved(false), 1500);
}
const secretHint = (set: boolean) => (set ? <span className="set"> · set</span> : null);
async function clearField(field: "qobuzToken" | "slskdApiKey" | "lastfmApiKey") {
@@ -192,6 +216,7 @@ export function SettingsForm() {
</div>
{tab === "Qobuz" ? (
<>
<form className="settings-section" onSubmit={saveCredentials}>
<div className="subhead">
Auth token
@@ -234,6 +259,55 @@ export function SettingsForm() {
{saved ? <span className="saved-note">Saved</span> : null}
</div>
</form>
<form className="settings-section" onSubmit={savePacing}>
<div className="subhead">
Pacing
<HelpButton title="Qobuz pacing">
Throttles Qobuz to a human-looking volume and cadence so the account isn&apos;t flagged
for bulk downloading. Within active hours it downloads up to the daily cap, spaced out
with randomized gaps; when throttled, albums fall through to Soulseek/YouTube and
upgrade to Qobuz hi-res later. The daily cap ramps up over the first two weeks from the
new-account start date.
</HelpButton>
</div>
<label className="field-inline">
<input type="checkbox" aria-label="pace qobuz downloads"
checked={pacingCfg["qobuz.pacing.enabled"] !== "false"}
onChange={(e) => setPacing("qobuz.pacing.enabled", e.target.checked ? "true" : "false")} />
<span>Pace Qobuz downloads</span>
</label>
<div className="field-grid">
<label className="field">
<span>Active hours — start (0–24)</span>
<input type="number" min={0} max={24} aria-label="qobuz active start hour"
value={pacingCfg["qobuz.pacing.activeStartHour"] ?? ""}
onChange={(e) => setPacing("qobuz.pacing.activeStartHour", e.target.value)} />
</label>
<label className="field">
<span>Active hours — end (0–24)</span>
<input type="number" min={0} max={24} aria-label="qobuz active end hour"
value={pacingCfg["qobuz.pacing.activeEndHour"] ?? ""}
onChange={(e) => setPacing("qobuz.pacing.activeEndHour", e.target.value)} />
</label>
<label className="field">
<span>Daily cap (steady)</span>
<input type="number" min={0} aria-label="qobuz daily cap"
value={pacingCfg["qobuz.pacing.dailyCap"] ?? ""}
onChange={(e) => setPacing("qobuz.pacing.dailyCap", e.target.value)} />
</label>
<label className="field">
<span>New-account start (warm-up)</span>
<input type="date" aria-label="qobuz warmup start date"
value={pacingCfg["qobuz.pacing.warmupStartDate"] ?? ""}
onChange={(e) => setPacing("qobuz.pacing.warmupStartDate", e.target.value)} />
</label>
</div>
<div className="save-row">
<button type="submit" className="btn accent">Save</button>
{pacingSaved ? <span className="saved-note">Saved</span> : null}
</div>
</form>
</>
) : null}
{tab === "Soulseek" ? (
@@ -394,6 +468,9 @@ export function SettingsForm() {
artists are processed per worker cycle (keeps job-claiming responsive on big rosters).
<b> Similar per seed</b> — candidates pulled per artist. <b>Albums per artist</b> —
releases surfaced per suggested artist. <b>Min score</b> — drops weak suggestions.{" "}
<b>Albums only</b> — suggest full-length albums only, hiding singles and EPs.{" "}
<b>Seed from library</b> — also find artists similar to ones you own albums by, not
just the artists you follow.{" "}
<b>ListenBrainz base URL</b> — leave blank for the default endpoint.
</HelpButton>
</div>
@@ -405,6 +482,22 @@ export function SettingsForm() {
</label>
))}
</div>
<label className="toggle" style={{ margin: "10px 0 0" }}>
<input
type="checkbox"
checked={discoverCfg["discover.albumsOnly"] !== "false"}
onChange={(e) => setCfg("discover.albumsOnly", e.target.checked ? "true" : "false")}
/>
Albums only (hide singles &amp; EPs)
</label>
<label className="toggle" style={{ margin: "6px 0 0" }}>
<input
type="checkbox"
checked={discoverCfg["discover.seedFromLibrary"] !== "false"}
onChange={(e) => setCfg("discover.seedFromLibrary", e.target.checked ? "true" : "false")}
/>
Seed from library (similar to albums you own)
</label>
<label className="field">
<span>ListenBrainz base URL (blank = default)</span>
<input value={discoverCfg["discover.listenBrainzUrl"] ?? ""}
web/src/app/wanted/wanted-client.tsx
+17
View File
@@ -65,6 +65,20 @@ export function WantedClient() {
refresh();
}
async function ignore(w: Wanted) {
const res = await fetch(`/api/releases/${w.id}`, {
method: "PATCH",
headers: { "content-type": "application/json" },
body: JSON.stringify({ ignored: true }),
}).catch(() => null);
if (res?.ok) {
setRows((list) => list.filter((x) => x.id !== w.id));
toast(`Ignoring ${w.album} — un-ignore from the artist’s discography`);
} else {
toast(`Couldn't ignore ${w.album}`);
}
}
return (
<div>
<PageHead title="Wanted" eyebrow="Cutting list · acquire & upgrade" />
@@ -109,6 +123,9 @@ export function WantedClient() {
<button className="btn sm ghost" onClick={() => searchNow(w)}>
Search now
</button>
<button className="btn sm ghost" onClick={() => ignore(w)}>
Ignore
</button>
</div>
</li>
))}
web/src/lib/artist-resolve.test.ts
+46
View File
@@ -0,0 +1,46 @@
import { describe, it, expect } from "vitest";
import { resolveArtistChoice, type ArtistHit } from "./artist-resolve";
const US: ArtistHit = { mbid: "us", name: "Looking Glass", disambiguation: "70's US pop group" };
const AU: ArtistHit = { mbid: "au", name: "Looking Glass", disambiguation: "Australian stoner rock" };
describe("resolveArtistChoice", () => {
it("is ambiguous when 2+ candidates share the exact name", () => {
expect(resolveArtistChoice("Looking Glass", [US, AU], "au")).toEqual({
kind: "ambiguous",
candidates: [US, AU],
});
});
it("uses the single exact-name match even over a disagreeing fallback mbid", () => {
const hit: ArtistHit = { mbid: "r", name: "Radiohead", disambiguation: "" };
expect(resolveArtistChoice("Radiohead", [hit], "wrong-lastfm-mbid")).toEqual({
kind: "single",
mbid: "r",
});
});
it("matches exact name case- and whitespace-insensitively", () => {
expect(resolveArtistChoice(" looking GLASS ", [US, AU])).toEqual({
kind: "ambiguous",
candidates: [US, AU],
});
});
it("falls back to the provided mbid when no candidate name matches exactly", () => {
const fuzzy: ArtistHit = { mbid: "x", name: "The Beatles", disambiguation: "" };
expect(resolveArtistChoice("Beatles", [fuzzy], "lastfm-mbid")).toEqual({
kind: "single",
mbid: "lastfm-mbid",
});
});
it("falls back to MB's top hit when there is no exact match and no fallback", () => {
const fuzzy: ArtistHit = { mbid: "top", name: "The Beatles", disambiguation: "" };
expect(resolveArtistChoice("Beatles", [fuzzy])).toEqual({ kind: "single", mbid: "top" });
});
it("is none when there are no candidates and no fallback", () => {
expect(resolveArtistChoice("Nobody Here", [])).toEqual({ kind: "none" });
});
});
web/src/lib/artist-resolve.ts
+34
View File
@@ -0,0 +1,34 @@
export type ArtistHit = { mbid: string; name: string; disambiguation: string };
export type Resolution =
| { kind: "single"; mbid: string }
| { kind: "ambiguous"; candidates: ArtistHit[] }
| { kind: "none" };
const norm = (s: string) => s.trim().toLowerCase();
/**
* Decide which MusicBrainz artist a Last.fm row refers to, given MB's search candidates.
*
* The bug this guards against: Last.fm's own MBID for an ambiguous name can point at the
* wrong same-named band (it mapped "Looking Glass" to an Australian stoner band, not the US
* pop group). So we prefer MB's own resolution:
* - 2+ candidates whose name EXACTLY matches (normalized) → ambiguous; let the user pick.
* - exactly 1 exact-name match → use it, even if it disagrees with `fallbackMbid`.
* - no exact-name match → keep prior behavior: the Last.fm `fallbackMbid`, else MB's top hit.
*
* `candidates` is assumed to arrive in MB relevance order (best first), so `candidates[0]`
* is the suggested default and exact-name matches keep that ordering.
*/
export function resolveArtistChoice(
name: string,
candidates: ArtistHit[],
fallbackMbid?: string | null,
): Resolution {
const exact = candidates.filter((a) => norm(a.name) === norm(name));
if (exact.length >= 2) return { kind: "ambiguous", candidates: exact };
if (exact.length === 1) return { kind: "single", mbid: exact[0].mbid };
if (fallbackMbid) return { kind: "single", mbid: fallbackMbid };
if (candidates.length > 0) return { kind: "single", mbid: candidates[0].mbid };
return { kind: "none" };
}
web/src/lib/auth.ts
+10
View File
@@ -37,6 +37,16 @@ export async function expectedToken(): Promise<string> {
return token;
}
/** Authenticate a request directly (for routes excluded from the auth middleware, e.g. large
* uploads). Returns true when auth is disabled or the request carries a valid session cookie. */
export async function isAuthed(request: Request): Promise<boolean> {
const token = await expectedToken();
if (!token) return true; // auth disabled
const cookie = request.headers.get("cookie") ?? "";
const m = cookie.match(new RegExp(`(?:^|;\\s*)${AUTH_COOKIE}=([^;]+)`));
return safeEqual(m ? decodeURIComponent(m[1]) : "", token);
}
/** Constant-time string compare (both operands are our own derived tokens). */
export function safeEqual(a: string, b: string): boolean {
if (a.length !== b.length) return false;
web/src/lib/library-ops.ts
+50
View File
@@ -0,0 +1,50 @@
import { rm, rmdir } from "node:fs/promises";
import { dirname, resolve, sep } from "node:path";
import { prisma } from "@/lib/db";
export type OpResult = { ok: boolean; status: number; error?: string };
/** Delete an owned album: remove its folder on disk (guarded to inside the library root),
* drop the LibraryItem, and stop monitoring the matching release so auto-monitor won't
* immediately re-grab it. Shared by the single DELETE route and bulk operations. */
export async function removeLibraryItem(id: string): Promise<OpResult> {
const item = await prisma.libraryItem.findUnique({ where: { id } });
if (!item) return { ok: false, status: 404, error: "not found" };
const root = resolve(process.env.LYRA_LIBRARY_ROOT ?? "/music");
const target = resolve(item.path);
if (target === root || !target.startsWith(root + sep)) {
return { ok: false, status: 400, error: "refusing to delete a path outside the library" };
}
try {
await rm(target, { recursive: true, force: true });
const parent = dirname(target);
if (parent !== root) await rmdir(parent).catch(() => {}); // prune emptied artist folder
} catch {
return { ok: false, status: 500, error: "failed to delete files on disk" };
}
await prisma.$transaction([
prisma.libraryItem.delete({ where: { id } }),
prisma.monitoredRelease.updateMany({
where: { artistName: item.artist, album: item.album },
data: { monitored: false, currentQualityClass: null },
}),
]);
return { ok: true, status: 200 };
}
/** Mark the release(s) matching an owned album do-not-monitor, so it drops off Wanted and is
* never enqueued. No-op (still ok) if the album has no matching release. */
export async function ignoreLibraryItem(id: string): Promise<OpResult> {
const item = await prisma.libraryItem.findUnique({ where: { id } });
if (!item) return { ok: false, status: 404, error: "not found" };
await prisma.monitoredRelease.updateMany({
where: item.rgMbid
? { rgMbid: item.rgMbid }
: { artistName: item.artist, album: item.album },
data: { ignored: true },
});
return { ok: true, status: 200 };
}
web/src/lib/library-path.ts
+23
View File
@@ -0,0 +1,23 @@
import { join, resolve } from "node:path";
const ILLEGAL = /[<>:"/\\|?*]/g;
const YEAR_DIR = /\((\d{4})\)\s*$/;
/** Filesystem-safe path segment — mirrors the worker's import naming so manual imports and
* edits produce the same on-disk layout the scanner expects (Artist/Album (Year)/…). */
export function safeName(s: string): string {
const cleaned = s.replace(ILLEGAL, "_").trim().replace(/^\.+|\.+$/g, "").trim();
return cleaned || "Unknown";
}
/** Absolute album folder for `{root}/{Artist}/{Album (Year)}` (year optional). */
export function albumDir(root: string, artist: string, album: string, year?: string): string {
const folder = year ? `${safeName(album)} (${year})` : safeName(album);
return join(resolve(root), safeName(artist), folder);
}
/** Pull a trailing "(YYYY)" out of an album folder path, if present. */
export function yearFromPath(path: string): string | null {
const m = YEAR_DIR.exec(path);
return m ? m[1] : null;
}
web/src/middleware.ts
+4 -2
View File
@@ -26,6 +26,8 @@ export async function middleware(req: NextRequest) {
}
export const config = {
// Run on everything except Next internals and static assets (which carry no secrets).
matcher: ["/((?!_next/static|_next/image|favicon.ico|icon.svg|fonts/).*)"],
// Run on everything except Next internals and static assets (which carry no secrets), plus
// /api/library/import — routing large album uploads through middleware caps the body at
// 10MB, so it's excluded here and authenticates itself inline instead.
matcher: ["/((?!_next/static|_next/image|favicon.ico|icon.svg|fonts/|api/library/import).*)"],
};
web/src/test/setup.ts
+3
View File
@@ -16,6 +16,9 @@ beforeEach(async () => {
await prisma.monitoredRelease.deleteMany();
await prisma.watchedArtist.deleteMany();
await prisma.discoverySuggestion.deleteMany();
await prisma.discoverySeedContribution.deleteMany();
await prisma.discoverySeed.deleteMany();
await prisma.config.deleteMany();
await prisma.apiCache.deleteMany(); // read-through cache must not leak across tests
await prisma.unmatchedAlbum.deleteMany();
});
worker/lyra_worker/_musicbrainz.py
+103 -9
View File
@@ -1,3 +1,4 @@
import time
from difflib import SequenceMatcher
from lyra_worker.types import MBTarget
@@ -5,10 +6,104 @@ from lyra_worker.types import MBTarget
_MIN_TITLE_RATIO = 0.6
def _with_retry(fn, *, attempts: int = 4, base_delay: float = 1.0, sleep=time.sleep):
"""Call `fn`, retrying transient MusicBrainz failures with exponential backoff.
A dropped TLS connection to the MusicBrainz webservice surfaces as
musicbrainzngs.NetworkError (`<urlopen error [SSL: UNEXPECTED_EOF_WHILE_READING]>`),
and MusicBrainz's own 503 rate-limit surfaces as ResponseError; both are
transient, so we retry rather than failing the whole pipeline for the job.
Any other exception (bug, non-transient) propagates immediately.
musicbrainzngs is imported lazily so importing this module stays offline.
"""
import musicbrainzngs
transient = (musicbrainzngs.NetworkError, musicbrainzngs.ResponseError)
for i in range(attempts):
try:
return fn()
except transient:
if i == attempts - 1:
raise
sleep(base_delay * (2**i))
def _ratio(a: str, b: str) -> float:
return SequenceMatcher(None, a.strip().casefold(), b.strip().casefold()).ratio()
def _is_album(g: dict) -> bool:
return (g.get("primary-type") or "").casefold() == "album"
def _rg_year(g: dict) -> int:
"""First-release year of a release-group, or 9999 when unknown (so undated groups lose ties)."""
frd = (g.get("first-release-date") or "")[:4]
return int(frd) if frd.isdigit() else 9999
def _credited_artist(g: dict) -> str:
"""Primary credited artist name of a release-group search result, or '' if absent."""
credit = g.get("artist-credit")
if isinstance(credit, list) and credit and isinstance(credit[0], dict):
return (credit[0].get("artist") or {}).get("name", "") or ""
return ""
def _best_release_group(artist: str, album: str, groups: list) -> dict | None:
"""Choose the best release-group match for `artist`/`album`.
Ranked, highest first, on `(artist_ratio, title_ratio, is_album)`:
* artist match dominates — MusicBrainz relevance ties same-titled release groups by
*different* artists (e.g. "Fun Machine" exists as a Lake Street Dive EP and an
unrelated band's Album), and the artist we followed is trustworthy, so a credited
artist that matches the request outranks everything else;
* title similarity is next;
* an Album primary-type only breaks ties *within* the same artist+title, so a famous
album (Michael Jackson's "Off the Wall") still isn't resolved to its same-named
single whose short tracklist would map positionally onto the album's files;
* finally, the EARLIEST release-group wins — MusicBrainz lists the original album and its
later reissues/anniversary editions as separate same-titled groups (Linkin Park "Hybrid
Theory": a 2000 original and a 2023 13-track reissue), which tie on everything above, so
MB's arbitrary relevance order used to pick a reissue whose longer tracklist/runtime then
failed completeness checks against the standard album a source delivers. Prefer the original.
Artist is a ranking signal, never a hard filter — a low match sinks a candidate but
never drops the release, so credited-name variations (feat., punctuation) still resolve.
Returns None below the title threshold. Pure — no network I/O."""
if not groups:
return None
best = max(
groups,
key=lambda g: (
_ratio(artist, _credited_artist(g)),
_ratio(album, g.get("title", "")),
_is_album(g),
-_rg_year(g), # tie-break: the original (earliest) group over later reissues
),
)
if _ratio(album, best.get("title", "")) < _MIN_TITLE_RATIO:
return None
return best
def _pick_release(releases: list) -> dict | None:
"""Choose which release of a release-group to read the tracklist from. MB returns them in an
arbitrary order and ``releases[0]`` is often a later reissue/deluxe/anniversary edition with
more tracks and a longer runtime than the standard album a source actually delivers — which
then makes a *complete* download fail the pipeline's track-count/duration completeness checks
and re-download forever (Linkin Park "Hybrid Theory": MB's arbitrary pick was a 2023 13-track
reissue at ~52 min vs the 2000 original's 12 tracks / ~38 min). Prefer the ORIGINAL standard
edition: an Official release, then the earliest date, then a dated one. Pure — unit-tested."""
if not releases:
return None
def _key(r: dict) -> tuple:
official = 0 if (r.get("status") == "Official") else 1 # official editions first
date = r.get("date") or ""
return (official, 0 if date else 1, date) # then dated, then earliest date
return min(releases, key=_key)
class MusicBrainzResolver:
"""Real MbResolver using the MusicBrainz webservice via musicbrainzngs.
@@ -26,12 +121,10 @@ class MusicBrainzResolver:
musicbrainzngs.set_useragent(self._app, self._version, self._contact)
res = musicbrainzngs.search_release_groups(query=album, artist=artist, limit=5)
res = _with_retry(lambda: musicbrainzngs.search_release_groups(query=album, artist=artist, limit=5))
groups = res.get("release-group-list", [])
if not groups:
return None
rg = max(groups, key=lambda g: _ratio(album, g.get("title", "")))
if _ratio(album, rg.get("title", "")) < _MIN_TITLE_RATIO:
rg = _best_release_group(artist, album, groups)
if rg is None:
return None
canonical_album = rg.get("title", album)
@@ -53,11 +146,12 @@ class MusicBrainzResolver:
titles: tuple[str, ...] = ()
rgid = rg.get("id")
if rgid:
rgfull = musicbrainzngs.get_release_group_by_id(rgid, includes=["releases"])
rgfull = _with_retry(lambda: musicbrainzngs.get_release_group_by_id(rgid, includes=["releases"]))
releases = rgfull.get("release-group", {}).get("release-list", [])
if releases:
rel = musicbrainzngs.get_release_by_id(
releases[0]["id"], includes=["recordings"]
chosen = _pick_release(releases)
if chosen:
rel = _with_retry(
lambda: musicbrainzngs.get_release_by_id(chosen["id"], includes=["recordings"])
).get("release", {})
tracks = []
for medium in rel.get("medium-list", []):
worker/lyra_worker/_mutagen.py
+32 -5
View File
@@ -1,21 +1,48 @@
import os
import re
from lyra_worker.library import safe_name
_AUDIO_EXT = {".flac", ".mp3", ".m4a", ".opus", ".ogg", ".aac", ".wav"}
# Leading "13.", "13 -", "13)" style track-number prefix streamrip bakes into a bonus-track
# filename. Requires an explicit delimiter so a real title that merely starts with a number
# (e.g. "99 Luftballons") is left alone.
_NUM_PREFIX = re.compile(r"^\s*\d+\s*[.):\-]\s*")
def plan_track_names(names: list[str], tracklist: tuple[str, ...]) -> list[tuple[str, str, str, int]]:
def _clean_extra_title(stem: str, artist: str = "") -> str:
"""Derive a display title for a track BEYOND the MusicBrainz tracklist (a bonus track that
streamrip named itself, e.g. `13. John Mayer - St. Patrick's Day (Album Version)`). Strip a
leading track-number prefix, then a leading `<artist> - ` if present, so the example yields
`St. Patrick's Day (Album Version)`. Falls back to the stem if nothing strips."""
s = _NUM_PREFIX.sub("", stem).strip()
if artist:
s = re.sub(r"^" + re.escape(artist) + r"\s*-\s*", "", s, flags=re.IGNORECASE).strip()
return s or stem
def plan_track_names(
names: list[str], tracklist: tuple[str, ...], artist: str = ""
) -> list[tuple[str, str, str, int]]:
"""Map audio filenames -> (old_name, new_name, title, track_number).
Files are sorted; titles come from `tracklist` by position when available,
else the filename stem. Pure — no filesystem or tag I/O.
Files are sorted; titles come from `tracklist` by position when available. A file BEYOND
the tracklist (a bonus track streamrip named itself) is canonicalized via `_clean_extra_title`
so it gets a clean `## Title.ext` name too; with no tracklist at all the on-disk stem is kept.
Pure — no filesystem or tag I/O.
"""
plan: list[tuple[str, str, str, int]] = []
audio = sorted(n for n in names if os.path.splitext(n)[1].lower() in _AUDIO_EXT)
for i, name in enumerate(audio):
ext = os.path.splitext(name)[1].lower()
title = tracklist[i] if i < len(tracklist) else os.path.splitext(name)[0]
stem = os.path.splitext(name)[0]
if i < len(tracklist):
title = tracklist[i]
elif tracklist: # a genuine bonus track past a known tracklist -> canonicalize its name
title = _clean_extra_title(stem, artist)
else: # no tracklist resolved at all -> keep the on-disk stem (renumbered)
title = stem
new_name = f"{i + 1:02d} {safe_name(title)}{ext}"
plan.append((name, new_name, title, i + 1))
return plan
@@ -29,7 +56,7 @@ class MutagenTagger:
import mutagen
names = os.listdir(album_dir)
for old_name, new_name, title, number in plan_track_names(names, target.tracklist):
for old_name, new_name, title, number in plan_track_names(names, target.tracklist, target.artist):
try:
path = os.path.join(album_dir, old_name)
audio = mutagen.File(path, easy=True)
worker/lyra_worker/adapters/_slskd.py
+201 -41
View File
@@ -2,6 +2,7 @@ import json
import os
import shutil
import time
import unicodedata
from typing import Callable
import requests
@@ -11,8 +12,25 @@ _DEFAULT_DOWNLOADS_ROOT = "/slskd-downloads"
_AUDIO_EXT = {"flac", "mp3", "m4a", "ogg", "opus", "wav", "aac"}
_LOSSLESS_EXT = {"flac", "wav"}
_SEARCH_POLLS = 40 # * 3s ≈ 2 min
_XFER_POLLS = 200 # * 3s ≈ 10 min
_STALL_POLLS = 30 # * 3s ≈ 90s of no byte progress → abandon a queued/dead peer and fall through
_POLL_SECONDS = 3
# Absolute backstop for a slow-but-progressing peer. Scaled to the album's total bytes (a flat
# cap starved big albums: a 500MB set from a ~400KB/s serial peer needs ~60-70min and was cut at
# a flat 60min, then re-downloaded from scratch — never finishing). Budget at a conservative floor
# throughput, clamped. The 90s stall timeout still abandons a truly dead/queued peer quickly.
_MIN_XFER_POLLS = 400 # * 3s ≈ 20 min floor (small albums)
_MAX_XFER_POLLS = 2400 # * 3s ≈ 2 h ceiling (bounds worst-case slot hold)
_BACKSTOP_FLOOR_BPS = 64_000 # budget the backstop assuming >= ~64 KB/s sustained
def _max_polls(total_bytes: int) -> int:
"""Poll budget (each _POLL_SECONDS) for the absolute download backstop, scaled to album size at
a conservative floor throughput so a large slow-but-healthy transfer isn't abandoned mid-flight.
Clamped to [_MIN_XFER_POLLS, _MAX_XFER_POLLS]. Pure — unit-tested."""
if total_bytes <= 0:
return _MIN_XFER_POLLS
budget = int(total_bytes / (_BACKSTOP_FLOOR_BPS * _POLL_SECONDS))
return max(_MIN_XFER_POLLS, min(budget, _MAX_XFER_POLLS))
def _basename(path: str) -> str:
@@ -27,6 +45,114 @@ def _dirname(path: str) -> str:
return ""
_ETA_CAP_SECONDS = 359_999 # ~100h; keeps the value well inside Job.downloadEtaSeconds (int4)
def _eta_seconds(total_bytes: int, bytes_now: int, speed_bps: float) -> int | None:
"""Seconds until the transfer finishes at the current measured rate, or None when it can't be
estimated (no throughput sample yet, unknown total, or already complete). Pure — unit-tested.
A near-stalled peer drives speed_bps toward zero, so the estimate is capped: an uncapped value
can exceed int4 and overflow the downloadEtaSeconds column write (poisoning the pipeline txn)."""
if speed_bps <= 0 or total_bytes <= 0 or bytes_now >= total_bytes:
return None
return min(int((total_bytes - bytes_now) / speed_bps), _ETA_CAP_SECONDS)
def _norm(s: str) -> str:
"""Lowercase, strip diacritics, and reduce to alphanumeric tokens, so 'Beyoncé' matches a
peer's 'Beyonce' folder and punctuation ('I Am… Sasha Fierce') doesn't defeat the match."""
s = unicodedata.normalize("NFKD", s or "")
s = "".join(ch for ch in s if not unicodedata.combining(ch))
s = "".join(ch if ch.isalnum() else " " for ch in s.lower())
return " ".join(s.split())
def _path_matches(needle: str, path: str) -> bool:
"""True if `needle` (an artist or album name) appears in a file path, tolerant of accents and
punctuation. Every significant token (≥3 chars) must be present; for a very short needle
(e.g. 'U2') the whole normalized string must appear."""
npath = _norm(path)
tokens = [t for t in _norm(needle).split() if len(t) >= 3]
if not tokens:
whole = _norm(needle)
return bool(whole) and whole in npath
return all(t in npath for t in tokens)
def _keep_matching(responses: list, needle: str) -> list:
"""Drop files — then now-empty responses — whose path doesn't match `needle`. Makes an album-
or artist-only fallback search precise so we never grab a same-titled release by a different
artist. Pure — unit-tested offline."""
out = []
for resp in responses:
files = [f for f in (resp.get("files") or []) if _path_matches(needle, f.get("filename", ""))]
if files:
out.append({**resp, "files": files})
return out
def _parse_search_responses(responses: list, artist: str = "") -> list[dict]:
"""Turn slskd search responses into album candidates (one per peer+directory), ordered so the
peers that will FINISH first come first: a free upload slot, then the shortest estimated
transfer time (album bytes ÷ the peer's advertised speed), then a shorter queue. Ranking on
estimated time — not raw speed — matters because Soulseek FLAC rips of the same album vary
~2x in size (a 24-bit/bloated rip vs a standard CD rip), and Lyra scores them the SAME quality
class (the search exposes no bit-depth), so a peer advertising high speed but serving huge
files can lose a race — and blow past the download backstop — against a peer with a smaller
standard rip. The pipeline tries candidates in this order, so this is what makes the Soulseek
fall-through converge on a copy that actually completes. Pure — no I/O — unit-tested offline."""
scored: list[tuple] = []
for resp in responses:
username = resp.get("username", "")
has_slot = 0 if resp.get("hasFreeUploadSlot") else 1 # 0 sorts first (free slot = starts now)
speed = int(resp.get("uploadSpeed") or 0)
queue = int(resp.get("queueLength") or 0)
by_dir: dict[str, list] = {}
for f in resp.get("files", []) or []:
name = f.get("filename", "")
ext = name.rsplit(".", 1)[-1].lower() if "." in name else ""
if ext not in _AUDIO_EXT:
continue
by_dir.setdefault(_dirname(name), []).append(
{"filename": name, "size": f.get("size", 0), "ext": ext}
)
for directory, dfiles in by_dir.items():
if not dfiles:
continue
# Estimated seconds to pull this album from this peer at its advertised rate. Unknown/
# zero speed → a low nominal (1 B/s) so total size still orders those peers last.
total_bytes = sum(int(f["size"] or 0) for f in dfiles)
est_seconds = total_bytes / (speed if speed > 0 else 1)
lossless = all(f["ext"] in _LOSSLESS_EXT for f in dfiles)
dirname = _basename(directory)
# Trust the REQUESTED artist when it appears anywhere in the path — many rips put the
# artist in a parent folder ("Rihanna\(2009) Rated R\…") or bury it ("2009 - Rihanna -
# Rated R"), which the "Artist - Album" folder parse gets wrong; that tanked the
# confidence score and lost real FLACs to clean-named MP3s. album stays the folder name
# (fuzzy-matched downstream); artist is what the parse gets wrong.
if artist and _path_matches(artist, dfiles[0]["filename"]):
guess_artist, guess_album = artist, dirname
elif " - " in dirname:
guess_artist, guess_album = (p.strip() for p in dirname.split(" - ", 1))
else:
guess_artist, guess_album = "", dirname
candidate = {
"source_ref": json.dumps({
"username": username,
"files": [{"filename": f["filename"], "size": f["size"]} for f in dfiles],
}),
"title": guess_album,
"artist": guess_artist,
"track_count": len(dfiles),
"format": "FLAC" if lossless else "MP3",
"bitrate": None,
}
scored.append((has_slot, est_seconds, queue, candidate))
# ascending: free slot first, then shortest estimated transfer, then shortest queue
scored.sort(key=lambda t: (t[0], t[1], t[2]))
return [c for *_rest, c in scored]
class SlskdClient:
"""Real Soulseek client talking to an slskd daemon's REST API.
@@ -58,11 +184,12 @@ class SlskdClient:
r.raise_for_status()
return r.json()
def search_album(self, artist: str, album: str) -> list[dict]:
def _run_search(self, text: str) -> list:
"""POST a search, wait for it to complete, and return the raw responses (a list)."""
r = requests.post(
f"{self._url}/api/v0/searches",
headers=self._headers(),
json={"searchText": f"{artist} {album}"},
json={"searchText": text},
timeout=30,
)
r.raise_for_status()
@@ -75,43 +202,27 @@ class SlskdClient:
break
responses = self._get(f"/api/v0/searches/{search_id}/responses")
candidates: list[dict] = []
for resp in responses:
username = resp.get("username", "")
by_dir: dict[str, list] = {}
for f in resp.get("files", []) or []:
name = f.get("filename", "")
ext = name.rsplit(".", 1)[-1].lower() if "." in name else ""
if ext not in _AUDIO_EXT:
return responses if isinstance(responses, list) else []
def search_album(self, artist: str, album: str) -> list[dict]:
responses = self._run_search(f"{artist} {album}")
if not responses:
# The Soulseek server silently drops searches containing certain blocklisted terms
# (major-label artists / album titles like Adele, Rihanna, Beyoncé, "Lemonade") →
# 0 responses. The blocked term poisons the whole query, but the OTHER field usually
# isn't blocked, so retry with each field alone (normalized) and keep only results
# whose path still matches the field we dropped. That recovers e.g. Beyoncé's
# "I Am… Sasha Fierce" via the album title, without ever grabbing a same-titled album
# by someone else.
for query, needle in ((album, artist), (artist, album)):
q = _norm(query)
if not q:
continue
by_dir.setdefault(_dirname(name), []).append(
{"filename": name, "size": f.get("size", 0), "ext": ext}
)
for directory, dfiles in by_dir.items():
if not dfiles:
continue
lossless = all(f["ext"] in _LOSSLESS_EXT for f in dfiles)
dirname = _basename(directory)
if " - " in dirname:
guess_artist, guess_album = (p.strip() for p in dirname.split(" - ", 1))
else:
guess_artist, guess_album = "", dirname
candidates.append(
{
"source_ref": json.dumps(
{
"username": username,
"files": [{"filename": f["filename"], "size": f["size"]} for f in dfiles],
}
),
"title": guess_album,
"artist": guess_artist,
"track_count": len(dfiles),
"format": "FLAC" if lossless else "MP3",
"bitrate": None,
}
)
return candidates
alt = _keep_matching(self._run_search(q), needle)
if alt:
responses = alt
break
return _parse_search_responses(responses, artist)
def download(self, source_ref: str, dest: str, on_progress: Callable[[float], None]) -> dict:
ref = json.loads(source_ref)
@@ -119,7 +230,17 @@ class SlskdClient:
files = ref["files"]
wanted = {f["filename"] for f in files}
os.makedirs(dest, exist_ok=True)
xfer_ids: set[str] = set() # slskd transfer ids we started, for cancel-on-abandon
try:
return self._download(username, files, wanted, dest, on_progress, xfer_ids)
except Exception:
# Abandoning this peer (timeout / error / the pipeline moving on): cancel the transfers
# we enqueued so they don't keep running and pile up in slskd as the job falls through
# to other peers/sources for the same album.
self._cancel_transfers(username, xfer_ids)
raise
def _download(self, username, files, wanted, dest, on_progress, xfer_ids: set) -> dict:
r = requests.post(
f"{self._url}/api/v0/transfers/downloads/{username}",
headers=self._headers(),
@@ -130,23 +251,49 @@ class SlskdClient:
on_progress(0.0)
total = len(files)
total_bytes = sum(int(f.get("size") or 0) for f in files)
completed = False
for _ in range(_XFER_POLLS):
last_bytes = -1
stall = 0
speed = 0.0 # bytes/sec, EWMA-smoothed so the ETA doesn't jitter poll to poll
prev_bytes = 0
prev_t = time.monotonic()
for _ in range(_max_polls(total_bytes)):
time.sleep(_POLL_SECONDS)
data = self._get(f"/api/v0/transfers/downloads/{username}")
states: list[str] = []
bytes_now = 0
for directory in data.get("directories", []) if isinstance(data, dict) else []:
for f in directory.get("files", []):
if f.get("filename") in wanted: # only the files we enqueued
states.append(str(f.get("state", "")))
bytes_now += int(f.get("bytesTransferred") or 0)
if f.get("id"): # remember the transfer so we can cancel it if abandoned
xfer_ids.add(str(f["id"]))
if states:
if any(bad in s for s in states for bad in ("Errored", "Rejected", "Cancelled")):
raise RuntimeError("soulseek transfer failed")
now = time.monotonic()
dt = now - prev_t
if dt > 0 and bytes_now >= prev_bytes:
inst = (bytes_now - prev_bytes) / dt
speed = inst if speed == 0.0 else 0.6 * speed + 0.4 * inst
prev_bytes, prev_t = bytes_now, now
done = sum(1 for s in states if "Completed" in s)
on_progress(min(1.0, done / max(total, 1)))
on_progress(min(1.0, done / max(total, 1)), _eta_seconds(total_bytes, bytes_now, speed))
if done >= total:
completed = True
break
# Stall detection: a peer making byte progress keeps its slot (up to the absolute
# backstop above) so a slow-but-working transfer can finish; one that sends no bytes
# for _STALL_POLLS (~90s) is queued/dead — abandon it fast and fall through.
if bytes_now > last_bytes:
last_bytes = bytes_now
stall = 0
else:
stall += 1
if stall >= _STALL_POLLS:
raise TimeoutError(f"soulseek transfer stalled for {username}")
if not completed:
raise TimeoutError(f"soulseek download did not complete for {username}")
@@ -162,6 +309,19 @@ class SlskdClient:
)
return {"track_count": moved, "path": dest}
def _cancel_transfers(self, username: str, xfer_ids: set) -> None:
"""Cancel the given in-flight slskd transfers for a peer. Best-effort: a cleanup failure
must never mask the original download failure."""
for tid in xfer_ids:
try:
requests.delete(
f"{self._url}/api/v0/transfers/downloads/{username}/{tid}",
headers=self._headers(),
timeout=15,
)
except Exception:
pass
def _retrieve(self, files: list[dict], dest: str) -> int:
"""Move the just-finished slskd files out of its downloads dir into `dest`.
worker/lyra_worker/adapters/_streamrip.py
+62
View File
@@ -34,6 +34,53 @@ def _run(coro):
return _loop.run_until_complete(coro)
class _ProgressAggregator:
"""Turn streamrip's per-track byte callbacks into one album-level 0..1 fraction.
streamrip downloads an album's tracks concurrently and reports progress only per track
(a byte delta per chunk), so the file-count poller sees 0 complete files until they all
land at once — the 0%→100% jump. Summing downloaded bytes over an *estimated* album total
(average started-track size × track count) gives a smooth, generally-climbing fraction.
Track sizes are similar, so the estimate is stable once a track or two has started; capped
at 0.99 because the pipeline sets 1.0 once the whole download returns. All streamrip
coroutines run on one event loop on the calling thread, so no locking is needed."""
def __init__(self, track_count: int, on_progress: Callable[[float], None]):
self._n = track_count or 0
self._on_progress = on_progress
self._seen_total = 0 # summed byte-size of tracks that have started downloading
self._seen_count = 0 # number of tracks that have started
self._done_bytes = 0 # summed bytes downloaded across all started tracks
def start_track(self, total) -> None:
self._seen_total += max(int(total or 0), 0)
self._seen_count += 1
def advance(self, delta) -> None:
self._done_bytes += max(int(delta or 0), 0)
if self._seen_count == 0:
return
avg = self._seen_total / self._seen_count
n = self._n if self._n > 0 else self._seen_count
est_total = avg * n
if est_total > 0:
self._on_progress(min(self._done_bytes / est_total, 0.99))
class _ProgressHandle:
"""streamrip's `Track.download` uses `with get_progress_callback(...) as cb:`; `cb` must be
the per-chunk byte callback. This stand-in yields that callback and no-ops on exit."""
def __init__(self, update: Callable[[int], None]):
self.update = update
def __enter__(self):
return self.update
def __exit__(self, *_):
return False
def _flatten_audio(dest: str) -> int:
"""streamrip writes into a nested `Artist - Album [...]/` subfolder. Move any audio
files up into `dest`, remove the now-empty subfolders, and return the count of audio
@@ -136,6 +183,7 @@ class StreamripClient:
async def _download(self, album_id: str, dest: str, on_progress: Callable[[float], None]) -> None:
os.makedirs(dest, exist_ok=True)
import streamrip.media.track as _track_mod
from streamrip.client import QobuzClient
from streamrip.db import Database, Dummy
from streamrip.media import PendingAlbum
@@ -153,7 +201,21 @@ class StreamripClient:
album = await pending.resolve()
if album is None:
raise RuntimeError(f"could not resolve Qobuz album {album_id}")
# Hook streamrip's per-track byte progress into a single album-level fraction so the
# Floor bar climbs smoothly instead of jumping 0→100. `Track.download` binds
# get_progress_callback at import, so patch it on the track module (not progress.py).
agg = _ProgressAggregator(len(getattr(album, "tracks", []) or []), on_progress)
_orig_cb = _track_mod.get_progress_callback
def _hook(_enabled, total, _desc):
agg.start_track(total)
return _ProgressHandle(agg.advance)
_track_mod.get_progress_callback = _hook
try:
await album.rip()
finally:
_track_mod.get_progress_callback = _orig_cb
on_progress(1.0)
finally:
await client.session.close()
worker/lyra_worker/adapters/fakes.py
+10 -3
View File
@@ -72,17 +72,24 @@ class FailingAdapter(_BaseFake):
class FakeMbBrowser:
"""In-memory MbBrowser for tests. `releases` maps artist mbid -> [ReleaseGroupInfo]."""
"""In-memory MbBrowser for tests. `releases` maps artist mbid -> [ReleaseGroupInfo].
`default_core=True` returns a synthetic core Album for any artist not in `releases`, so
scoring tests don't trip the "no core discography → drop the suggestion" filter."""
def __init__(self, artists=None, releases=None):
def __init__(self, artists=None, releases=None, default_core=False):
self._artists = list(artists or [])
self._releases = dict(releases or {})
self._default_core = default_core
def search_artist(self, name: str) -> list[ArtistHit]:
return list(self._artists)
def browse_release_groups(self, artist_mbid: str) -> list[ReleaseGroupInfo]:
return list(self._releases.get(artist_mbid, []))
if artist_mbid in self._releases:
return list(self._releases[artist_mbid])
if self._default_core:
return [ReleaseGroupInfo(f"rg-{artist_mbid}", "Album", "Album", (), "2000-01-01")]
return []
class FakeAudioProbe:
worker/lyra_worker/claim.py
+38 -1
View File
@@ -24,13 +24,50 @@ def reclaim_stuck_jobs(conn: psycopg.Connection) -> int:
return n
_MAX_ATTEMPTS = 5
def requeue_or_fail(
conn: psycopg.Connection, job_id: str, error: str, max_attempts: int = _MAX_ATTEMPTS
) -> None:
"""Recover a job whose pipeline raised an *unhandled* exception (e.g. a dropped MusicBrainz
TLS connection surfacing as an SSL EOF). Without this the job is left in its mid-pipeline
'matching' state; since claim_next only picks 'requested' it is orphaned until the next
restart's reclaim_stuck_jobs — showing "Searching sources…" on the press indefinitely.
Such errors are usually transient, so requeue to 'requested' until the attempt cap, then fall
to 'needs_attention' so a persistently-failing job stays visible instead of looping forever.
`attempts` is bumped by claim_next on every claim, so it already counts tries."""
with conn.cursor() as cur:
cur.execute('SELECT attempts, "requestId" FROM "Job" WHERE id = %s', (job_id,))
row = cur.fetchone()
if row is None:
return
attempts, request_id = row
if attempts >= max_attempts:
cur.execute(
'UPDATE "Job" SET state = \'needs_attention\', error = %s, "claimedAt" = NULL, '
'"updatedAt" = now() WHERE id = %s',
(error, job_id),
)
cur.execute('UPDATE "Request" SET status = \'needs_attention\' WHERE id = %s', (request_id,))
else:
cur.execute(
'UPDATE "Job" SET state = \'requested\', "currentStage" = \'intake\', error = NULL, '
'"claimedAt" = NULL, "downloadProgress" = 0, "updatedAt" = now() WHERE id = %s',
(job_id,),
)
cur.execute('UPDATE "Request" SET status = \'pending\' WHERE id = %s', (request_id,))
conn.commit()
def claim_next(conn: psycopg.Connection) -> str | None:
"""Atomically claim the oldest queued job. Returns its id or None."""
with conn.cursor() as cur:
cur.execute(
"""
SELECT id FROM "Job"
WHERE state = 'requested'
WHERE state = 'requested' AND NOT paused
ORDER BY "createdAt" ASC
FOR UPDATE SKIP LOCKED
LIMIT 1
worker/lyra_worker/discovery.py
+160 -28
View File
@@ -17,6 +17,8 @@ class DiscoveryConfig:
similar_per_seed: int = 20
albums_per_artist: int = 1
min_score: float = 0.0
albums_only: bool = True
seed_from_library: bool = True
@classmethod
def from_config(cls, config: dict) -> "DiscoveryConfig":
@@ -32,6 +34,12 @@ class DiscoveryConfig:
except (KeyError, TypeError, ValueError):
return default
def _bool(key: str, default: bool) -> bool:
raw = config.get(key)
if raw is None:
return default
return str(raw).strip().lower() in _TRUE
return cls(
enabled=str(config.get("discover.enabled", "")).strip().lower() in _TRUE,
interval_hours=_int("discover.intervalHours", 168),
@@ -39,6 +47,8 @@ class DiscoveryConfig:
similar_per_seed=_int("discover.similarPerSeed", 20),
albums_per_artist=_int("discover.albumsPerArtist", 1),
min_score=_float("discover.minScore", 0.0),
albums_only=_bool("discover.albumsOnly", True),
seed_from_library=_bool("discover.seedFromLibrary", True),
)
@@ -63,15 +73,37 @@ def _healthy_sources(sources):
def _seed_artists(conn: psycopg.Connection, cfg: DiscoveryConfig):
"""Return up to chunk_size due seeds as (mbid, name, source) where source is 'watched'
(a followed artist, throttled on WatchedArtist.lastDiscoveredAt) or 'library' (an artist
you own albums by but don't follow, throttled on DiscoverySeed.lastDiscoveredAt). Both are
ordered least-recently-swept first so a sweep drains all eligible seeds across chunks."""
interval = cfg.interval_hours
with conn.cursor() as cur:
cur.execute(
'SELECT mbid, name FROM "WatchedArtist" '
'WHERE "lastDiscoveredAt" IS NULL '
' OR "lastDiscoveredAt" < now() - make_interval(hours => %s) '
'ORDER BY "lastDiscoveredAt" ASC NULLS FIRST LIMIT %s',
(cfg.interval_hours, cfg.chunk_size),
'SELECT mbid, name, \'watched\' AS source, "lastDiscoveredAt" FROM "WatchedArtist" '
'WHERE "lastDiscoveredAt" IS NULL OR "lastDiscoveredAt" < now() - make_interval(hours => %s) ',
(interval,),
)
return cur.fetchall()
rows = cur.fetchall()
if cfg.seed_from_library:
# Distinct owned-album artists (with a captured MBID) that we don't already follow,
# left-joined to their throttle row. A brand-new library seed (no DiscoverySeed row)
# is immediately eligible via NULLS FIRST.
cur.execute(
'SELECT DISTINCT ON (li."artistMbid") li."artistMbid", li.artist, \'library\', ds."lastDiscoveredAt" '
'FROM "LibraryItem" li '
'LEFT JOIN "DiscoverySeed" ds ON ds.mbid = li."artistMbid" '
'WHERE li."artistMbid" IS NOT NULL '
' AND li."artistMbid" NOT IN (SELECT mbid FROM "WatchedArtist") '
' AND (ds."lastDiscoveredAt" IS NULL OR ds."lastDiscoveredAt" < now() - make_interval(hours => %s))',
(interval,),
)
rows += cur.fetchall()
# Merge both sources, least-recently-swept first (NULL = never), cap at chunk_size.
rows.sort(key=lambda r: (r[3] is not None, r[3]))
return [(mbid, name, source) for mbid, name, source, _last in rows[: cfg.chunk_size]]
def _followed_mbids(conn: psycopg.Connection) -> set[str]:
@@ -80,6 +112,14 @@ def _followed_mbids(conn: psycopg.Connection) -> set[str]:
return {r[0] for r in cur.fetchall()}
def _owned_artist_mbids(conn: psycopg.Connection) -> set[str]:
"""MBIDs of artists you own albums by (captured on LibraryItem). Used both as extra
discovery seeds and to keep already-owned artists out of the suggestion candidates."""
with conn.cursor() as cur:
cur.execute('SELECT DISTINCT "artistMbid" FROM "LibraryItem" WHERE "artistMbid" IS NOT NULL')
return {r[0] for r in cur.fetchall()}
def _upsert_artist(conn: psycopg.Connection, cand: dict) -> int:
dedupe = f"artist:{cand['mbid']}:"
with conn.cursor() as cur:
@@ -103,27 +143,57 @@ def _existing_rg_mbids(conn: psycopg.Connection) -> set[str]:
return {r[0] for r in cur.fetchall()}
def _upsert_album(conn: psycopg.Connection, artist: dict, rg) -> int:
def _upsert_album(conn: psycopg.Connection, artist: dict, rg, reason: str) -> int:
dedupe = f"album:{artist['mbid']}:{rg.rg_mbid}"
with conn.cursor() as cur:
cur.execute(
'INSERT INTO "DiscoverySuggestion" (id, kind, "artistMbid", "artistName", '
'"rgMbid", album, "primaryType", "secondaryTypes", "firstReleaseDate", '
'score, "seedCount", sources, status, "dedupeKey", "createdAt", "updatedAt") '
"VALUES (gen_random_uuid()::text, 'album', %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, "
'score, "seedCount", sources, "albumReason", status, "dedupeKey", "createdAt", "updatedAt") '
"VALUES (gen_random_uuid()::text, 'album', %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, "
"'pending', %s, now(), now()) "
'ON CONFLICT ("dedupeKey") DO UPDATE SET score = EXCLUDED.score, '
'"updatedAt" = now() '
'"albumReason" = EXCLUDED."albumReason", "updatedAt" = now() '
"WHERE \"DiscoverySuggestion\".status = 'pending'",
(artist["mbid"], artist["name"], rg.rg_mbid, rg.title,
rg.primary_type or None, list(rg.secondary_types), rg.first_release_date or None,
artist["score"], artist["seed_count"], sorted(artist["sources"]), dedupe),
artist["score"], artist["seed_count"], sorted(artist["sources"]), reason, dedupe),
)
return cur.rowcount
def _album_kind_ok(g, albums_only: bool) -> bool:
"""Which release groups qualify as discovery album suggestions.
Discovery-only policy: when albums_only (default), suggest full studio Albums
only — no Singles/EPs, no secondary types (live/compilation/etc.). Falls back
to the broader is_core_release (Album/Single/EP) when the toggle is off. This
deliberately does NOT touch is_core_release, which the monitor/scan still use."""
if albums_only:
return (g.primary_type or "") == "Album" and len(g.secondary_types or ()) == 0
return is_core_release(g.primary_type, g.secondary_types)
def _norm_title(s: str) -> str:
return (s or "").strip().lower()
def _popular_pick(conn, popularity, artist_name: str, core: list) -> str | None:
"""The rg_mbid of the artist's most-played qualifying un-owned album (Last.fm), matched
into the already-browsed `core` by MBID first then normalized title. None if unavailable."""
if popularity is None or not core:
return None
by_mbid = {g.rg_mbid: g for g in core}
by_title = {_norm_title(g.title): g for g in core}
for name, mbid, _plays in popularity.top_albums(conn, artist_name):
g = (by_mbid.get(mbid) if mbid else None) or by_title.get(_norm_title(name))
if g is not None:
return g.rg_mbid
return None
def _derive_albums(conn: psycopg.Connection, browser: MbBrowser, surfaced: list[dict],
cfg: DiscoveryConfig) -> int:
cfg: DiscoveryConfig, popularity=None) -> int:
if cfg.albums_per_artist <= 0:
return 0
have = _existing_rg_mbids(conn)
@@ -134,11 +204,43 @@ def _derive_albums(conn: psycopg.Connection, browser: MbBrowser, surfaced: list[
except Exception as e: # one artist's browse failure must not abort the sweep
print(f"worker: discovery album browse failed for {artist['mbid']}: {e}", flush=True)
continue
# Drop suggestions for "artists" with no real discography of their own — producers,
# session players, band members, soundtrack-only credits. ListenBrainz surfaces them
# (they co-occur in listens) but they have no core release-group to follow or want, and
# their page is empty. Require at least one core release (Album/Single/EP, no secondary
# type); on browse failure we keep the suggestion rather than drop a real artist.
if not any(is_core_release(g.primary_type, g.secondary_types) for g in groups):
with conn.cursor() as cur:
cur.execute("DELETE FROM \"DiscoverySuggestion\" WHERE status = 'pending' "
'AND "artistMbid" = %s', (artist["mbid"],))
continue
core = [g for g in groups
if is_core_release(g.primary_type, g.secondary_types) and g.rg_mbid not in have]
if _album_kind_ok(g, cfg.albums_only) and g.rg_mbid not in have]
core.sort(key=lambda g: g.first_release_date or "", reverse=True)
# newest picks (existing behavior) + the single most-played album (Last.fm); merged so
# an album that is both newest AND most-played is one row tagged "newest,popular".
reasons: dict[str, str] = {}
for rg in core[: cfg.albums_per_artist]:
albums += _upsert_album(conn, artist, rg)
reasons[rg.rg_mbid] = "newest"
popular = _popular_pick(conn, popularity, artist["name"], core)
if popular is not None:
reasons[popular] = "newest,popular" if reasons.get(popular) == "newest" else "popular"
# Re-deriving replaces this artist's album set: drop any pending album suggestion of
# theirs that is no longer a pick (stale singles/EPs from older sweeps, or an album now
# owned), keeping wanted/dismissed rows untouched. Then upsert the current picks.
with conn.cursor() as cur:
cur.execute(
"DELETE FROM \"DiscoverySuggestion\" WHERE kind = 'album' AND status = 'pending' "
'AND "artistMbid" = %s AND NOT ("rgMbid" = ANY(%s))',
(artist["mbid"], list(reasons.keys())),
)
for rg in core: # preserve newest-first order among the chosen
reason = reasons.get(rg.rg_mbid)
if reason is not None:
albums += _upsert_album(conn, artist, rg, reason)
have.add(rg.rg_mbid)
return albums
@@ -161,11 +263,18 @@ def _record_contributions(conn: psycopg.Connection, seed_mbid: str, live, follow
except Exception as e: # a bad source/seed must not abort the sweep
print(f"worker: discovery source {src.name} failed for {seed_mbid}: {e}", flush=True)
continue
for sa in sorted(similar, key=lambda a: a.score, reverse=True)[: cfg.similar_per_seed]:
top = sorted(similar, key=lambda a: a.score, reverse=True)[: cfg.similar_per_seed]
# Max-normalize each source to [0, 1] by its own top score before summing across
# sources, so sources on wildly different scales contribute comparably (Last.fm's
# `match` is 0–1; ListenBrainz co-occurrence counts run to the thousands — raw sums
# let ListenBrainz drown Last.fm's ranking signal).
max_score = max((sa.score for sa in top), default=0.0)
for sa in top:
if sa.mbid in followed:
continue
norm = sa.score / max_score if max_score > 0 else 0.0
c = per_cand.setdefault(sa.mbid, {"name": sa.name, "score": 0.0, "sources": set()})
c["score"] += sa.score
c["score"] += norm
c["sources"].add(src.name)
if sa.name and not c["name"]:
c["name"] = sa.name
@@ -216,34 +325,57 @@ def _recompute_suggestions(conn: psycopg.Connection, affected: set[str],
return surfaced
def reset_pending(conn: psycopg.Connection) -> None:
"""Clear discovery state so the next sweep rebuilds from scratch with current logic. Drops
all PENDING suggestions (followed/wanted/dismissed user decisions are kept), every seed
contribution + library-seed throttle, and marks each watched artist due again."""
with conn.cursor() as cur:
cur.execute("DELETE FROM \"DiscoverySuggestion\" WHERE status = 'pending'")
cur.execute('DELETE FROM "DiscoverySeedContribution"')
cur.execute('DELETE FROM "DiscoverySeed"')
cur.execute('UPDATE "WatchedArtist" SET "lastDiscoveredAt" = NULL')
conn.commit()
def run_discovery(conn: psycopg.Connection, sources: list[SimilaritySource],
browser: MbBrowser, cfg: DiscoveryConfig) -> DiscoveryResult:
browser: MbBrowser, cfg: DiscoveryConfig, popularity=None) -> DiscoveryResult:
"""Record each seed's similar-artist contributions, then recompute affected
suggestions from the summed contributions — so a candidate's score is the sum of
every seed's latest contribution, independent of chunk and sweep boundaries."""
seeds = _seed_artists(conn, cfg)
followed = _followed_mbids(conn)
owned = _owned_artist_mbids(conn) if cfg.seed_from_library else set()
# Artists you already have (followed or owned) are seeds/inputs, never suggested as
# candidates — discovery only surfaces genuinely new artists.
known = followed | owned
live = _healthy_sources(sources)
# Evict contributions from seeds no longer followed — unfollow/removal has no
# FK cascade, so without this a removed seed's score would linger and inflate
# the aggregate forever. Seed `affected` with the touched candidates so their
# suggestion scores are recomputed from the remaining live seeds this sweep.
# Evict contributions whose seed is no longer valid — an unfollowed artist AND (when
# library seeding is on) an artist whose last owned album was deleted. Neither has an FK
# cascade, so without this a stale seed's score would linger and inflate the aggregate.
# Seed `affected` with the touched candidates so their scores recompute from live seeds.
valid_seeds = list(known)
with conn.cursor() as cur:
cur.execute('SELECT DISTINCT "candidateMbid" FROM "DiscoverySeedContribution" '
'WHERE "seedMbid" NOT IN (SELECT mbid FROM "WatchedArtist")')
'WHERE NOT ("seedMbid" = ANY(%s))', (valid_seeds,))
affected = {r[0] for r in cur.fetchall()}
cur.execute('DELETE FROM "DiscoverySeedContribution" '
'WHERE "seedMbid" NOT IN (SELECT mbid FROM "WatchedArtist")')
cur.execute('DELETE FROM "DiscoverySeedContribution" WHERE NOT ("seedMbid" = ANY(%s))',
(valid_seeds,))
for seed_mbid, _seed_name in seeds:
affected |= _record_contributions(conn, seed_mbid, live, followed, cfg)
for seed_mbid, _seed_name, _source in seeds:
affected |= _record_contributions(conn, seed_mbid, live, known, cfg)
surfaced = _recompute_suggestions(conn, affected, cfg)
albums = _derive_albums(conn, browser, surfaced, cfg)
albums = _derive_albums(conn, browser, surfaced, cfg, popularity)
with conn.cursor() as cur:
for seed_mbid, _seed_name in seeds:
for seed_mbid, _seed_name, source in seeds:
if source == "library":
cur.execute(
'INSERT INTO "DiscoverySeed" (mbid, "lastDiscoveredAt") VALUES (%s, now()) '
'ON CONFLICT (mbid) DO UPDATE SET "lastDiscoveredAt" = now()',
(seed_mbid,))
else:
cur.execute('UPDATE "WatchedArtist" SET "lastDiscoveredAt" = now() WHERE mbid = %s',
(seed_mbid,))
conn.commit()
worker/lyra_worker/floor.py
+42
View File
@@ -0,0 +1,42 @@
import os
from lyra_worker.config import get_config
_TRUE = {"1", "true", "yes", "on"}
def press_paused(config: dict[str, str]) -> bool:
"""True when the whole press is paused — the worker stops claiming new jobs."""
return str(config.get("floor.paused", "")).strip().lower() in _TRUE
def job_delay_seconds(config: dict[str, str]) -> float:
"""Seconds to wait between finishing one download and claiming the next. Invalid or
absent -> 0.0 (back-to-back). Negatives are clamped to 0."""
try:
return max(0.0, float(int(config.get("floor.jobDelaySeconds", "") or 0)))
except (TypeError, ValueError):
return 0.0
def kill_requested(config: dict[str, str]) -> bool:
"""True when a force-stop of the in-flight download has been requested."""
return str(config.get("floor.killRequested", "")).strip().lower() in _TRUE
def consume_kill_switch(conn, exit_fn=os._exit) -> bool:
"""If a force-stop is pending, clear the flag and terminate the process (default
os._exit) so the in-flight download is aborted. The container restart policy brings the
worker back; startup clears staging and requeues the aborted job. Returns whether it
fired. exit_fn is injectable for tests."""
config = get_config(conn)
if not kill_requested(config):
return False
with conn.cursor() as cur:
cur.execute(
'UPDATE "Config" SET value = \'false\', "updatedAt" = now() WHERE key = %s',
("floor.killRequested",),
)
conn.commit()
exit_fn(0)
return True
worker/lyra_worker/main.py
+44 -5
View File
@@ -6,16 +6,17 @@ import uuid
import psycopg
from lyra_worker.claim import claim_next, reclaim_stuck_jobs
from lyra_worker.claim import claim_next, reclaim_stuck_jobs, requeue_or_fail
from lyra_worker.config import get_config
from lyra_worker.crypto import secret_key_problem
from lyra_worker.db import wait_for_db
from lyra_worker.discovery import DiscoveryConfig, DiscoveryResult, run_discovery
from lyra_worker.discovery import DiscoveryConfig, DiscoveryResult, reset_pending, run_discovery
from lyra_worker.floor import press_paused, job_delay_seconds, consume_kill_switch
from lyra_worker.library import clear_staging_root
from lyra_worker.monitor import MonitorConfig, fulfill_owned_releases, reconcile, sweep
from lyra_worker.pipeline import run_pipeline
from lyra_worker.registry import (
build_adapters, build_browser, build_probe, build_resolver,
build_adapters, build_album_popularity, build_browser, build_probe, build_resolver,
build_similarity_sources, build_tagger,
)
from lyra_worker.scan import build_worklist, clear_worklist, scan_chunk
@@ -115,8 +116,10 @@ def maybe_run_scan(conn, resolver, probe, browser, config, dest_root: str = DEST
def _run_discovery(conn, sources, browser, config=None) -> DiscoveryResult:
cfg = DiscoveryConfig.from_config(config if config is not None else get_config(conn))
return run_discovery(conn, sources, browser, cfg)
resolved = config if config is not None else get_config(conn)
cfg = DiscoveryConfig.from_config(resolved)
popularity = build_album_popularity(resolved)
return run_discovery(conn, sources, browser, cfg, popularity=popularity)
def finish_job(conn, job_id, mcfg) -> None:
@@ -145,6 +148,10 @@ def maybe_run_discovery(conn, sources, browser, config, dcfg, now, last_discover
starting = not in_progress
if starting: # begin a fresh sweep
if str(config.get("discover.resetRequested", "")).strip().lower() in _TRUE:
reset_pending(conn) # wipe stale pending suggestions before rebuilding
_set_config(conn, "discover.resetRequested", "false")
print("worker: discovery reset — cleared pending suggestions, rebuilding", flush=True)
_set_config(conn, "discover.inProgress", "true")
_set_config(conn, "discover.requested", "false")
@@ -220,9 +227,30 @@ def _liveness_heartbeat() -> None:
time.sleep(HEARTBEAT_SECONDS)
KILL_POLL_SECONDS = 1.5
def _kill_switch_watch() -> None:
"""Poll floor.killRequested on a background daemon thread (its own DB connection) so a
force-stop aborts the process even while the main loop is blocked in a download. On any
DB hiccup, reconnect and keep watching."""
conn = wait_for_db()
while True:
try:
consume_kill_switch(conn)
except Exception:
try:
conn.close()
except Exception:
pass
conn = wait_for_db()
time.sleep(KILL_POLL_SECONDS)
def run_forever() -> None:
_require_secret_key()
threading.Thread(target=_liveness_heartbeat, daemon=True).start()
threading.Thread(target=_kill_switch_watch, daemon=True).start()
conn = wait_for_db()
clear_staging_root(STAGING_ROOT) # sweep any staging dirs orphaned by a prior crash
reclaimed = reclaim_stuck_jobs(conn) # requeue jobs a prior crash left mid-pipeline
@@ -246,6 +274,7 @@ def run_forever() -> None:
last_discover_tick = 0.0
last_heartbeat = 0.0
last_prune = 0.0
next_job_allowed = 0.0 # monotonic time before which no new job is claimed (inter-job delay)
try:
while True:
# A transient DB connection loss (e.g. AdminShutdown when Postgres restarts) raises
@@ -277,6 +306,12 @@ def run_forever() -> None:
maybe_run_scan(conn, resolver, probe, browser, config)
# Press paused (downloads only) or the inter-job delay hasn't elapsed: keep the
# loop alive (monitor/scan/discovery above still run) but claim nothing.
if press_paused(config) or now < next_job_allowed:
time.sleep(IDLE_SLEEP)
continue
job_id = claim_next(conn)
if job_id is None:
time.sleep(IDLE_SLEEP)
@@ -288,8 +323,12 @@ def run_forever() -> None:
except Exception as e: # one bad job must not take down the worker loop
print(f"worker: pipeline failed for job {job_id}: {e}", flush=True)
conn.rollback()
# Don't orphan the job in its mid-pipeline 'matching' state (claim_next only
# picks 'requested'): requeue for a bounded retry, then needs_attention.
requeue_or_fail(conn, job_id, str(e))
finish_job(conn, job_id, mcfg)
print(f"worker: finished job {job_id}", flush=True)
next_job_allowed = time.monotonic() + job_delay_seconds(config)
except psycopg.OperationalError as e:
print(f"worker: db connection lost ({e}); reconnecting...", flush=True)
try:
worker/lyra_worker/monitor.py
+60 -17
View File
@@ -86,16 +86,40 @@ def _set_state(conn: psycopg.Connection, rel_id: str, state: str) -> None:
cur.execute('UPDATE "MonitoredRelease" SET state = %s WHERE id = %s', (state, rel_id))
def _in_library_at_cutoff(conn: psycopg.Connection, artist: str, album: str, cutoff: int) -> bool:
def _in_library_at_cutoff(
conn: psycopg.Connection, artist: str, album: str, rg_mbid: str, cutoff: int
) -> bool:
"""True if the album is owned at/above `cutoff`. Matches on the release-group MBID when the
library row has one — robust against MusicBrainz artist-name canonicalization (e.g. the
followed 'Kanye West' imports as 'Ye') — and falls back to exact artist+album for scanned
rows that predate MBID capture."""
with conn.cursor() as cur:
cur.execute(
'SELECT 1 FROM "LibraryItem" WHERE artist = %s AND album = %s '
'AND "qualityClass" >= %s LIMIT 1',
(artist, album, cutoff),
'SELECT 1 FROM "LibraryItem" WHERE "qualityClass" >= %s AND ('
' ("rgMbid" IS NOT NULL AND "rgMbid" = %s) '
' OR ("rgMbid" IS NULL AND artist = %s AND album = %s)) LIMIT 1',
(cutoff, rg_mbid, artist, album),
)
return cur.fetchone() is not None
def _retire_failed_attempts(conn: psycopg.Connection, rel_id: str) -> None:
"""Delete prior failed (needs_attention) Request rows for this release (Job/Candidate cascade).
A hard-to-match release is re-pressed every retry interval; each failed attempt used to leave
a needs_attention Request behind, piling up phantom "needs attention" cards on the press even
after a later attempt imported the album. Retiring them when the release is fulfilled or
re-attempted keeps the press showing one live attempt at a time. Manual (non-monitor) requests
have no monitoredReleaseId and are never touched."""
with conn.cursor() as cur:
cur.execute(
'DELETE FROM "Request" r USING "Job" j '
'WHERE j."requestId" = r.id AND r."monitoredReleaseId" = %s '
"AND j.state = 'needs_attention'",
(rel_id,),
)
def _enqueue_one(conn: psycopg.Connection, rel_id: str, artist: str, album: str) -> None:
with conn.cursor() as cur:
cur.execute(
@@ -118,9 +142,11 @@ def enqueue_due(conn: psycopg.Connection, cfg: MonitorConfig) -> int:
cur.execute(
'SELECT mr.id, mr."artistName", mr.album, mr.state, mr."currentQualityClass", '
' (mr."firstGrabbedAt" IS NOT NULL '
' AND mr."firstGrabbedAt" < now() - make_interval(days => %s)) AS window_expired '
' AND mr."firstGrabbedAt" < now() - make_interval(days => %s)) AS window_expired, '
' mr."rgMbid" '
'FROM "MonitoredRelease" mr '
'WHERE mr.monitored = true '
' AND NOT mr.ignored ' # user-ignored releases are never enqueued
" AND mr.state <> 'fulfilled' "
' AND (mr."lastSearchedAt" IS NULL '
' OR mr."lastSearchedAt" < now() - make_interval(hours => %s)) '
@@ -133,8 +159,11 @@ def enqueue_due(conn: psycopg.Connection, cfg: MonitorConfig) -> int:
rows = cur.fetchall()
enqueued = 0
for rel_id, artist, album, state, quality, window_expired in rows:
if _in_library_at_cutoff(conn, artist, album, cfg.quality_cutoff):
for rel_id, artist, album, state, quality, window_expired, rg_mbid in rows:
# A due release is about to be fulfilled or re-attempted; either way any earlier failed
# attempt is superseded, so retire it before deciding.
_retire_failed_attempts(conn, rel_id)
if _in_library_at_cutoff(conn, artist, album, rg_mbid, cfg.quality_cutoff):
_set_state(conn, rel_id, "fulfilled")
continue
if state == "grabbed" and (window_expired or (quality is not None and quality >= cfg.quality_cutoff)):
@@ -153,12 +182,19 @@ def fulfill_owned_releases(conn: psycopg.Connection, cfg: MonitorConfig) -> int:
with conn.cursor() as cur:
cur.execute(
'UPDATE "MonitoredRelease" mr SET state = \'fulfilled\', '
' "currentQualityClass" = li.q, '
' "currentQualityClass" = owned.q, '
' "firstGrabbedAt" = COALESCE(mr."firstGrabbedAt", now()) '
'FROM (SELECT artist, album, max("qualityClass") AS q FROM "LibraryItem" '
' GROUP BY artist, album) li '
'WHERE li.artist = mr."artistName" AND li.album = mr.album '
" AND mr.monitored = true AND mr.state <> 'fulfilled' AND li.q >= %s",
'FROM ('
' SELECT mr2.id AS mid, max(li."qualityClass") AS q '
' FROM "MonitoredRelease" mr2 JOIN "LibraryItem" li ON ('
# match on rgMbid where the library row has one (survives artist-name canonicalization),
# else fall back to exact artist+album for scanned rows lacking an MBID
' (li."rgMbid" IS NOT NULL AND li."rgMbid" = mr2."rgMbid") '
' OR (li."rgMbid" IS NULL AND li.artist = mr2."artistName" AND li.album = mr2.album)) '
" WHERE mr2.monitored = true AND mr2.state <> 'fulfilled' "
' GROUP BY mr2.id HAVING max(li."qualityClass") >= %s'
') owned '
'WHERE mr.id = owned.mid',
(cfg.quality_cutoff,),
)
fixed = cur.rowcount
@@ -170,21 +206,28 @@ def reconcile(conn: psycopg.Connection, job_id: str, cfg: MonitorConfig) -> None
"""Feed a finished job's outcome back into its MonitoredRelease (if any)."""
with conn.cursor() as cur:
cur.execute(
'SELECT j.state, r."monitoredReleaseId", r.artist, r.album '
'FROM "Job" j JOIN "Request" r ON r.id = j."requestId" WHERE j.id = %s',
'SELECT j.state, r."monitoredReleaseId", r.artist, r.album, mr."rgMbid" '
'FROM "Job" j JOIN "Request" r ON r.id = j."requestId" '
'LEFT JOIN "MonitoredRelease" mr ON mr.id = r."monitoredReleaseId" '
'WHERE j.id = %s',
(job_id,),
)
row = cur.fetchone()
if row is None:
return
state, rel_id, artist, album = row
state, rel_id, artist, album, rg_mbid = row
if rel_id is None or state != "imported":
return # needs_attention (or in-flight) leaves the release wanted
# A successful import supersedes any earlier failed attempts for this release.
_retire_failed_attempts(conn, rel_id)
with conn.cursor() as cur:
cur.execute(
'SELECT "qualityClass" FROM "LibraryItem" WHERE artist = %s AND album = %s',
(artist, album),
'SELECT max("qualityClass") FROM "LibraryItem" WHERE '
' ("rgMbid" IS NOT NULL AND "rgMbid" = %s) '
' OR ("rgMbid" IS NULL AND artist = %s AND album = %s)',
(rg_mbid, artist, album),
)
lib = cur.fetchone()
quality = lib[0] if lib else None
worker/lyra_worker/pipeline.py
+220 -45
View File
@@ -1,21 +1,70 @@
import os
import random
import shutil
import threading
import time
from concurrent.futures import ThreadPoolExecutor
from dataclasses import replace
from typing import Sequence
from typing import Callable, Sequence
import psycopg
from lyra_worker.adapters.base import SourceAdapter
from lyra_worker.confidence import score_confidence
from lyra_worker.library import album_dir, import_album, list_audio_files, staging_dir
from lyra_worker.qobuz_gate import gate_open, record_download
from lyra_worker.quality import quality_class
from lyra_worker.ranker import rank_candidates
from lyra_worker.types import Candidate, MBTarget
_AUDIO_EXT = {".flac", ".mp3", ".m4a", ".opus", ".ogg", ".aac", ".wav"}
# Reject a download whose total playtime falls below this fraction of MusicBrainz's expected
# total — catches truncated files / preview clips substituted for real tracks that keep the
# track COUNT right. Generous, so edition/encoding differences don't false-positive; bonus
# tracks (over-long) are always fine. Kept well below 1.0 because a source legitimately delivers a
# different edition than MB's resolved release (shorter radio edits, no bonus/live tracks), and only
# a genuinely truncated/preview-filled download (playtime a fraction of expected) should be caught.
_DURATION_MIN_RATIO = 0.65
# Cap how many ranked candidates a single job will actually download+verify before giving up.
# Without this, the incomplete-download fall-through can grind through dozens of sources (a popular
# album returns ~200 Soulseek candidates), each a slow attempt that also leaves an abandoned slskd
# transfer. The best candidates rank first, so 6 attempts is plenty.
_MAX_DOWNLOAD_ATTEMPTS = 6
# The best quality any non-Qobuz source offers (Soulseek FLAC = CD-lossless). A library copy at or
# above this can only be improved by hi-res Qobuz, so when Qobuz is gated by pacing an upgrade of
# such a copy is deferred without even searching.
_CD_LOSSLESS_CLASS = 2
def _measure_staged_duration_s(staging: str) -> float | None:
"""Sum the playtime (seconds) of the non-empty staged audio via mutagen. Returns None if
nothing could be measured (no audio, or an unreadable container) so the caller skips the
check rather than failing a good download on a probe hiccup. Lazy mutagen import; not
exercised by the offline fakes (which stage no real audio)."""
try:
import mutagen
except Exception:
return None
total = 0.0
measured = False
for root, _dirs, files in os.walk(staging):
for f in files:
if os.path.splitext(f)[1].lower() not in _AUDIO_EXT:
continue
path = os.path.join(root, f)
try:
if os.path.getsize(path) == 0:
continue
audio = mutagen.File(path)
length = getattr(getattr(audio, "info", None), "length", None)
except Exception:
continue
if length:
total += float(length)
measured = True
return total if measured else None
def _count_staged_audio(staging: str) -> int:
"""Count NON-EMPTY audio files anywhere under the staging dir (streamrip nests them in a
@@ -34,6 +83,40 @@ def _count_staged_audio(staging: str) -> int:
return n
def _download_problem(result, staging: str, target, winner, measure_duration) -> str | None:
"""Why a completed download is unusable — ``"incomplete download"`` / ``"download too short
(...)"`` — or None if it's good. Completeness is judged against the CHOSEN SOURCE's own track
count (``winner.track_count``), NOT MusicBrainz's canonical release: MB picks an arbitrary
releases[0] that is often a deluxe/expanded edition with more tracks (and runtime) than the
standard album a source legitimately delivers, so gating on ``target.track_count`` falsely
rejects complete standard-edition downloads (The Script "No Sound Without Silence" 11 vs MB 12,
Skillet "Unleashed" 12 vs MB's 20-track deluxe). Pure — no DB I/O — so the download loop can
call it per candidate and fall through to the next source on a problem."""
expected = winner.track_count
# (a) Truncated: fewer files than the source promised (a track failed mid-download). Count the
# NON-EMPTY audio on disk too: a silently-skipped track leaves a 0-byte placeholder the adapter
# still counts. (`staged and ...` keeps fake adapters that stage nothing on the count path.)
staged = _count_staged_audio(staging)
truncated = result.track_count < expected or bool(staged and staged < expected)
# (b) Implausibly small vs MB's album — a lone-track "full album" video / wrong match, as
# distinct from a legitimately smaller edition. A real edition keeps more than half MB's tracks.
too_small = target.track_count is not None and expected * 2 <= target.track_count
if truncated or too_small:
return "incomplete download"
# Duration: a truncated file or short preview substituted for a real track shows up as playtime
# well under the expected total. Scale MB's total to the DELIVERED edition's size so a smaller
# edition isn't judged against a larger one's runtime. Only when the total is known (measured is
# None for the offline fakes).
if target.total_duration_s:
expected_total = target.total_duration_s
if target.track_count:
expected_total *= winner.track_count / target.track_count
measured = measure_duration(staging)
if measured is not None and measured < expected_total * _DURATION_MIN_RATIO:
return f"download too short ({measured / 60:.0f} of ~{expected_total / 60:.0f} min)"
return None
def _poll_download_progress(job_id: str, staging: str, expected: int, stop: "threading.Event",
reports: "threading.Event") -> None:
"""Until `stop`, periodically write Job.downloadProgress = files-in-staging / expected
@@ -59,6 +142,16 @@ def _poll_download_progress(job_id: str, staging: str, expected: int, stop: "thr
conn.close()
def _search_adapter(adapter: SourceAdapter, target: MBTarget) -> list[Candidate]:
"""Search one adapter, swallowing failures — a down source contributes no candidates rather
than crashing the job. Runs on a worker thread (searches are parallelized across sources)."""
try:
return list(adapter.search(target))
except Exception as e:
print(f"pipeline: adapter {adapter.name} search failed: {e}", flush=True)
return []
def _set_state(conn: psycopg.Connection, job_id: str, state: str, stage: str) -> None:
with conn.cursor() as cur:
cur.execute(
@@ -68,30 +161,42 @@ def _set_state(conn: psycopg.Connection, job_id: str, state: str, stage: str) ->
conn.commit()
def _set_download_progress(conn: psycopg.Connection, job_id: str, frac: float) -> None:
def _set_download_progress(
conn: psycopg.Connection, job_id: str, frac: float, eta_seconds: int | None = None
) -> None:
with conn.cursor() as cur:
cur.execute('UPDATE "Job" SET "downloadProgress" = %s WHERE id = %s', (frac, job_id))
cur.execute(
'UPDATE "Job" SET "downloadProgress" = %s, "downloadEtaSeconds" = %s WHERE id = %s',
(frac, eta_seconds, job_id),
)
conn.commit()
def _make_on_progress(conn: psycopg.Connection, job_id: str, reports: "threading.Event"):
"""Build the on_progress(pct) callback threaded into adapter.download. It marks the
adapter as reporting real byte-level progress (so the file-count poller yields) and
throttle-writes Job.downloadProgress. Called synchronously from adapter.download on the
pipeline's own thread, so it safely reuses `conn`."""
"""Build the on_progress(pct, eta_seconds=None) callback threaded into adapter.download. It
marks the adapter as reporting real byte-level progress (so the file-count poller yields) and
throttle-writes Job.downloadProgress (+ the download ETA when the adapter measures one — slskd
does; others pass None). Called synchronously from adapter.download on the pipeline's own
thread, so it safely reuses `conn`."""
state = {"frac": -1.0, "t": 0.0}
def on_progress(pct: float) -> None:
def on_progress(pct: float, eta_seconds: int | None = None) -> None:
reports.set()
frac = 0.0 if pct < 0 else 1.0 if pct > 1 else float(pct)
now = time.monotonic()
# throttle DB writes: on a >=1% move or every 0.5s, not once per received byte
if frac - state["frac"] >= 0.01 or (now - state["t"]) >= 0.5:
# throttle DB writes: on a >=1% move, a new ETA, or every 0.5s — not once per received byte
if frac - state["frac"] >= 0.01 or eta_seconds is not None or (now - state["t"]) >= 0.5:
state["frac"] = frac
state["t"] = now
try:
_set_download_progress(conn, job_id, frac)
_set_download_progress(conn, job_id, frac, eta_seconds)
except Exception as e: # a progress write must never fail the download
# Roll back so a failed write can't leave the shared pipeline conn in an aborted
# txn (which would cascade "current transaction is aborted" into every later query).
try:
conn.rollback()
except Exception:
pass
print(f"pipeline: on_progress write failed: {e}", flush=True)
return on_progress
@@ -146,6 +251,19 @@ def _is_upgrade_job(conn: psycopg.Connection, job_id: str) -> bool:
return bool(row and row[0])
def _is_force_job(conn: psycopg.Connection, job_id: str) -> bool:
"""A user-forced re-acquire (Library 'Replace / upgrade'): skip the already-in-library
dedupe so an owned album is re-downloaded. The import step still keeps the new copy only
if it's higher quality, so a forced upgrade can never downgrade what's on disk."""
with conn.cursor() as cur:
cur.execute(
'SELECT force FROM "Request" WHERE id = (SELECT "requestId" FROM "Job" WHERE id = %s)',
(job_id,),
)
row = cur.fetchone()
return bool(row and row[0])
def _already_in_library_at_cutoff(conn: psycopg.Connection, target: MBTarget, cutoff: int) -> bool:
with conn.cursor() as cur:
cur.execute(
@@ -166,6 +284,17 @@ def _library_quality(conn: psycopg.Connection, target: MBTarget) -> int | None:
return row[0] if row else None
def _keep_existing_and_complete(conn: psycopg.Connection, job_id: str) -> None:
"""Finish a job without importing — keep the existing library copy — and mark its request
completed. Used when a (deferred) upgrade has nothing better to offer right now; the release
stays below the cutoff and is re-attempted later."""
_set_state(conn, job_id, "imported", "import")
with conn.cursor() as cur:
cur.execute("UPDATE \"Request\" SET status = 'completed' WHERE id = %s",
(_request_id(conn, job_id),))
conn.commit()
def _persist_candidates(conn: psycopg.Connection, job_id: str, candidates: list[Candidate]) -> None:
with conn.cursor() as cur:
for c in candidates:
@@ -197,15 +326,17 @@ def _import(conn: psycopg.Connection, job_id: str, target: MBTarget,
with conn.cursor() as cur:
cur.execute(
'INSERT INTO "LibraryItem" (id, "requestId", artist, album, path, source, '
'format, "qualityClass", "trackNames", "importedAt") '
"VALUES (gen_random_uuid()::text, %s, %s, %s, %s, %s, %s, %s, %s, now()) "
'format, "qualityClass", "trackNames", "rgMbid", "artistMbid", "importedAt") '
"VALUES (gen_random_uuid()::text, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, now()) "
'ON CONFLICT (artist, album) DO UPDATE SET '
' "requestId" = EXCLUDED."requestId", path = EXCLUDED.path, source = EXCLUDED.source, '
' format = EXCLUDED.format, "qualityClass" = EXCLUDED."qualityClass", '
' "trackNames" = EXCLUDED."trackNames", "importedAt" = now() '
' "trackNames" = EXCLUDED."trackNames", "rgMbid" = COALESCE(EXCLUDED."rgMbid", "LibraryItem"."rgMbid"), '
' "artistMbid" = COALESCE(EXCLUDED."artistMbid", "LibraryItem"."artistMbid"), "importedAt" = now() '
'WHERE EXCLUDED."qualityClass" > "LibraryItem"."qualityClass"',
(request_id, target.artist, target.album, path, winner.source,
winner.quality.fmt, quality_class(winner.quality), track_names),
winner.quality.fmt, quality_class(winner.quality), track_names,
target.rg_mbid or None, target.artist_mbid or None),
)
cur.execute(
"UPDATE \"Request\" SET status = 'completed' WHERE id = %s", (request_id,)
@@ -223,6 +354,7 @@ def run_pipeline(
dest_root: str = "/music",
staging_root: str | None = None,
upgrade_cutoff: int | None = None,
measure_duration: Callable[[str], float | None] = _measure_staged_duration_s,
) -> None:
"""Real staged acquisition using source-agnostic adapters. Fakes in this plan."""
names = [a.name for a in adapters]
@@ -237,8 +369,11 @@ def run_pipeline(
if resolved is not None:
target = resolved
# dedupe: a monitor-driven upgrade job proceeds unless the existing copy already meets
# the cutoff; a plain request short-circuits on any existing copy (slice-1 behavior).
if upgrade_cutoff is not None and _is_upgrade_job(conn, job_id):
# the cutoff; a plain request short-circuits on any existing copy (slice-1 behavior). A
# user-forced re-acquire skips the dedupe entirely (keep-if-better still guards the import).
if _is_force_job(conn, job_id):
_dedupe_hit = False
elif upgrade_cutoff is not None and _is_upgrade_job(conn, job_id):
_dedupe_hit = _already_in_library_at_cutoff(conn, target, upgrade_cutoff)
else:
_dedupe_hit = _already_in_library(conn, target)
@@ -252,19 +387,43 @@ def run_pipeline(
conn.commit()
return
# 2. match — search all adapters; score + persist EVERY candidate found
# Pre-search upgrade skip: a >= CD-lossless copy can only be improved by hi-res Qobuz. If Qobuz
# is gated by pacing, don't even search (no other source can beat it) — defer to when Qobuz
# frees up. This avoids re-searching every already-owned album (~2 min Soulseek search each) on
# every monitor cycle. A user-forced re-acquire always proceeds.
_existing_q = _library_quality(conn, target)
if _existing_q is not None and _existing_q >= _CD_LOSSLESS_CLASS \
and not _is_force_job(conn, job_id) and not gate_open(conn):
_keep_existing_and_complete(conn, job_id)
return
# 2. match — search all adapters CONCURRENTLY (Soulseek's search polls up to ~2 min; run it
# alongside Qobuz's instant search so total match time = the slowest source, not their sum).
# Only Qobuz touches streamrip's shared asyncio loop and there's a single Qobuz adapter, so no
# two threads drive that loop at once. Results are collected in adapter order (map preserves
# input order) so candidate ordering stays deterministic.
_set_state(conn, job_id, "matching", "match")
found: list[Candidate] = []
for adapter in adapters:
try:
results = adapter.search(target)
except Exception as e: # a down source contributes no candidates, never crashes the job
print(f"pipeline: adapter {adapter.name} search failed: {e}", flush=True)
continue
with ThreadPoolExecutor(max_workers=max(1, len(adapters))) as pool:
per_adapter = list(pool.map(lambda a: (a, _search_adapter(a, target)), adapters))
for adapter, results in per_adapter:
for c in results:
found.append(replace(c, source_tier=adapter.tier, confidence=score_confidence(target, c)))
# Drop candidates that can't be the full album — a result with far fewer tracks than the release
# has (e.g. a single-song Soulseek folder named like the album). Same threshold as the
# completeness 'too small' guard, applied up front so we never waste a download attempt on it.
# Unknown counts (0) and legitimately smaller editions (e.g. 11 vs 12) are kept.
if target.track_count:
found = [c for c in found if c.track_count == 0 or c.track_count * 2 > target.track_count]
_persist_candidates(conn, job_id, found)
# Qobuz pacing: when the budget gate is closed (off-hours / daily cap / spacing), drop Qobuz
# candidates so the job falls through to another source and Qobuz stays under the radar — UNLESS
# Qobuz is the only option (nothing to fall through to, so don't strand the album).
if any(c.source == "qobuz" for c in found) and any(c.source != "qobuz" for c in found):
if not gate_open(conn):
found = [c for c in found if c.source != "qobuz"]
# 3. rank
_set_state(conn, job_id, "matched", "rank")
ranked = rank_candidates(target, found, min_confidence)
@@ -272,12 +431,23 @@ def run_pipeline(
_fail(conn, job_id, "no candidate above confidence threshold")
return
# Upgrade guard: for an album already in the library, skip the download when no currently-
# available source beats the copy we have (ranked[0] is the highest-quality candidate). This
# happens when Qobuz is gated by pacing and only a same-quality Soulseek copy is left — the
# download would be discarded by keep-if-better, so defer it. The release stays below the cutoff
# and is re-attempted when a better source frees up. A user-forced re-acquire always proceeds.
existing_q = _library_quality(conn, target)
if existing_q is not None and not _is_force_job(conn, job_id) \
and quality_class(ranked[0].quality) <= existing_q:
_keep_existing_and_complete(conn, job_id) # keep the existing copy; nothing better available
return
# 4. download (fall-through) into an isolated per-job staging dir
_set_state(conn, job_id, "downloading", "download")
by_source = {a.name: a for a in adapters}
staging = staging_dir(staging_root or f"{dest_root}/.staging", job_id)
winner = None
result = None
last_problem = None # why the most recent downloaded-but-rejected candidate was unusable
_set_download_progress(conn, job_id, 0.0) # reset for this run
expected = target.track_count or (ranked[0].track_count if ranked else 0)
_stop = threading.Event()
@@ -288,36 +458,46 @@ def run_pipeline(
_poller.start()
try:
try:
attempts = 0
for candidate in ranked:
adapter = by_source.get(candidate.source)
if adapter is None:
continue
if attempts >= _MAX_DOWNLOAD_ATTEMPTS:
break # don't grind through every peer/source of a popular album
attempts += 1
_mark_chosen(conn, job_id, candidate.source_ref) # reflect the source now in flight
shutil.rmtree(staging, ignore_errors=True) # clean slate per attempt
_reports.clear() # this attempt hasn't reported yet → poller estimates until it does
result = adapter.download(candidate, staging, _make_on_progress(conn, job_id, _reports))
if result.ok:
if not result.ok:
continue
# Verify completeness on the staging copy. A download that lands incomplete or too
# short falls through to the NEXT-ranked source (e.g. Qobuz repeatedly fails one
# track → try the deluxe edition or Soulseek) rather than failing the whole job on
# the first source's shortfall.
last_problem = _download_problem(result, staging, target, candidate, measure_duration)
if last_problem is None:
winner = candidate
break
finally:
_stop.set()
_poller.join(timeout=3)
if winner is None or result is None or not result.ok:
_fail(conn, job_id, "all downloads failed")
if winner is None:
# No source produced a complete album: surface why the last one was rejected (incomplete
# / too short), or "all downloads failed" if none even downloaded.
_fail(conn, job_id, last_problem or "all downloads failed")
return
_set_download_progress(conn, job_id, 1.0) # download done — UI shows 100% into Finishing
if winner.source == "qobuz":
# Count this Qobuz download against today's budget and set the next randomized spacing.
try:
record_download(conn, random.random())
except Exception as e: # pacing bookkeeping must never fail a good download
print(f"pipeline: qobuz pacing record failed for job {job_id}: {e}", flush=True)
# 5. verify completeness on the staging copy, then promote + tag
# 5. promote + tag the verified winner
_set_state(conn, job_id, "tagging", "tag")
expected = target.track_count if target.track_count is not None else winner.track_count
# Also count the NON-EMPTY audio actually on disk: a silently-skipped track leaves a
# 0-byte placeholder that the adapter still counts, so trust the files when any are
# staged. (`staged and ...` keeps fake adapters that stage nothing on the count path.)
staged = _count_staged_audio(staging)
if result.track_count < expected or (staged and staged < expected):
_fail(conn, job_id, "incomplete download")
return
final = album_dir(dest_root, target)
existing_q = _library_quality(conn, target)
new_q = quality_class(winner.quality)
@@ -331,12 +511,7 @@ def run_pipeline(
print(f"pipeline: tagging failed for job {job_id}: {e}", flush=True)
_import(conn, job_id, target, winner, final)
else:
# an existing copy is already at >= this quality — keep it untouched, just complete the request
with conn.cursor() as cur:
cur.execute(
"UPDATE \"Request\" SET status = 'completed' WHERE id = %s",
(_request_id(conn, job_id),),
)
conn.commit()
# an existing copy is already at >= this quality — keep it untouched, just complete
_keep_existing_and_complete(conn, job_id)
finally:
shutil.rmtree(staging, ignore_errors=True) # a partial/failed download never persists
worker/lyra_worker/qobuz_gate.py
+159
View File
@@ -0,0 +1,159 @@
"""Qobuz pacing ("budget gate").
Keep using Qobuz for hi-res, but at a human-looking volume and cadence so the account isn't
flagged for bulk downloading (which got it 403 USER_BLOCKED). One gate decides, per album,
whether Qobuz is an eligible source right now: within active hours, under today's cap (a warm-up
ramp of a steady cap), and past a randomized spacing since the last Qobuz download. When the gate
is closed the pipeline drops Qobuz candidates and falls through to the next source; monitored
albums upgrade to Qobuz later once budget frees up.
Pure decision logic (`today_cap`, `is_allowed`, `next_gap_seconds`) is separated from the two DB
helpers (`gate_open`, `record_download`) so the policy is unit-tested without a clock or DB.
"""
from dataclasses import dataclass
from datetime import date
_TRUE = {"1", "true", "yes", "on"}
_JITTER = 0.3 # ± fraction around the spread interval
_GAP_FLOOR_S = 60.0
@dataclass(frozen=True)
class QobuzPacing:
enabled: bool = True
active_start: int = 8
active_end: int = 23
daily_cap: int = 40
warmup_start_date: str = "" # ISO "YYYY-MM-DD", or "" for no warm-up ramp
@classmethod
def from_config(cls, config: dict) -> "QobuzPacing":
def _int(key, default):
try:
return int(config[key])
except (KeyError, TypeError, ValueError):
return default
return cls(
enabled=str(config.get("qobuz.pacing.enabled", "true")).strip().lower() in _TRUE,
active_start=_int("qobuz.pacing.activeStartHour", 8),
active_end=_int("qobuz.pacing.activeEndHour", 23),
daily_cap=_int("qobuz.pacing.dailyCap", 40),
warmup_start_date=str(config.get("qobuz.pacing.warmupStartDate", "") or "").strip(),
)
def today_cap(pacing: QobuzPacing, today: date) -> int:
"""Today's Qobuz download cap: the steady cap, ramped up by account age (days since
``warmup_start_date``). No warm-up date -> steady cap. A small steady cap is never exceeded."""
steady = max(0, pacing.daily_cap)
if not pacing.warmup_start_date:
return steady
try:
start = date.fromisoformat(pacing.warmup_start_date)
except ValueError:
return steady
days = max(0, (today - start).days)
if days < 3:
return min(5, steady)
if days < 7:
return min(10, steady)
if days < 14:
return min(20, steady)
return steady
def is_allowed(pacing: QobuzPacing, now_hour: int, count_today: int, cap: int,
now_epoch: float, next_allowed_epoch: float) -> bool:
"""Pure gate decision. Qobuz is eligible only within active hours, under today's cap, and past
the randomized spacing since the last download. Pacing disabled -> always allowed."""
if not pacing.enabled:
return True
if not (pacing.active_start <= now_hour < pacing.active_end):
return False
if count_today >= cap:
return False
if now_epoch < next_allowed_epoch:
return False
return True
def next_gap_seconds(pacing: QobuzPacing, cap: int, rand01: float) -> float:
"""Seconds until the next Qobuz download is allowed: the day's budget spread across the active
window (``activeWindow / cap``), ± _JITTER, floored at _GAP_FLOOR_S. ``rand01`` in [0,1] is the
injected randomness (so tests are deterministic; callers pass random.random())."""
window_s = max(1, pacing.active_end - pacing.active_start) * 3600
base = window_s / max(1, cap)
lo, hi = base * (1 - _JITTER), base * (1 + _JITTER)
gap = lo + (hi - lo) * rand01
return max(_GAP_FLOOR_S, gap)
def _now_parts(conn):
"""(hour:int, today:date, epoch:float) from the DB clock — avoids container-TZ ambiguity."""
with conn.cursor() as cur:
cur.execute("SELECT extract(hour from now())::int, current_date, extract(epoch from now())")
hour, today, epoch = cur.fetchone()
return int(hour), today, float(epoch)
def _read(conn, keys):
with conn.cursor() as cur:
cur.execute('SELECT key, value FROM "Config" WHERE key = ANY(%s)', (list(keys),))
return dict(cur.fetchall())
def gate_open(conn) -> bool:
"""Whether Qobuz may be used for a download right now (reads pacing config + counter/spacing
state from Config and the DB clock). A counter from a previous day is treated as 0."""
from lyra_worker.config import get_config
pacing = QobuzPacing.from_config(get_config(conn))
hour, today, epoch = _now_parts(conn)
state = _read(conn, ("qobuz.pacing.countDay", "qobuz.pacing.countToday", "qobuz.pacing.nextAllowedAt"))
count = 0
if state.get("qobuz.pacing.countDay") == today.isoformat():
try:
count = int(state.get("qobuz.pacing.countToday") or 0)
except (TypeError, ValueError):
count = 0
try:
next_allowed = float(state.get("qobuz.pacing.nextAllowedAt") or 0)
except (TypeError, ValueError):
next_allowed = 0.0
return is_allowed(pacing, hour, count, today_cap(pacing, today), epoch, next_allowed)
def record_download(conn, rand01: float) -> None:
"""Record a successful Qobuz download: bump today's counter (resetting it if the stored day is
stale) and set the randomized next-allowed time. ``rand01`` is injected randomness."""
from lyra_worker.config import get_config
pacing = QobuzPacing.from_config(get_config(conn))
_hour, today, epoch = _now_parts(conn)
state = _read(conn, ("qobuz.pacing.countDay", "qobuz.pacing.countToday"))
if state.get("qobuz.pacing.countDay") == today.isoformat():
try:
count = int(state.get("qobuz.pacing.countToday") or 0)
except (TypeError, ValueError):
count = 0
else:
count = 0
count += 1
next_allowed = epoch + next_gap_seconds(pacing, today_cap(pacing, today), rand01)
_write(conn, {
"qobuz.pacing.countDay": today.isoformat(),
"qobuz.pacing.countToday": str(count),
"qobuz.pacing.nextAllowedAt": repr(next_allowed),
})
def _write(conn, kv: dict) -> None:
with conn.cursor() as cur:
for key, value in kv.items():
cur.execute(
'INSERT INTO "Config"(key,value,secret,"updatedAt") VALUES (%s,%s,false,now()) '
'ON CONFLICT (key) DO UPDATE SET value=EXCLUDED.value, "updatedAt"=now()',
(key, value),
)
conn.commit()
worker/lyra_worker/registry.py
+8
View File
@@ -13,6 +13,7 @@ from lyra_worker.browser import MbBrowser
from lyra_worker.probe import AudioProbe
from lyra_worker.resolver import MbResolver
from lyra_worker.similarity._lastfm import LastfmSource
from lyra_worker.similarity._lastfm_albums import LastfmAlbumPopularity
from lyra_worker.similarity._listenbrainz import ListenBrainzSource
from lyra_worker.similarity.base import SimilaritySource
from lyra_worker.tagger import Tagger
@@ -67,3 +68,10 @@ def build_similarity_sources(config: dict, dsn: str | None = None) -> list[Simil
if key:
sources.append(LastfmSource(api_key=key, browser=build_browser(dsn)))
return sources
def build_album_popularity(config: dict, dsn: str | None = None):
"""Last.fm album-popularity provider for discovery, or None when no Last.fm key is set
(discovery then surfaces newest albums only)."""
key = config.get("lastfm.api_key")
return LastfmAlbumPopularity(api_key=key) if key else None
worker/lyra_worker/scan.py
+46 -8
View File
@@ -84,13 +84,16 @@ def _record_owned(conn, target, quality_class: int, fmt: str, path: str, watched
track_names = list_audio_files(path) # capture the real on-disk tracks
cur.execute(
'INSERT INTO "LibraryItem" (id, "requestId", artist, album, path, source, format, '
'"qualityClass", "trackNames", "importedAt") '
"VALUES (gen_random_uuid()::text, NULL, %s, %s, %s, 'scan', %s, %s, %s, now()) "
# refresh trackNames on re-scan (populates items imported before this column);
# xmax=0 ⇒ this was an INSERT (a genuinely new library item) vs an UPDATE.
'ON CONFLICT (artist, album) DO UPDATE SET "trackNames" = EXCLUDED."trackNames" '
'"qualityClass", "trackNames", "rgMbid", "artistMbid", "importedAt") '
"VALUES (gen_random_uuid()::text, NULL, %s, %s, %s, 'scan', %s, %s, %s, %s, %s, now()) "
# refresh trackNames + MBIDs on re-scan (populates items recorded before these
# columns); xmax=0 ⇒ this was an INSERT (a genuinely new library item) vs an UPDATE.
'ON CONFLICT (artist, album) DO UPDATE SET "trackNames" = EXCLUDED."trackNames", '
' "rgMbid" = COALESCE(EXCLUDED."rgMbid", "LibraryItem"."rgMbid"), '
' "artistMbid" = COALESCE(EXCLUDED."artistMbid", "LibraryItem"."artistMbid") '
"RETURNING (xmax = 0) AS inserted",
(target.artist, target.album, path, fmt, quality_class, track_names),
(target.artist, target.album, path, fmt, quality_class, track_names,
target.rg_mbid or None, target.artist_mbid or None),
)
newly = cur.fetchone()[0] # a new LibraryItem means this album wasn't recorded before
conn.commit()
@@ -134,6 +137,32 @@ def _process_album(conn, resolver, probe, browser, artist_name, album_folder, al
return "imported" if _record_owned(conn, target, pr.quality_class, pr.fmt, album_path, watched_id) else "recorded"
def _record_unmatched(conn, scan_id: str, artist: str, album: str, path: str, reason: str) -> None:
"""Persist an album folder the scan couldn't resolve, so /library can surface it.
Upsert by path; the current scan's id tags the row so stale ones can be pruned."""
with conn.cursor() as cur:
cur.execute(
'INSERT INTO "UnmatchedAlbum" (id, artist, album, path, reason, "scanId", "scannedAt") '
"VALUES (gen_random_uuid()::text, %s, %s, %s, %s, %s, now()) "
'ON CONFLICT (path) DO UPDATE SET artist = EXCLUDED.artist, album = EXCLUDED.album, '
'reason = EXCLUDED.reason, "scanId" = EXCLUDED."scanId", "scannedAt" = now()',
(artist or "", album or "", path, reason, scan_id),
)
def _clear_unmatched(conn, path: str) -> None:
"""Drop an album's unmatched row once it resolves (matched, or fixed + re-scanned)."""
with conn.cursor() as cur:
cur.execute('DELETE FROM "UnmatchedAlbum" WHERE path = %s', (path,))
def _prune_unmatched(conn, scan_id: str) -> None:
"""Once a scan completes, drop unmatched rows it did not touch — a prior scan's rows,
or folders removed/fixed since — so the list reflects only the latest scan."""
with conn.cursor() as cur:
cur.execute('DELETE FROM "UnmatchedAlbum" WHERE "scanId" <> %s', (scan_id,))
def build_worklist(conn: psycopg.Connection, scan_id: str, dest_root: str = "/music") -> int:
"""Walk the tree once and persist this scan's album worklist. Idempotent
(ON CONFLICT DO NOTHING), so a re-run after a crash adds only missing rows.
@@ -169,6 +198,7 @@ def scan_chunk(conn: psycopg.Connection, resolver, probe: AudioProbe, browser,
imported = skipped = 0
browsed: set[str] = set() # artists whose discography we've populated this chunk
for item_id, artist_name, album_folder, album_path in items:
reason = "no MusicBrainz match"
try:
outcome = _process_album(conn, resolver, probe, browser, artist_name, album_folder, album_path, browsed)
except Exception as e:
@@ -179,11 +209,15 @@ def scan_chunk(conn: psycopg.Connection, resolver, probe: AudioProbe, browser,
except Exception:
pass
print(f"scan: skipping {artist_name}/{album_folder}: {e}", flush=True)
reason = f"unreadable: {e}"
outcome = "skipped"
if outcome == "imported":
imported += 1
elif outcome == "skipped":
if outcome == "skipped":
skipped += 1
_record_unmatched(conn, scan_id, artist_name, album_folder, album_path, reason)
elif outcome in ("imported", "recorded"):
_clear_unmatched(conn, album_path) # it resolved — no longer unmatched
with conn.cursor() as cur:
cur.execute('UPDATE "ScanWorkItem" SET done = true WHERE id = %s', (item_id,))
conn.commit()
@@ -191,7 +225,11 @@ def scan_chunk(conn: psycopg.Connection, resolver, probe: AudioProbe, browser,
with conn.cursor() as cur:
cur.execute('SELECT count(*) FROM "ScanWorkItem" WHERE "scanId" = %s AND NOT done', (scan_id,))
remaining = cur.fetchone()[0]
return imported, skipped, remaining == 0
done = remaining == 0
if done:
_prune_unmatched(conn, scan_id)
conn.commit()
return imported, skipped, done
def clear_worklist(conn: psycopg.Connection, scan_id: str) -> None:
worker/lyra_worker/similarity/_lastfm.py
+1
View File
@@ -31,6 +31,7 @@ class LastfmSource:
params={"method": method, "api_key": self._key, "format": "json", **params},
timeout=self._timeout,
)
res.raise_for_status() # a 4xx/5xx (rate limit, auth) → RequestException, handled by callers
return res.json()
def health(self) -> bool:

Some files were not shown because too many files have changed in this diff Show More