feat(worker): separate download staging volume (STAGING_DIR)

Staging is no longer forced under the library. The worker reads STAGING_ROOT
(container path, default /music/.staging) and stages per-job downloads there;
docker-compose mounts ${STAGING_DIR:-${MUSIC_DIR}/.staging} at /staging with
STAGING_ROOT=/staging. Set STAGING_DIR to a fast local disk when the library
(MUSIC_DIR) is a network share so temp download I/O stays off the share.

Safe cross-volume: import_album already assembles into a temp dir on the
library volume and swaps atomically there, so partial downloads never touch
the library and the final swap stays atomic. clear_staging_root now clears the
root's contents (mount-safe) rather than removing the root. Documented in
.env.example; new tests cover the separate-root path.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
Jonathan
2026-07-12 23:44:27 +02:00
parent 891793953f
commit e825e3d7ad
7 changed files with 84 additions and 16 deletions
+20 -6
View File
@@ -26,14 +26,28 @@ _COVER_STEMS = {"cover", "folder", "front"}
_COVER_EXT = {".jpg", ".jpeg", ".png"}
def staging_dir(dest_root: str, job_id: str) -> str:
"""A per-job download staging directory, on the same filesystem as the library."""
return f"{dest_root}/.staging/{job_id}"
def staging_dir(staging_root: str, job_id: str) -> str:
"""A per-job download staging directory under `staging_root`. The staging root may be a
volume separate from the library (e.g. local disk while the library is an SMB share):
import_album assembles into a temp dir *on the library volume* and swaps atomically there,
so cross-volume staging stays safe."""
return f"{staging_root}/{job_id}"
def clear_staging_root(dest_root: str) -> None:
"""Remove every staging dir (called at worker startup to sweep crash orphans)."""
shutil.rmtree(f"{dest_root}/.staging", ignore_errors=True)
def clear_staging_root(staging_root: str) -> None:
"""Remove every per-job staging dir under `staging_root` (worker-startup crash-orphan sweep).
Clears the contents but not the root itself, so a mounted staging volume stays mounted."""
if not os.path.isdir(staging_root):
return
for name in os.listdir(staging_root):
path = os.path.join(staging_root, name)
if os.path.isdir(path):
shutil.rmtree(path, ignore_errors=True)
else:
try:
os.remove(path)
except OSError:
pass
def _find_cover(staging: str) -> str | None: