docs: add configuration & secrets plan (slice 1, plan 3a)
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -0,0 +1,706 @@
|
||||
# Lyra Configuration & Secrets Implementation Plan
|
||||
|
||||
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
|
||||
|
||||
**Goal:** Give Lyra an encrypted key/value configuration store so the upcoming real source adapters can read credentials (Qobuz login, slskd URL + API key) that the user enters through a settings page — secrets encrypted at rest and shared across the Node web app (which writes them) and the Python worker (which reads them).
|
||||
|
||||
**Architecture:** A Prisma-owned `Config` table holds `key → value` rows, with a `secret` flag marking values stored as an AES-256-GCM envelope. Both runtimes share one 32-byte key from the `LYRA_SECRET_KEY` env var and one envelope format (`{iv, ct, tag}`, all base64). The Next.js settings page posts credentials to `PUT /api/config`, which encrypts secret fields before upserting; `GET /api/config` never returns secret values, only whether each is set. The worker's `config.get_config(conn)` reads and decrypts everything for the adapters.
|
||||
|
||||
**Tech Stack:** Prisma 6 / PostgreSQL 17; Next.js App Router + Node `crypto` (AES-256-GCM); Python 3.12 + the `cryptography` package (`AESGCM`). Tests: Vitest (web), pytest (worker), including a cross-language test proving the Python worker decrypts a Node-produced envelope.
|
||||
|
||||
## Global Constraints
|
||||
|
||||
- Prisma owns the schema; the worker uses raw psycopg v3 SQL only (no DDL).
|
||||
- **Crypto contract (both runtimes MUST match exactly):** AES-256-GCM. Key = 32 raw bytes decoded from base64 env var `LYRA_SECRET_KEY`. Nonce/IV = 12 random bytes per encryption. Envelope = a JSON string `{"iv": b64, "ct": b64, "tag": b64}` where `ct` is the ciphertext WITHOUT the tag and `tag` is the 16-byte GCM tag. (Node keeps the tag separate via `getAuthTag()`; Python's `AESGCM` appends the tag to ciphertext, so Python code must split/join the last 16 bytes — shown in the tasks.)
|
||||
- Secrets NEVER leave the backend in plaintext: `GET /api/config` returns booleans (`*Set`) for secret fields, never their values.
|
||||
- Config keys (fixed): `qobuz.email` (plain), `qobuz.password` (secret), `slskd.url` (plain), `slskd.api_key` (secret). Later adapter plans read these.
|
||||
- `LYRA_SECRET_KEY` is provided to BOTH the `web` and `worker` services via environment.
|
||||
- Every task ends with a commit.
|
||||
|
||||
## Shared interfaces (defined by tasks below; listed for consistency)
|
||||
|
||||
```
|
||||
web/src/lib/crypto.ts (Task 2): encryptSecret(plaintext: string): string ; decryptSecret(envelope: string): string
|
||||
web/src/app/api/config/route.ts (Task 3): GET -> masked config ; PUT (partial settings) -> upsert (encrypt secrets)
|
||||
worker/lyra_worker/crypto.py (Task 5): encrypt_secret(plaintext: str) -> str ; decrypt_secret(envelope: str) -> str
|
||||
worker/lyra_worker/config.py (Task 6): get_config(conn) -> dict[str, str] # secrets decrypted
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Task 1: Config schema + migration
|
||||
|
||||
**Files:**
|
||||
- Modify: `web/prisma/schema.prisma`
|
||||
|
||||
**Interfaces:**
|
||||
- Produces: a `Config` table — `key String @id`, `value String`, `secret Boolean @default(false)`, `updatedAt DateTime @updatedAt`.
|
||||
|
||||
- [ ] **Step 1: Add the model**
|
||||
|
||||
Append to `web/prisma/schema.prisma`:
|
||||
```prisma
|
||||
model Config {
|
||||
key String @id
|
||||
value String
|
||||
secret Boolean @default(false)
|
||||
updatedAt DateTime @updatedAt
|
||||
}
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Create and apply the migration**
|
||||
|
||||
Run (Postgres up via `docker compose up -d db`):
|
||||
```bash
|
||||
cd web
|
||||
export DATABASE_URL=postgresql://lyra:lyra@localhost:5432/lyra
|
||||
npx prisma migrate dev --name add_config
|
||||
```
|
||||
Expected: migration created under `web/prisma/migrations/` and applied; "Your database is now in sync with your schema."
|
||||
|
||||
- [ ] **Step 3: Verify the table exists**
|
||||
|
||||
Run:
|
||||
```bash
|
||||
cd web
|
||||
export DATABASE_URL=postgresql://lyra:lyra@localhost:5432/lyra
|
||||
npx prisma db execute --stdin <<'SQL'
|
||||
SELECT 1 FROM "Config" LIMIT 0;
|
||||
SQL
|
||||
echo "table OK"
|
||||
```
|
||||
Expected: no error; prints `table OK`.
|
||||
|
||||
- [ ] **Step 4: Commit**
|
||||
|
||||
```bash
|
||||
git add web/prisma/schema.prisma web/prisma/migrations
|
||||
git commit -m "feat: add Config schema"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Task 2: Web crypto util + secret-key plumbing
|
||||
|
||||
**Files:**
|
||||
- Create: `web/src/lib/crypto.ts`
|
||||
- Test: `web/src/lib/crypto.test.ts`
|
||||
- Modify: `.env.example`
|
||||
- Modify: `docker-compose.yml`
|
||||
|
||||
**Interfaces:**
|
||||
- Consumes: `LYRA_SECRET_KEY` env var.
|
||||
- Produces: `encryptSecret(plaintext: string): string` and `decryptSecret(envelope: string): string` implementing the Global-Constraints crypto contract.
|
||||
|
||||
- [ ] **Step 1: Add LYRA_SECRET_KEY to env + compose**
|
||||
|
||||
In `.env.example`, add a line (with a documented generator):
|
||||
```
|
||||
# 32 random bytes, base64. Generate with: openssl rand -base64 32
|
||||
LYRA_SECRET_KEY=MDEyMzQ1Njc4OWFiY2RlZjAxMjM0NTY3ODlhYmNkZWY=
|
||||
```
|
||||
In `docker-compose.yml`, add `LYRA_SECRET_KEY: ${LYRA_SECRET_KEY}` to the `environment:` block of BOTH the `web` and `worker` services (next to `DATABASE_URL`).
|
||||
|
||||
- [ ] **Step 2: Write the failing test**
|
||||
|
||||
`web/src/lib/crypto.test.ts`:
|
||||
```ts
|
||||
import { describe, it, expect, beforeAll } from "vitest";
|
||||
import { encryptSecret, decryptSecret } from "./crypto";
|
||||
|
||||
beforeAll(() => {
|
||||
// base64 of the 32-byte ASCII string "0123456789abcdef0123456789abcdef"
|
||||
process.env.LYRA_SECRET_KEY = "MDEyMzQ1Njc4OWFiY2RlZjAxMjM0NTY3ODlhYmNkZWY=";
|
||||
});
|
||||
|
||||
describe("crypto", () => {
|
||||
it("round-trips a secret", () => {
|
||||
const env = encryptSecret("hunter2");
|
||||
expect(env).not.toContain("hunter2");
|
||||
expect(decryptSecret(env)).toBe("hunter2");
|
||||
});
|
||||
|
||||
it("produces a well-formed envelope", () => {
|
||||
const env = JSON.parse(encryptSecret("x"));
|
||||
expect(typeof env.iv).toBe("string");
|
||||
expect(typeof env.ct).toBe("string");
|
||||
expect(typeof env.tag).toBe("string");
|
||||
});
|
||||
|
||||
it("rejects a tampered envelope", () => {
|
||||
const env = JSON.parse(encryptSecret("secret"));
|
||||
env.ct = Buffer.from("garbage").toString("base64");
|
||||
expect(() => decryptSecret(JSON.stringify(env))).toThrow();
|
||||
});
|
||||
});
|
||||
```
|
||||
|
||||
- [ ] **Step 3: Run the test to verify it fails**
|
||||
|
||||
Run: `cd web && npx vitest run src/lib/crypto.test.ts`
|
||||
Expected: FAIL — cannot import `./crypto`.
|
||||
|
||||
- [ ] **Step 4: Implement the crypto util**
|
||||
|
||||
`web/src/lib/crypto.ts`:
|
||||
```ts
|
||||
import { createCipheriv, createDecipheriv, randomBytes } from "node:crypto";
|
||||
|
||||
function key(): Buffer {
|
||||
const b64 = process.env.LYRA_SECRET_KEY;
|
||||
if (!b64) throw new Error("LYRA_SECRET_KEY is not set");
|
||||
const k = Buffer.from(b64, "base64");
|
||||
if (k.length !== 32) throw new Error("LYRA_SECRET_KEY must decode to 32 bytes");
|
||||
return k;
|
||||
}
|
||||
|
||||
export function encryptSecret(plaintext: string): string {
|
||||
const iv = randomBytes(12);
|
||||
const cipher = createCipheriv("aes-256-gcm", key(), iv);
|
||||
const ct = Buffer.concat([cipher.update(plaintext, "utf8"), cipher.final()]);
|
||||
const tag = cipher.getAuthTag();
|
||||
return JSON.stringify({
|
||||
iv: iv.toString("base64"),
|
||||
ct: ct.toString("base64"),
|
||||
tag: tag.toString("base64"),
|
||||
});
|
||||
}
|
||||
|
||||
export function decryptSecret(envelope: string): string {
|
||||
const { iv, ct, tag } = JSON.parse(envelope) as { iv: string; ct: string; tag: string };
|
||||
const decipher = createDecipheriv("aes-256-gcm", key(), Buffer.from(iv, "base64"));
|
||||
decipher.setAuthTag(Buffer.from(tag, "base64"));
|
||||
const pt = Buffer.concat([decipher.update(Buffer.from(ct, "base64")), decipher.final()]);
|
||||
return pt.toString("utf8");
|
||||
}
|
||||
```
|
||||
|
||||
- [ ] **Step 5: Run the test to verify it passes**
|
||||
|
||||
Run: `cd web && npx vitest run src/lib/crypto.test.ts`
|
||||
Expected: PASS — all three tests green.
|
||||
|
||||
- [ ] **Step 6: Commit**
|
||||
|
||||
```bash
|
||||
git add web/src/lib/crypto.ts web/src/lib/crypto.test.ts .env.example docker-compose.yml
|
||||
git commit -m "feat: add web AES-256-GCM secret crypto and LYRA_SECRET_KEY plumbing"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Task 3: Config API (get masked / put encrypted)
|
||||
|
||||
**Files:**
|
||||
- Create: `web/src/app/api/config/route.ts`
|
||||
- Test: `web/src/app/api/config/route.test.ts`
|
||||
|
||||
**Interfaces:**
|
||||
- Consumes: `prisma` (`@/lib/db`), `encryptSecret` (Task 2).
|
||||
- Produces:
|
||||
- `PUT /api/config` — body may include any of `{ qobuzEmail, qobuzPassword, slskdUrl, slskdApiKey }` (strings). Each provided field is upserted into `Config`: plain fields (`qobuz.email`→`qobuzEmail`, `slskd.url`→`slskdUrl`) stored as-is with `secret=false`; secret fields (`qobuz.password`→`qobuzPassword`, `slskd.api_key`→`slskdApiKey`) stored `encryptSecret(value)` with `secret=true`. An empty string for a field means "leave unchanged" (skip it). Responds `200 { ok: true }`.
|
||||
- `GET /api/config` — responds `200 { qobuzEmail: string, slskdUrl: string, qobuzPasswordSet: boolean, slskdApiKeySet: boolean }` (secret values never returned; `*Set` is whether a row exists for that key).
|
||||
|
||||
- [ ] **Step 1: Write the failing test**
|
||||
|
||||
`web/src/app/api/config/route.test.ts`:
|
||||
```ts
|
||||
import { describe, it, expect, beforeAll } from "vitest";
|
||||
import { GET, PUT } from "./route";
|
||||
import { prisma } from "@/lib/db";
|
||||
|
||||
beforeAll(() => {
|
||||
process.env.LYRA_SECRET_KEY = "MDEyMzQ1Njc4OWFiY2RlZjAxMjM0NTY3ODlhYmNkZWY=";
|
||||
});
|
||||
|
||||
function putReq(body: unknown) {
|
||||
return new Request("http://localhost/api/config", {
|
||||
method: "PUT",
|
||||
headers: { "content-type": "application/json" },
|
||||
body: JSON.stringify(body),
|
||||
});
|
||||
}
|
||||
|
||||
describe("config API", () => {
|
||||
it("stores plain fields verbatim and secret fields encrypted", async () => {
|
||||
await PUT(putReq({ qobuzEmail: "me@example.com", qobuzPassword: "hunter2", slskdUrl: "http://slskd:5030", slskdApiKey: "abc" }));
|
||||
|
||||
const email = await prisma.config.findUnique({ where: { key: "qobuz.email" } });
|
||||
const pw = await prisma.config.findUnique({ where: { key: "qobuz.password" } });
|
||||
expect(email!.value).toBe("me@example.com");
|
||||
expect(email!.secret).toBe(false);
|
||||
expect(pw!.secret).toBe(true);
|
||||
expect(pw!.value).not.toContain("hunter2"); // encrypted envelope
|
||||
});
|
||||
|
||||
it("GET masks secrets and reports set-ness", async () => {
|
||||
await PUT(putReq({ qobuzEmail: "me@example.com", qobuzPassword: "hunter2" }));
|
||||
const res = await GET();
|
||||
const body = await res.json();
|
||||
expect(body.qobuzEmail).toBe("me@example.com");
|
||||
expect(body.qobuzPasswordSet).toBe(true);
|
||||
expect(body.slskdApiKeySet).toBe(false);
|
||||
expect(JSON.stringify(body)).not.toContain("hunter2");
|
||||
});
|
||||
|
||||
it("empty string leaves a field unchanged", async () => {
|
||||
await PUT(putReq({ qobuzPassword: "first" }));
|
||||
await PUT(putReq({ qobuzPassword: "" }));
|
||||
const pw = await prisma.config.findUnique({ where: { key: "qobuz.password" } });
|
||||
expect(pw).not.toBeNull(); // still present, not wiped
|
||||
});
|
||||
});
|
||||
```
|
||||
|
||||
Add to `web/src/test/setup.ts`'s `beforeEach` cleanup (it currently deletes Job/Request) a line to also clear Config:
|
||||
```ts
|
||||
await prisma.config.deleteMany();
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Run the test to verify it fails**
|
||||
|
||||
Run:
|
||||
```bash
|
||||
cd web
|
||||
export DATABASE_URL=postgresql://lyra:lyra@localhost:5432/lyra
|
||||
npx vitest run src/app/api/config/route.test.ts
|
||||
```
|
||||
Expected: FAIL — cannot import `./route`.
|
||||
|
||||
- [ ] **Step 3: Implement the route**
|
||||
|
||||
`web/src/app/api/config/route.ts`:
|
||||
```ts
|
||||
import { prisma } from "@/lib/db";
|
||||
import { encryptSecret } from "@/lib/crypto";
|
||||
|
||||
// settings field -> (config key, is secret)
|
||||
const FIELDS: Record<string, { key: string; secret: boolean }> = {
|
||||
qobuzEmail: { key: "qobuz.email", secret: false },
|
||||
qobuzPassword: { key: "qobuz.password", secret: true },
|
||||
slskdUrl: { key: "slskd.url", secret: false },
|
||||
slskdApiKey: { key: "slskd.api_key", secret: true },
|
||||
};
|
||||
|
||||
export async function PUT(request: Request) {
|
||||
let body: Record<string, unknown>;
|
||||
try {
|
||||
body = (await request.json()) as Record<string, unknown>;
|
||||
} catch {
|
||||
return Response.json({ error: "invalid JSON" }, { status: 400 });
|
||||
}
|
||||
|
||||
for (const [field, { key, secret }] of Object.entries(FIELDS)) {
|
||||
const raw = body[field];
|
||||
if (typeof raw !== "string" || raw === "") continue; // absent or "unchanged"
|
||||
const value = secret ? encryptSecret(raw) : raw;
|
||||
await prisma.config.upsert({
|
||||
where: { key },
|
||||
create: { key, value, secret },
|
||||
update: { value, secret },
|
||||
});
|
||||
}
|
||||
|
||||
return Response.json({ ok: true });
|
||||
}
|
||||
|
||||
export async function GET() {
|
||||
const rows = await prisma.config.findMany();
|
||||
const byKey = new Map(rows.map((r) => [r.key, r]));
|
||||
return Response.json({
|
||||
qobuzEmail: byKey.get("qobuz.email")?.value ?? "",
|
||||
slskdUrl: byKey.get("slskd.url")?.value ?? "",
|
||||
qobuzPasswordSet: byKey.has("qobuz.password"),
|
||||
slskdApiKeySet: byKey.has("slskd.api_key"),
|
||||
});
|
||||
}
|
||||
```
|
||||
|
||||
- [ ] **Step 4: Run the test to verify it passes**
|
||||
|
||||
Run:
|
||||
```bash
|
||||
cd web
|
||||
export DATABASE_URL=postgresql://lyra:lyra@localhost:5432/lyra
|
||||
npx vitest run src/app/api/config/route.test.ts
|
||||
```
|
||||
Expected: PASS — all three tests green.
|
||||
|
||||
- [ ] **Step 5: Run the full web suite**
|
||||
|
||||
Run: `cd web && export DATABASE_URL=postgresql://lyra:lyra@localhost:5432/lyra && npx vitest run`
|
||||
Expected: PASS — config + requests suites all green.
|
||||
|
||||
- [ ] **Step 6: Commit**
|
||||
|
||||
```bash
|
||||
git add web/src/app/api/config web/src/test/setup.ts
|
||||
git commit -m "feat: add config API with encrypted secret storage"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Task 4: Settings page
|
||||
|
||||
**Files:**
|
||||
- Create: `web/src/app/settings/page.tsx`
|
||||
- Create: `web/src/app/settings/settings-form.tsx`
|
||||
- Modify: `web/src/app/page.tsx` (add a link to settings)
|
||||
|
||||
**Interfaces:**
|
||||
- Consumes: `GET`/`PUT /api/config` (Task 3).
|
||||
- Produces: a `/settings` page with a client form pre-filled from `GET /api/config` (plain fields shown; secret fields show a "•••• set" placeholder when `*Set` is true and are left blank otherwise), submitting changed fields via `PUT`.
|
||||
|
||||
- [ ] **Step 1: Write the settings form client component**
|
||||
|
||||
`web/src/app/settings/settings-form.tsx`:
|
||||
```tsx
|
||||
"use client";
|
||||
|
||||
import { useEffect, useState } from "react";
|
||||
|
||||
type Loaded = { qobuzEmail: string; slskdUrl: string; qobuzPasswordSet: boolean; slskdApiKeySet: boolean };
|
||||
|
||||
export function SettingsForm() {
|
||||
const [qobuzEmail, setQobuzEmail] = useState("");
|
||||
const [qobuzPassword, setQobuzPassword] = useState("");
|
||||
const [slskdUrl, setSlskdUrl] = useState("");
|
||||
const [slskdApiKey, setSlskdApiKey] = useState("");
|
||||
const [pwSet, setPwSet] = useState(false);
|
||||
const [keySet, setKeySet] = useState(false);
|
||||
const [saved, setSaved] = useState(false);
|
||||
|
||||
useEffect(() => {
|
||||
fetch("/api/config")
|
||||
.then((r) => r.json())
|
||||
.then((c: Loaded) => {
|
||||
setQobuzEmail(c.qobuzEmail);
|
||||
setSlskdUrl(c.slskdUrl);
|
||||
setPwSet(c.qobuzPasswordSet);
|
||||
setKeySet(c.slskdApiKeySet);
|
||||
});
|
||||
}, []);
|
||||
|
||||
async function submit(e: React.FormEvent) {
|
||||
e.preventDefault();
|
||||
await fetch("/api/config", {
|
||||
method: "PUT",
|
||||
headers: { "content-type": "application/json" },
|
||||
body: JSON.stringify({ qobuzEmail, qobuzPassword, slskdUrl, slskdApiKey }),
|
||||
});
|
||||
setQobuzPassword("");
|
||||
setSlskdApiKey("");
|
||||
setSaved(true);
|
||||
setPwSet(pwSet || qobuzPassword !== "");
|
||||
setKeySet(keySet || slskdApiKey !== "");
|
||||
}
|
||||
|
||||
return (
|
||||
<form onSubmit={submit}>
|
||||
<label>Qobuz email <input aria-label="qobuz email" value={qobuzEmail} onChange={(e) => setQobuzEmail(e.target.value)} /></label>
|
||||
<label>Qobuz password <input aria-label="qobuz password" type="password" placeholder={pwSet ? "•••• (set)" : ""} value={qobuzPassword} onChange={(e) => setQobuzPassword(e.target.value)} /></label>
|
||||
<label>slskd URL <input aria-label="slskd url" value={slskdUrl} onChange={(e) => setSlskdUrl(e.target.value)} /></label>
|
||||
<label>slskd API key <input aria-label="slskd api key" type="password" placeholder={keySet ? "•••• (set)" : ""} value={slskdApiKey} onChange={(e) => setSlskdApiKey(e.target.value)} /></label>
|
||||
<button type="submit">Save</button>
|
||||
{saved ? <span> Saved.</span> : null}
|
||||
</form>
|
||||
);
|
||||
}
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Write the settings page**
|
||||
|
||||
`web/src/app/settings/page.tsx`:
|
||||
```tsx
|
||||
import { SettingsForm } from "./settings-form";
|
||||
|
||||
export default function SettingsPage() {
|
||||
return (
|
||||
<main>
|
||||
<h1>Settings</h1>
|
||||
<SettingsForm />
|
||||
<p><a href="/">← Back to queue</a></p>
|
||||
</main>
|
||||
);
|
||||
}
|
||||
```
|
||||
|
||||
- [ ] **Step 3: Add a settings link to the home page**
|
||||
|
||||
In `web/src/app/page.tsx`, add a link above `<Queue />` (inside `<main>`, after `<h1>Lyra</h1>`):
|
||||
```tsx
|
||||
<p><a href="/settings">Settings</a></p>
|
||||
```
|
||||
|
||||
- [ ] **Step 4: Verify the app builds**
|
||||
|
||||
Run: `cd web && npm run build`
|
||||
Expected: build completes; `/settings` compiled as a route, no type errors.
|
||||
|
||||
- [ ] **Step 5: Commit**
|
||||
|
||||
```bash
|
||||
git add web/src/app/settings web/src/app/page.tsx
|
||||
git commit -m "feat: add settings page for credentials"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Task 5: Worker crypto util (cross-language)
|
||||
|
||||
**Files:**
|
||||
- Create: `worker/lyra_worker/crypto.py`
|
||||
- Test: `worker/tests/test_crypto.py`
|
||||
- Modify: `worker/requirements.txt` (add `cryptography`)
|
||||
|
||||
**Interfaces:**
|
||||
- Consumes: `LYRA_SECRET_KEY` env var.
|
||||
- Produces: `encrypt_secret(plaintext: str) -> str` and `decrypt_secret(envelope: str) -> str` using the SAME envelope format as `web/src/lib/crypto.ts`, so the worker can decrypt what the web app encrypted.
|
||||
|
||||
- [ ] **Step 1: Add the dependency**
|
||||
|
||||
Append to `worker/requirements.txt`:
|
||||
```
|
||||
cryptography>=42,<46
|
||||
```
|
||||
Then install into the venv: `worker/.venv/bin/pip install -r worker/requirements.txt`
|
||||
|
||||
- [ ] **Step 2: Generate the cross-language fixture**
|
||||
|
||||
The Python test must prove it can decrypt an envelope produced by Node using the SAME algorithm/contract as `web/src/lib/crypto.ts`. Generate one with the FIXED test key via a throwaway plain-JS script (no tsx/TS execution needed — plain Node runs `.mjs`):
|
||||
```bash
|
||||
cat > /tmp/lyra-gen-fixture.mjs <<'EOF'
|
||||
import { createCipheriv, randomBytes } from "node:crypto";
|
||||
const key = Buffer.from("MDEyMzQ1Njc4OWFiY2RlZjAxMjM0NTY3ODlhYmNkZWY=", "base64");
|
||||
const iv = randomBytes(12);
|
||||
const cipher = createCipheriv("aes-256-gcm", key, iv);
|
||||
const ct = Buffer.concat([cipher.update("cross-lang-secret", "utf8"), cipher.final()]);
|
||||
const tag = cipher.getAuthTag();
|
||||
console.log(JSON.stringify({ iv: iv.toString("base64"), ct: ct.toString("base64"), tag: tag.toString("base64") }));
|
||||
EOF
|
||||
node /tmp/lyra-gen-fixture.mjs
|
||||
```
|
||||
This uses the identical AES-256-GCM `{iv,ct,tag}` contract as `crypto.ts`. Copy the printed JSON envelope — paste it into the test below as `NODE_FIXTURE` (a one-time captured value; it's deterministic to decrypt). Then `rm /tmp/lyra-gen-fixture.mjs`.
|
||||
|
||||
- [ ] **Step 3: Write the failing test**
|
||||
|
||||
`worker/tests/test_crypto.py`:
|
||||
```python
|
||||
import pytest
|
||||
|
||||
from lyra_worker.crypto import decrypt_secret, encrypt_secret
|
||||
|
||||
TEST_KEY = "MDEyMzQ1Njc4OWFiY2RlZjAxMjM0NTY3ODlhYmNkZWY=" # 32 bytes, base64
|
||||
# Produced by web/src/lib/crypto.ts encryptSecret("cross-lang-secret") with TEST_KEY (see Step 2).
|
||||
NODE_FIXTURE = "PASTE_THE_ENVELOPE_FROM_STEP_2_HERE"
|
||||
|
||||
|
||||
@pytest.fixture(autouse=True)
|
||||
def _key(monkeypatch):
|
||||
monkeypatch.setenv("LYRA_SECRET_KEY", TEST_KEY)
|
||||
|
||||
|
||||
def test_round_trip():
|
||||
env = encrypt_secret("hunter2")
|
||||
assert "hunter2" not in env
|
||||
assert decrypt_secret(env) == "hunter2"
|
||||
|
||||
|
||||
def test_decrypts_node_produced_envelope():
|
||||
# Cross-language contract: the worker must decrypt what the web app encrypted.
|
||||
assert decrypt_secret(NODE_FIXTURE) == "cross-lang-secret"
|
||||
|
||||
|
||||
def test_rejects_tampered_envelope():
|
||||
import json
|
||||
env = json.loads(encrypt_secret("secret"))
|
||||
env["ct"] = "Z2FyYmFnZQ==" # "garbage"
|
||||
with pytest.raises(Exception):
|
||||
decrypt_secret(json.dumps(env))
|
||||
```
|
||||
|
||||
- [ ] **Step 4: Run the test to verify it fails**
|
||||
|
||||
Run:
|
||||
```bash
|
||||
cd worker
|
||||
worker/.venv/bin/python -m pytest tests/test_crypto.py -v
|
||||
```
|
||||
Expected: FAIL — `ModuleNotFoundError: No module named 'lyra_worker.crypto'`.
|
||||
|
||||
- [ ] **Step 5: Implement the crypto util**
|
||||
|
||||
`worker/lyra_worker/crypto.py`:
|
||||
```python
|
||||
import base64
|
||||
import json
|
||||
import os
|
||||
|
||||
from cryptography.hazmat.primitives.ciphers.aead import AESGCM
|
||||
|
||||
|
||||
def _key() -> bytes:
|
||||
b64 = os.environ.get("LYRA_SECRET_KEY")
|
||||
if not b64:
|
||||
raise RuntimeError("LYRA_SECRET_KEY is not set")
|
||||
k = base64.b64decode(b64)
|
||||
if len(k) != 32:
|
||||
raise RuntimeError("LYRA_SECRET_KEY must decode to 32 bytes")
|
||||
return k
|
||||
|
||||
|
||||
def encrypt_secret(plaintext: str) -> str:
|
||||
iv = os.urandom(12)
|
||||
full = AESGCM(_key()).encrypt(iv, plaintext.encode("utf-8"), None) # ct || tag(16)
|
||||
ct, tag = full[:-16], full[-16:]
|
||||
return json.dumps(
|
||||
{
|
||||
"iv": base64.b64encode(iv).decode(),
|
||||
"ct": base64.b64encode(ct).decode(),
|
||||
"tag": base64.b64encode(tag).decode(),
|
||||
}
|
||||
)
|
||||
|
||||
|
||||
def decrypt_secret(envelope: str) -> str:
|
||||
e = json.loads(envelope)
|
||||
iv = base64.b64decode(e["iv"])
|
||||
ct = base64.b64decode(e["ct"])
|
||||
tag = base64.b64decode(e["tag"])
|
||||
pt = AESGCM(_key()).decrypt(iv, ct + tag, None) # AESGCM expects ct || tag
|
||||
return pt.decode("utf-8")
|
||||
```
|
||||
|
||||
- [ ] **Step 6: Run the test to verify it passes**
|
||||
|
||||
Run:
|
||||
```bash
|
||||
cd worker
|
||||
worker/.venv/bin/python -m pytest tests/test_crypto.py -v
|
||||
```
|
||||
Expected: PASS — round-trip, the Node-fixture decrypt, and the tamper-rejection all green. (If the Node-fixture test fails, the envelope formats disagree — re-check Step 2's output against the `{iv,ct,tag}` contract.)
|
||||
|
||||
- [ ] **Step 7: Commit**
|
||||
|
||||
```bash
|
||||
git add worker/lyra_worker/crypto.py worker/tests/test_crypto.py worker/requirements.txt
|
||||
git commit -m "feat: add worker AES-256-GCM crypto matching the web envelope"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Task 6: Worker config reader
|
||||
|
||||
**Files:**
|
||||
- Create: `worker/lyra_worker/config.py`
|
||||
- Test: `worker/tests/test_config.py`
|
||||
|
||||
**Interfaces:**
|
||||
- Consumes: the `Config` table, `decrypt_secret` (Task 5), the `conn` fixture.
|
||||
- Produces: `get_config(conn) -> dict[str, str]` — reads all `Config` rows, returning `{key: value}` where rows with `secret=true` have their `value` decrypted via `decrypt_secret`. Missing keys are simply absent.
|
||||
|
||||
- [ ] **Step 1: Write the failing test**
|
||||
|
||||
`worker/tests/test_config.py`:
|
||||
```python
|
||||
import pytest
|
||||
|
||||
from lyra_worker.config import get_config
|
||||
from lyra_worker.crypto import encrypt_secret
|
||||
|
||||
TEST_KEY = "MDEyMzQ1Njc4OWFiY2RlZjAxMjM0NTY3ODlhYmNkZWY="
|
||||
|
||||
|
||||
@pytest.fixture(autouse=True)
|
||||
def _key(monkeypatch):
|
||||
monkeypatch.setenv("LYRA_SECRET_KEY", TEST_KEY)
|
||||
|
||||
|
||||
def _put(conn, key, value, secret):
|
||||
with conn.cursor() as cur:
|
||||
cur.execute(
|
||||
'INSERT INTO "Config" (key, value, secret, "updatedAt") VALUES (%s, %s, %s, now()) '
|
||||
'ON CONFLICT (key) DO UPDATE SET value = EXCLUDED.value, secret = EXCLUDED.secret',
|
||||
(key, value, secret),
|
||||
)
|
||||
conn.commit()
|
||||
|
||||
|
||||
def test_reads_plain_and_decrypts_secret(conn):
|
||||
_put(conn, "slskd.url", "http://slskd:5030", False)
|
||||
_put(conn, "slskd.api_key", encrypt_secret("topsecret"), True)
|
||||
|
||||
cfg = get_config(conn)
|
||||
assert cfg["slskd.url"] == "http://slskd:5030"
|
||||
assert cfg["slskd.api_key"] == "topsecret" # decrypted
|
||||
|
||||
|
||||
def test_missing_keys_absent(conn):
|
||||
cfg = get_config(conn)
|
||||
assert "qobuz.password" not in cfg
|
||||
```
|
||||
|
||||
The `conn` fixture cleans Job/Request but not Config. Add Config cleanup: in `worker/tests/conftest.py`, in the `conn` fixture's pre-`yield` cleanup AND post-`yield` teardown, add `cur.execute('DELETE FROM "Config"')` alongside the existing deletes.
|
||||
|
||||
- [ ] **Step 2: Run the test to verify it fails**
|
||||
|
||||
Run:
|
||||
```bash
|
||||
cd worker
|
||||
export DATABASE_URL=postgresql://lyra:lyra@localhost:5432/lyra
|
||||
worker/.venv/bin/python -m pytest tests/test_config.py -v
|
||||
```
|
||||
Expected: FAIL — `ModuleNotFoundError: No module named 'lyra_worker.config'`.
|
||||
|
||||
- [ ] **Step 3: Implement the config reader**
|
||||
|
||||
`worker/lyra_worker/config.py`:
|
||||
```python
|
||||
import psycopg
|
||||
|
||||
from lyra_worker.crypto import decrypt_secret
|
||||
|
||||
|
||||
def get_config(conn: psycopg.Connection) -> dict[str, str]:
|
||||
"""Return all config as {key: value}, decrypting rows marked secret."""
|
||||
with conn.cursor() as cur:
|
||||
cur.execute('SELECT key, value, secret FROM "Config"')
|
||||
rows = cur.fetchall()
|
||||
out: dict[str, str] = {}
|
||||
for key, value, secret in rows:
|
||||
out[key] = decrypt_secret(value) if secret else value
|
||||
return out
|
||||
```
|
||||
|
||||
- [ ] **Step 4: Run the test to verify it passes**
|
||||
|
||||
Run:
|
||||
```bash
|
||||
cd worker
|
||||
export DATABASE_URL=postgresql://lyra:lyra@localhost:5432/lyra
|
||||
worker/.venv/bin/python -m pytest tests/test_config.py -v
|
||||
```
|
||||
Expected: PASS — both tests green.
|
||||
|
||||
- [ ] **Step 5: Run the full worker suite**
|
||||
|
||||
Run:
|
||||
```bash
|
||||
cd worker
|
||||
export DATABASE_URL=postgresql://lyra:lyra@localhost:5432/lyra
|
||||
worker/.venv/bin/python -m pytest -v
|
||||
```
|
||||
Expected: PASS — every suite green (config, crypto, and the existing pipeline/adapter/etc. suites), output pristine.
|
||||
|
||||
- [ ] **Step 6: Commit**
|
||||
|
||||
```bash
|
||||
git add worker/lyra_worker/config.py worker/tests/test_config.py worker/tests/conftest.py
|
||||
git commit -m "feat: add worker config reader with secret decryption"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Notes for the next plans (adapters)
|
||||
|
||||
Real adapters (Plans 3b–3d) call `get_config(conn)` (or receive the resolved config) to obtain `qobuz.email`/`qobuz.password`, `slskd.url`/`slskd.api_key`, etc., and construct themselves accordingly. `registry.build_adapters()` will read config and include an adapter only when its required keys are present (using each adapter's `health()`), so an unconfigured source is simply skipped rather than failing the pipeline. `LYRA_SECRET_KEY` must be set in the environment of both services (already plumbed here) — document generating a real one with `openssl rand -base64 32` for deployment.
|
||||
Reference in New Issue
Block a user