feat(mail): IDLE-daemon for real-time Legend mail-protection

Standalone ESM-daemon that:
- Connects via ImapFlow IDLE to all active Legend mailboxes
- Triggers /api/mail/scan-internal on new-mail events (real-time)
- Auto-renew IDLE every 25min (RFC 3501 limit), exponential-backoff reconnect
- DB-refresh every 5min for new/removed connections

Plus deploy-pipeline:
- GH-Actions artifact-upload + scp to /srv/rebreak/backend/imap-idle/
- npm install --production on server (imapflow + pg)
- pm2 startOrReload via ecosystem.config.js
- start-idle-staging.sh wrapper for Infisical secret-injection

Replaces 30min-cron polling for Legend tier -- Casino-mails now blocked
within seconds, fulfilling Legend tier marketing promise.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
chahinebrini 2026-05-09 20:48:33 +02:00
parent e5c9fadd1d
commit de701677b2
7 changed files with 513 additions and 1 deletions

View File

@ -67,6 +67,13 @@ jobs:
path: backend-output.tar.gz path: backend-output.tar.gz
retention-days: 7 retention-days: 7
- name: Upload imap-idle artifact
uses: actions/upload-artifact@v4
with:
name: imap-idle
path: backend/imap-idle/
retention-days: 7
# ── 2. Deploy: Artifact zum Hetzner pushen + extract + pm2 restart ────────── # ── 2. Deploy: Artifact zum Hetzner pushen + extract + pm2 restart ──────────
deploy: deploy:
name: Deploy zu Hetzner name: Deploy zu Hetzner
@ -79,6 +86,12 @@ jobs:
with: with:
name: backend-output name: backend-output
- name: Download imap-idle artifact
uses: actions/download-artifact@v4
with:
name: imap-idle
path: imap-idle/
- name: Setup SSH - name: Setup SSH
env: env:
SSH_PRIVATE_KEY: ${{ secrets.HETZNER_SSH_KEY }} SSH_PRIVATE_KEY: ${{ secrets.HETZNER_SSH_KEY }}
@ -102,6 +115,14 @@ jobs:
scp -i ~/.ssh/id_ed25519 backend-output.tar.gz \ scp -i ~/.ssh/id_ed25519 backend-output.tar.gz \
"$SSH_USER@$SSH_HOST:/srv/rebreak/backend/.output-incoming.tar.gz" "$SSH_USER@$SSH_HOST:/srv/rebreak/backend/.output-incoming.tar.gz"
- name: Upload imap-idle zu Hetzner
env:
SSH_HOST: ${{ vars.HETZNER_HOST }}
SSH_USER: ${{ vars.HETZNER_USER }}
run: |
scp -r -i ~/.ssh/id_ed25519 imap-idle/ \
"$SSH_USER@$SSH_HOST:/srv/rebreak/backend/imap-idle/"
- name: Server-side deploy (extract + migrate + pm2 restart) - name: Server-side deploy (extract + migrate + pm2 restart)
env: env:
SSH_HOST: ${{ vars.HETZNER_HOST }} SSH_HOST: ${{ vars.HETZNER_HOST }}

View File

@ -0,0 +1,51 @@
# rebreak-imap-idle
Standalone IMAP IDLE Daemon für Rebreak.
## Was er macht
- Hält pro aktiver `MailConnection` (DB) eine persistente IMAP-IDLE-Session
- Reagiert in Echtzeit auf `EXISTS`-Events (neue Mail im Postfach)
- Feuert bei jedem Event `POST /api/mail/scan-internal` gegen das lokale Backend
- Das Backend entscheidet ob und welche Mails gelöscht werden (Gambling-Keywords + Blocklist)
- Aktualisiert alle 5 min die Connection-Liste (neue User → neue Sessions, entfernte → geschlossen)
- IDLE wird alle 25 min erneuert (RFC 3501 Server-Timeout liegt bei 29 min)
## Env-Vars
| Variable | Pflicht | Beschreibung |
|---------------------|---------|-----------------------------------------------------------|
| `DATABASE_URL` | ja | Postgres-Connection-String (Supabase Pooler oder direkt) |
| `ADMIN_SECRET` | ja | Shared Secret für /api/mail/scan-internal Header |
| `ENCRYPTION_KEY` | ja | AES-256 Key (identisch zum Backend-Key, 32+ Zeichen) |
| `BACKEND_URL` | nein | Default: http://127.0.0.1:3016 (staging) / 3015 (prod) |
| `NODE_ENV` | nein | `production` → BACKEND_URL default port 3015 |
## Lokal starten (Entwicklung)
```bash
cd backend/imap-idle
npm install
DATABASE_URL=<...> ADMIN_SECRET=<...> ENCRYPTION_KEY=<...> node index.mjs
```
Via Infisical (analog zu start-staging.sh):
```bash
infisical run --env=staging -- node index.mjs
```
## PM2 (Produktion)
Wird via ecosystem.config.js gestartet — siehe `docs/internal/MAIL_DAEMON_DEPLOYMENT.md`.
## Logs (pm2)
```
[idle/<email>] connected (imap.gmail.com:993)
[idle/<email>] exists-event received (new mail)
[idle/<email>] scan-triggered → scanned=12 blocked=1
[idle/<email>] idle renewing (25min threshold)
[idle/<email>] reconnecting in 5s (attempt 2)
[idle/db] refreshed — 47 active connections, 47 sessions
```

353
backend/imap-idle/index.mjs Normal file
View File

@ -0,0 +1,353 @@
/**
* rebreak-imap-idle IMAP IDLE Daemon
*
* Hält pro aktivem MailConnection-Eintrag eine persistente IMAP-IDLE-Session.
* Wenn der Server "EXISTS" meldet (neue Mail), feuert der Daemon sofort
* POST /api/mail/scan-internal gegen das lokale Backend ohne 30min-Warte.
*
* Env-Vars (via Infisical-Wrapper):
* DATABASE_URL Postgres-Connection-String
* ADMIN_SECRET Header-Secret für /api/mail/scan-internal
* ENCRYPTION_KEY AES-256 Key (gleicher wie im Backend)
* BACKEND_URL z.B. http://127.0.0.1:3016 (default: 3016)
* NODE_ENV production / staging
*
* Starten:
* node index.mjs
*
* Prozess-Signale:
* SIGTERM / SIGINT graceful shutdown (alle IMAP-Sessions schließen)
*/
import { ImapFlow } from "imapflow";
import pg from "pg";
import { createDecipheriv } from "crypto";
// ─── Config ─────────────────────────────────────────────────────────────────
const BACKEND_URL =
process.env.BACKEND_URL ||
(process.env.NODE_ENV === "production"
? "http://127.0.0.1:3015"
: "http://127.0.0.1:3016");
const ADMIN_SECRET =
process.env.NUXT_ADMIN_SECRET || process.env.ADMIN_SECRET || "";
const DB_REFRESH_INTERVAL_MS = 5 * 60 * 1000; // 5 min — neue Connections entdecken
const IDLE_RENEW_INTERVAL_MS = 25 * 60 * 1000; // 25 min — RFC 3501 max = 29min
const RECONNECT_DELAYS_MS = [1000, 5000, 30_000]; // exponential backoff, danach 60s loop
const RECONNECT_LOOP_DELAY_MS = 60 * 1000;
// ─── DB-Pool (direktes pg, kein Prisma — Daemon ist kein Nitro-Kontext) ────
const pool = new pg.Pool({ connectionString: process.env.DATABASE_URL });
async function loadActiveConnections() {
const { rows } = await pool.query(
`SELECT id, "userId", email, "imapHost", "imapPort",
"passwordEncrypted", "rejectUnauthorized", "useStarttls"
FROM rebreak."MailConnection"
WHERE "isActive" = true`,
);
return rows;
}
// ─── Crypto (analog zu server/utils/crypto.ts) ──────────────────────────────
const AES_ALGO = "aes-256-gcm";
const KEY_LENGTH = 32;
function getKey() {
const raw =
process.env.NUXT_ENCRYPTION_KEY || process.env.ENCRYPTION_KEY || "";
if (!raw || raw.length < KEY_LENGTH) {
return Buffer.alloc(
KEY_LENGTH,
raw.padEnd(KEY_LENGTH, "0").slice(0, KEY_LENGTH),
);
}
return Buffer.from(raw.slice(0, KEY_LENGTH), "utf8");
}
function decrypt(stored) {
const parts = stored.split(":");
if (parts.length !== 3) throw new Error("Invalid encrypted format");
const [ivHex, tagHex, dataHex] = parts;
const decipher = createDecipheriv(
AES_ALGO,
getKey(),
Buffer.from(ivHex, "hex"),
);
decipher.setAuthTag(Buffer.from(tagHex, "hex"));
return decipher.update(Buffer.from(dataHex, "hex")) + decipher.final("utf8");
}
// ─── Logging ────────────────────────────────────────────────────────────────
function log(email, msg) {
// NIEMALS password/credentials loggen. email ist safe (kein secret).
console.log(`[idle/${email}] ${msg}`);
}
function logError(email, msg, err) {
const errMsg = err?.message ?? String(err);
// Credentials tauchen nie in err.message auf (ImapFlow maskiert sie nicht,
// aber wir liefern pass nur an ImapFlow — nicht in eigenen log-calls).
console.error(`[idle/${email}] ${msg}: ${errMsg}`);
}
// ─── Session-Registry ────────────────────────────────────────────────────────
// Map<connectionId, SessionHandle>
const sessions = new Map();
let shuttingDown = false;
// ─── Scan-Trigger ────────────────────────────────────────────────────────────
async function triggerScan(conn) {
try {
const res = await fetch(`${BACKEND_URL}/api/mail/scan-internal`, {
method: "POST",
headers: {
"Content-Type": "application/json",
"x-admin-secret": ADMIN_SECRET,
},
body: JSON.stringify({ userId: conn.userId }),
});
if (!res.ok) {
log(conn.email, `scan-trigger HTTP ${res.status}`);
} else {
const data = await res.json().catch(() => ({}));
log(
conn.email,
`scan-triggered → scanned=${data.scanned ?? "?"} blocked=${data.blocked ?? "?"}`,
);
}
} catch (err) {
logError(conn.email, "scan-trigger failed", err);
}
}
// ─── IDLE-Session ────────────────────────────────────────────────────────────
/**
* Startet eine einzelne IDLE-Session für eine MailConnection.
* Reconnect-Loop läuft intern diese Funktion returned nie (bis shutdown).
*/
async function runSession(conn) {
let attempt = 0;
while (!shuttingDown) {
let password;
try {
password = decrypt(conn.passwordEncrypted);
} catch (err) {
logError(conn.email, "decrypt failed — session aborted", err);
return; // Kein retry bei Decrypt-Fehler (kaputtes Credential)
}
const useImplicitTls = !conn.useStarttls;
const imap = new ImapFlow({
host: conn.imapHost,
port: conn.imapPort,
secure: useImplicitTls,
...(conn.useStarttls ? { requireTLS: true } : {}),
auth: { user: conn.email, pass: password },
logger: false,
tls: { rejectUnauthorized: conn.rejectUnauthorized ?? true },
// Outlook schließt idle-connections aggressiv — disableCompression
// verhindert edge-cases bei partial reads nach reconnect
disableCompression: conn.imapHost.includes("office365"),
});
// Referenz ablegen damit shutdown() darauf zugreifen kann
const handle = sessions.get(conn.id);
if (handle) handle.imap = imap;
try {
await imap.connect();
log(conn.email, `connected (${conn.imapHost}:${conn.imapPort})`);
attempt = 0; // Reset nach erfolgreicher Verbindung
await imap.getMailboxLock("INBOX");
// IDLE-Loop: alle IDLE_RENEW_INTERVAL_MS erneuern
while (!shuttingDown && sessions.has(conn.id)) {
let idleAbort;
const idlePromise = new Promise((resolve, reject) => {
imap
.idle()
.then(resolve)
.catch(reject);
idleAbort = () => {
// ImapFlow.idle() bricht ab wenn die Connection getrennt wird
imap.close();
resolve();
};
});
// exists-event → sofort scannen
const onExists = () => {
log(conn.email, "exists-event received (new mail)");
triggerScan(conn); // fire-and-forget
};
imap.on("exists", onExists);
// IDLE nach 25min erneuern (RFC 3501: Server darf nach 29min droppen)
const renewTimer = setTimeout(() => {
log(conn.email, "idle renewing (25min threshold)");
imap.close(); // Unterbricht idle() → Loop iteriert → reconnect
}, IDLE_RENEW_INTERVAL_MS);
try {
await idlePromise;
} finally {
clearTimeout(renewTimer);
imap.removeListener("exists", onExists);
}
if (shuttingDown || !sessions.has(conn.id)) break;
// Kurze Pause vor Reconnect (idle() returned auch bei normalem timeout)
await sleep(500);
}
try { await imap.logout(); } catch { /* ignore */ }
return; // Sauberer Exit (shutdown oder Connection entfernt)
} catch (err) {
logError(conn.email, "connection error", err);
try { imap.close(); } catch { /* ignore */ }
}
if (shuttingDown || !sessions.has(conn.id)) return;
// Exponential backoff
const delay =
attempt < RECONNECT_DELAYS_MS.length
? RECONNECT_DELAYS_MS[attempt]
: RECONNECT_LOOP_DELAY_MS;
attempt++;
log(conn.email, `reconnecting in ${delay / 1000}s (attempt ${attempt})`);
await sleep(delay);
}
}
// ─── Session-Management ──────────────────────────────────────────────────────
function startSession(conn) {
if (sessions.has(conn.id)) return; // bereits aktiv
log(conn.email, "session starting");
const handle = { conn, imap: null, promise: null };
sessions.set(conn.id, handle);
handle.promise = runSession(conn).catch((err) => {
logError(conn.email, "session crashed (unhandled)", err);
sessions.delete(conn.id);
});
}
async function stopSession(connectionId, email) {
const handle = sessions.get(connectionId);
if (!handle) return;
log(email ?? connectionId, "session stopping");
sessions.delete(connectionId);
if (handle.imap) {
try { handle.imap.close(); } catch { /* ignore */ }
}
await handle.promise.catch(() => {});
}
// ─── DB-Refresh-Loop ─────────────────────────────────────────────────────────
async function refreshConnections() {
let rows;
try {
rows = await loadActiveConnections();
} catch (err) {
console.error("[idle/db] loadActiveConnections failed:", err.message);
return;
}
const activeIds = new Set(rows.map((r) => r.id));
// Neue Connections starten
for (const row of rows) {
if (!sessions.has(row.id)) {
startSession(row);
}
}
// Entfernte Connections stoppen
for (const [id, handle] of sessions.entries()) {
if (!activeIds.has(id)) {
await stopSession(id, handle.conn?.email);
}
}
console.log(
`[idle/db] refreshed — ${activeIds.size} active connections, ${sessions.size} sessions`,
);
}
// ─── Graceful Shutdown ───────────────────────────────────────────────────────
async function shutdown(signal) {
console.log(`[idle] received ${signal} — shutting down gracefully`);
shuttingDown = true;
const stopPromises = [];
for (const [id, handle] of sessions.entries()) {
stopPromises.push(stopSession(id, handle.conn?.email));
}
await Promise.allSettled(stopPromises);
await pool.end().catch(() => {});
console.log("[idle] shutdown complete");
process.exit(0);
}
process.on("SIGTERM", () => shutdown("SIGTERM"));
process.on("SIGINT", () => shutdown("SIGINT"));
// ─── Startup ─────────────────────────────────────────────────────────────────
function sleep(ms) {
return new Promise((resolve) => setTimeout(resolve, ms));
}
function assertEnv() {
const missing = [];
if (!process.env.DATABASE_URL) missing.push("DATABASE_URL");
if (!ADMIN_SECRET) missing.push("ADMIN_SECRET / NUXT_ADMIN_SECRET");
if (missing.length > 0) {
console.error(
`[idle] FEHLER: fehlende Env-Vars: ${missing.join(", ")}. Daemon startet nicht.`,
);
process.exit(1);
}
}
async function main() {
assertEnv();
console.log(
`[idle] starting — backend=${BACKEND_URL} env=${process.env.NODE_ENV ?? "unknown"}`,
);
// Initialer Load
await refreshConnections();
// Periodischer DB-Refresh
setInterval(() => {
if (!shuttingDown) refreshConnections();
}, DB_REFRESH_INTERVAL_MS);
}
main().catch((err) => {
console.error("[idle] fatal startup error:", err);
process.exit(1);
});

View File

@ -0,0 +1,18 @@
{
"name": "rebreak-imap-idle",
"version": "1.0.0",
"private": true,
"type": "module",
"description": "IMAP IDLE Daemon — Real-time Gambling-Mail-Detection für Legend-User",
"main": "index.mjs",
"scripts": {
"start": "node index.mjs"
},
"dependencies": {
"imapflow": "^1.2.18",
"pg": "^8.16.3"
},
"engines": {
"node": ">=20.0.0"
}
}

View File

@ -0,0 +1,38 @@
#!/bin/bash
# rebreak-imap-idle Staging — Infisical-Secret-Injection
#
# Wird von pm2 als Script-Interpreter gestartet (ecosystem.config.js).
# Injiziert DATABASE_URL, ADMIN_SECRET, ENCRYPTION_KEY via Infisical-staging.
# ENCRYPTION_KEY muss identisch zum Backend-Key sein (AES-256-GCM).
set -euo pipefail
source /etc/environment
if [[ -z "${INFISICAL_CLIENT_ID:-}" || -z "${INFISICAL_CLIENT_SECRET:-}" ]]; then
echo "[idle] FEHLER: INFISICAL_CLIENT_ID / INFISICAL_CLIENT_SECRET nicht in /etc/environment gesetzt" >&2
exit 1
fi
INFISICAL_TOKEN=$(infisical login \
--method=universal-auth \
--client-id="${INFISICAL_CLIENT_ID}" \
--client-secret="${INFISICAL_CLIENT_SECRET}" \
--silent --plain 2>/dev/null)
[[ -z "$INFISICAL_TOKEN" ]] && { echo "[idle] Infisical login fehlgeschlagen" >&2; exit 1; }
NODE_BIN="/root/.nvm/versions/node/v24.11.1/bin/node"
DAEMON="/srv/rebreak/backend/imap-idle/index.mjs"
exec infisical run \
--projectId="${INFISICAL_PROJECT_ID:-14b11b35-ef59-4b8a-a16b-398f0cc3ad93}" \
--env=staging \
--token="$INFISICAL_TOKEN" \
-- bash -c '
set -e
export DATABASE_URL="${DATABASE_URL:-${NUXT_DATABASE_URL:-}}"
export ADMIN_SECRET="${ADMIN_SECRET:-${NUXT_ADMIN_SECRET:-}}"
export ENCRYPTION_KEY="${ENCRYPTION_KEY:-${NUXT_ENCRYPTION_KEY:-}}"
export BACKEND_URL="http://127.0.0.1:3016"
exec '"$NODE_BIN"' '"$DAEMON"'
'

View File

@ -86,6 +86,22 @@ module.exports = {
max_memory_restart: "128M", max_memory_restart: "128M",
}, },
// ─── IMAP IDLE Daemon Staging ───────────────────────────────────────────
// Deployment-Anleitung: docs/internal/MAIL_DAEMON_DEPLOYMENT.md
// start-idle-staging.sh injiziert Infisical-Secrets (DATABASE_URL, ADMIN_SECRET, ENCRYPTION_KEY).
// Initialer Start via: pm2 startOrReload /srv/rebreak/ecosystem.config.js
{
name: "rebreak-idle-staging",
script: `${APP_DIR}/imap-idle/start-idle-staging.sh`,
interpreter: "bash",
cwd: `${APP_DIR}/imap-idle`,
instances: 1,
autorestart: true,
watch: false,
max_memory_restart: "256M",
env: { NODE_ENV: "production" },
},
// ─── DNS-Blocker (auskommentiert bis DNS-Daemons aufgesetzt sind) ────── // ─── DNS-Blocker (auskommentiert bis DNS-Daemons aufgesetzt sind) ──────
// { // {
// name: "dns-rebreak", // name: "dns-rebreak",

View File

@ -101,6 +101,19 @@ CI=true "${PNPM_BIN}" install --frozen-lockfile 2>&1 || {
} }
log "pnpm install done" log "pnpm install done"
# 3b. imap-idle Runtime-Deps installieren (imapflow + pg, standalone package.json)
IDLE_DIR="${APP_DIR}/imap-idle"
if [[ -d "$IDLE_DIR" && -f "$IDLE_DIR/package.json" ]]; then
log "Step 3b: npm install (imap-idle runtime-deps)..."
cd "$IDLE_DIR"
npm install --production --prefer-offline 2>&1
# scp preserviert Permissions nicht immer -- explizit setzen
[[ -f "$IDLE_DIR/start-idle-staging.sh" ]] && chmod +x "$IDLE_DIR/start-idle-staging.sh"
log "imap-idle npm install done"
else
log "Step 3b: imap-idle-Verzeichnis nicht gefunden -- skip (wird via GH-Actions deployed)"
fi
# 4. Artifact extrahieren -- atomisches mv (gleicher Pattern wie deploy.sh) # 4. Artifact extrahieren -- atomisches mv (gleicher Pattern wie deploy.sh)
log "Step 4: Artifact extrahieren..." log "Step 4: Artifact extrahieren..."
cd "${APP_DIR}" cd "${APP_DIR}"
@ -129,7 +142,9 @@ log "rebreak-staging restarted"
# 6. Optional services (best-effort, Mo's Scope) # 6. Optional services (best-effort, Mo's Scope)
log "Step 6: Optional services restart..." log "Step 6: Optional services restart..."
"${PM2_BIN}" restart rebreak-imap-staging 2>/dev/null || true "${PM2_BIN}" restart rebreak-imap-staging 2>/dev/null || true
"${PM2_BIN}" restart rebreak-idle-staging 2>/dev/null || true # rebreak-idle-staging: startOrReload (erster Deploy hat keinen laufenden Prozess)
"${PM2_BIN}" restart rebreak-idle-staging --update-env 2>/dev/null || \
"${PM2_BIN}" start "${REPO_ROOT}/ecosystem.config.js" --only rebreak-idle-staging
"${PM2_BIN}" restart dns-rebreak-staging 2>/dev/null || true "${PM2_BIN}" restart dns-rebreak-staging 2>/dev/null || true
"${PM2_BIN}" restart dns-rebreak 2>/dev/null || true "${PM2_BIN}" restart dns-rebreak 2>/dev/null || true