feat(gen): streaming + crash-resilient resume for full-scale data gen

The original gen_synthetic.py held all N samples in a [None]*n list and
only wrote at the end — three full-scale runs were killed mid-way (env
reaps long sessions) and lost everything (0 bytes on disk each time).

Fixes (proven on the 500k+100k run that just completed):
- run() now streams: each completed future is written to disk the instant
  it's produced, via an O(workers) reorder buffer that keeps output in
  index order. Peak RAM drops from O(n) (~2.7GB for 50k) to O(workers).
- --resume flag: counts existing lines and appends only the missing
  suffix (same seed => deterministic prefix, so resume is content-safe).
- scripts/gen_full_supervisor.sh: loops 'run --resume -> check count'
  until target reached. Each surviving iteration adds >=1 line, so the
  run converges even across multiple kills (took 2 kills + 2 resumes to
  finish the 500k align). On this host: 2 partial runs (50k + 437k) then
  a final resume filled the last 62k.
- Verified: order-deterministic (same-seed runs byte-identical), resume
  appends without rewriting the prefix, target-already-met => no-op.

Full-scale data now on disk: align 500k (27G) + sft 100k (5.4G) +
eval_synth 2k held-out (untouched).

scripts/gen_full_supervisor.sh

@@ -0,0 +1,39 @@
+#!/usr/bin/env bash
+# Crash-resilient full-scale generation supervisor.
+#
+# gen_synthetic.py now (a) streams each sample to disk the instant it's produced
+# (O(workers) memory, not O(n)) and (b) supports --resume: it counts existing
+# lines and only generates+appends the missing suffix. So a kill mid-run keeps
+# all progress on disk; re-running --resume continues from where it stopped.
+#
+# This supervisor loops "run --resume -> check count" until the target line
+# count is reached. If the supervisor itself is killed, the data already on
+# disk survives; relaunch this script and it picks up. Convergence is guaranteed:
+# every iteration that survives adds >= 1 line.
+#
+# Usage: scripts/gen_full_supervisor.sh
+set -uo pipefail   # NB: no -e — a killed gen sub-call must NOT abort the loop
+cd "$(dirname "$0")/.."
+export PYTHONPATH=src
+W=$(nproc)
+
+gen() {  # target_n  out  seed  [extra flags...]
+  local target=$1 out=$2 seed=$3; shift 3
+  while :; do
+    local cur=0
+    if [ -f "$out" ]; then cur=$(wc -l < "$out"); fi
+    if [ "$cur" -ge "$target" ]; then
+      echo "[sup] $out done: $cur/$target lines"; return 0
+    fi
+    echo "[sup] $out at $cur/$target — launching --resume $(date '+%H:%M:%S')"
+    .venv/bin/python scripts/gen_synthetic.py \
+      --n "$target" --out "$out" --seed "$seed" --workers "$W" --resume "$@"
+    # loop: re-check count (gen may have been killed or completed)
+  done
+}
+
+echo "[sup] === START $(date '+%H:%M:%S') ==="
+gen 500000 data/align.jsonl 0
+gen 100000 data/sft.jsonl   0 --evolve
+echo "[sup] === ALL DONE $(date '+%H:%M:%S') ==="
+echo "[sup] align=$(wc -l < data/align.jsonl) sft=$(wc -l < data/sft.jsonl) eval=$(wc -l < data/eval_synth.jsonl)"