The original gen_synthetic.py held all N samples in a [None]*n list and
only wrote at the end — three full-scale runs were killed mid-way (env
reaps long sessions) and lost everything (0 bytes on disk each time).
Fixes (proven on the 500k+100k run that just completed):
- run() now streams: each completed future is written to disk the instant
it's produced, via an O(workers) reorder buffer that keeps output in
index order. Peak RAM drops from O(n) (~2.7GB for 50k) to O(workers).
- --resume flag: counts existing lines and appends only the missing
suffix (same seed => deterministic prefix, so resume is content-safe).
- scripts/gen_full_supervisor.sh: loops 'run --resume -> check count'
until target reached. Each surviving iteration adds >=1 line, so the
run converges even across multiple kills (took 2 kills + 2 resumes to
finish the 500k align). On this host: 2 partial runs (50k + 437k) then
a final resume filled the last 62k.
- Verified: order-deterministic (same-seed runs byte-identical), resume
appends without rewriting the prefix, target-already-met => no-op.
Full-scale data now on disk: align 500k (27G) + sft 100k (5.4G) +
eval_synth 2k held-out (untouched).
Critical eval-pipeline bugs found while running the stage-2 model:
1. parse_anomaly_segments only recognized {"segments\:[[s,e]]}; the training
data (data/instruct.py) emits {"anomaly_regions":[{"start","end"}]}.
The model learned the correct format but scored 0 → VUS-PR=0. Now accepts
both schemas + a balanced-brace JSON scanner (the flat regex could not
match nested objects the model emits).
2. qa_judge._extract_json had the same nesting bug — reuse the scanner.
3. Generation truncated at 64-96 tokens (too short for stats JSON) → bumped
to 192 in report + instruct_check.
Report enhancements:
- wire the pure-LLM baseline (series-as-text, no TS modality) so design §5.4
'model vs pure LLM' is actually computed. Heavy predictors built/evaluated
one at a time with GPU cleanup (single 12GB card).
- add QA-judge table across all 6 categories (design §6.5 问答均分).
- add --pure_llm / --max_per_cat flags.
Results (reports/eval-2026-06-30.md): model QA-judge mean 2.45 vs pure_llm
0.88 (2.8x); anomaly localization (VUS-PR) is the weak point — neither
model nor pure_llm localize well; TS-modality ablation change_rate 0.98.
张宗平