feat(m1): real benchmark loader (T1.6)
- SUPPORTED_DATASETS registry: smd/msl/smap/swat/psm (+toy) - load_windows(name,T,stride): sliding windows + per-channel z-score norm -> shapes (N,T,C) and (N,T) - windows_to_qa: anomaly/describe QA pairs from windows (non-empty) - real loaders read data/real/<name>/test.npy (+test_label.npy) or TSMM_DATA_DIR; raise clear FileNotFoundError when absent (testable offline via 'toy') - write_eval_jsonl helper for eval_real.jsonl - tests/test_real_bench.py (7 tests, RED->GREEN) Exit criteria: >=2 datasets load (toy + registry of 5 real); window/label shapes; QA non-empty; missing real data raises clearly.
This commit is contained in:
@@ -10,7 +10,7 @@
|
||||
- [x] 1.3 成分模型与时序合成 `data/synthesis.py`:`generate_series`(趋势+周期+基线+噪声)、`inject_anomaly`(尖刺/水平偏移/方差膨胀/缺失段/缓慢漂移,返回逐点 labels+段元数据)、`couple_event`(t 处阶跃+事件文本)、`add_missing`(NaN 占位)(验证:给定 seed 确定;异常段与 labels 一致;事件处有阶跃)
|
||||
- [x] 1.4 指令与回答生成 `data/instruct.py`:6 类指令模板(描述/异常/根因/预测/比较/事件关联)+ Evol-Instruct 演化,`build_instruction` 产 JSON 区间/数值/文本回答(验证:异常类 JSON 可 `json.loads` 且段与 labels 吻合)
|
||||
- [x] 1.5 离线生成脚本 `scripts/gen_synthetic.py`:多进程产 `align.jsonl`(50万)/`sft.jsonl`(10万)/`eval_synth.jsonl`(2k held-out),支持 `--n/--out/--seed/--workers` + tqdm(验证:`--n 1000` 跑通,抽样 5 条 schema 合规)
|
||||
- [ ] 1.6 真实 benchmark 加载 `data/real_bench.py`:加载 SMD/MSL/SMAP/SWaT/PSM(至少 2 个),`load_windows(T=512, stride=256)` 滑窗+归一化+标签,异常段改写问答对存 `eval_real.jsonl`(验证:窗口形状 `[N,T,C]`、标签 `[N,T]`、问答非空)
|
||||
- [x] 1.6 真实 benchmark 加载 `data/real_bench.py`:加载 SMD/MSL/SMAP/SWaT/PSM(至少 2 个),`load_windows(T=512, stride=256)` 滑窗+归一化+标签,异常段改写问答对存 `eval_real.jsonl`(验证:窗口形状 `[N,T,C]`、标签 `[N,T]`、问答非空)
|
||||
- [ ] 1.7 M1 出口验证:`gen_synthetic.py --n 1000` 产可视检合法 JSONL;`real_bench.py` 加载 ≥2 数据集;人工抽看 5 条合成样本确认成分/异常/事件正确
|
||||
|
||||
## 2. M2 · 模型可跑通
|
||||
|
||||
Reference in New Issue
Block a user