 张宗平
|
4c4a8f6435
|
feat(m5): full eval pipeline — parse/metrics/judge/baselines/report (T5.1-T5.5)
- src/tsmm/eval/parse_answer.py: text→JSON segments→point scores (JSON-first,
regex fallback, all-zero on fail).
- src/tsmm/eval/ts_metrics.py: VUS-PR (main, threshold-free; constant-score →
prevalence, NOT inflated), AUC-PR, Point-F1 (no PA), PA-F1 (control only),
self-contained Affiliation-F1.
- src/tsmm/eval/qa_judge.py: RuleJudge (anomaly IoU / describe+forecast numeric
tolerance) + LLMJudge (stub backend = deterministic offline heuristic for the
no-network host; openai backend = GPT-4o 0-5; pluggable callable).
- src/tsmm/eval/baselines.py: trivial (Random/Constant/AllReport) + ts_to_text
+ pure_llm_answer glue; Time-LLM/ChatTS marked not-reproduced (honest).
- src/tsmm/eval/report.py + scripts/run_eval.sh: trained model + all baselines
through the SAME pipeline → reports/eval-YYYYMMDD.md (VUS-PR main, PA-F1
explicitly labelled control, trivial baselines included).
- tests: 45 new (parse 13, metrics 12, judge 12, baselines 8). 145 passing.
- Smoke: run_eval on 40 samples produces valid markdown; PA-F1≈1.0 vs
VUS-PR≈0.35 for all_report demonstrates why PA must not be the headline.
|
2026-06-30 03:18:14 +00:00 |
|
|
张宗平
|
357e2e60bd
|
feat(m4): stage 2 SFT training script (LoRA r=16) + smoke (T4.1)
- src/tsmm/train/stage2.py: loads stage1 ckpt (optional), freeze_llm +
enable_lora(r=16) on q/v_proj, masked LM loss on sft.jsonl, bs=4/grad_accum=8/
ctx=1024/BF16/gradient checkpointing, ckpt every 2000 steps (encoder+projector
state + peft LoRA adapter via save_pretrained).
- scripts/train_stage2.sh: wrapper with design defaults.
- Smoke verified: --max_steps 4, LoRA injected, finite loss, peak 3.29 GB
(≤11GB budget), ckpt+adapter saved.
|
2026-06-30 03:11:00 +00:00 |
|
|
张宗平
|
bf95ac18e4
|
feat(m3): stage 1 alignment training script + smoke (T3.2)
- src/tsmm/train/stage1.py: streams align.jsonl, freeze_llm, trains
Encoder+Projector (AdamW lr=1e-4), bs=8/grad_accum=4/ctx=512/BF16,
gradient checkpointing + enable_input_require_grads, loss = lm + lambda*InfoNCE
(original vs perturbed TS representation), ckpt + tensorboard every 2000 steps.
- scripts/train_stage1.sh: wrapper with design defaults.
- src/tsmm/data/collator.py: handle JSON null (missing markers) -> NaN -> fill.
- .gitignore: /checkpoints/ and data/*.jsonl.
- Smoke verified: --max_steps 5 on 64 samples, finite loss, peak 5.52 GB (≤6GB).
- 100 tests passing.
|
2026-06-30 02:22:32 +00:00 |
|
|
张宗平
|
9cfda37275
|
feat(m1): offline generation pipeline + CLI (T1.5)
- tsmm.data.gen_pipeline.gen_one_sample: composes attrs/series/anomaly/event/missing/instruct
into one JSON-serializable sample (full schema)
- write_jsonl helper
- scripts/gen_synthetic.py: multiprocessing + tqdm CLI (--n/--out/--seed/--workers)
- regression: build_instruction('event') with empty events no longer raises (TDD RED first)
- tests/test_gen_pipeline.py (6 tests, RED->GREEN)
Exit criteria: python scripts/gen_synthetic.py --n 1000 -> 1000 samples, schema-OK,
all 6 categories balanced (~130-190 each).
|
2026-06-29 23:29:04 +08:00 |
|
|
张宗平
|
748bf41c53
|
feat(m1): scaffold tsmm package (T1.1)
- pyproject.toml (src-layout, deps per plan)
- src/tsmm/{data,model,train,eval} skeletons
- tests/test_smoke.py (RED->GREEN: importable + subpackages)
- configs/, scripts/, data/, checkpoints/ + .gitignore
- project venv (.venv) for reproducible env (M1 deps only; CUDA torch @ M2)
Verifications: import tsmm OK; pytest tests/ 3 passed.
|
2026-06-29 23:21:47 +08:00 |
|