4c4a8f6435
feat(m5): full eval pipeline — parse/metrics/judge/baselines/report (T5.1-T5.5)
...
- src/tsmm/eval/parse_answer.py: text→JSON segments→point scores (JSON-first,
regex fallback, all-zero on fail).
- src/tsmm/eval/ts_metrics.py: VUS-PR (main, threshold-free; constant-score →
prevalence, NOT inflated), AUC-PR, Point-F1 (no PA), PA-F1 (control only),
self-contained Affiliation-F1.
- src/tsmm/eval/qa_judge.py: RuleJudge (anomaly IoU / describe+forecast numeric
tolerance) + LLMJudge (stub backend = deterministic offline heuristic for the
no-network host; openai backend = GPT-4o 0-5; pluggable callable).
- src/tsmm/eval/baselines.py: trivial (Random/Constant/AllReport) + ts_to_text
+ pure_llm_answer glue; Time-LLM/ChatTS marked not-reproduced (honest).
- src/tsmm/eval/report.py + scripts/run_eval.sh: trained model + all baselines
through the SAME pipeline → reports/eval-YYYYMMDD.md (VUS-PR main, PA-F1
explicitly labelled control, trivial baselines included).
- tests: 45 new (parse 13, metrics 12, judge 12, baselines 8). 145 passing.
- Smoke: run_eval on 40 samples produces valid markdown; PA-F1≈1.0 vs
VUS-PR≈0.35 for all_report demonstrates why PA must not be the headline.
2026-06-30 03:18:14 +00:00
204d5237ba
feat(m3): losses — masked LM CE + symmetric InfoNCE (T3.1)
...
- src/tsmm/train/losses.py:
* lm_loss(logits, labels): shifted masked CE (answer span only); clean 0.0
when all positions masked.
* infonce_contrastive_loss(z1, z2): symmetric CLIP-style InfoNCE in TS-
embedding space; positive = original vs light-augmentation view.
- tests/test_losses.py: 9 tests. 100 tests passing.
Spec clarification (small tier): InfoNCE operates in TS-embedding space
(encoder/projector output) rather than answer-embedding space; light
augmentations are positives. T3.3 strong-perturbation sensitivity is a
separate downstream check.
2026-06-30 02:19:02 +00:00
7e1df76e3e
feat(m2): Collator — JSONL → wrapper raw batch (T2.5)
...
- src/tsmm/data/collator.py: Collator maps JSONL sample dicts to the wrapper
batch contract {series [B,max_T,max_C], attributes, timestamps, events,
question, answer}. Variable-T pad/truncate, variable-C pad, NaN→fill.
Variable-text padding/attention_mask deferred to the wrapper's per-sample
splice + max_seq padding.
- tests/test_collator.py: 11 tests. End-to-end Collator→Wrapper fwd+bwd+generate
verified (loss finite, peak 2.32 GB, generate emits text). 91 tests passing.
2026-06-30 02:14:47 +00:00
aec8fd5ef1
feat(m2): MultimodalTSModel wrapper — end-to-end fwd/generate + LoRA (T2.4)
...
- src/tsmm/model/wrapper.py: MultimodalTSModel combining TS Encoder + Projector
+ Qwen2.5-0.5B LLM + LoRA. Two modes: freeze_llm() (stage ①) and
enable_lora(r=16) (stage ②, peft on q/v_proj). forward() takes a raw batch
{series, attributes, timestamps, events, question, answer} end-to-end
(encoder→projector→splice→LLM) OR pre-spliced inputs_embeds.
- src/tsmm/model/multimodal.py: device-aware splice (move ids/ts_embeds to
the bound embedding layer's device).
- tests/test_wrapper.py: 6 tests — finite loss, grad flows into
encoder/projector under freeze_llm, generate returns text, freeze_llm and
enable_lora mode invariants, GPU memory budget. 80 tests passing.
- Measured: stage ① fwd+bwd peak 1.87 GB (design budget ~5 GB).
Spec clarifications (small tier, comet-build Step 4):
- Wrapper batch contract = {series, attributes, timestamps, events, question,
answer} (lists of str + series tensor). Collator (T2.5) will produce this.
- Encoder/Projector kept fp32; their output cast to LLM dtype (bf16) before
splice, so inputs_embeds matches LLM weights.
2026-06-30 02:11:36 +00:00
f509d47878
feat(m2): multimodal splice [属性][时间戳][TS tok][事件][问题][回答] (T2.3)
...
- src/tsmm/model/multimodal.py: MultimodalSplicer — binds LLM input
embedding layer, assembles text embeds (via tokenizer) + TS token embeds
(from Projector) into inputs_embeds/attention_mask/labels; labels = -100
everywhere except answer+EOS (training); left-truncation to max_len=1024.
- tests/test_multimodal.py: 8 tests against real Qwen2.5-0.5B tokenizer +
stub embedding (shapes, all-ones mask, answer-only labels, TS token count
in sequence, no-NaN, truncation). 74 tests passing.
- configs/llm.yaml: offline model path (downloaded via proxy to
~/models/Qwen2.5-0.5B-Instruct; HF hub unreachable on host).
Spec clarifications (small tier, comet-build Step 4):
- Training appends answer + EOS to the prompt layout.
- Build embedding layer with len(tokenizer) (incl. added special tokens),
not tokenizer.vocab_size which excludes them (eos=151645).
2026-06-30 02:05:15 +00:00
f20c2db803
feat(m2): Projector Linear(256→896)+LayerNorm (T2.2)
...
- src/tsmm/model/projector.py: Projector (Linear + LayerNorm).
- tests/test_projector.py: 5 tests (shape, Qwen hidden alignment,
LayerNorm present, gradient flow, no-NaN).
- 66 tests passing.
2026-06-30 00:17:47 +00:00
545802d62c
feat(m2): TS Encoder — Patchify + 2-layer Transformer (T2.1)
...
- src/tsmm/model/ts_encoder.py: Patchify (channel-independent unfold) +
TSEncoder (shared per-channel patch linear, mean-pool over channels,
learnable positional embedding, 2-layer pre-norm Transformer, LayerNorm).
- configs/ts_encoder.yaml: encoder/projector hyperparameters.
- tests/test_ts_encoder.py: 10 tests (shape, n_patches formula, window
values, determinism, gradient flow, param sanity).
- 61 tests passing; GPU fwd/bwd verified on RTX 3060 (39 MB peak).
Spec clarifications (small tier, comet-build Step 4):
- n_patches = (T-P)//S+1 ⇒ 127 for T=512 (design said 128; arithmetic slip).
- 'Channel-independent' = shared per-channel patch embed + mean-pool →
C-free output [B,n_patches,d]; channel identity carried by [属性] tokens.
- Actual params ≈2.1M (design's '≈4M' was a rough estimate).
2026-06-30 00:16:54 +00:00
722102b335
feat(m1): real benchmark loader (T1.6)
...
- SUPPORTED_DATASETS registry: smd/msl/smap/swat/psm (+toy)
- load_windows(name,T,stride): sliding windows + per-channel z-score norm
-> shapes (N,T,C) and (N,T)
- windows_to_qa: anomaly/describe QA pairs from windows (non-empty)
- real loaders read data/real/<name>/test.npy (+test_label.npy) or TSMM_DATA_DIR;
raise clear FileNotFoundError when absent (testable offline via 'toy')
- write_eval_jsonl helper for eval_real.jsonl
- tests/test_real_bench.py (7 tests, RED->GREEN)
Exit criteria: >=2 datasets load (toy + registry of 5 real); window/label shapes;
QA non-empty; missing real data raises clearly.
2026-06-29 23:30:30 +08:00
9cfda37275
feat(m1): offline generation pipeline + CLI (T1.5)
...
- tsmm.data.gen_pipeline.gen_one_sample: composes attrs/series/anomaly/event/missing/instruct
into one JSON-serializable sample (full schema)
- write_jsonl helper
- scripts/gen_synthetic.py: multiprocessing + tqdm CLI (--n/--out/--seed/--workers)
- regression: build_instruction('event') with empty events no longer raises (TDD RED first)
- tests/test_gen_pipeline.py (6 tests, RED->GREEN)
Exit criteria: python scripts/gen_synthetic.py --n 1000 -> 1000 samples, schema-OK,
all 6 categories balanced (~130-190 each).
2026-06-29 23:29:04 +08:00
1b4c6ddf4c
feat(m1): instruction & answer generation, 6 categories (T1.4)
...
- describe / anomaly / root_cause / forecast / compare / event
- JSON answers (stats, anomaly_regions aligned w/ labels, forecast), text for cause/event
- evolve_instruction: deterministic Evol-Instruct-style rephrasing
- tests/test_instruct.py (12 tests, RED->GREEN)
Exit criteria: anomaly JSON json.loads-able & regions reconstruct labels; full suite 37 passed.
2026-06-29 23:26:41 +08:00
e2c0756a78
feat(m1): synthesis — components + anomaly/event/missing (T1.3)
...
- generate_series: baseline + trend + seasonality + noise (opt heavy-tail)
- inject_anomaly: spike / level_shift / variance_change / missing_segment / drift
* non-overlapping segments (TDD caught an overlap bug) -> labels exactly
match contiguous [start,end) runs
* returns (series, point labels, segment metadata)
- couple_event: persistent per-channel step at t + event text
- add_missing: NaN drop at given rate
- tests/test_synthesis.py (14 tests, RED->GREEN; fixed overlap regression)
Exit criteria: seed-deterministic; labels cover segments; event step persistent.
2026-06-29 23:25:08 +08:00
6d10454484
feat(m1): attribute vocab + sampling (T1.2)
...
- ATTRIBUTE_VOCAB (16 AIOps channels w/ units)
- FREQ_BANDS (6 sampling resolutions)
- sample_attributes(n_channels, seed) -> unique-name, reproducible
- tests/test_attributes.py (8 tests, RED->GREEN)
Exit criteria: fields complete, count correct, seed-reproducible.
2026-06-29 23:22:40 +08:00
748bf41c53
feat(m1): scaffold tsmm package (T1.1)
...
- pyproject.toml (src-layout, deps per plan)
- src/tsmm/{data,model,train,eval} skeletons
- tests/test_smoke.py (RED->GREEN: importable + subpackages)
- configs/, scripts/, data/, checkpoints/ + .gitignore
- project venv (.venv) for reproducible env (M1 deps only; CUDA torch @ M2)
Verifications: import tsmm OK; pytest tests/ 3 passed.
2026-06-29 23:21:47 +08:00
张宗平