张宗平
4c4a8f6435
- src/tsmm/eval/parse_answer.py: text→JSON segments→point scores (JSON-first, regex fallback, all-zero on fail). - src/tsmm/eval/ts_metrics.py: VUS-PR (main, threshold-free; constant-score → prevalence, NOT inflated), AUC-PR, Point-F1 (no PA), PA-F1 (control only), self-contained Affiliation-F1. - src/tsmm/eval/qa_judge.py: RuleJudge (anomaly IoU / describe+forecast numeric tolerance) + LLMJudge (stub backend = deterministic offline heuristic for the no-network host; openai backend = GPT-4o 0-5; pluggable callable). - src/tsmm/eval/baselines.py: trivial (Random/Constant/AllReport) + ts_to_text + pure_llm_answer glue; Time-LLM/ChatTS marked not-reproduced (honest). - src/tsmm/eval/report.py + scripts/run_eval.sh: trained model + all baselines through the SAME pipeline → reports/eval-YYYYMMDD.md (VUS-PR main, PA-F1 explicitly labelled control, trivial baselines included). - tests: 45 new (parse 13, metrics 12, judge 12, baselines 8). 145 passing. - Smoke: run_eval on 40 samples produces valid markdown; PA-F1≈1.0 vs VUS-PR≈0.35 for all_report demonstrates why PA must not be the headline.
tsmm
Time-series as a Modality — an experimental, single-GPU multimodal model that treats multivariate time series as an independent modality, jointly injected into a small LLM (Qwen2.5-0.5B) for time-series question answering and reasoning (ChatTS route).
Status
M1 (data pipeline) under construction. See
docs/superpowers/specs/2026-06-29-ts-as-modality-design.md for the design and
docs/superpowers/plans/2026-06-29-ts-as-modality-plan.md for the task plan.
Layout
src/tsmm/ package (data, model, train, eval)
configs/ training / data configs
scripts/ offline generation & training entrypoints
tests/ pytest suite
data/ generated datasets (gitignored)
checkpoints/ model checkpoints (gitignored)
Install (dev)
python3 -m venv .venv
. .venv/bin/activate
pip install -e . # full deps (incl. torch/transformers/peft/...)
# M1-only lightweight install:
pip install numpy tqdm pyyaml pytest && pip install -e . --no-deps