 张宗平
|
f509d47878
|
feat(m2): multimodal splice [属性][时间戳][TS tok][事件][问题][回答] (T2.3)
- src/tsmm/model/multimodal.py: MultimodalSplicer — binds LLM input
embedding layer, assembles text embeds (via tokenizer) + TS token embeds
(from Projector) into inputs_embeds/attention_mask/labels; labels = -100
everywhere except answer+EOS (training); left-truncation to max_len=1024.
- tests/test_multimodal.py: 8 tests against real Qwen2.5-0.5B tokenizer +
stub embedding (shapes, all-ones mask, answer-only labels, TS token count
in sequence, no-NaN, truncation). 74 tests passing.
- configs/llm.yaml: offline model path (downloaded via proxy to
~/models/Qwen2.5-0.5B-Instruct; HF hub unreachable on host).
Spec clarifications (small tier, comet-build Step 4):
- Training appends answer + EOS to the prompt layout.
- Build embedding layer with len(tokenizer) (incl. added special tokens),
not tokenizer.vocab_size which excludes them (eos=151645).
|
2026-06-30 02:05:15 +00:00 |
|
|
张宗平
|
545802d62c
|
feat(m2): TS Encoder — Patchify + 2-layer Transformer (T2.1)
- src/tsmm/model/ts_encoder.py: Patchify (channel-independent unfold) +
TSEncoder (shared per-channel patch linear, mean-pool over channels,
learnable positional embedding, 2-layer pre-norm Transformer, LayerNorm).
- configs/ts_encoder.yaml: encoder/projector hyperparameters.
- tests/test_ts_encoder.py: 10 tests (shape, n_patches formula, window
values, determinism, gradient flow, param sanity).
- 61 tests passing; GPU fwd/bwd verified on RTX 3060 (39 MB peak).
Spec clarifications (small tier, comet-build Step 4):
- n_patches = (T-P)//S+1 ⇒ 127 for T=512 (design said 128; arithmetic slip).
- 'Channel-independent' = shared per-channel patch embed + mean-pool →
C-free output [B,n_patches,d]; channel identity carried by [属性] tokens.
- Actual params ≈2.1M (design's '≈4M' was a rough estimate).
|
2026-06-30 00:16:54 +00:00 |
|
|
张宗平
|
748bf41c53
|
feat(m1): scaffold tsmm package (T1.1)
- pyproject.toml (src-layout, deps per plan)
- src/tsmm/{data,model,train,eval} skeletons
- tests/test_smoke.py (RED->GREEN: importable + subpackages)
- configs/, scripts/, data/, checkpoints/ + .gitignore
- project venv (.venv) for reproducible env (M1 deps only; CUDA torch @ M2)
Verifications: import tsmm OK; pytest tests/ 3 passed.
|
2026-06-29 23:21:47 +08:00 |
|