 张宗平
|
f509d47878
|
feat(m2): multimodal splice [属性][时间戳][TS tok][事件][问题][回答] (T2.3)
- src/tsmm/model/multimodal.py: MultimodalSplicer — binds LLM input
embedding layer, assembles text embeds (via tokenizer) + TS token embeds
(from Projector) into inputs_embeds/attention_mask/labels; labels = -100
everywhere except answer+EOS (training); left-truncation to max_len=1024.
- tests/test_multimodal.py: 8 tests against real Qwen2.5-0.5B tokenizer +
stub embedding (shapes, all-ones mask, answer-only labels, TS token count
in sequence, no-NaN, truncation). 74 tests passing.
- configs/llm.yaml: offline model path (downloaded via proxy to
~/models/Qwen2.5-0.5B-Instruct; HF hub unreachable on host).
Spec clarifications (small tier, comet-build Step 4):
- Training appends answer + EOS to the prompt layout.
- Build embedding layer with len(tokenizer) (incl. added special tokens),
not tokenizer.vocab_size which excludes them (eos=151645).
|
2026-06-30 02:05:15 +00:00 |
|