feat(m2): multimodal splice [属性][时间戳][TS tok][事件][问题][回答] (T2.3)
- src/tsmm/model/multimodal.py: MultimodalSplicer — binds LLM input embedding layer, assembles text embeds (via tokenizer) + TS token embeds (from Projector) into inputs_embeds/attention_mask/labels; labels = -100 everywhere except answer+EOS (training); left-truncation to max_len=1024. - tests/test_multimodal.py: 8 tests against real Qwen2.5-0.5B tokenizer + stub embedding (shapes, all-ones mask, answer-only labels, TS token count in sequence, no-NaN, truncation). 74 tests passing. - configs/llm.yaml: offline model path (downloaded via proxy to ~/models/Qwen2.5-0.5B-Instruct; HF hub unreachable on host). Spec clarifications (small tier, comet-build Step 4): - Training appends answer + EOS to the prompt layout. - Build embedding layer with len(tokenizer) (incl. added special tokens), not tokenizer.vocab_size which excludes them (eos=151645).
This commit is contained in:
@@ -0,0 +1,10 @@
|
||||
# LLM backbone config (T2.3 / T2.4)
|
||||
# Design ref: docs/superpowers/specs/2026-06-29-ts-as-modality-design.md §2.2
|
||||
# Qwen2.5-0.5B-Instruct: hidden=896, 24 layers, vocab=151936, BF16 ~1GB
|
||||
|
||||
# Local offline path (downloaded via proxy; HF hub unreachable on this host)
|
||||
model_path: /home/zhangzp/models/Qwen2.5-0.5B-Instruct
|
||||
hf_model_id: Qwen/Qwen2.5-0.5B-Instruct # for reference; offline use model_path
|
||||
|
||||
hidden_size: 896
|
||||
torch_dtype: bfloat16
|
||||
Reference in New Issue
Block a user