 张宗平
|
357e2e60bd
|
feat(m4): stage 2 SFT training script (LoRA r=16) + smoke (T4.1)
- src/tsmm/train/stage2.py: loads stage1 ckpt (optional), freeze_llm +
enable_lora(r=16) on q/v_proj, masked LM loss on sft.jsonl, bs=4/grad_accum=8/
ctx=1024/BF16/gradient checkpointing, ckpt every 2000 steps (encoder+projector
state + peft LoRA adapter via save_pretrained).
- scripts/train_stage2.sh: wrapper with design defaults.
- Smoke verified: --max_steps 4, LoRA injected, finite loss, peak 3.29 GB
(≤11GB budget), ckpt+adapter saved.
|
2026-06-30 03:11:00 +00:00 |
|