Qwen3.5-397B-A17B LoRA SFT Adapter (v2 - 3-view, 32K)
LoRA adapter fine-tuned on AMD GPU kernel optimization agent trajectories using LLaMA-Factory.
Training Details
| Metric | Value |
|---|---|
| Base model | Qwen/Qwen3.5-397B-A17B (397B MoE, 17B active) |
| Final train loss | 0.0848 |
| LoRA rank / alpha | 16 / 32 |
| Target modules | all (13 types: q/k/v/o_proj, gate/up/down_proj, shared_expert_gate, in_proj) |
| Trainable params | 64.2M / 396.9B (0.016%) |
| cutoff_len | 32,768 |
| Training examples | 296 (3-view extraction from 102 raw trajectories) |
| Epochs | 3 |
| Runtime | 1h 32 min |
| Hardware | 8x AMD Instinct MI355X, DeepSpeed ZeRO-3 |
| wandb | amdpilot/amdpilot-sft/runs/8hh4pmqx |
Usage
from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer
base_model = AutoModelForCausalLM.from_pretrained(
"Qwen/Qwen3.5-397B-A17B", device_map="auto", torch_dtype="bfloat16"
)
model = PeftModel.from_pretrained(base_model, "JinnP/Qwen3.5-397B-A17B-LoRA-SFT-v2")
- Downloads last month
- 1
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support
Model tree for JinnP/Qwen3.5-397B-A17B-LoRA-SFT-v2
Base model
Qwen/Qwen3.5-397B-A17B