Hugging Face's logo

stmasson
/
training-scripts

Model card Files
xet
Community
stmasson commited on
Commit
1ee1b2b
·
verified ·
1 Parent(s): e237980

Upload train_ministral_n8n.py with huggingface_hub

Browse files
Files changed (1) hide show
  1. train_ministral_n8n.py +1 -1
train_ministral_n8n.py CHANGED
@@ -97,7 +97,7 @@ model = AutoModelForCausalLM.from_pretrained(
97
  device_map="auto",
98
  torch_dtype=torch.bfloat16,
99
  trust_remote_code=True,
100
- attn_implementation="flash_attention_2",
101
  )
102
  model = prepare_model_for_kbit_training(model)
103
 
 
97
  device_map="auto",
98
  torch_dtype=torch.bfloat16,
99
  trust_remote_code=True,
100
+ attn_implementation="sdpa", # Use PyTorch SDPA (no flash-attn install needed)
101
  )
102
  model = prepare_model_for_kbit_training(model)
103