Spaces:

wjbmattingly
/

NuMarkdown-8B-Thinking-Demo

Running on Zero

William Mattingly commited on Aug 8

Commit

0e717ef

1 Parent(s): 3138c9c

removed flash attn

Files changed (1) hide show

app.py CHANGED Viewed

@@ -17,7 +17,7 @@ processor = AutoProcessor.from_pretrained(
 model = Qwen2_5_VLForConditionalGeneration.from_pretrained(
     model_id,
     torch_dtype=torch.bfloat16,
-    attn_implementation="flash_attention_2",
     device_map="auto",
     trust_remote_code=True,
 )

 model = Qwen2_5_VLForConditionalGeneration.from_pretrained(
     model_id,
     torch_dtype=torch.bfloat16,
+    attn_implementation="sdpa",
     device_map="auto",
     trust_remote_code=True,
 )