Spaces:

AstroMLab
/

AstroSage-8B

Runtime error

Tijmen2 commited on Nov 20, 2024

Commit

b39668d

verified ·

1 Parent(s): a05c286

Update app.py

Files changed (1) hide show

app.py CHANGED Viewed

@@ -24,7 +24,7 @@ model = AutoModelForCausalLM.from_pretrained(
     device_map="auto",
     use_safetensors=True,
     trust_remote_code=True,
-    load_in_4bit=True,
     torch_dtype=torch.bfloat16
 )

     device_map="auto",
     use_safetensors=True,
     trust_remote_code=True,
+    load_in_8bit=True,
     torch_dtype=torch.bfloat16
 )