mrfakename

/

Ministral-3-8B-Instruct-2512-Llamafied-TextOnly

like 1

what went wrong?

what went wrong?

Using https://huggingface.co/spaces/ggml-org/gguf-my-repo

I get the following error:
Error converting to fp16: INFO:hf-to-gguf:Loading model: Ministral-3-8B-Instruct-2512-Llamafied-TextOnly
INFO:hf-to-gguf:Model architecture: LlamaForCausalLM
INFO:hf-to-gguf:gguf: loading model weight map from 'model.safetensors.index.json'
INFO:hf-to-gguf:gguf: indexing model part 'model.safetensors'
INFO:gguf.gguf_writer:gguf: This GGUF file is for Little Endian only
INFO:hf-to-gguf:Exporting model...
INFO:hf-to-gguf:output.weight, torch.bfloat16 --> F16, shape = {4096, 131072}
INFO:hf-to-gguf:token_embd.weight, torch.bfloat16 --> F16, shape = {4096, 131072}
INFO:hf-to-gguf:blk.0.attn_norm.weight, torch.bfloat16 --> F32, shape = {4096}
INFO:hf-to-gguf:blk.0.ffn_down.weight, torch.float8_e4m3fn --> F16, shape = {14336, 4096}
Traceback (most recent call last):
File "/home/user/app/./llama.cpp/convert_hf_to_gguf.py", line 10437, in
main()
File "/home/user/app/./llama.cpp/convert_hf_to_gguf.py", line 10431, in main
model_instance.write()
File "/home/user/app/./llama.cpp/convert_hf_to_gguf.py", line 660, in write
self.prepare_tensors()
File "/home/user/app/./llama.cpp/convert_hf_to_gguf.py", line 2511, in prepare_tensors
super().prepare_tensors()
File "/home/user/app/./llama.cpp/convert_hf_to_gguf.py", line 531, in prepare_tensors
for new_name, data_torch in (self.modify_tensors(data_torch, name, bid)):
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/user/app/./llama.cpp/convert_hf_to_gguf.py", line 2478, in modify_tensors
return [(self.map_tensor_name(name), data_torch)]
^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/user/app/./llama.cpp/convert_hf_to_gguf.py", line 490, in map_tensor_name
raise ValueError(f"Can not map tensor {name!r}")
ValueError: Can not map tensor 'model.layers.0.mlp.down_proj.weight_scale_inv'

Hmm... I'm not sure... I can try re-converting it though I'm not sure that is the issue

I didn't expect it to work with the other one after this failed but it did and I got the gguf to run.

Can you run this model normally (non-GGUF)?
And are you able to convert the 3B reasoning model?

Can you run this model normally (non-GGUF)?
And are you able to convert the 3B reasoning model?

I don't have a setup to run non gguf models, I could try the 3b one later but why try the reasoning one if the instruct one makes trouble?

Alright, let me reconvert it.

Ah, my bad. Looks like only half of the model got uploaded.

It actually looks like it is an issue because the Instruct models are FP8 quantized. I will take a look and see if I can dequantize them and re-upload.

Hi, I've uploaded the latest checkpoint that should be fixed. Let me know if this is still a problem

Hi, I've uploaded the latest checkpoint that should be fixed. Let me know if this is still a problem

I tried it again the error is different this time it seems to be the tokenizer but I see that you have a tokenizer file so it's not missing.

File "/home/user/.pyenv/versions/3.11.14/lib/python3.11/site-packages/transformers/models/auto/tokenization_auto.py", line 1153, in from_pretrained
raise ValueError(
ValueError: Tokenizer class TokenizersBackend does not exist or is not currently imported.