CohereLabs

/

tiny-aya-global

like 50

Follow

Cohere Labs 2.17k

Hi CohereLabs

Congratulations to get 67 languages into such a 'tiny' model!

Thank you for already providing the GGUF quants of your new model. 👍 Would you perhaps consider to also provide MLX quants for people with Apple Macs and iPhones?

I would suggest BF16 for Apple Macs, and 4-bit versions for iPhones.

# Install MLX
pip install -U mlx-lm

# BF16 quants for Apple Macs
mlx_lm.convert --hf-path CohereLabs/tiny-aya-global --mlx-path tiny-aya-global-mlx-bf16 --dtype bfloat16
mlx_lm.convert --hf-path CohereLabs/tiny-aya-water --mlx-path tiny-aya-water-mlx-bf16 --dtype bfloat16
mlx_lm.convert --hf-path CohereLabs/tiny-aya-earth --mlx-path tiny-aya-earth-mlx-bf16 --dtype bfloat16
mlx_lm.convert --hf-path CohereLabs/tiny-aya-fire --mlx-path tiny-aya-fire-mlx-bf16 --dtype bfloat16

# 4-bit quants for Apple iPhones
mlx_lm.convert --hf-path CohereLabs/tiny-aya-global --mlx-path tiny-aya-global-mlx-4bit -q --q-bits 4
mlx_lm.convert --hf-path CohereLabs/tiny-aya-water --mlx-path tiny-aya-water-mlx-4bit -q --q-bits 4
mlx_lm.convert --hf-path CohereLabs/tiny-aya-earth --mlx-path tiny-aya-earth-mlx-4bit -q --q-bits 4
mlx_lm.convert --hf-path CohereLabs/tiny-aya-fire --mlx-path tiny-aya-fire-mlx-4bit -q --q-bits 4