README.md

---
title: README
emoji: 🦀
colorFrom: red
colorTo: yellow
pinned: false
---

# rustformers - Large Language Models in Rust

rustformers is a group that wants to make it easy for Rust developers to access the power of large language models (LLMs).
As of June 2023, the focus is on keeping pace with the fast-moving [GGML](https://github.com/ggerganov/ggml) ecosystem -
a C library that uses quantization to make it possible to run LLMs on the CPU.
In the future, there will be support for other backends that allow running LLMs on GPUs, in Wasm environments, and more.

This 🤗 space will host models vetted by the rustformers team.

If you're a Rust developer and are looking to build with LLMs, keep reading to learn about `llm` - a Rust crate that is the entry point to the rustformers ecosystem.

## `llm`

The `llm` crate exposes an interface for working with LLMs and provides implementations for many popular model architectures,
including BLOOM, GPT-2, GPT-J, GPT-NeoX, LLaMA, and MPT.
The `llm` project includes a simple CLI for [interacting with LLMs](https://github.com/rustformers/llm#usage),
as well as [examples](https://github.com/rustformers/llm/tree/main/crates/llm/examples) of how to use `llm` in a Rust project.
To learn more about `llm`, visit its [GitHub repository](https://github.com/rustformers/llm)
or check out the [official documentation](https://docs.rs/llm) for released versions.

## Get in Touch!

We're building a community of enthusiastic developers and would love for you to join! If you're looking to chat, check out our [Discord](https://discord.gg/YB9WaXYAWU).
We also actively monitor GitHub [Issues](https://github.com/rustformers/llm/issues/new) and [Discussions](https://github.com/rustformers/llm/discussions/new).