Husnain
commited on
⚡ [Enhance] Use nous-mixtral-8x7b as default model
Browse files
README.md
CHANGED
|
@@ -13,10 +13,12 @@ app_port: 23333
|
|
| 13 |
|
| 14 |
Huggingface LLM Inference API in OpenAI message format.
|
| 15 |
|
|
|
|
|
|
|
| 16 |
## Features
|
| 17 |
|
| 18 |
-
- Available Models (2024/04/
|
| 19 |
-
- `mistral-7b`, `mixtral-8x7b`, `nous-mixtral-8x7b`, `gemma-7b`, `gpt-3.5-turbo`
|
| 20 |
- Adaptive prompt templates for different models
|
| 21 |
- Support OpenAI API format
|
| 22 |
- Enable api endpoint via official `openai-python` package
|
|
@@ -77,7 +79,7 @@ api_key = "hf_xxxxxxxxxxxxxxxx"
|
|
| 77 |
|
| 78 |
client = OpenAI(base_url=base_url, api_key=api_key)
|
| 79 |
response = client.chat.completions.create(
|
| 80 |
-
model="mixtral-8x7b",
|
| 81 |
messages=[
|
| 82 |
{
|
| 83 |
"role": "user",
|
|
@@ -116,7 +118,7 @@ api_key = "hf_xxxxxxxxxxxxxxxx"
|
|
| 116 |
|
| 117 |
requests_headers = {}
|
| 118 |
requests_payload = {
|
| 119 |
-
"model": "mixtral-8x7b",
|
| 120 |
"messages": [
|
| 121 |
{
|
| 122 |
"role": "user",
|
|
|
|
| 13 |
|
| 14 |
Huggingface LLM Inference API in OpenAI message format.
|
| 15 |
|
| 16 |
+
Project link: https://github.com/Niansuh/HF-LLM-API
|
| 17 |
+
|
| 18 |
## Features
|
| 19 |
|
| 20 |
+
- Available Models (2024/04/20):
|
| 21 |
+
- `mistral-7b`, `mixtral-8x7b`, `nous-mixtral-8x7b`, `gemma-7b`, `command-r-plus`, `llama3-70b`, `zephyr-141b`, `gpt-3.5-turbo`
|
| 22 |
- Adaptive prompt templates for different models
|
| 23 |
- Support OpenAI API format
|
| 24 |
- Enable api endpoint via official `openai-python` package
|
|
|
|
| 79 |
|
| 80 |
client = OpenAI(base_url=base_url, api_key=api_key)
|
| 81 |
response = client.chat.completions.create(
|
| 82 |
+
model="nous-mixtral-8x7b",
|
| 83 |
messages=[
|
| 84 |
{
|
| 85 |
"role": "user",
|
|
|
|
| 118 |
|
| 119 |
requests_headers = {}
|
| 120 |
requests_payload = {
|
| 121 |
+
"model": "nous-mixtral-8x7b",
|
| 122 |
"messages": [
|
| 123 |
{
|
| 124 |
"role": "user",
|