LM Studio Support with Q4_K_S please?

#5
by Cagannn - opened
Files changed (1) hide show
  1. README.md +2 -4
README.md CHANGED
@@ -25,8 +25,6 @@ library_name: transformers
25
  | IQuest-Coder-V1-40B-Instruct | [🤗 Hugging Face](https://huggingface.co/IQuestLab/IQuest-Coder-V1-40B-Instruct) |
26
  | IQuest-Coder-V1-40B-Loop-Instruct | [🤗 Hugging Face](https://huggingface.co/IQuestLab/IQuest-Coder-V1-40B-Loop-Instruct) |
27
 
28
- [Clarification: Regarding the Performance of IQuest-Coder-V1](https://github.com/IQuestLab/IQuest-Coder-V1/issues/14#issuecomment-3705756919)
29
-
30
  ## Sampling Parameters
31
  For the IQuest-Coder-V1-Instruct: We suggest using Temperature=0.6, TopP=0.85, TopK=20.
32
 
@@ -124,13 +122,13 @@ model_name = "IQuest/IQuest-Coder-V1-40B-Thinking"
124
  For production deployment, you can use vLLM to create an OpenAI-compatible API endpoint. Please refer to the [vLLM PR](https://github.com/vllm-project/vllm/pull/31575/files) for implementation details.
125
 
126
  ```bash
127
- vllm serve IQuestLab/IQuest-Coder-V1-40B-Instruct --tensor-parallel-size 8
128
  ```
129
 
130
  For Thinking models with reasoning support:
131
 
132
  ```bash
133
- vllm serve IQuestLab/IQuest-Coder-V1-40B-Thinking --reasoning-parser qwen3 --tensor-parallel-size 8
134
  ```
135
 
136
 
 
25
  | IQuest-Coder-V1-40B-Instruct | [🤗 Hugging Face](https://huggingface.co/IQuestLab/IQuest-Coder-V1-40B-Instruct) |
26
  | IQuest-Coder-V1-40B-Loop-Instruct | [🤗 Hugging Face](https://huggingface.co/IQuestLab/IQuest-Coder-V1-40B-Loop-Instruct) |
27
 
 
 
28
  ## Sampling Parameters
29
  For the IQuest-Coder-V1-Instruct: We suggest using Temperature=0.6, TopP=0.85, TopK=20.
30
 
 
122
  For production deployment, you can use vLLM to create an OpenAI-compatible API endpoint. Please refer to the [vLLM PR](https://github.com/vllm-project/vllm/pull/31575/files) for implementation details.
123
 
124
  ```bash
125
+ vllm serve IQuest/IQuest-Coder-V1-40B-Instruct --tensor-parallel-size 8
126
  ```
127
 
128
  For Thinking models with reasoning support:
129
 
130
  ```bash
131
+ vllm serve IQuest/IQuest-Coder-V1-40B-Thinking --reasoning-parser qwen3 --tensor-parallel-size 8
132
  ```
133
 
134