| | --- |
| | license: cc-by-sa-4.0 |
| | metrics: |
| | - accuracy |
| | pipeline_tag: text-generation |
| | tags: |
| | - code |
| | --- |
| | # Exllama v2 defog/llama-3-sqlcoder-8b |
| |
|
| | Using <a href="https://github.com/turboderp/exllamav2/releases/tag/v0.0.21">turboderp's ExLlamaV2 v0.0.21</a> for quantization. |
| |
|
| | <b>The "main" branch only contains the measurement.json, download one of the other branches for the model</b> |
| |
|
| | Each branch contains an individual bits per weight, with the main one containing only the meaurement.json for further conversions. |
| |
|
| | Original model: <a href="https://huggingface.co/defog/llama-3-sqlcoder-8b">defog/llama-3-sqlcoder-8b</a><br> |
| | Calibration dataset is based on: <a href="https://huggingface.co/datasets/m-a-p/CodeFeedback-Filtered-Instruction">m-a-p/CodeFeedback-Filtered-Instruction</a> |
| |
|
| | ## Available sizes |
| |
|
| | | Branch | Bits | lm_head bits | Description | |
| | | ----- | ---- | ------- | ------------ | |
| | | [8_0](https://huggingface.co/Apel-sin/llama-3-sqlcoder-8B-exl2/tree/8_0) | 8.0 | 8.0 | Maximum quality that ExLlamaV2 can produce, near unquantized performance. | |
| | | [6_5](https://huggingface.co/Apel-sin/llama-3-sqlcoder-8B-exl2/tree/6_5) | 6.5 | 8.0 | Very similar to 8.0, good tradeoff of size vs performance, **recommended**. | |
| | | [5_5](https://huggingface.co/Apel-sin/llama-3-sqlcoder-8B-exl2/tree/5_5) | 5.5 | 8.0 | Slightly lower quality vs 6.5, but usable on 8GB cards. | |
| | |
| | A capable language model for text to SQL generation for Postgres, Redshift and Snowflake that is on-par with the most capable generalist frontier models. |
| | |
| |  |
| | |
| | ## Model Description |
| | |
| | Developed by: Defog, Inc |
| | Model type: [Text to SQL] |
| | License: [CC-by-SA-4.0] |
| | Finetuned from model: [Meta-Llama-3-8B-Instruct] |
| | |
| | ## Demo Page |
| | [https://defog.ai/sqlcoder-demo/](https://defog.ai/sqlcoder-demo/) |
| | |
| | ## Ideal prompt and inference parameters |
| | Set temperature to 0, and do not do sampling. |
| | |
| | ### Prompt |
| | ``` |
| | <|begin_of_text|><|start_header_id|>user<|end_header_id|> |
| | |
| | Generate a SQL query to answer this question: `{user_question}` |
| | {instructions} |
| |
|
| | DDL statements: |
| | {create_table_statements}<|eot_id|><|start_header_id|>assistant<|end_header_id|> |
| | |
| | The following SQL query best answers the question `{user_question}`: |
| | ```sql |
| | |
| | ``` |
| |
|
| | ## Evaluation |
| | This model was evaluated on SQL-Eval, a PostgreSQL based evaluation framework developed by Defog for testing and alignment of model capabilities. |
| |
|
| | You can read more about the methodology behind SQLEval [here](https://defog.ai/blog/open-sourcing-sqleval/). |
| |
|
| | ## Contact |
| | Contact us on X at [@defogdata](https://twitter.com/defogdata), or on email at founders@defog.ai |
