Update README.md

README.md

@@ -31,11 +31,11 @@
 
 ---
 
-# Model Card for Model ID
+# Model Card for LLM Instruction‑Tuning for Text Classification (LoRA + QLoRA)
 
 <!-- Provide a quick summary of what the model is/does. -->
 
-This modelcard aims to be a base template for new models. It has been generated using [this raw template](https://github.com/huggingface/huggingface_hub/blob/main/src/huggingface_hub/templates/modelcard_template.md?plain=1).
+This repository provides code and configuration to fine‑tune a decoder‑only LLM (default: `meta-llama/Llama-3.2-1B`) for **instruction‑style text classification** using **LoRA/QLoRA**. Rather than training a task‑specific classifier head, the project formulates classification as a short instruction → answer generation task and evaluates by **exact string match** against the label. It includes simple training/inference scripts, a 5‑label arXiv‑style demo, and optional Amazon SageMaker entrypoints.
 
 ## Model Details
 
@@ -43,185 +43,225 @@ This modelcard aims to be a base template for new models. It has been generated
 
 <!-- Provide a longer summary of what this model is. -->
 
+This project instruction‑tunes a base, decoder‑only LLM with **LoRA adapters** loaded in **4‑bit NF4** precision for memory‑efficient training and inference. Supervised fine‑tuning is performed with TRL’s `SFTTrainer`. Prompts ask the model to “return the answer as the exact text label,” so predictions are decoded as plain text and compared by string match.
 
+- **Developed by:** Amirhossein Yousefi (GitHub: `amirhossein-yousefi`)
+- **Model type:** Decoder‑only LLM fine‑tuned with LoRA for single‑label text classification via instruction‑following
+- **Language(s) (NLP):** English by default (demo dataset uses arXiv titles/abstracts); broader multilingual coverage depends on the chosen base model
+- **License:** The repository itself does not include an explicit OSS license; the **base model** `meta-llama/Llama-3.2-1B` is governed by the **Llama 3.2 Community License**. You must accept and comply with Meta’s license to access and use the weights. 
+- **Finetuned from model :** `meta-llama/Llama-3.2-1B` (configurable)
 
-- **Developed by:** [More Information Needed]
-- **Funded by [optional]:** [More Information Needed]
-- **Shared by [optional]:** [More Information Needed]
-- **Model type:** [More Information Needed]
-- **Language(s) (NLP):** [More Information Needed]
-- **License:** [More Information Needed]
-- **Finetuned from model [optional]:** [More Information Needed]
-
-### Model Sources [optional]
+### Model Sources 
 
 <!-- Provide the basic links for the model. -->
 
-- **Repository:** [More Information Needed]
-- **Paper [optional]:** [More Information Needed]
-- **Demo [optional]:** [More Information Needed]
+- **Repository:** https://github.com/amirhossein-yousefi/LLM-Instruction-Tuning-Text-Classification
+- **Demo :** The repo includes an arXiv‑style 5‑label demo and example results; no hosted demo is provided.
 
 ## Uses
 
-<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
-
 ### Direct Use
 
-<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+- Fine‑tune LoRA adapters on your own CSV dataset for **single‑label text classification** (e.g., topic/category detection) using the provided `scripts/train.py`.
+- Run inference/evaluation with `scripts/predict.py` to generate deterministic label strings and compute **accuracy**, **micro/macro F1**, a **classification report**, and a **confusion matrix**.
+- Optional **Amazon SageMaker** utilities let you run managed training and deploy a real‑time endpoint with the LoRA adapters attached at load time.
 
-[More Information Needed]
+### Downstream Use 
 
-### Downstream Use [optional]
+- Integrate the trained LoRA adapters into applications where explainable, instruction‑driven classification is helpful (e.g., routing, tagging, moderation). 
+- Swap the base model (any compatible decoder‑only LLM on the Hugging Face Hub) and re‑train with the same prompt template.
+- Extend label sets without architectural changes—only prompt/label lists need to be updated.
 
-<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+### Out-of-Scope Use
 
-[More Information Needed]
+- **CPU‑only** training/inference with this repo as‑is (4‑bit `bitsandbytes` path expects NVIDIA CUDA GPUs).
+- **Multi‑label** classification (comma‑separated outputs) is not implemented out of the box (listed as a roadmap idea).
+- **Open‑domain generation** or safety‑critical decision‑making; this project focuses on label selection with short inputs.
 
-### Out-of-Scope Use
+## Bias, Risks, and Limitations
 
-<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+- Outputs mirror biases in the **training corpus** you provide and in the **base model**. If your labels or examples are imbalanced or ambiguous, the model may propagate that bias.
+- Exact‑match decoding can be brittle to **tokenization/typo** effects—ensure labels are short, canonical strings and restrict the decoding space.
+- The base Llama 3.2 model has its own safety limitations and license‑based usage constraints (e.g., attribution and acceptable‑use provisions). 
+- The demo dataset is limited to **5 arXiv‑style labels** and relatively short academic texts; generalizing beyond this domain requires additional data.
 
-[More Information Needed]
+### Recommendations
 
-## Bias, Risks, and Limitations
+- Curate balanced datasets; consider **stratified splits** and per‑class metrics.
+- Keep **temperature = 0.0** for deterministic label decoding; constrain generation length (e.g., `max_new_tokens=8`).
+- Validate robustness with **label synonyms/aliases** and adversarial cases; consider post‑processing that maps variants to canonical labels.
+- Review and comply with the **Llama 3.2 Community License** (and any other upstream licenses) when distributing adapters/derivatives.
 
-<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+## How to Get Started with the Model
 
-[More Information Needed]
+**Install & train**
 
-### Recommendations
+```bash
+python -m venv .venv
+source .venv/bin/activate   # Windows: .venv\Scripts\Activate.ps1
+pip install --upgrade pip
+pip install -r requirements.txt
 
-<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+# If the base model is gated, export an HF token
+export HF_TOKEN=YOUR_HF_ACCESS_TOKEN
 
-Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+# One‑command training on CSVs
+python scripts/train.py   --base_path dataset   --train_file train.csv   --val_file validation.csv   --test_file test.csv   --label_column label_name   --text_fields title abstract   --base_model_name meta-llama/Llama-3.2-1B   --output_dir llama-3.2-1b-arxiver-lora
+```
 
-## How to Get Started with the Model
+**Inference & evaluation**
+
+```bash
+python scripts/predict.py   --base_path dataset   --test_file test.csv   --base_model_name meta-llama/Llama-3.2-1B   --output_dir llama-3.2-1b-arxiver-lora   --save_csv predictions.csv
+```
 
-Use the code below to get started with the model.
+**SageMaker **
 
-[More Information Needed]
+```bash
+# Train a managed job
+python sagemaker/train_sm.py   --source_dir .   --dataset_dir dataset   --train_file train.csv   --val_file validation.csv   --test_file test.csv   --label_column label_name   --text_fields title abstract   --base_model_id meta-llama/Llama-3.2-1B   --instance_type ml.g5.2xlarge   --instance_count 1
+
+# Deploy a real‑time endpoint
+python sagemaker/deploy_sm.py   --training_job_name <your-job>   --base_model_id meta-llama/Llama-3.2-1B   --instance_type ml.g5.2xlarge   --default_labels_json '["cs.CL","cs.CV","cs.LG","hep-ph","quant-ph"]'
+```
 
 ## Training Details
 
 ### Training Data
 
-<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
-
-[More Information Needed]
+- Expected input: three CSV files under a base path: `train.csv`, `validation.csv`, `test.csv`.
+- Required columns: a **label** column (default `label_name`) and one or more text fields (defaults: `title`, `abstract`). Missing/blank text fields are skipped; text fields are concatenated with punctuation.
+- The repository ships utilities to prepare a **5‑class arXiv‑style demo** (labels: `['cs.CL','cs.CV','cs.LG','hep-ph','quant-ph']`).
 
 ### Training Procedure
 
-<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
-
-#### Preprocessing [optional]
-
-[More Information Needed]
+#### Preprocessing 
 
+- Prompts are constructed as short instruction → answer pairs:
+  - **Train:** includes the gold label after `label:`.
+  - **Inference:** leaves `label:` empty and decodes the generated label.
 
 #### Training Hyperparameters
 
-- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
-
-#### Speeds, Sizes, Times [optional]
+- **Training regime:** mixed precision with `fp16=True`, `tf32=True`; 4‑bit NF4 quantization with bfloat16 compute (QLoRA‑style).
+- **Selected defaults (single‑GPU):**
+  - `num_train_epochs=1`
+  - `per_device_train_batch_size=8`, `per_device_eval_batch_size=8`
+  - `gradient_accumulation_steps=2` (effective 16 per step, per device)
+  - `learning_rate=2e-4`, `weight_decay=1e-3`, `warmup_ratio=0.03`
+  - `logging_steps=10`, `evaluation_strategy="epoch"`, `save_strategy="epoch"`, `save_total_limit=2`
+  - LoRA: `r=2`, `alpha=2`, `dropout=0.0`
+  - Quantization: `load_in_4bit=True`, `bnb_4bit_quant_type="nf4"`, `bnb_4bit_compute_dtype="bfloat16"`, `bnb_4bit_use_double_quant=True`
+  - Generation (eval): `temperature=0.0`, `max_new_tokens=8`, `do_sample=False`
 
-<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+#### Speeds, Sizes, Times 
 
-[More Information Needed]
+- Example environment: Laptop RTX 3080 Ti (16 GB VRAM), CUDA 12.9, PyTorch 2.8.0+cu129.
+- Example run stats: ~6,314 seconds wall‑clock training, with TensorBoard logs under the run directory.
+- Total training FLOPs (example): ~3.69e16 (as reported by the training logs).
 
 ## Evaluation
 
-<!-- This section describes the evaluation protocols and provides the results. -->
-
 ### Testing Data, Factors & Metrics
 
 #### Testing Data
 
-<!-- This should link to a Dataset Card if possible. -->
-
-[More Information Needed]
+- The example evaluation uses the provided arXiv‑style 5‑label test split.
 
 #### Factors
 
-<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
-
-[More Information Needed]
+- Per‑class metrics are reported for `cs.CL`, `cs.CV`, `cs.LG`, `hep-ph`, `quant-ph`.
 
 #### Metrics
 
-<!-- These are the evaluation metrics being used, ideally with a description of why. -->
-
-[More Information Needed]
+- Accuracy, micro F1, macro F1, per‑class precision/recall/F1, and a confusion matrix.
 
 ### Results
 
-[More Information Needed]
+- **Overall:** Accuracy 93.8%, Micro‑F1 0.938, Macro‑F1 0.950.
+- **Per‑class (Precision / Recall / F1 / Support):**
+  - `cs.CL`: 0.914 / 0.963 / 0.938 / 432
+  - `cs.CV`: 0.935 / 0.923 / 0.929 / 545
+  - `cs.LG`: 0.917 / 0.890 / 0.903 / 536
+  - `hep-ph`: 0.994 / 0.988 / 0.991 / 164
+  - `quant-ph`: 0.986 / 0.990 / 0.988 / 293
 
 #### Summary
 
+The LoRA‑tuned 1B parameter Llama 3.2 model achieves strong performance on short academic texts while keeping training/inference affordable due to 4‑bit quantization. Performance is consistent across most classes, with particularly high scores for physics categories.
 
+## Model Examination 
 
-## Model Examination [optional]
-
-<!-- Relevant interpretability work for the model goes here -->
-
-[More Information Needed]
+- The repo includes utilities for a **classification report** and **confusion matrix**. Inspect misclassifications to refine label definitions or add examples. Consider probing sensitivity to prompt wording.
 
 ## Environmental Impact
 
-<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
-
-Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+*(Approximate; depends on your hardware and run length.)*  
+Use the [MLCO2 Impact calculator](https://mlco2.github.io/impact#compute) with your GPU model, power draw, and wall‑clock runtime.
 
-- **Hardware Type:** [More Information Needed]
-- **Hours used:** [More Information Needed]
-- **Cloud Provider:** [More Information Needed]
-- **Compute Region:** [More Information Needed]
-- **Carbon Emitted:** [More Information Needed]
+- **Hardware Type:** Single NVIDIA GPU (example: RTX 3080 Ti Laptop 16 GB)
+- **Hours used:** ~1.75 hours (example)
+- **Cloud Provider:** N/A (local) in example; SageMaker supported
+- **Compute Region:** N/A (local) or your chosen AWS region
+- **Carbon Emitted:** Not estimated
 
-## Technical Specifications [optional]
+## Technical Specifications 
 
 ### Model Architecture and Objective
 
-[More Information Needed]
+- **Architecture:** Decoder‑only Transformer (Llama 3.2 family when using the default base)
+- **Objective:** Supervised instruction‑tuning for **single‑label classification** via generative decoding with exact‑match evaluation
+- **Context length:** 512 tokens (config default; pass explicitly to trainer to ensure enforcement)
 
 ### Compute Infrastructure
 
-[More Information Needed]
-
 #### Hardware
 
-[More Information Needed]
+- NVIDIA CUDA GPU required for 4‑bit `bitsandbytes` training/inference  
+  (CPU‑only runs are not supported by the included scripts).
 
 #### Software
 
-[More Information Needed]
+- Python ≥ 3.10, PyTorch, `transformers`, `trl`, `peft`, `bitsandbytes`, `accelerate`, and standard scientific Python packages.
+- Optional: Astral’s `uv` for faster, reproducible dependency management (the repo also ships `requirements.txt`).
 
-## Citation [optional]
+## Citation 
 
-<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+If you use this repository, please cite the GitHub project and the base model as appropriate.
 
-**BibTeX:**
-
-[More Information Needed]
-
-**APA:**
+**BibTeX (project):**
+```bibtex
+@software{yousefi_2025_llm_instruction_tuning_text_classification,
+  author    = {Yousefi, Amirhossein},
+  title     = {LLM Instruction-Tuning for Text Classification (LoRA + QLoRA)},
+  year      = {2025},
+  publisher = {GitHub},
+  url       = {https://github.com/amirhossein-yousefi/LLM-Instruction-Tuning-Text-Classification}
+}
+```
 
-[More Information Needed]
+**APA (project):**  
+Yousefi, A. (2025). *LLM Instruction‑Tuning for Text Classification (LoRA + QLoRA)*. GitHub. https://github.com/amirhossein-yousefi/LLM-Instruction-Tuning-Text-Classification
 
-## Glossary [optional]
+**Base model:** Meta AI. (2024). *Llama 3.2‑1B* [Computer software]. Meta. https://huggingface.co/meta-llama/Llama-3.2-1B
 
-<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+## Glossary 
 
-[More Information Needed]
+- **LoRA:** Low‑Rank Adapters for parameter‑efficient fine‑tuning.
+- **QLoRA:** LoRA training with quantized base weights (typically 4‑bit NF4) and higher‑precision compute.
+- **SFT:** Supervised Fine‑Tuning.
+- **Exact‑match decoding:** Evaluates whether the generated label text exactly matches the gold label string.
 
-## More Information [optional]
+## More Information 
 
-[More Information Needed]
+- Amazon SageMaker scripts are included for managed training and deployment.
+- Roadmap ideas include multi‑label support and few‑shot exemplars in prompts.
 
-## Model Card Authors [optional]
+## Model Card Authors 
 
-[More Information Needed]
+- Drafted by: ChatGPT (based on the repository’s README and code structure)
+- Repository author: Amirhossein Yousefi
 
 ## Model Card Contact
 
-[More Information Needed]
+- Open an issue on the GitHub repository for questions or contributions.