Update README.md

README.md

@@ -3,17 +3,90 @@ library_name: transformers
 tags:
 - finance
 - personal_finance
+- lora
+- Deepseek-Distill
 license: mit
 datasets:
 - Akhil-Theerthala/PersonalFinance_v2
 base_model:
 - deepseek-ai/DeepSeek-R1-Distill-Qwen-14B
+pipeline_tag: text-generation
 ---
 
+# Personal Finance Reasoning LoRA Model
+
+This model is a LoRA-adapted version of `deepseek-ai/DeepSeek-R1-Distill-Qwen-14b`, specifically fine-tuned to enhance reasoning capabilities for complex queries within the personal finance domain.
+
 ## Model Details
 
-### Model Description
-This model is trained upon the personal finance dataset accesible here. 
-This is a LoRA adapted version of the model.
+- **Developed by:** Akhil-Theerthala
+- **Model type:** Language model fine-tuned for causal language modeling and reasoning.
+- **Language(s) (NLP):** English
+- **License:** MIT
+- **Finetuned from model:** `deepseek-ai/DeepSeek-R1-Distill-Qwen-14b`
+- **Model Description:** Built upon the `deepseek-ai/DeepSeek-R1-Distill-Qwen-14b` base model, this adaptation is specifically engineered to address nuanced personal finance questions. By fine-tuning with the `Akhil-Theerthala/PersonalFinance_v2` dataset, which comprises diverse scenarios requiring contextual understanding and multi-step reasoning, the model aims to generate more relevant and coherent responses to financial inquiries. The use of LoRA facilitates this specialization efficiently, targeting key aspects of financial reasoning such as query analysis and context processing inherent in the training data.
+- **Resources for more information:**
+    - Dataset: [Akhil-Theerthala/PersonalFinance_v2](https://huggingface.co/datasets/Akhil-Theerthala/PersonalFinance_v2)
+    - Base Model: [deepseek-ai/DeepSeek-R1-Distill-Qwen-14b](https://huggingface.co/deepseek-ai/DeepSeek-R1-Distill-Qwen-14b)
+
+## Training Data
+The model was fine-tuned on the `Akhil-Theerthala/PersonalFinance_v2` dataset.
+
+- **Dataset Description:** This dataset contains approximately ~7.04k samples designed for personal finance reasoning. It includes diverse scenarios and queries related to financial advice, planning, and problem-solving. The data is structured to facilitate tasks like question answering and text generation requiring financial reasoning, often involving analysis of context and complex query structures.
+- **Data Size:** ~7,000 training instances.
+- **Format:** JSON
+- **Source:** [Akhil-Theerthala/PersonalFinance_v2 on Hugging Face](https://huggingface.co/datasets/Akhil-Theerthala/PersonalFinance_v2)
+
+## Training Procedure
+
+### Adaptation Method:
+The base model `DeepSeek-R1-Distill-Qwen-14b` (14B parameters) was adapted using Low-Rank Adaptation (LoRA). This method was chosen due to the dataset size (~7k samples), making it suitable for parameter-efficient fine-tuning rather than full fine-tuning.
+
+### Hyperparameters:
+- **LoRA r (rank):** 64
+- **LoRA alpha:** 128
+- **LoRA dropout:** 0.1
+- **Target modules:** all-linear
+- **Optimizer:** Adam
+- **Learning rate:** 1e-4
+- **Batch size:** 40
+- **Number of epochs:** 8
 
-{Further details will be added soon}
+
+## Evaluation
+- **The Evaluation is currently under process, with the major method of evaluation using LLMs-as-Judge framework, preferably Gemini-2.5-Flash as the judge. The Benchmarks used and the comparisons done will be shared shortly.** 
+### Intended Uses:
+- Assisting users with personal finance queries by providing reasoned explanations and advice.
+- Educational tool for understanding financial concepts.
+- Generating text related to financial scenarios.
+- Research in domain-specific reasoning and parameter-efficient fine-tuning.
+
+### Limitations:
+
+- **Knowledge Cutoff:** The model's knowledge is limited to the information present in its training data (both the base model's pre-training and the `PersonalFinance_v2` fine-tuning dataset). It may not be aware of very recent financial events, regulations, or products.
+- **Potential Biases:** The model may reflect biases present in the training data.
+- **Hallucinations:** Like all large language models, it may occasionally generate plausible-sounding but incorrect information (hallucinations).
+- **Domain Specificity:** While specialized for personal finance, its reasoning capabilities might be less robust outside this domain or for highly niche financial topics not well-represented in the training data.
+- **Dataset Limitations:** The current version of the dataset is only ~7k which is barely enough to get quality results from LoRA adaptation. Further refinement and scaling up of the dataset is necessary.
+
+**Cost Incurred:** As an independent data scientist trying to develop this project, the entire cost of development till now totals up to ~$80. The cost was mostly reduced by the usage of free/open-source methods whenever available. (Though those had resource constraints with my 16GB M2 Mac Mini). Further development of this process, definitely needs some better, more cost-efficient approaches.
+
+## Further Information & Collaboration
+- **Contact:** akhiltvsn@gmail.com
+- **Future Work:**
+    - Refining and expanding the `PersonalFinance_v2` dataset. From 7k to at least 50k samples.
+    - Exploring Mixture of Experts (MoE) methods for further model development.
+- **Call for Collaboration:** I am a solo dude just randomly working on this project during my free time. If you are interested in this project, and want to expand the scope, then do ping me here, on Linkedin or just send me a mail.
+
+---
+## Citation 
+```BibTex
+@misc{akhil_theerthala_2025,
+	author       = { Akhil Theerthala },
+	title        = { Kuvera-14B-v0.1.0 (Revision 3fb04b9) },
+	year         = 2025,
+	url          = { https://huggingface.co/Akhil-Theerthala/Kuvera-14B-v0.1.0 },
+	doi          = { 10.57967/hf/5707 },
+	publisher    = { Hugging Face }
+}
+```