End of training

README.md

@@ -1,36 +1,60 @@
 ---
-license: apache-2.0
+library_name: peft
+base_model: microsoft/codebert-base
 tags:
-- code
-- roberta
-- peft
-- lora
-- fine-tuned
-- emotion
-- natural-language-processing
-- code-generation
-- prompt-engineering
-- code-understanding
-- text-to-prompt
-- instruction-tuning
+- generated_from_trainer
 datasets:
-- CM/codexglue_code2text_php
-- CM/codexglue_code2text_python
-- CM/codexglue_code2text_javascript
-- microsoft/codexglue_method_generation
-language:
-- th
-- en
-metrics:
-- accuracy
-- bertscore
-- bleu
-base_model:
-- microsoft/codebert-base
-pipeline_tag: text2text-generation
-library_name: adapter-transformers
+- code_search_net
+model-index:
+- name: codebert-model
+  results: []
 ---
 
-# CodeBERT V2 (LoRA Fine-Tuned)
+<!-- This model card has been generated automatically according to the information the Trainer had access to. You
+should probably proofread and complete it, then remove this comment. -->
 
-Fine-tuned version of CodeBERT with LoRA on `dair-ai/emotion` dataset. Trained using Hugging Face Transformers and PEFT.
+# codebert-model
+
+This model is a fine-tuned version of [microsoft/codebert-base](https://huggingface.co/microsoft/codebert-base) on the code_search_net dataset.
+It achieves the following results on the evaluation set:
+- eval_loss: 0.8346
+- eval_model_preparation_time: 0.0057
+- eval_accuracy: {'accuracy': 0.21967491508976225}
+- eval_f1: {'f1': 0.0}
+- eval_runtime: 9384.6382
+- eval_samples_per_second: 0.878
+- eval_steps_per_second: 0.11
+- step: 0
+
+## Model description
+
+More information needed
+
+## Intended uses & limitations
+
+More information needed
+
+## Training and evaluation data
+
+More information needed
+
+## Training procedure
+
+### Training hyperparameters
+
+The following hyperparameters were used during training:
+- learning_rate: 5e-05
+- train_batch_size: 8
+- eval_batch_size: 8
+- seed: 42
+- optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
+- lr_scheduler_type: linear
+- num_epochs: 3
+
+### Framework versions
+
+- PEFT 0.15.2
+- Transformers 4.51.3
+- Pytorch 2.6.0+cu124
+- Datasets 3.6.0
+- Tokenizers 0.21.1

adapter_config.json

@@ -1,7 +1,7 @@
 {
   "alpha_pattern": {},
   "auto_mapping": null,
-  "base_model_name_or_path": "Athipan01/codebert-model",
+  "base_model_name_or_path": "microsoft/codebert-base",
   "bias": "none",
   "corda_config": null,
   "eva_config": null,
@@ -13,7 +13,7 @@
   "layers_pattern": null,
   "layers_to_transform": null,
   "loftq_config": {},
-  "lora_alpha": 16,
+  "lora_alpha": 32,
   "lora_bias": false,
   "lora_dropout": 0.1,
   "megatron_config": null,
@@ -27,8 +27,8 @@
   "rank_pattern": {},
   "revision": null,
   "target_modules": [
-    "value",
-    "query"
+    "query",
+    "value"
   ],
   "task_type": "SEQ_CLS",
   "trainable_token_indices": null,

adapter_model.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:16c0c47103351969fb000f9300207536eef5fdcc26d55c3b817dde8af0b5c081
-size 3567808
+oid sha256:64c1852e9de5ac6ebc9a8aa5faf0c85e3d8ce81aef69b36350dba25275eea1cc
+size 3555504

training_args.bin

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:164d9c8a6e014f5d4ddb7e7fbfd90f0ff7765bf99b7467b521e1ff7ca8170cfc
-size 5304
+oid sha256:e5d2992ae7cf00ab35f42ff5caa407f1021c209acfca749548cdb8b0180e72f0
+size 5368