sigridjineth commited on
Commit
a1b935a
ยท
verified ยท
1 Parent(s): 65b4710

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +32 -17
README.md CHANGED
@@ -23,23 +23,38 @@ The model demonstrated stable and consistent improvement throughout the training
23
 
24
  The true power of the fine-tuned model is its ability to understand semantic context beyond simple keyword matching. In the following challenging example, the fine-tuned model correctly infers the answer, while the original base model fails.
25
 
26
- * **์ฟผ๋ฆฌ (Query):**
27
-
28
- ```
29
- "์ผ๋ก  ๋จธ์Šคํฌ๊ฐ€ ์„ค๋ฆฝํ•œ ์ „๊ธฐ์ฐจ ํšŒ์‚ฌ๋Š” ์–ด๋””์•ผ?"
30
- ```
31
-
32
- * **โœ… Fine-tuned Model Results:**
33
-
34
- 1. **`Score: 10.00`**: **ํ…Œ์Šฌ๋ผ**๋Š” ๋ชจ๋ธ S, 3, X, Y๋ฅผ ์ƒ์‚ฐํ•˜๋ฉฐ ์˜คํ† ํŒŒ์ผ๋Ÿฟ ๊ธฐ๋Šฅ์œผ๋กœ ์œ ๋ช…ํ•ฉ๋‹ˆ๋‹ค.
35
- 2. **`Score: 9.51`**: **์ŠคํŽ˜์ด์ŠคX**๋Š” ์žฌ์‚ฌ์šฉ ๊ฐ€๋Šฅํ•œ ๋กœ์ผ“์„ ๊ฐœ๋ฐœํ•˜์—ฌ ์šฐ์ฃผ ํƒ์‚ฌ ๋น„์šฉ์„ ํฌ๊ฒŒ ๋‚ฎ์ท„์Šต๋‹ˆ๋‹ค.
36
- 3. **`Score: 8.57`**: ์•„๋งˆ์กด ์›น ์„œ๋น„์Šค(AWS)๋Š” ํด๋ผ์šฐ๋“œ ์ปดํ“จํŒ… ์‹œ์žฅ์˜ ์„ ๋‘์ฃผ์ž์ž…๋‹ˆ๋‹ค.
37
-
38
- * **โŒ Original Model Results:**
39
-
40
- 1. **`Score: 8.55`**: ์ˆ˜๋„๊ถŒ ์ „์ฒ ์€ ์„œ์šธ๊ณผ ์ฃผ๋ณ€ ๋„์‹œ๋ฅผ ์—ฐ๊ฒฐํ•˜๋Š” ์ค‘์š”ํ•œ ๊ตํ†ต์ˆ˜๋‹จ์ž…๋‹ˆ๋‹ค.
41
- 2. **`Score: 8.54`**: **ํ…Œ์Šฌ๋ผ**๋Š” ๋ชจ๋ธ S, 3, X, Y๋ฅผ ์ƒ์‚ฐํ•˜๋ฉฐ ์˜คํ† ํŒŒ์ผ๋Ÿฟ ๊ธฐ๋Šฅ์œผ๋กœ ์œ ๋ช…ํ•ฉ๋‹ˆ๋‹ค.
42
- 3. **`Score: 8.39`**: **์ŠคํŽ˜์ด์ŠคX**๋Š” ์žฌ์‚ฌ์šฉ ๊ฐ€๋Šฅํ•œ ๋กœ์ผ“์„ ๊ฐœ๋ฐœํ•˜์—ฌ ์šฐ์ฃผ ํƒ์‚ฌ ๋น„์šฉ์„ ํฌ๊ฒŒ ๋‚ฎ์ท„์Šต๋‹ˆ๋‹ค.
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
43
 
44
  **Analysis**: The fine-tuned model correctly identifies 'Tesla' by understanding the semantic relationship between the query and the document, even with no direct keyword overlap. In contrast, the original model is easily confused by distractors and fails to rank the correct answer first, demonstrating the significant impact of the ColBERT fine-tuning process.
45
 
 
23
 
24
  The true power of the fine-tuned model is its ability to understand semantic context beyond simple keyword matching. In the following challenging example, the fine-tuned model correctly infers the answer, while the original base model fails.
25
 
26
+ ```
27
+ $ python inference.py
28
+ Using device: cuda
29
+
30
+ Loading fine-tuned model...
31
+ Fine-tuned model loaded.
32
+
33
+ Loading original (pre-trained) model for comparison...
34
+ Original model loaded.
35
+
36
+ ==================================================
37
+ Query: ์ผ๋ก  ๋จธ์Šคํฌ๊ฐ€ ์„ค๋ฆฝํ•œ ์ „๊ธฐ์ฐจ ํšŒ์‚ฌ๋Š” ์–ด๋””์•ผ?
38
+ ==================================================
39
+
40
+ --- 1. โœ… Fine-tuned Model Results ---
41
+ Rank 1 (Score: 9.00): ํ…Œ์Šฌ๋ผ๋Š” ๋ชจ๋ธ S, 3, X, Y๋ฅผ ์ƒ์‚ฐํ•˜๋ฉฐ ์˜คํ† ํŒŒ์ผ๋Ÿฟ ๊ธฐ๋Šฅ์œผ๋กœ ์œ ๋ช…ํ•ฉ๋‹ˆ๋‹ค.
42
+ Rank 2 (Score: 7.92): ์ŠคํŽ˜์ด์ŠคX๋Š” ์žฌ์‚ฌ์šฉ ๊ฐ€๋Šฅํ•œ ๋กœ์ผ“์„ ๊ฐœ๋ฐœํ•˜์—ฌ ์šฐ์ฃผ ํƒ์‚ฌ ๋น„์šฉ์„ ํฌ๊ฒŒ ๋‚ฎ์ท„์Šต๋‹ˆ๋‹ค.
43
+ Rank 3 (Score: 7.72): ์•„๋งˆ์กด ์›น ์„œ๋น„์Šค(AWS)๋Š” ํด๋ผ์šฐ๋“œ ์ปดํ“จํŒ… ์‹œ์žฅ์˜ ์„ ๋‘์ฃผ์ž์ž…๋‹ˆ๋‹ค.
44
+ Rank 4 (Score: 7.23): ์ˆ˜๋„๊ถŒ ์ „์ฒ ์€ ์„œ์šธ๊ณผ ์ฃผ๋ณ€ ๋„์‹œ๋ฅผ ์—ฐ๊ฒฐํ•˜๋Š” ์ค‘์š”ํ•œ ๊ตํ†ต์ˆ˜๋‹จ์ž…๋‹ˆ๋‹ค.
45
+ Rank 5 (Score: 5.77): ๋Œ€ํ•œ๋ฏผ๊ตญ์˜ ์ˆ˜๋„๋Š” ์„œ์šธ์ž…๋‹ˆ๋‹ค. ์„œ์šธ์€ ๊ฒฝ์ œ์™€ ๋ฌธํ™”์˜ ์ค‘์‹ฌ์ง€์ž…๋‹ˆ๋‹ค.
46
+ Rank 6 (Score: 5.43): ์ผ๋ณธ์˜ ์ˆ˜๋„๋Š” ๋„์ฟ„์ž…๋‹ˆ๋‹ค. ๋ฒš๊ฝƒ์ด ์•„๋ฆ„๋‹ค์šด ๋„์‹œ์ฃ .
47
+ Rank 7 (Score: 5.40): ํ”„๋ž‘์Šค์˜ ์ˆ˜๋„๋Š” ํŒŒ๋ฆฌ์ด๋ฉฐ, ์—ํŽ ํƒ‘์œผ๋กœ ์œ ๋ช…ํ•ฉ๋‹ˆ๋‹ค.
48
+
49
+ --- 2. โŒ Original Model Results ---
50
+ Rank 1 (Score: 9.13): ์ˆ˜๋„๊ถŒ ์ „์ฒ ์€ ์„œ์šธ๊ณผ ์ฃผ๋ณ€ ๋„์‹œ๋ฅผ ์—ฐ๊ฒฐํ•˜๋Š” ์ค‘์š”ํ•œ ๊ตํ†ต์ˆ˜๋‹จ์ž…๋‹ˆ๋‹ค.
51
+ Rank 2 (Score: 8.79): ํ…Œ์Šฌ๋ผ๋Š” ๋ชจ๋ธ S, 3, X, Y๋ฅผ ์ƒ์‚ฐํ•˜๋ฉฐ ์˜คํ† ํŒŒ์ผ๋Ÿฟ ๊ธฐ๋Šฅ์œผ๋กœ ์œ ๋ช…ํ•ฉ๋‹ˆ๋‹ค.
52
+ Rank 3 (Score: 8.77): ์ผ๋ณธ์˜ ์ˆ˜๋„๋Š” ๋„์ฟ„์ž…๋‹ˆ๋‹ค. ๋ฒš๊ฝƒ์ด ์•„๋ฆ„๋‹ค์šด ๋„์‹œ์ฃ .
53
+ Rank 4 (Score: 8.71): ๋Œ€ํ•œ๋ฏผ๊ตญ์˜ ์ˆ˜๋„๋Š” ์„œ์šธ์ž…๋‹ˆ๋‹ค. ์„œ์šธ์€ ๊ฒฝ์ œ์™€ ๋ฌธํ™”์˜ ์ค‘์‹ฌ์ง€์ž…๋‹ˆ๋‹ค.
54
+ Rank 5 (Score: 8.53): ์•„๋งˆ์กด ์›น ์„œ๋น„์Šค(AWS)๋Š” ํด๋ผ์šฐ๋“œ ์ปดํ“จํŒ… ์‹œ์žฅ์˜ ์„ ๋‘์ฃผ์ž์ž…๋‹ˆ๋‹ค.
55
+ Rank 6 (Score: 8.48): ์ŠคํŽ˜์ด์ŠคX๋Š” ์žฌ์‚ฌ์šฉ ๊ฐ€๋Šฅํ•œ ๋กœ์ผ“์„ ๊ฐœ๋ฐœํ•˜์—ฌ ์šฐ์ฃผ ํƒ์‚ฌ ๋น„์šฉ์„ ํฌ๊ฒŒ ๋‚ฎ์ท„์Šต๋‹ˆ๋‹ค.
56
+ Rank 7 (Score: 8.24): ํ”„๋ž‘์Šค์˜ ์ˆ˜๋„๋Š” ํŒŒ๋ฆฌ์ด๋ฉฐ, ์—ํŽ ํƒ‘์œผ๋กœ ์œ ๋ช…ํ•ฉ๋‹ˆ๋‹ค.
57
+ ```
58
 
59
  **Analysis**: The fine-tuned model correctly identifies 'Tesla' by understanding the semantic relationship between the query and the document, even with no direct keyword overlap. In contrast, the original model is easily confused by distractors and fails to rank the correct answer first, demonstrating the significant impact of the ColBERT fine-tuning process.
60