kernelpool commited on
Commit
d185a17
·
verified ·
1 Parent(s): 1b3084d

Add files using upload-large-folder tool

Browse files
Files changed (2) hide show
  1. README.md +12 -0
  2. minimax4_3e-7.png +0 -0
README.md CHANGED
@@ -11,6 +11,18 @@ tags:
11
 
12
  This model was quantized to 4-bit using DWQ with mlx-lm version **0.28.4**.
13
 
 
 
 
 
 
 
 
 
 
 
 
 
14
  ## Use with mlx
15
 
16
  ```bash
 
11
 
12
  This model was quantized to 4-bit using DWQ with mlx-lm version **0.28.4**.
13
 
14
+ | Parameter | Value |
15
+ |---------------------------|--------------------------------|
16
+ | DWQ learning rate | 3e-7 |
17
+ | Batch size | 1 |
18
+ | Dataset | `allenai/tulu-3-sft-mixture` |
19
+ | Initial validation loss | 0.069 |
20
+ | Final validation loss | 0.047 |
21
+ | Relative KL reduction | ≈32 % |
22
+ | Tokens processed | ≈1.09 M |
23
+
24
+ <img src="minimax4_3e-7.png" width="600" alt="Training loss curve">
25
+
26
  ## Use with mlx
27
 
28
  ```bash
minimax4_3e-7.png ADDED