Hugging Face's logo

tangledgroup
/
tangled-alpha-0.1-core

Text Generation
Transformers
chat
core
base
instruct
reason
conversational
Model card Files
xet
Community
tangled-alpha-0.1-core

Commit History

out/pretrain-core/final
c2fc44c

Marko Tasic commited on

reqs
9b182f7

Marko Tasic commited on

grokadamw.GrokAdamW
da80ae1

mtasic85 commited on

grokadamw.GrokAdamW
1386dd6

mtasic85 commited on

global_batch_size: 256; micro_batch_size: 2
afa8f6e

mtasic85 commited on

global_batch_size: 256; micro_batch_size: 4
bee417a

mtasic85 commited on

micro_batch_size: 3
0f5ef2e

mtasic85 commited on

micro_batch_size: 4
1c9b116

mtasic85 commited on

micro_batch_size: 1
056e2c6

mtasic85 commited on

class_path: torch.optim.AdamW
fcc1668

mtasic85 commited on

micro_batch_size: 2
578a7be

mtasic85 commited on

class_path: bitsandbytes.optim.AdamW8bit
ed2b433

mtasic85 commited on

class_path: bitsandbytes.optim.PagedAdamW8bit
14f2503

mtasic85 commited on

micro_batch_size: 4
50de401

mtasic85 commited on

class_path: torchao.prototype.low_bit_optim.AdamW8bit
dfc4418

mtasic85 commited on

class_path: torchao.prototype.low_bit_optim.AdamW4bit
4afec17

mtasic85 commited on

class_path: torchao.prototype.low_bit_optim.AdamW8bit
779fd25

mtasic85 commited on

max_seq_length: 8192
b68070c

mtasic85 commited on

pretrain model
193a28c

mtasic85 commited on

git config
8432ba4

mtasic85 commited on

initial commit
3d391d5
verified

mtasic85 commited on