Pretraining Context Compressor for Large Language Models with Embedding-Based Memory
YuHong Dai
BroAlanTaps
AI & ML interests
CV(2D-Object Detection), NLP(Infer Accelerate)
Recent Activity
updated
a collection
2 days ago
PCC-Pretrained
updated
a dataset
2 days ago
BroAlanTaps/Warmup-GPT2-Large-Llama3-8B-Redpajama
published
a dataset
2 days ago
BroAlanTaps/Warmup-GPT2-Large-Llama3-8B-Redpajama
Organizations
models
129
BroAlanTaps/wikipedia-visual-en-qwen2.5-2b-instruct-lora-sft-wikipedia-samples
Image-to-Text
•
2B
•
Updated
•
1
BroAlanTaps/Stage2-PCC-Large-16x
Updated
BroAlanTaps/Stage2-PCC-Large-128x
Updated
BroAlanTaps/Stage2-PCC-Large-4x
Updated
BroAlanTaps/PCC-Large-Encoder-Llama3-8B-Instruct
8B
•
Updated
•
54
BroAlanTaps/Stage1-PCC-Large-128x
Updated
BroAlanTaps/Stage2-PCC-Lite-256x
0.8B
•
Updated
•
1
BroAlanTaps/Stage2-PCC-Lite-128x
0.8B
•
Updated
•
35
BroAlanTaps/Stage2-PCC-Lite-64x
0.8B
•
Updated
BroAlanTaps/Stage2-PCC-Lite-16x
0.8B
•
Updated
•
44
datasets
7
BroAlanTaps/Warmup-GPT2-Large-Llama3-8B-Redpajama
Viewer
•
Updated
•
1.55M
•
17
BroAlanTaps/pretrain-corpus-test
Viewer
•
Updated
•
576
•
1
BroAlanTaps/GPT2-Large-Llama3-8B-fineweb-256-5Btokens
Viewer
•
Updated
•
19.2M
•
1.13k
BroAlanTaps/efficiency_samples_8k
Viewer
•
Updated
•
1k
•
7
BroAlanTaps/Stage2-PCC-Large-SFT-Squad
Viewer
•
Updated
•
92.7k
•
11
BroAlanTaps/Stage2-PCC-SFT-HotpotQA
Viewer
•
Updated
•
90.4k
•
51
BroAlanTaps/Stage2-PCC-Lite-SFT-Squad
Viewer
•
Updated
•
92.7k
•
301