Hugging Face's logo
Andrey's picture
In a Training Loop 🔄
17 3

Andrey

Bochkov
musicakamusic's profile picture radiocpp's profile picture NikolayKozloff's profile picture
·

AI & ML interests

None yet

Recent Activity

reacted to sergiopaniego's post with 🔥 9 days ago
New REPL environment in OpenEnv available! ✨ Used in the Recursive Language Models (RLM) paper by Alex Zhang. Ready for inference & post-training using trajectories. Handles long contexts: > Run Python code in a sandbox > Make recursive calls to LMs > Explore data programmatically > Return final result Docs: https://meta-pytorch.org/OpenEnv/environments/repl/ Inference script: https://github.com/meta-pytorch/OpenEnv/blob/main/examples/repl_oolong_simple.py
posted an update 9 days ago
Curious reproducible fact: I trained a GPT-like decoder-only Transformer where the entire input embedding table is frozen and reduced to a 16‑D binary token-ID code (0/1) — this is NOT 16-bit quantization. Key details: - vocab_size = 65536, n_embed = 16 (2^16 = 65536 unique IDs) - deterministic expansion 16 → d_model=1024 via repeat_interleave (scale=64) - full embedding table is published (embeddings.txt) for auditability Repro note + verification script: https://huggingface.co/blog/Bochkov/emergent-semantics-beyond-token-embeddings Model repo: https://huggingface.co/Bochkov/emergent-semantics-model-16-bit-269m License: Apache-2.0
upvoted a paper 12 days ago
NVIDIA Nemotron Nano 2: An Accurate and Efficient Hybrid Mamba-Transformer Reasoning Model
View all activity

Organizations

None yet