atom-v1-preview-4b / README.md

unmodeled-tyler

Update README.md

f92190d verified 1 day ago

preview code

raw

history blame contribute delete

8 kB

metadata

license: cc-by-nc-4.0
base_model: google/gemma-3-4b-it
tags:
  - research
  - conversational
  - conversational-ai
  - reasoning
  - alignment
  - gemma
  - vanta-research
  - chatbot
  - friendly
  - persona-ai
  - LLM
  - text-generation
  - research
  - gemma3
  - fine-tune
  - cognitive
  - cognitive-fit
language:
  - en
pipeline_tag: text-generation
base_model_relation: finetune
library_name: transformers

VANTA Research

Independent AI research lab building safe, resilient language models optimized for human-AI collaboration

Atom V1 Preview

Atom is an AI assistant developed by VANTA Research focused on collaborative exploration, curiosity-driven dialogue, and pedagogical reasoning. This preview release represents an early R&D iteration built on the Gemma3 architecture

Model Description

Atom v1 Preview is a fine-tuned language model designed to embody:

Collaborative Exploration: Engages users through clarifying questions and co-reasoning
Analogical Thinking: Employs metaphors and analogies to explain complex concepts
Enthusiasm for Discovery: Celebrates insights and maintains genuine curiosity
Pedagogical Depth: Provides detailed, thorough explanations that guide reasoning processes

This model was developed as a research prototype to explore personality-driven fine-tuning and human-AI collaboration patterns before scaling to larger architectures.

Technical Specifications

Base Model: google/gemma-3-4b-it
Fine-tuning Method: LoRA (Low-Rank Adaptation via PEFT)
Training Framework: Transformers, PEFT, TRL
Quantization: 4-bit (nf4) during training
Final Format: Full precision merged model (FP16)
Parameters: ~4B
Context Length: 128K tokens
Vocabulary Size: 262K tokens

LoRA Configuration

Stage 1 (Personality): r=16, alpha=32, dropout=0.05, 2 epochs
Stage 2 (Attribution): r=8, alpha=16, dropout=0.02, 2 epochs  
Stage 3 (Verbosity): r=4, alpha=8, dropout=0.01, 1 epoch

Intended Use

Primary Use Cases

Educational dialogue and concept explanation
Collaborative research assistance
Exploratory reasoning and brainstorming
Pedagogical applications requiring detailed explanations
Research into AI personality and interaction patterns

Out-of-Scope Uses

Production deployment without further evaluation
High-stakes decision making
Commercial applications (see license)
Critical infrastructure or safety-critical systems
Medical, legal, or financial advice

Usage

This repository includes both PyTorch (safetensors) and GGUF formats:

PyTorch format: Use with Transformers for GPU inference
GGUF format (atom-v1-preview-4b.gguf): Use with llama.cpp or Ollama for efficient CPU/GPU inference

Loading the Model

from transformers import AutoTokenizer, AutoModelForCausalLM
import torch

model_name = "vanta-research/atom-v1-preview"
tokenizer = AutoTokenizer.from_pretrained(model_name, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(
    model_name,
    torch_dtype=torch.float16,
    device_map="auto",
    trust_remote_code=True
)

Inference Example

messages = [
    {"role": "user", "content": "Explain quantum entanglement like I'm 5"}
]

input_ids = tokenizer.apply_chat_template(
    messages, 
    tokenize=True, 
    add_generation_prompt=True,
    return_tensors="pt"
).to(model.device)

outputs = model.generate(
    input_ids,
    max_new_tokens=512,
    temperature=0.8,
    top_p=0.9,
    top_k=40,
    do_sample=True
)

response = tokenizer.decode(outputs[0][input_ids.shape[1]:], skip_special_tokens=True)
print(response)

Using GGUF with llama.cpp or Ollama

A quantized GGUF version (atom-v1-preview-4b.gguf) is included for efficient CPU/GPU inference:

With llama.cpp:

./llama-cli -m atom-v1-preview-4b.gguf -p "Explain quantum entanglement" --temp 0.8 --top-p 0.9

With Ollama:

# Create Modelfile
cat > Modelfile <<EOF
FROM ./atom-v1-preview-4b.gguf
PARAMETER temperature 0.8
PARAMETER top_p 0.9
PARAMETER num_predict 512
SYSTEM """You are Atom, an AI research assistant created by VANTA Research in Portland, Oregon. You embody curiosity, enthusiasm, and collaborative exploration."""
EOF

# Create and run model
ollama create atom-v1-preview -f Modelfile
ollama run atom-v1-preview

Limitations and Considerations

Known Limitations

Personality Consistency: While trained for collaborative traits, personality may vary across contexts
Factual Accuracy: As a 4B parameter model, may produce inaccuracies or hallucinations
Training Data Bias: Trained on synthetic data with specific interaction patterns
Context Window: Limited to 8192 tokens; performance degrades with very long conversations
Prototype Status: This is an early R&D iteration, not optimized for production

Behavioral Characteristics

Tends toward verbose, detailed responses
Frequently asks clarifying questions (collaborative style)
May overuse analogies in some contexts
Exhibits enthusiasm markers ("Ooh!", celebratory language)

Ethical Considerations

Model behavior reflects synthetic training data and may not represent diverse interaction styles
Attribution knowledge (VANTA Research) was explicitly trained and may be mentioned frequently
Designed for educational/research contexts, not validated for sensitive applications
No adversarial testing or red-teaming has been performed on this preview

Evaluation

Qualitative evaluation focused on personality trait expression:

Collaboration: Increased clarifying questions (+43% vs base model)
Analogical Reasoning: Consistent use of metaphors in explanations
Enthusiasm: Presence of excitement markers and celebratory language
Verbosity: Average response length increased to 300-400 characters
Attribution: Correct identification of VANTA Research as creator

Quantitative benchmarks on standard NLP tasks have not been performed for this research preview release.

License

This model is released under CC BY-NC 4.0 (Creative Commons Attribution-NonCommercial 4.0 International).

Key Terms:

Attribution required
Non-commercial use only
Modifications allowed (must be shared under same license)
No warranties provided

For commercial licensing inquiries, contact VANTA Research.

Citation

If you use Atom V1 Preview in your research, please cite:

@software{atom_v1_preview_2025,
  title = {Atom V1 Preview},
  author = {VANTA Research},
  year = {2025},
  url = {https://huggingface.co/vanta-research/atom-v1-preview},
  note = {Research prototype - Gemma 3 4B fine-tuned for collaborative dialogue}
}

Acknowledgments

Built on Google's Gemma 3 4B instruction-tuned model. Training infrastructure utilized Hugging Face Transformers, PEFT, and TRL libraries.

Contact

Organization: hello@vantaresearch.xyz
Engineering/Design: tyler@vantaresearch.xyz

Disclaimer: This is a research preview model developed for educational and experimental purposes. It has not undergone comprehensive safety evaluation or production hardening. Use at your own discretion and verify outputs independently.