hilarl/psyche-siren-4b

Psyche Siren 4B

Model Description

Siren is a family of state-of-the-art models, designed to understand and analyze human creativity across audio, visual, textual, and temporal dimensions. The Psyche Siren model is a specialized for psychology and mental health applications, using Google's Gemma 3 model as a base. The model has been trained on comprehensive psychology datasets including personality psychology, social psychology, and behavioral science research.

Training Data

• Size: 1M+ psychology research records
• Domains:
  • Personality Psychology (Big Five, HEXACO, etc.)
  • Social Psychology
  • Behavioral Science
  • Mental Health Research
  • Cross-cultural Psychology

Model Details

• Base Model: google/gemma-3-4b-it
• Fine-tuning Method: LoRA (Low-Rank Adaptation)
• Training Framework: Axolotl
• Hardware: NVIDIA H100 80GB
• Training Time: ~11 hours

Usage

from transformers import AutoTokenizer, AutoModelForCausalLM
from peft import PeftModel

# Load model
base_model = "google/gemma-3-4b-it"
model = AutoModelForCausalLM.from_pretrained(base_model, torch_dtype=torch.bfloat16)
model = PeftModel.from_pretrained(model, "hilarl/psyche-siren-4b")
tokenizer = AutoTokenizer.from_pretrained(base_model)

# Generate response
prompt = "What are the Big Five personality traits?"
inputs = tokenizer(prompt, return_tensors="pt")
outputs = model.generate(**inputs, max_new_tokens=256)
response = tokenizer.decode(outputs[0], skip_special_tokens=True)

Intended Use

This model is designed for:

• Educational purposes in psychology
• Research applications
• Mental health awareness (not diagnosis)
• Personality assessment insights
• Psychological concept explanation

Limitations

• This model is for informational and educational purposes only
• Not a substitute for professional psychological assessment or therapy
• Should not be used for clinical diagnosis
• May reflect biases present in training data
• Requires human oversight for sensitive applications

Ethical Considerations

• Always encourage users to seek professional help for serious mental health concerns
• Maintain appropriate boundaries in therapeutic contexts
• Be aware of potential biases in psychological research data
• Use responsibly with vulnerable populations

Training Configuration

base_model: google/gemma-3-4b-it
model_type: Gemma3ForConditionalGeneration
load_in_8bit: false
load_in_4bit: false
strict: false

lora_config:
  r: 32
  lora_alpha: 16
  lora_dropout: 0.05
  target_modules:
    - q_proj
    - k_proj
    - v_proj
    - o_proj
    - gate_proj
    - up_proj
    - down_proj

training_args:
  learning_rate: 3e-5
  num_train_epochs: 1
  per_device_train_batch_size: 2
  gradient_accumulation_steps: 8
  warmup_steps: 500
  logging_steps: 50
  save_steps: 500

Citation

If you use this model in your research, please cite:

@misc{psyche-siren-4b,
  title={Psyche Siren},
  author={Hilal Agil},
  year={2025},
  url={https://huggingface.co/hilarl/psyche-siren-4b}
}

License

This model is released under the Gemma license, following the base model's licensing terms.