psyche-siren-4b / README.md

hilarl

Update README.md

963c3ee verified 3 months ago

preview code

raw

history blame contribute delete

3.45 kB

metadata

language:
  - en
license: gemma
base_model: google/gemma-3-4b-it
tags:
  - psychology
  - mental-health
  - personality
  - social-psychology
  - fine-tuned
  - gemma
  - peft
  - lora
datasets:
  - custom-psychology-dataset
library_name: transformers
pipeline_tag: text-generation

Psyche Siren 4B

Model Description

Siren is a family of state-of-the-art models, designed to understand and analyze human creativity across audio, visual, textual, and temporal dimensions. The Psyche Siren model is a specialized for psychology and mental health applications, using Google's Gemma 3 model as a base. The model has been trained on comprehensive psychology datasets including personality psychology, social psychology, and behavioral science research.

Training Data

Size: 1M+ psychology research records
Domains:
- Personality Psychology (Big Five, HEXACO, etc.)
- Social Psychology
- Behavioral Science
- Mental Health Research
- Cross-cultural Psychology

Model Details

Base Model: google/gemma-3-4b-it
Fine-tuning Method: LoRA (Low-Rank Adaptation)
Training Framework: Axolotl
Hardware: NVIDIA H100 80GB
Training Time: ~11 hours

Usage

from transformers import AutoTokenizer, AutoModelForCausalLM
from peft import PeftModel

# Load model
base_model = "google/gemma-3-4b-it"
model = AutoModelForCausalLM.from_pretrained(base_model, torch_dtype=torch.bfloat16)
model = PeftModel.from_pretrained(model, "hilarl/psyche-siren-4b")
tokenizer = AutoTokenizer.from_pretrained(base_model)

# Generate response
prompt = "What are the Big Five personality traits?"
inputs = tokenizer(prompt, return_tensors="pt")
outputs = model.generate(**inputs, max_new_tokens=256)
response = tokenizer.decode(outputs[0], skip_special_tokens=True)

Intended Use

This model is designed for:

Educational purposes in psychology
Research applications
Mental health awareness (not diagnosis)
Personality assessment insights
Psychological concept explanation

Limitations

This model is for informational and educational purposes only
Not a substitute for professional psychological assessment or therapy
Should not be used for clinical diagnosis
May reflect biases present in training data
Requires human oversight for sensitive applications

Ethical Considerations

Always encourage users to seek professional help for serious mental health concerns
Maintain appropriate boundaries in therapeutic contexts
Be aware of potential biases in psychological research data
Use responsibly with vulnerable populations

Training Configuration

base_model: google/gemma-3-4b-it
model_type: Gemma3ForConditionalGeneration
load_in_8bit: false
load_in_4bit: false
strict: false

lora_config:
  r: 32
  lora_alpha: 16
  lora_dropout: 0.05
  target_modules:
    - q_proj
    - k_proj
    - v_proj
    - o_proj
    - gate_proj
    - up_proj
    - down_proj

training_args:
  learning_rate: 3e-5
  num_train_epochs: 1
  per_device_train_batch_size: 2
  gradient_accumulation_steps: 8
  warmup_steps: 500
  logging_steps: 50
  save_steps: 500

Citation

If you use this model in your research, please cite:

@misc{psyche-siren-4b,
  title={Psyche Siren},
  author={Hilal Agil},
  year={2025},
  url={https://huggingface.co/hilarl/psyche-siren-4b}
}

License

This model is released under the Gemma license, following the base model's licensing terms.