keras/qwen3_moe_30b_a3b_en

This is a Qwen3Moe model uploaded using the KerasHub library and can be used with JAX, TensorFlow, and PyTorch backends. This model is related to a CausalLM task.

Model config:

• name: qwen3_moe_backbone
• trainable: True
• dtype: {'module': 'keras', 'class_name': 'DTypePolicy', 'config': {'name': 'float32'}, 'registered_name': None}
• vocabulary_size: 151936
• num_layers: 48
• num_query_heads: 32
• head_dim: 128
• hidden_dim: 2048
• intermediate_dim: 6144
• moe_intermediate_dim: 768
• rope_max_wavelength: 1000000.0
• num_key_value_heads: 4
• rope_scaling_factor: 1.0
• layer_norm_epsilon: 1e-06
• dropout: 0
• tie_word_embeddings: False
• sliding_window_size: None
• num_experts: 128
• top_k: 8
• norm_top_k_prob: True
• decoder_sparse_step: 1
• mlp_only_layers: []
• router_aux_loss_coefficient: 0.001

This model card has been generated automatically and should be completed by the model author. See Model Cards documentation for more information.