Multi-modal model with moe related domains (1)