Zero-Shot Style Transfer for Multimodal Data-Driven Gesture Synthesis - Perception,INteraction, Robotique sociales Accéder directement au contenu
Communication Dans Un Congrès Année : 2023

Zero-Shot Style Transfer for Multimodal Data-Driven Gesture Synthesis

Résumé

We propose a multimodal speech driven approach to generate 2D upper-body gestures for virtual agents, in the communicative style of different speakers, seen or unseen by our model during training. Upper-body gestures of a source speaker are generated based on the content of his/her multimodal data-speech acoustics and text semantics. The synthesized source speaker's gestures are conditioned on the multimodal style representation of the target speaker. Our approach is zero-shot, and can generalize the style transfer to new unseen speakers, without any additional training. An objective evaluation is conducted to validate our approach.
Fichier principal
Vignette du fichier
SIVA_2023(2).pdf (344.17 Ko) Télécharger le fichier
Origine : Fichiers produits par l'(les) auteur(s)

Dates et versions

hal-03972560 , version 1 (03-02-2023)

Identifiants

  • HAL Id : hal-03972560 , version 1

Citer

Mireille Fares, Nicolas Obin, Catherine Pelachaud. Zero-Shot Style Transfer for Multimodal Data-Driven Gesture Synthesis. International Conference on Automatic Face and Gesture Recognition 2023, Workshop on Socially Interactive Human-like Virtual Agents, Jan 2023, WAIKOLOA (Hawaii), United States. ⟨hal-03972560⟩
63 Consultations
144 Téléchargements

Partager

Gmail Facebook X LinkedIn More