Detail publikace

MixedEmotions: An Open-Source Toolbox for Multimodal Emotion Analysis

BUITELAAR, P. WOOD, I. NEGI, S. ARCAN, M. MCCRAE, J. ABELE, A. ROBIN, C. ANDRYUSHECHKIN, V. ZIAD, H. SAGHA, H. SCHMITT, M. SCHULLER, B. SÁNCHEZ-RADA, J. IGLESIAS, C. NAVARRO, C. GIEFER, A. HEISE, N. MASUCCI, V. DANZA, F. CATERINO, C. SMRŽ, P. HRADIŠ, M. POVOLNÝ, F. KLIMEŠ, M. MATĚJKA, P. TUMMARELLO, G.

Originální název

MixedEmotions: An Open-Source Toolbox for Multimodal Emotion Analysis

Anglický název

MixedEmotions: An Open-Source Toolbox for Multimodal Emotion Analysis

Jazyk

en

Originální abstrakt

Recently, there is an increasing tendency to embed functionalities for recognizing emotions from user-generated media content in automated systems such as call-centre operations, recommendations, and assistive technologies, providing richer and more informative user and content profiles. However, to date, adding these functionalities was a tedious, costly, and time-consuming effort, requiring identification and integration of diverse tools with diverse interfaces as required by the use case at hand. The MixedEmotions Toolbox leverages the need for such functionalities by providing tools for text, audio, video, and linked data processing within an easily integrable plug-and-play platform. These functionalities include: 1) for text processing: emotion and sentiment recognition; 2) for audio processing: emotion, age, and gender recognition; 3) for video processing: face detection and tracking, emotion recognition, facial landmark localization, head pose estimation, face alignment, and body pose estimation; and 4) for linked data: knowledge graph integration. Moreover, the MixedEmotions Toolbox is open-source and free. In this paper, we present this toolbox in the context of the existing landscape, and provide a range of detailed benchmarks on standard test-beds showing its state-of-the-art performance. Furthermore, three real-world use cases show its effectiveness, namely, emotion-driven smart TV, call center monitoring, and brand reputation analysis.

Anglický abstrakt

Recently, there is an increasing tendency to embed functionalities for recognizing emotions from user-generated media content in automated systems such as call-centre operations, recommendations, and assistive technologies, providing richer and more informative user and content profiles. However, to date, adding these functionalities was a tedious, costly, and time-consuming effort, requiring identification and integration of diverse tools with diverse interfaces as required by the use case at hand. The MixedEmotions Toolbox leverages the need for such functionalities by providing tools for text, audio, video, and linked data processing within an easily integrable plug-and-play platform. These functionalities include: 1) for text processing: emotion and sentiment recognition; 2) for audio processing: emotion, age, and gender recognition; 3) for video processing: face detection and tracking, emotion recognition, facial landmark localization, head pose estimation, face alignment, and body pose estimation; and 4) for linked data: knowledge graph integration. Moreover, the MixedEmotions Toolbox is open-source and free. In this paper, we present this toolbox in the context of the existing landscape, and provide a range of detailed benchmarks on standard test-beds showing its state-of-the-art performance. Furthermore, three real-world use cases show its effectiveness, namely, emotion-driven smart TV, call center monitoring, and brand reputation analysis.

Dokumenty

BibTex


@article{BUT155798,
  author="Ian {Wood} and Cecile {Robin} and Pavel {Smrž} and Michal {Hradiš} and Filip {Povolný} and Marek {Klimeš} and Pavel {Matějka}",
  title="MixedEmotions: An Open-Source Toolbox for Multimodal Emotion Analysis",
  annote="Recently, there is an increasing tendency to embed functionalities for
recognizing emotions from user-generated media content in automated systems such
as call-centre operations, recommendations, and assistive technologies, providing
richer and more informative user and content profiles. However, to date, adding
these functionalities was a tedious, costly, and time-consuming effort, requiring
identification and integration of diverse tools with diverse interfaces as
required by the use case at hand. The MixedEmotions Toolbox leverages the need
for such functionalities by providing tools for text, audio, video, and linked
data processing within an easily integrable plug-and-play platform. These
functionalities include: 1) for text processing: emotion and sentiment
recognition; 2) for audio processing: emotion, age, and gender recognition; 3)
for video processing: face detection and tracking, emotion recognition, facial
landmark localization, head pose estimation, face alignment, and body pose
estimation; and 4) for linked data: knowledge graph integration. Moreover, the
MixedEmotions Toolbox is open-source and free. In this paper, we present this
toolbox in the context of the existing landscape, and provide a range of detailed
benchmarks on standard test-beds showing its state-of-the-art performance.
Furthermore, three real-world use cases show its effectiveness, namely,
emotion-driven smart TV, call center monitoring, and brand reputation analysis.",
  address="NEUVEDEN",
  chapter="155798",
  doi="10.1109/TMM.2018.2798287",
  edition="NEUVEDEN",
  howpublished="print",
  institution="NEUVEDEN",
  number="9",
  volume="20",
  year="2018",
  month="august",
  pages="2454--2465",
  publisher="NEUVEDEN",
  type="journal article in Web of Science"
}