Recognizing the emotion an image evokes in the observer has long attracted the interest of the community for its many potential applications. However, it is a challenging task mainly due to the inherent complexity and subjectivity of human feelings. Such a difficulty is exacerbated in the domain of visual arts, mainly because of their abstract nature. In this work, we propose a new version of the artistic knowledge graph we were working on, namely ArtGraph, obtained by integrating the emotion labels provided by the ArtEmis dataset. The proposed graph enables emotion-based information retrieval and knowledge discovery even without training a learning model. In addition, we propose an artwork emotion classification system that jointly exploits visual features and knowledge graph-embeddings. Experimental evaluation revealed that while improvements in emotion classification depend mainly on the use of visual features, the prediction of style, genre and emotion can benefit from the simultaneous exploitation of visual and contextual features and can assist each other in a synergistic way.

Recognizing the Emotions Evoked by Artworks Through Visual Features and Knowledge Graph-Embeddings

Aslan S.;
2022

Abstract

Recognizing the emotion an image evokes in the observer has long attracted the interest of the community for its many potential applications. However, it is a challenging task mainly due to the inherent complexity and subjectivity of human feelings. Such a difficulty is exacerbated in the domain of visual arts, mainly because of their abstract nature. In this work, we propose a new version of the artistic knowledge graph we were working on, namely ArtGraph, obtained by integrating the emotion labels provided by the ArtEmis dataset. The proposed graph enables emotion-based information retrieval and knowledge discovery even without training a learning model. In addition, we propose an artwork emotion classification system that jointly exploits visual features and knowledge graph-embeddings. Experimental evaluation revealed that while improvements in emotion classification depend mainly on the use of visual features, the prediction of style, genre and emotion can benefit from the simultaneous exploitation of visual and contextual features and can assist each other in a synergistic way.
International Conference on Image Analysis and Processing 2022 Proceeding
File in questo prodotto:
Non ci sono file associati a questo prodotto.

I documenti in ARCA sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione.

Utilizza questo identificativo per citare o creare un link a questo documento: http://hdl.handle.net/10278/5004013
Citazioni
  • ???jsp.display-item.citation.pmc??? ND
  • Scopus 0
  • ???jsp.display-item.citation.isi??? ND
social impact