Contributo in atti di convegno, 2017, ITA, 10.1109/ICCVW.2017.45

Cross-media learning for image sentiment analysis in the wild

Vadicamo L.; Carrara F.; Falchi F.; Cimino A.; Dell'Orletta F.; Cresci S.; Tesconi M.

CNR-ISTI, Pisa, Italy; CNR-ISTI, Pisa, Italy; CNR-ISTI, Pisa, Italy; CNR-ILC, Pisa, Italy; CNR-ILC, Pisa, Italy; CNR-IIT, Pisa, Italy; CNR-IIT, Pisa, Italy

Much progress has been made in the field of sentiment analysis in the past years. Researchers relied on textual data for this task, while only recently they have started investigating approaches to predict sentiments from multimedia content. With the increasing amount of data shared on social media, there is also a rapidly growing interest in approaches that work "in the wild", i.e. that are able to deal with uncontrolled conditions. In this work, we faced the challenge of training a visual sentiment classifier starting from a large set of user-generated and unlabeled contents. In particular, we collected more than 3 million tweets containing both text and images, and we leveraged on the sentiment polarity of the textual contents to train a visual sentiment classifier. To the best of our knowledge, this is the first time that a cross-media learning approach is proposed and tested in this context. We assessed the validity of our model by conducting comparative studies and evaluations on a benchmark for visual sentiment analysis. Our empirical study shows that although the text associated to each image is often noisy and weakly correlated with the image content, it can be profitably exploited to train a deep Convolutional Neural Network that effectively predicts the sentiment polarity of previously unseen images.

ICCV 2017 IEEE International Conference on Computer Vision Workshops, Venezia, Italy, 22-29 October 2017

Keywords

Big data, Data Mining, Sentiment Analysis, Social Media Analysis

CNR authors

Cimino Andrea, Vadicamo Lucia, Carrara Fabio, Tesconi Maurizio, Falchi Fabrizio, Dell Orletta Felice, Cresci Stefano

CNR institutes

IIT – Istituto di informatica e telematica, ILC – Istituto di linguistica computazionale "Antonio Zampolli", ISTI – Istituto di scienza e tecnologie dell'informazione "Alessandro Faedo"

ID: 375139

Year: 2017

Type: Contributo in atti di convegno

Creation: 2017-08-30 13:40:03.000

Last update: 2023-11-06 19:32:38.000

External IDs

CNR OAI-PMH: oai:it.cnr:prodotti:375139

DOI: 10.1109/ICCVW.2017.45

ISI Web of Science (WOS): 000425239600038

Scopus: 2-s2.0-85046302366