News
14/2/2019: the slides of the presentation at EVALITA 2018 have been published here.
30/9/2018: the annotated test set and the evaluation script have been published here.
26/9/2018: the results of the participant systems have been published here (bottom of the page).
20/9/2018: please fill in this short survey on your submitted system: https://goo.gl/forms/
17/9/2018: the instructions to prepare and submit your technical report have been published here.
In recent years, many websites started offering a high level interaction with users, who are no more a passive audience, but can actively produce new content. For instance, platforms like Amazon or TripAdvisor allow people to express their opinions on products, such as food, electronic items, clothes or services such as hotels and restaurants.
Sentiment Analysis (or Opinion Mining) is the task of automatically extract subjective opinions from a text. In its most basic form, a SA system takes a natural language text in input and classifies it with a label indicating whether the text is expressing a positive or negative sentiment, or neither (neutral, or objective, text).
Recently, the spread of e-commerce and sharing economy platforms led to the creation of vast collections of content on the Web expressing subjective opinions on a wide variety of products and services. Such reviews are often quite detailed in expressing the reviewer’s opinion on several aspects of the target entity.
Aspect-based Sentiment Analysis (ABSA) is an evolution of Sentiment Analysis that aims at capturing the aspect-level opinions expressed in natural language texts.
At the international level, ABSA was introduced as a shared task at SemEval, the most prominent evaluation campaign in the Natural Language Processing field, in 2014 (SE-ABSA14), providing a benchmark dataset of reviews in English . Datasets of computer laptops and restaurant reviews were annotated with aspect terms (both fine-grained, e.g. “hard disk”, “pizza”, and coarse-grained, e.g., “food”) and their polarity (positive or negative).
The task was repeated in SemEval 2015 (SE-ABSA15) and 2016 (SE-ABSA16), aiming to facilitate more in-depth research by providing a new ABSA framework to investigate the relations between the identified constituents of the expressed opinions and growing up to include languages other than English and different domains.
ABSA is a novel task at EVALITA. However, it bears some similarities with at least two tasks from the previous editions of the campaign. SENTIPOLC, featured in the 2014 and 2016 editions of EVALITA, is a shared task on the polarity classification of social media content. In SENTIPOLC, participant systems had to label short messages from tweets as subjective or objective, expressing a positive or negative sentiment, and containing forms of ironic language. In the task we are proposing, participants are similarly invited to tag texts containing subjective opinions (i.e., reviews) with sentiment labels, although at a more fine-grained level.
The other EVALITA task related to ABSA is NEEL-it, held at EVALITA 2016. NEEL-it is the task of Named Entity Recognition and Linking, that is, the task of identifying the spans of an input text that refer to named entities, and linking them to entries in a knowledge base, e.g., pages of Wikipedia. As part of the ABSA task, participant systems will also have to identify the portions of text that refer to specific aspects, before computing the sentiment expressed towards each of them.