Abstract

This article discusses the problem of extracting sentiment and opinions about a collection of articles on scientific reviews conducted under an international conference on computing in Spanish language. The aim of this analysis is on the one hand to automatically determine the orientation of a review of an article and contrast this approach with the assessment made by the reviewer of the article. This would allow scientists to characterize and compare reviews crosswise, and more objectively support the overall assessment of a scientific article. A hybrid approach that combines an unsupervised machine learning algorithm with techniques from natural language processing is proposed to analyze reviews, and part-of-speech (POS) tagging to obtain the syntactic structure of a sentence. This syntactic structure, along with the use of dictionaries, allows to determine the semantic orientation of the review through a scoring algorithm. A set of experiments were conducted to evaluate the capability and performance of the proposed approaches relative to a baseline, using standard metrics, such as accuracy, precision, recall, and the F1-score. The results show improvements in the case of binary, ternary and a 5-point scale classification in relation to classical machine learning algorithms such as SVM and NB, but they also present a challenge to improve the multiclass classification in this domain.