Overall Rating Prediction from Review Texts using Category-oriented Japanese Sentiment Polarity Dictionary

Abstract

Hotel booking sites provide evaluations, including textual reviews and numerical ratings by hotel guests. However, some evaluations do not include numerical ratings, and there are some evaluations in which textual reviews and numerical ratings are inconsistent (i.e., a positive review text is posted along with a low rating, or vice versa). Such evaluations may need to be
clarified for site users. To resolve such problems, we propose three highly accurate methods to predict an overall numerical rating from a textual review. Our new proposal is to use Categoryoriented Sentiment Polarity Dictionaries (CSPD), which are automatically compiled for each category using a Rakuten Travel review database. The CSPD gives the sentiment polarity value
(i.e., the positivity/negativity value) for each sentiment word for each category. Our proposed methods first predict category ratings from the BERT vector for the review and the CSPD. After that, based on the predicted category ratings and the BERT vector, our methods predict the overall rating. We conducted evaluation experiments using the Rakuten Travel review dataset
for 2014-2019. Our experimental results show that our methods achieve higher accuracy than using only BERT vectors and successfully detect inconsistent evaluations.