cs.CLFeb 24, 2026
SaveBeyond the Star Rating: A Scalable Framework for Aspect-Based Sentiment Analysis Using LLMs and Text Classification
Organizations: University of Southern California, Marina del Rey, 90292, CA, USA.
Abstract
Customer-provided reviews have become an important source of information for business owners and other customers alike. However, effectively analyzing millions of unstructured reviews remains challenging. While large language models (LLMs) show promise for natural language understanding, their application to large-scale review analysis has been limited by computational costs and scalability concerns. This study proposes a hybrid approach that uses LLMs for aspect identification while employing classic machine-learning methods for sentiment classification at scale. Using ChatGPT to analyze sampled restaurant reviews, we identified key aspects of dining experiences and developed sentiment classifiers using human-labeled reviews, which we subsequently applied to 4.7 million reviews collected over 17 years from a major online platform. Regression analysis reveals that our machine-labeled aspects significantly explain variance in overall restaurant ratings across different aspects of dining experiences, cuisines, and geographical regions. Our findings demonstrate that combining LLMs with traditional machine learning approaches can effectively automate aspect-based sentiment analysis of large-scale customer feedback, suggesting a practical framework for both researchers and practitioners in the hospitality industry and potentially, other service sectors.
Figures & tables
Figure 1: Example of aspect-based sentiment analysis on two restaurant reviews. Identified aspects in the review text, such as service , food , and ambiance , are highlighted in bold. Positive sentiments are highlighted in green, and negative sentiments in red.
| Fields Used in the Study | |
|---|---|
| Variable | Description |
| review_id | Unique identifier for each review |
| user_id | Unique identifier for each user |
| business_id | Unique identifier for each business |
| review_rating | Star rating assigned by the user to the business (1-5 scale) |
| text | Full text of the review provided by the user |
Table 1: Description of variables from the Yelp Open Dataset used for analysis.
| GPT-4o | |
|---|---|
| Bussiness ID | Extracted Aspects |
| cOoTI*******VZzdw | Service, Food Quality, Atmosphere/Ambiance, Value for Money, Speed of Service, Accuracy of Order, Drink Quality, Friendliness of Staff, Consistency, Special Dietary Offerings |
| 2N9WH*******-qN_A | Food Quality, Service, Beverage Quality, Ambiance, Price/Value, Accuracy of Order, Consistency, Dietary Options, Wait time |
| 6EizP*******ZMI9Lw | Service, Food Quality, Ambiance, Presentation, Value for Money, Timeliness, Menu Variety, Consistency, Portion Size, Drink Selection |
| cpaML*******_GXow | Service, Boba Quality, Price, Sushi Quality, Location Convenience, Atmosphere/Ambiance, Portion Size, Authenticity, Customization Options |
| RXA9e*******KHyIQ | Food Quality, Service, Atmosphere/Ambiance, Cleanliness, Portion Size, Value for Money, Speed/Timeliness, Authenticity, Consistency, Curb Appeal, Recommendations, Specialty Items |
Table 2: Key aspects extracted by GPT-4o and 4o-mini for five randomly selected businesses based on customer reviews. Business IDs are partially masked.
| Positive Sentiment Group | |
|---|---|
| Topic 1 | great, food, place, service, restaurant, good, time, chicken, santa, chase |
| Topic 2 | best, clam, chowder, waiter, manhattan, italian, barbara, santa, favorite, great |
| Topic 3 | delicious, food, perfect, place, kitchen, great, love, cherry, little, half |
| Topic 4 | great, place, food, service, lunch, loved, drink, cheese, jonathan, good |
| Topic 5 | calamari, happy, hour, appetizer, great, drink, chicken, bartender, day, good |
Table 3: Examples of the most relevant words in topics generated by Latent Dirichlet Allocation (LDA) across the three sentiment groups (positive, negative, neutral) for restaurant reviews. Each group consists of five topics, with the ten most frequently associated words shown for each topic.
| Review | Service | Food | Ambiance | Wait | Price | Menu |
| Quality | Time | Variety | ||||
| The New Mexican style cuisine here is damn good. I love the breakfast, especially with a bloody mary. Yum! The green and red sauces are delicious but SPICY so if you don’t like spice consider the sour cream green chili sauce, or half and half. The staff are pretty great too. | 1 | 1 | NA | NA | NA | 1 |
| Great atmosphere, very cool looking on the inside, makes you think you are in a submarine or that’s how I see it. Food was good and they have many beers on tap. The only downside was slow service, but maybe it was just a tough night for the staff. Besides that, a great place to come with friends or family. | -1 | 1 | 1 | NA | 0 | 1 |
| Really good fries…about a dozen varieties from Spanish, Irish, Mexican, pizza & more. Wings (their specialty) were great. Our waitress, Camilla, was quick, attentive, friendly & helpful. Plus she was as cute as a button. Bathrooms were clean. 50% off wings if you check in on Yelp!! We shall return. | 1 | 1 | 1 | 1 | 1 | 1 |
| The one positive: the view. Service was horrible, the drinks watered down and the food was horrible. Order beer by the bottle and don’t expect much beyond a cool location. | -1 | -1 | 1 | NA | NA | NA |
Table 4: Example of sentiment labels for reviews assigned by one reviewer, with aspects rated by sentiment score: -1 (negative), 0 (neutral), 1 (positive), and ‘NA’ for ‘irrelevant’.
| Service (Mean = 0.855, p < 0.05) A1 A2 A3 A4 A5 A1 1 .83 *** .95 *** .88 *** .78 *** A2 1 .91 *** .84 *** .83 *** A3 1 .90 *** .85 *** A4 1 .85 *** A5 1 | Food Quality (Mean = 0.883, p < 0.01) A1 A2 A3 A4 A5 A1 1 .90 *** .92 *** .86 *** .82 *** A2 1 .92 *** .90 *** .92 *** A3 1 .81 *** .90 *** A4 1 .88 *** A5 1 |
| Ambiance (Mean = 0.864, p < 0.05) A1 A2 A3 A4 A5 A1 1 .90 *** .98 *** .79 *** .98 *** A2 1 .91 *** .72 *** .96 *** A3 1 .72 *** .95 *** A4 1 .75 *** A5 1 | Wait Time (Mean = 0.745, p < 0.05) A1 A2 A3 A4 A5 A1 1 .80 *** .54 * .80 *** .90 *** A2 1 .52 * .85 *** .92 *** A3 1 .55 * .61 ** A4 1 .88 *** A5 1 |
| Price (Mean = 0.918, p < 0.01) A1 A2 A3 A4 A5 A1 1 .94 *** .91 *** .99 *** .98 *** A2 1 .85 *** .91 *** .92 *** A3 1 .90 *** .93 *** A4 1 .96 *** A5 1 | Menu Variety (Mean = 0.872, p < 0.05) A1 A2 A3 A4 A5 A1 1 .90 *** .89 *** .85 *** .96 *** A2 1 .86 *** .84 *** .88 *** A3 1 .82 *** .88 *** A4 1 .90 *** A5 1 |
| * p < 0.05, ** p < 0.01, *** p < 0.001 | |
Table 5: Inter-annotator agreement results measured by pairwise Pearson correlations across six dining aspects: service , food quality , ambiance , wait time , price , and menu variety .
| Business ID | Serv-ice | Qual-ity | Ambi-ance | Wait Time | Price | Menu | Over-all | State | Cuisine |
| I8…WsWA | 0.53 | 0.80 | 0.46 | 0.00 | 0.06 | 0.00 | 4.5 | NV | American |
| UJ…RlA | 0.27 | 0.77 | 0.07 | -0.01 | -0.01 | 0.03 | 4.5 | LA | Indian |
| 2L…dzw | -0.84 | -0.30 | 0.00 | -0.15 | 0.00 | 0.00 | 1.5 | FL | Other |
| 8T…FMg | 0.25 | 0.50 | 0.06 | -0.01 | 0.00 | 0.01 | 3.5 | PA | Italian |
| BF…sQ | 0.19 | 0.80 | 0.00 | -0.01 | 0.01 | 0.03 | 4.0 | FL | Mediter-ranean |
Table 6: Example average predicted ratings for each restaurant across six aspects: service , food quality(quality) , ambiance , wait time , price , and menu variety (menu) The Overall column represents the restaurant’s overall rating. Business IDs are partially masked for anonymity, and the State and Cuisine columns indicate the location and main cuisine type offered, respectively.
Figure 2: Average classification accuracy of machine learning algorithms using different text vectorization methods (TF-IDF with and without class imbalance adjustment, and fastText) across all six aspects.
Figure 3: Performance comparison of machine learning models across restaurant review aspects and sentiment classifications. This figure presents F1-scores and overall accuracy for Support Vector Machines (SVM), Logistic Regression (L.R.), and different vectorization approaches (TF-IDF with/without class imbalance handling, and FastText) across six restaurant aspects. For each aspect, F1-scores are shown for three sentiment categories (Negative, Neutral/Irrelevant, Positive).
| One-Stage Classifier | ||||||
|---|---|---|---|---|---|---|
| Metric | Service | Ambiance | Quality | Menu | Wait Time | Price |
| Negative | ||||||
| Precision | 0.81 | 0.62 | 0.68 | 0 | 0.64 | 0.79 |
| Recall | 0.75 | 0.11 | 0.58 | 0 | 0.46 | 0.13 |
| F1-Score | 0.78 | 0.19 | 0.63 | 0 | 0.54 | 0.23 |
| Support | 191 | 45 | 160 | 26 | 84 | 82 |
Table 7: Performance metrics, including accuracy, class-specific precision, recall, and F1-score, with support of the one-stage classifier across six aspects of restaurant reviews: service, ambiance, food quality, menu, wait time, and price .
| Two-Stage Classifier | ||||||
|---|---|---|---|---|---|---|
| Metric | Service | Ambiance | Quality | Menu | Wait Time | Price |
| Stage 1: Relevance | ||||||
| Relevant | ||||||
| Precision | 0.76 | 0.75 | 0.91 | 0.69 | 0.73 | 0.82 |
| Recall | 0.94 | 0.47 | 1 | 0.12 | 0.17 | 0.06 |
| F1-Score | 0.84 | 0.58 | 0.95 | 0.21 | 0.27 | 0.11 |
Table 8: Performance metrics, including accuracy, class-specific precision, recall, F1-score, and support for stage one (relevance classification (relevant vs. irrelevant), as well as sentiment classification (negative, neutral, positive). The two-stage classifier operates across six aspects of restaurant reviews: service, ambiance, food quality, menu, wait time, and price . Note that Stage 2 was trained exclusively on relevant reviews.
| Aspect | Model 1 | Model 2 | Model 3 | Model 4 |
|---|---|---|---|---|
| Service | 0.75 (0.73, 0.77)*** | 0.74 (0.72, 0.76)*** | 0.78 (0.76, 0.80)*** | 0.77 (0.76, 0.79)*** |
| Food Quality | 1.58 (1.56, 1.60)*** | 1.59 (1.57, 1.60)*** | 1.58 (1.56, 1.60)*** | 1.58 (1.57, 1.60)*** |
| Ambiance | -0.31 (-0.33, -0.29)*** | -0.31 (-0.33, -0.28)*** | -0.27 (-0.30, -0.25)*** | -0.27 (-0.29, -0.25)*** |
| Wait Time | 0.22 (0.15, 0.28)*** | 0.24 (0.17, 0.30)*** | 0.18 (0.12, 0.25)*** | 0.22 (0.15, 0.28)*** |
| Price | 0.06 (-0.05, 0.17) | 0.05 (-0.06, 0.16) | 0.08 (-0.03, 0.19) | 0.07 (-0.04, 0.18) |
| Menu Variety | 0.69 (0.64, 0.74)*** | 0.70 (0.64, 0.75)*** | 0.66 (0.61, 0.72)*** | 0.67 (0.61, 0.72)*** |
Table 9: Regression coefficients predicting overall restaurant ratings (one-stage classifier). This table presents the estimated coefficients, 95% confidence intervals, and significance levels for the six aspects across four regression models. These aspects represent the primary predictors, while control variables (cuisine and state) are analyzed separately in Figure 4 . *** indicates significance level of .
Figure 4: Visualization of control variable effects from the main regression models. Panel (a) shows the effects of cuisine from Model 2, and panel (b) shows the effects of state from Model 3. Horizontal bars represent effect sizes, with 95% confidence intervals shown as error bars. Categories are ordered by effect magnitude, with numerical estimates displayed alongside each bar. Gray bars indicate non-significant effects ( ). Significance levels: *** , ** , * .
| Service | Food Quality | Atmosphere | ||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Model | Class | Prec | Rec | F1 | Acc | Supp | Prec | Rec | F1 | Acc | Supp | Prec | Rec | F1 | Acc | Supp |
| MNB+TFIDF(w) | Pos | .66 | .71 | .68 | .65 | 411 | .69 | .98 | .81 | .69 | 621 | .75 | .17 | .28 | .72 | 263 |
| Neu | .60 | .66 | .63 | 398 | .62 | .29 | .39 | 219 | .72 | .98 | .83 | 692 | ||||
| Neg | .81 | .50 | .62 | 191 | .74 | .12 | .21 | 160 | .00 | .00 | .00 | 45 | ||||
| LR+TFIDF(w) | Pos | .74 | .73 | .73 | .71 | 411 | .80 | .92 | .86 | .73 | 621 | .60 | .43 | .50 | .74 | 263 |
| Neu | .67 | .73 | .73 | 398 | .56 | .41 | .47 | 219 | .78 | .90 | .84 | 692 | ||||
Table 10: Complete model evaluation results for sentiment classification. Results include Positive (Pos), Neutral (Neu), and Negative (Neg) sentiment metrics (Precision, Recall, F1, and Accuracy) across all model combinations: TF-IDF (with and without class imbalance) and FastText vectorization, paired with Multinomial Naive Bayes (MNB), Logistic Regression (LR), Random Forest (RF), and Support Vector Machines (SVM). Metrics are presented for the aspects: Service , Food Quality , and Atmosphere . Class distributions are provided in the Support (Supp) column.
| Wait Time | Price | Menu | ||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Model | Class | Prec | Rec | F1 | Acc | Supp | Prec | Rec | F1 | Acc | Supp | Prec | Rec | F1 | Acc | Supp |
| MNB+TFIDF(w) | Pos | .00 | .00 | .00 | .85 | 78 | .00 | .00 | .00 | .73 | 184 | .86 | .03 | .05 | .74 | 237 |
| Neu | .85 | 1.00 | .92 | 838 | .73 | 1.00 | .85 | 734 | .74 | 1.00 | .85 | 737 | ||||
| Neg | .81 | .20 | .32 | 84 | .00 | .00 | .00 | 82 | .00 | .00 | .00 | 26 | ||||
| LR+TFIDF(w) | Pos | .73 | .10 | .18 | .86 | 78 | .58 | .16 | .25 | .75 | 184 | .55 | .30 | .39 | .75 | 237 |
| Neu | .87 | .98 | .93 | 838 | .77 | .98 | .86 | 734 | .79 | .93 | .85 | 737 | ||||
Table 11: Complete model evaluation results for sentiment classification (continued). Performance metrics for Wait Time, Price , and Menu aspects across all model and vectorization combinations.
| Aspect/Variable | Model 1 | Model 2 | Model 3 | Model 4 |
|---|---|---|---|---|
| Main Aspects | ||||
| Service | 0.587 (0.011)*** | 0.464 (0.010)*** | 0.449 (0.011)*** | 0.450 (0.011)*** |
| Food Quality | 1.315 (0.009)*** | 1.438 (0.009)*** | 1.466 (0.009)*** | 1.465 (0.009)*** |
| Ambiance | -0.379 (0.012)*** | -0.374 (0.012)*** | -0.341 (0.012)*** | -0.336 (0.012)*** |
| Wait Time | 0.626 (0.027)*** | 0.637 (0.026)*** | 0.588 (0.026)*** | 0.594 (0.026)*** |
| Price | 0.142 (0.041)*** | 0.106 (0.040)** | 0.075 (0.039). | 0.065 (0.039). |
Table 12: Regression results predicting overall restaurant ratings (two-stage classifier). Coefficients shown with standard errors in parentheses. Model 1: Global regression without controls; Model 2: Including state fixed effects; Model 3: Including cuisine fixed effects; Model 4: Including both state and cuisine fixed effects.
Figure 5: Workflow visualization for the classification and regression analysis of Yelp restaurant reviews, detailing the training of one-stage and two-stage classifiers, prediction processes based on review relevance and sentiment, and a final regression analysis to model overall business ratings from aggregated sentiment predictions.
More in this topic
Aspect-Based Sentiment Analysis
Aspect-Based Sentiment Analysis (ABSA) identifies what people express opinions about and the sentiment or emotion attached to each aspect.
See all 52 papers