cs.CLMar 1, 2026
SaveMOSAIC: Modular Opinion Summarization using Aspect Identification and Clustering
Organizations: Viator, TripAdvisor / London
Abstract
Reviews are central to how travelers evaluate products on online marketplaces, yet existing summarization research often emphasizes end-to-end quality while overlooking benchmark reliability and the practical utility of granular insights. To address this, we propose MOSAIC, a scalable, modular framework designed for industrial deployment that decomposes summarization into interpretable components, including theme discovery, structured opinion extraction, and grounded summary generation. We validate the practical impact of our approach through online A/B tests on live product pages, showing that surfacing intermediate outputs improves customer experience and delivers measurable value even prior to full summarization deployment. We further conduct extensive offline experiments to demonstrate that MOSAIC achieves superior aspect coverage and faithfulness compared to strong baselines for summarization. Crucially, we introduce opinion clustering as a system-level component and show that it significantly enhances faithfulness, particularly under the noisy and redundant conditions typical of user reviews. Finally, we identify reliability limitations in the standard SPACE dataset and release a new open-source tour experience dataset (TRECS) to enable more robust evaluation.
Figures & tables
Figure 1: Industrial deployment of MOSAIC on a live product page. The interface surfaces intermediate outputs directly to users, including actionable Traveler Tips and interactive Review Themes (e.g., Couple-friendly, Amazing Sights) that highlight relevant text using opinions.
Figure 2: Example of structured ABSA output, where the LLM decomposes a single review into multiple theme–aspect–opinion–sentiment tuples that capture distinct facets of the customer experience.
Figure 3: Overview of the proposed opinion - summarisation pipeline using example from travel experience domain. Structured ABSA tuples (theme, aspect, opinion, sentiment) are collated by theme (e.g., “Guide”, “Pickup”), clustered to merge semantically similar opinions, and then summarised into concise theme - level summaries, which are finally composed into a coherent product - level summary.
| Methods | Coverage | G-Eval | AlignScore-R/M |
|---|---|---|---|
| Sentiment CoT-GPT-4o li2024sentiment | 0.96 | 0.75 | 0.72 / 0.08 |
| FT-Llama 8B touvron2023llama | 0.87 | 0.60 | 0.33 / 0.06 |
| Aspect-aware decomposition-GPT-4o li2025decomposed | 0.95 | 0.76 | 0.68 / 0.06 |
| MOSAIC-GPT-4o (ours) | 0.99 | 0.84 | 0.81 / 0.16 |
| Automatic decomposition-Llama 70B khot2022decomposed | 0.59 | 0.31 | 0.51 / 0.03 |
| Chunk-wise decomposition-Llama 70B khot2022decomposed | 0.84 | 0.72 | 0.65 / 0.06 |
Table 1: Results on PeerSum - The first section of the table presents results for GPT-4o and state-of-the-art models. The second section has results for Llama 70B. Underlined scores denote best in section per metric while bold scores denote best overall. AlignScore-R calculates AlignScore against source reviews, while AlignScore-M is computed against reference meta-reviews.
| Methods | Coverage | G-Eval | AlignScore-R/M |
|---|---|---|---|
| HIRO-abs hosking2024hierarchical | 0.87 | 0.62 | 0.83 / 0.24 |
| TCG bhaskar2022prompted | 0.98 | 0.66 | 0.66/0.11 |
| Aspect-aware decomposition-GPT-4o li2025decomposed | 1.00 | 0.90 | 0.81/0.10 |
| MOSAIC GPT-4o (ours) | 1.00 | 0.90 | 0.77/0.07 |
| Automatic decomposition-Llama 70B khot2022decomposed | 0.63 | 0.38 | 0.70/ 0.22 |
| Chunk-wise decomposition-Llama 70B khot2022decomposed | 0.93 | 0.84 | 0.65/0.01 |
Table 2: Results on SPACE - The first section of the table presents results for GPT-4o and state-of-the-art models. The second section has results for Llama 70B. Underlined scores denote best in section per metric, while bold scores denote best overall. AlignScore-R calculates AlignScore against source reviews, while AlignScore-M is computed against reference meta-reviews.
| G-Eval | AlignScore-R/M | |
|---|---|---|
| Without OC (GPT-4o) | 0.859 | 0.904/0.48 |
| With OC (GPT-4o) | 0.858 | 0.909 / 0.518 |
| Without OC (Llama 70B) | 0.631 | 0.935 /0.546 |
| With OC (Llama 70B) | 0.684 | 0.933/ 0.575 |
Table 3: Results on TRECS dataset. First section of the table presents results with and without opinion clustering (OC) and using GPT-4o for summarization, and second section has results for Llama3.1-70B.
Figure 4: Preference of three LLMs (GPT-4o-mini, GPT-4.1-mini, and Llama-3.1-70B) when selecting between summaries generated from redundant opinions and those generated from deduplicated opinions, based on aspect coverage across 10 datasets. For each dataset (X-axis), bars represent the percentage of model responses favoring the non-redundant summary (blue), the redundant summary (red), or indicating equal coverage (green).
Figure 5: Preference of three LLMs (GPT-4o-mini, GPT-4.1-mini, and Llama-3.1-70B) when selecting between summaries generated from redundant opinions and those generated from deduplicated opinions, based on aspect faithfulness across 10 datasets. For each dataset (X-axis), bars represent the percentage of model responses favoring the non-redundant summary (blue), the redundant summary (red), or indicating equal coverage (green).
Figure 6: Distribution of sentiment scores for theme-level summaries and of the number of themes identified in product summaries for SPACE and MOSAIC, both evaluated using Llama-3.1 - 70B
Appendix figures & tables8 assets
Supplementary material from the paper’s appendix.
Appendix
| Dataset | #Train / Dev / Test | #Reviews | MetaL | #Aspects |
|---|---|---|---|---|
| PeerSum | 22,420 / 50 / 100 | 14.9 | 156.1 | 5 |
| SPACE | 0 / 25 / 25 | 100 | 75.7 | 6 |
| TRECS | 14,0631 | 409 | 94.8 | 36 |
Table 4: Statistics of our experimental datasets. #Train/Dev/Test denote the number of training, development, and test instances; #Reviews is the average number of reviews per entity; MetaL to the average meta-review length; #Aspects denotes the number of aspects covered.
| Theme & Count | |
|---|---|
| Tour Guide or Instructors | 320 |
| Tour Informativeness | 287 |
| Tour Planning & Coordination | 252 |
| Point of Interest | 237 |
| Natural Landscape Viewing | 174 |
| Food & Beverage | 172 |
Table 5: Distribution of the top 10 most frequent themes in the TRECS dataset. This illustrates the thematic diversity of the travel experience domain, with ’Tour Guide’ and ’Tour Informativeness’ being the most discussed categories.
| Dataset Number | Duplication Vector | Description |
|---|---|---|
| 1 | [3000, 5, 5, 5, 5, 5, 5, 5, 5, 3000] | Extreme redundancy at boundaries |
| 2 | [5, 5, 5, 3000, 3000, 5, 5, 5, 5, 5] | Extreme redundancy in the middle |
| 3 | [3000, 3000, 5, 5, 5, 5, 5, 5, 5, 5] | Extreme redundancy at the start |
| 4 | [3000, 5, 5, 3000, 5, 5, 5, 5, 5, 5] | Extreme redundancy scattered across positions |
| 5 | [2000, 2000, 5, 5, 5, 5, 5, 5, 5, 2000] | High redundancy at boundaries |
| 6 | [2000, 2000, 2000, 5, 5, 5, 5, 5, 5, 5] | High redundancy at the start |
Table 6: Synthetic redundancy patterns designed to stress-test model robustness with respect to opinion redundancy. The vectors specify extreme duplication counts at various positions (start, middle, end, or scattered) to isolate the effects of redundancy and input ordering on summarization quality.
| ID | v1 | v2 | v3 |
|---|---|---|---|
| 1 | The staff are friendly and exceptional. Every room (lobby included) was very clean. They are spacious, very quiet, and come with a coffee maker. Though, the rooms are outdated in decor. The hotel itself is conveniently close to the airport and restaurants. There’s a chocolate-chip cookie at arrival, and for the prices, the experience is a good value . | Service was exceptional and the quality was great! The rooms are always clean, quiet and spacious with nicely appointed bathrooms. The location is across the street from the airport, was within walking distance to a Denny’s and other restaurants. The hotel interior itself is a bit outdated, but the room we stayed was modern. | All the staff was exceptionally helpful, courteous, and friendly, keeping the rooms clean and well-prepared. The interior of the hotel needs updating, but the rooms themselves were very spacious, modern, and comfortable to stay in. The hotel itself is conveniently located near the airport, a steak restaurant, fast food, and has a free shuttle service for broader access to Seattle. |
| 2 | Hotel staff were all wonderfully accommodating and friendly. The entire hotel was very clean and the rooms even smell “fresh”. The rooms were large, with large beds, and modern, essential amenities that made it feel cozy. The swim-up pool bar has great, grilled food and reasonably priced drinks; and the continental breakfast was plentiful. (There are not enough reviews available to mention the location.) | The staff was helpful and friendly, recommending many places to eat. The entire property is very clean, and the room was clean and comfy. The rooms are large, with comfortable beds, and plenty of amenities. The breakfast was plentiful and great, as was the food at the swim-up bar and grill. The location is excellent, close to the airport, but the pool was a bit small, and loud from the planes. It has free WiFi, and overall the hotel is an incredible value . | The staff was very professional and helpful to us. The hotel was very clean and well kept. The rooms were spacious and comfortable. There is a continental breakfast buffet included is great and there is a swim-up bar and grill which has very good food, too. The location is excellent but the Pool was a bit loud from the planes as it is near the airport. |
| 3 | Service was excellent; They were helpful, friendly, and gave great tips on where to go & how to get there. Our room was very nice and large, with a lovely bathroom. It even had a small provision for skylight. Breakfast was ample and delicious if you like the typical European breakfast. The hotel was situated in a great location, in a calm area away from the busy parts of the city, but still comfortable walking distance to everything. It’s a very good value for price . | The staff were pleasant and helpful, especially when pointing out places of interest. Everything was very clean; the rooms even smelled good. The rooms as a whole were very large, with a lovely bathroom, and comfortable enough for a good night’s sleep. The European style breakfast menu was ample and delicious. The hotel was in a quiet location, but close to the Museum of Natural History and within walking distance to everything else. | Staff were very friendly and helpful to show us the direction to places of interest and restaurants. Overall the room- Smelled good, clean. The bathroom is newly renovated and very clean and functional. Our room was very large with a lovely bathroom with a small provision for skylight. If you like the typical European breakfast, then it is, free, delicious and ample. The hotel is located by the Museum of Natural History (Museaodi Storia Nautrale). The hotel is also situated in a calm area away from the busy parts of the city but still in comfortable walking distance to everything. |
| 4 | The service was excellent. The staff was very pleasant and helpful. The rooms were very clean. The room was quite big, the bed was comfy, and it had a kitchenette with a good size fridge. The hotel is in walking distance to most major attractions and the public transportation is easily accessible. The hotel was very nice, and looks like they are doing renovations. The rooms need it. It’s not a high-end hotel, but it’s great value for the price. | The hotel, which is within walking distance to most major attractions, looked dated, like it needed renovations done. Aside from that, the staff was very friendly and helpful, going out of their way to please and assist. No room service, though. The rooms are quite clean and spacious, with comfy beds and a breadth of amenities such as a kitchenette and free internet | Service at the front desk is very friendly and even arranges taxi service; overall just helpful. Everything was real clean, bright, and maintained daily. The rooms were large, with comfy beds, but the furniture was dated. The rest of the hotel could use renovations, it was all dated. The hotel is walking distance to major attractions and public transportation. |
| 5 | Staff were incredibly friendly which made our stay more comfortable. Room was a clean and tidy suite. I was plwasantly surprised by the room. The size was ok and it was small, there was a small kitchen, living room and separate bedroom. It also had big comfortable beds. Room service from the adjoining Moxie’s Grill has good food. The breakfast was very good, too, and can be taken up to a leisurely 2pm at the weekend. The hotel is also in a great location with all major downtown attractions within reasonable walking distance | The hotel is in a great location with all major downtown attractions within reasonable walking distance. The relatively small rooms are comfortable, clean, and convenient, with a kitchen and washer/dryer at your service. The staff on reception were always friendly and helpful, and informative with any question you have to ask them. Room service from the adjoining Moxie’s Grill has good food, fast delivery and friendly service. | The hotel staff was friendly, helpful, and informative. The housekeeping service was excellent and fast. The rooms and hotel are very clean, but people dislike that there are no suite cleaning services (no fresh towels/linens). Although small, rooms include a small kitchen, living room, and bedroom. The bed is big and comfy. There’s a good view from the balcony. Room service, breakfast, and Moxie’s Grill next door all have good food. The hotel is in a great location; all major downtown attractions are within walking distance. The price is good value , compared to other hotels in Vancouver. |
| 6 | The location is excellent, right next to Placa Catalunia. The rooms are nice and clean and big, with great amenities and comfortable beds to satisfy. Every member of staff, from reception to bar staff were extremely professional and helpful, and the breakfast was good, with a sufficient range of offers. | Tha hotel staff was delightful. They were always willing to help answer my questions or provide recommendations. The hotel, rooms, and bathrooms and clean and well maintained. The room was a good size, quite, and had very comfortable beds. Breakfast was good, with a sufficient range of offers. The location was excellent. It was close to everything, including shopping and the Placa Catalunia. The value and reasonable quality makes it hard to beat . | Reception, bar, and housekeeping staff were all delightful and helpful. The hotel was comfortably clean and fresh. The rooms were a good size, with very comfortable beds and great amenities. The hotel is located next to Placa Catalunia, and close enough to get a great shopping and cultural experience. |
Table 7: Qualitative examples of product summaries for eight hotels from the SPACE dataset where the “Value for Money” theme is explicitly discussed (e.g., “good value”, “reasonably priced”). These examples demonstrate that “Value for Money” is a salient and frequently recurring theme in user feedback, underscoring the limitation of the standard SPACE schema which excludes this dimension from evaluation.
| Category | Count |
|---|---|
| rooms | 9820 |
| service | 5999 |
| location | 4534 |
| overall_experience | 4176 |
| food | 3244 |
| amenities | 1847 |
Table 8: Frequency distribution of the top 22 raw themes identified in the SPACE dataset during the initial unconstrained generation phase (Section 3.1.1 ). These counts represent the unfiltered output before refinement and semantic deduplication.
| Final Theme | Original Count | Updated Count |
|---|---|---|
| rooms | 9820 | 9820 |
| service | 5999 | 5999 |
| location | 4534 | 4534 |
| overall_experience | 4176 | 4176 |
| food | 3244 | 3244 |
| amenities | 1847 | 1847 |
Table 9: Frequency distribution of the top 22 raw themes identified in the SPACE dataset during the initial “Unconstrained Theme Generation” phase 3.1.1 . The counts reflect the raw output before and after semantic deduplication, highlighting the importance of semantic deduplication
| Theme | Count |
|---|---|
| Value for Money | 25 |
| Bathroom | 13 |
| Quietness | 8 |
| Transportation | 2 |
Table 10: Frequency of newly identified themes that appear in the top-six most frequent themes per product as per MOSAIC, but are absent from the original SPACE dataset.
| Dataset | Model | Avg. Opinions / Theme | Avg. Themes / Product |
|---|---|---|---|
| SPACE | GPT-4o | 1411 | 10.88 |
| SPACE | GPT-4.1 | 1687 | 10.92 |
| SPACE | LLaMA 3.1 70B | 1349 | 10.88 |
| PeerSum | GPT-4o | 738 | 5.76 |
| PeerSum | GPT-4.1 | 1107 | 5.70 |
| PeerSum | LLaMA 3.1 70B | 621 | 5.79 |
Table 11: Dataset statistics across different models, showing the average number of opinions per theme and the average number of themes per product using section 3.1.1
More in this topic
Text Summarization
Text summarization develops methods for condensing documents while preserving the information, perspective, and evidence readers need.
See all 142 papers