Auto-AdvER: Disentangling Condition, History, and Commercial Offers in Automotive Advertisements
Organizations: Manchester Metropolitan University, John Dalton tower, Chester St, Manchester M1 5GD, UK · Auto Trader UK, 1 Tony Wilson Place, Manchester M15 4FN, UK
Abstract
This paper presents a case study on the development of Auto-AdvER, a specialised span-labelling schema, along with its corresponding dataset for the textual car advertisement domain. Developed with industry needs in mind, Auto-AdvER is designed to enhance text mining analytics in this domain and contributes a linguistically unique sequence-labelling dataset. We introduce a schema consisting of three labels: 'Condition', 'Historic', and 'Sales Options'. We outline the guiding principles for annotation, describe the methodology for schema development, and present the results of an annotation study demonstrating an inter-annotator agreement of 78%. Furthermore, we compare the performance of various models, including transformer encoder models (BERT, DeBERTaV3) and transformer decoder-based open- and closed-source large language models: Llama, Qwen, GPT, and Gemini, as well as conditional random fields. Our results show that a fine-tuned DeBERTaV3 encoder achieves the highest F1-score, followed closely by conditional random fields, while the large language models reach the highest precision but lower recall. We present this work as a stepping stone towards more fine-grained analysis and discuss Auto-AdvER's potential impact on advertisement analytics, pricing, and customer insights. Potential applications include the analysis of market dynamics and data-driven predictive maintenance.
Figures & tables
| Number of | CONDITION | HISTORIC | SALES OPTIONS |
|---|---|---|---|
| # Labels | 594 | 810 | 2,174 |
| # Tokens (spaCy) | 2,466 | 3,985 | 6,216 |
| # Sub-word tokens (DeBERTaV3) | 2,957 | 4,559 | 6,875 |
| Label | COR | INC | PAR | MIS | SPU |
|---|---|---|---|---|---|
| CONDITION | 72% | 7% | 12% | 5% | 5% |
| HISTORIC | 63% | 5% | 20% | 6% | 6% |
| SALES_ | 60% | 2% | 22% | 8% | 8% |
| OPTIONS |
| Model | Precision | Recall | F1-score |
|---|---|---|---|
| BERT-base | 52.6 1.7 | 63.5 2.4 | 57.5 1.2 |
| DeBERTaV3- base | 63.6 1.2 | 72.4 2.0 | 67.7 0.8 |
| GPT-4o | 73.8 1.6 | 54.4 1.3 | 62.6 0.3 |
| Gemini 1.5 Flash | 72.4 2.1 | 49.9 3.2 | 59.0 2.0 |
| Llama 3.1 70B it | 75.9 1.1 | 44.5 2.3 | 56.1 2.1 |
| Qwen 2.5 72B it | 76.8 3.2 | 34.8 1.1 | 47.9 1.6 |
| Model | F1-score (mean std) | ||
|---|---|---|---|
| Historic | Sales Option | Condition | |
| DeBERTaV3- base | 58.1 2.3 | 74.0 2.7 | 57.8 5.7 |
| GPT-4o | 53.0 2.2 | 68.4 1.9 | 52.2 5.2 |
| Gemini 1.5 Flash | 51.6 2.4 | 66.7 4.1 | 38.5 5.2 |
| CRF | 59.5 3.8 | 73.2 2.8 | 46.5 4.4 |
| Model | COR | INC | PAR | MIS | SPU |
|---|---|---|---|---|---|
| DeBERTaV3-base | 48.7 | 4.9 | 19.8 | 7.7 | 18.8 |
| GPT-4o | 43.3 | 3.9 | 14.6 | 31.2 | 7.0 |
| Gemini 1.5 Flash | 40.6 | 4.6 | 11.2 | 35.9 | 7.6 |
| CRF | 47.0 | 4.1 | 17.2 | 23.2 | 8.4 |
Appendix figures & tables8 assets
Supplementary material from the paper’s appendix.
Appendix
| PoS Tag | HISTORIC | CONDITION | SALES OPTIONS |
|---|---|---|---|
| NN | 23.9% | 23.7% | 37.7% |
| JJ | 14.9% | 12.5% | 10.9% |
| NNP | 14.5% | 13.1% | 25.7% |
| CD | 13.4% | 14.1% | 10.2% |
| NNS | 12.6% | 11.7% | 9.9% |
| IN | 7.5% | 10.2% | 2.2% |
| Label | Match | P | R | F 1 |
|---|---|---|---|---|
| Condition | Exact | 75.60 | 75.60 | 75.60 |
| Partial | 81.64 | 81.64 | 81.64 | |
| Sales_Options | Exact | 64.88 | 64.88 | 64.88 |
| Partial | 76.83 | 76.83 | 76.83 | |
| Historic | Exact | 67.28 | 67.28 | 67.28 |
| Partial | 77.83 | 77.83 | 77.83 |
| Mention | Condition | Historic | Sales Options |
|---|---|---|---|
| MOT | “MOT Until October” | “never failed an MOT” | “6 months MOT” |
| service | “next service due September 2025” | “serviced every 6 months” | “full service history” |
| tyre | “good tyres” | “Tyres recently fitted” | “tyre repair kit” |
| mileage | “18,500 miles” | “new front discs at 64k miles” | “mileage verification” |
| Model | Label | COR | INC | PAR | MIS | SPU |
|---|---|---|---|---|---|---|
| DeBERTaV3-base | Historic | 37.8 | 9.5 | 23.5 | 5.7 | 23.5 |
| Sales Options | 55.6 | 1.3 | 17.4 | 7.8 | 17.8 | |
| Condition | 38.3 | 11.7 | 23.8 | 9.7 | 16.4 | |
| GPT-4o | Historic | 32.3 | 6.0 | 19.3 | 36.7 | 5.7 |
| Sales Options | 49.4 | 1.5 | 13.7 | 27.8 | 7.6 | |
| Condition | 34.7 | 9.8 | 12.5 | 37.0 | 6.0 |
| Feature | Value | Description |
|---|---|---|
| token.lower() | reconditioning | Current token is “reconditioning” |
| token.lower() | securing | Current token is “securing” |
| token.lower() | fsh | Current token is “fsh” (Full Service History) |
| token.lower() | warranty | Current token is “warranty” |
| token.lower() | detailing | Current token is “detailing” |
| -1:token.lower() | visual | Previous token is “visual” |
| Feature | Value | Description |
|---|---|---|
| token.lower() | unregistered | Current token is “unregistered” |
| -1:token.lower() | category | Previous token is “category” |
| token.lower() | category | Current token is “category” |
| +1:token.lower() | but | Next token is “but” |
| -1:token.lower() | excellent | Previous token is “excellent” |
| +1:token.lower() | Next token is a tick emoji |
| Feature | Value | Description |
|---|---|---|
| -1:token.lower() | fronts | Previous token is “fronts” |
| -1:token.lower() | shaft | Previous token is “shaft” |
| +1:token.lower() | refurbished | Next token is “refurbished” |
| token.lower() | motorway | Current token is “motorway” |
| +1:token.lower() | advisories | Next token is “advisories” |
| token[-3:] | NER | Last 3 characters are “NER”, e.g. from “OWNER” |
Document Information Extraction
Document Information Extraction turns text, tables, forms, and images into structured, searchable data.
See all 290 papers