Auto-AdvER: Disentangling Condition, History, and Commercial Offers in Automotive Advertisements

Filippos Ventirozos; Ioanna Nteka; Tania Nandy; Jozef Baca; Charlie Roadhouse; Peter A. Appleby; Matthew Shardlow

cs.CLDec 7, 2024

Auto-AdvER: Disentangling Condition, History, and Commercial Offers in Automotive Advertisements

Authors: Filippos Ventirozos, Ioanna Nteka, Tania Nandy, Jozef Baca, Charlie Roadhouse, Peter A. Appleby, Matthew Shardlow

Organizations: Manchester Metropolitan University, John Dalton tower, Chester St, Manchester M1 5GD, UK · Auto Trader UK, 1 Tony Wilson Place, Manchester M15 4FN, UK

Abstract

This paper presents a case study on the development of Auto-AdvER, a specialised span-labelling schema, along with its corresponding dataset for the textual car advertisement domain. Developed with industry needs in mind, Auto-AdvER is designed to enhance text mining analytics in this domain and contributes a linguistically unique sequence-labelling dataset. We introduce a schema consisting of three labels: 'Condition', 'Historic', and 'Sales Options'. We outline the guiding principles for annotation, describe the methodology for schema development, and present the results of an annotation study demonstrating an inter-annotator agreement of 78%. Furthermore, we compare the performance of various models, including transformer encoder models (BERT, DeBERTaV3) and transformer decoder-based open- and closed-source large language models: Llama, Qwen, GPT, and Gemini, as well as conditional random fields. Our results show that a fine-tuned DeBERTaV3 encoder achieves the highest F1-score, followed closely by conditional random fields, while the large language models reach the highest precision but lower recall. We present this work as a stepping stone towards more fine-grained analysis and discuss Auto-AdvER's potential impact on advertisement analytics, pricing, and customer insights. Potential applications include the analysis of market dynamics and data-driven predictive maintenance.

Figures & tables

Number of	CONDITION	HISTORIC	SALES OPTIONS
# Labels	594	810	2,174
# Tokens (spaCy)	2,466	3,985	6,216
# Sub-word tokens (DeBERTaV3)	2,957	4,559	6,875

Table 1: Distribution of labels in our dataset, with the spaCy word tokens they cover and the corresponding sub-word tokens of the deberta-v3-base vocabulary.

Label	COR	INC	PAR	MIS	SPU
CONDITION	72%	7%	12%	5%	5%
HISTORIC	63%	5%	20%	6%	6%
SALES_	60%	2%	22%	8%	8%
OPTIONS	60%	2%	22%	8%	8%

Table 2: Pairwise inter-annotator agreement averaged across all three annotators for each label. Scores are computed over all ordered annotator pairs and report the proportion of correct (cor), incorrect (inc), partial (par), missing (mis), and spurious (spu) span matches following the evaluation scheme in Section 4.1 .

Panel of Figure 1 — Figure 1: Prompt part 1/3: the system instruction used for the LLMs.

Panel of Figure 2 — Figure 2: Prompt part 2/3: the prompt that introduces the task and the label’s definition (label_definition). The label definitions are in Appendix B .

Panel of Figure 3 — Figure 3: Prompt part 3/3: the in-context examples and the advertisement to be annotated. The figure shows only two ICL examples for illustration. These would typically be longer—an entire car advert, which might comprise one to several sentences—rather than a single sentence. The last User utterance points to a sentence (i.e., advertisement) that we expect the LLM to annotate with the special characters “@@” and “##” to obtain the token spans that correspond to the label of interest.

Model	Precision	Recall	F1-score
BERT-base	52.6 $\pm$ 1.7	63.5 $\pm$ 2.4	57.5 $\pm$ 1.2
DeBERTaV3- base	63.6 $\pm$ 1.2	72.4 $\pm$ 2.0	67.7 $\pm$ 0.8
GPT-4o	73.8 $\pm$ 1.6	54.4 $\pm$ 1.3	62.6 $\pm$ 0.3
Gemini 1.5 Flash	72.4 $\pm$ 2.1	49.9 $\pm$ 3.2	59.0 $\pm$ 2.0
Llama 3.1 70B it	75.9 $\pm$ 1.1	44.5 $\pm$ 2.3	56.1 $\pm$ 2.1
Qwen 2.5 72B it	76.8 $\pm$ 3.2	34.8 $\pm$ 1.1	47.9 $\pm$ 1.6

Table 3: Mean and population standard deviation over the three folds of precision, recall and

F_{1}

-score (%) for each model; “it” stands for instruction-tuned; bold marks the highest value in each column.

Model	F1-score (mean $\pm$ std)
Model	Historic	Sales Option	Condition
DeBERTaV3- base	58.1 $\pm$ 2.3	74.0 $\pm$ 2.7	57.8 $\pm$ 5.7
GPT-4o	53.0 $\pm$ 2.2	68.4 $\pm$ 1.9	52.2 $\pm$ 5.2
Gemini 1.5 Flash	51.6 $\pm$ 2.4	66.7 $\pm$ 4.1	38.5 $\pm$ 5.2
CRF	59.5 $\pm$ 3.8	73.2 $\pm$ 2.8	46.5 $\pm$ 4.4

Table 4:

F_{1}

-score performance across different labels for the top four performing models. Values are shown as percentages. The highest scores in each column are bolded.

Model	COR	INC	PAR	MIS	SPU
DeBERTaV3-base	48.7	4.9	19.8	7.7	18.8
GPT-4o	43.3	3.9	14.6	31.2	7.0
Gemini 1.5 Flash	40.6	4.6	11.2	35.9	7.6
CRF	47.0	4.1	17.2	23.2	8.4

Table 5: Match types per model as percentages of its matches pooled over the three test folds, following Table 2 (COR correct, INC incorrect label, PAR partial overlap, MIS missing, SPU spurious); Appendix G gives the breakdown per label.

Appendix figures & tables8 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 4: An example of the annotation interface, illustrating a customised NER annotation task implemented using Prodigy, displayed within a reduced-size window.

PoS Tag	HISTORIC	CONDITION	SALES OPTIONS
NN	23.9%	23.7%	37.7%
JJ	14.9%	12.5%	10.9%
NNP	14.5%	13.1%	25.7%
CD	13.4%	14.1%	10.2%
NNS	12.6%	11.7%	9.9%
IN	7.5%	10.2%	2.2%

Appendix

Table 6: Top PoS Tag Frequencies as Percentages for Each Label. The detailed PoS tags can be found in https://web.archive.org/web/20190206204307/https://www.clips.uantwerpen.be/pages/mbsp-tags .

Label	Match	P	R	F 1
Condition	Exact	75.60	75.60	75.60
Condition	Partial	81.64	81.64	81.64
Sales_Options	Exact	64.88	64.88	64.88
Sales_Options	Partial	76.83	76.83	76.83
Historic	Exact	67.28	67.28	67.28
Historic	Partial	77.83	77.83	77.83

Appendix

Table 7: Inter-annotator agreement (IAA) across all annotator pairs, reported as exact and partial match precision (P), recall (R), and F 1 (%) per entity label. Partial match awards half credit to overlapping spans.

Mention	Condition	Historic	Sales Options
MOT	“MOT Until October”	“never failed an MOT”	“6 months MOT”
service	“next service due September 2025”	“serviced every 6 months”	“full service history”
tyre	“good tyres”	“Tyres recently fitted”	“tyre repair kit”
mileage	“18,500 miles”	“new front discs at 64k miles”	“mileage verification”

Appendix

Table 8: The same mention type under each of the three labels: gold spans from the dataset. An MOT can be the certificate the car holds now, a test it passed in the past, or a promise made with the sale.

Model	Label	COR	INC	PAR	MIS	SPU
DeBERTaV3-base	Historic	37.8	9.5	23.5	5.7	23.5
	Sales Options	55.6	1.3	17.4	7.8	17.8
	Condition	38.3	11.7	23.8	9.7	16.4
GPT-4o	Historic	32.3	6.0	19.3	36.7	5.7
	Sales Options	49.4	1.5	13.7	27.8	7.6
	Condition	34.7	9.8	12.5	37.0	6.0

Appendix

Table 9: Match types per model and label, as percentages of the model’s matches for that label pooled over the three test folds (COR correct, INC incorrect label, PAR partial overlap, MIS missing, SPU spurious).

Feature	Value	Description
token.lower()	reconditioning	Current token is “reconditioning”
token.lower()	securing	Current token is “securing”
token.lower()	fsh	Current token is “fsh” (Full Service History)
token.lower()	warranty	Current token is “warranty”
token.lower()	detailing	Current token is “detailing”
-1:token.lower()	visual	Previous token is “visual”

Appendix

Table 10: Top ten most significant CRF features for the Sales Options class.

Feature	Value	Description
token.lower()	unregistered	Current token is “unregistered”
-1:token.lower()	category	Previous token is “category”
token.lower()	category	Current token is “category”
+1:token.lower()	but	Next token is “but”
-1:token.lower()	excellent	Previous token is “excellent”
+1:token.lower()		Next token is a tick emoji

Appendix

Table 11: Top ten most significant CRF features for the Condition class.

Feature	Value	Description
-1:token.lower()	fronts	Previous token is “fronts”
-1:token.lower()	shaft	Previous token is “shaft”
+1:token.lower()	refurbished	Next token is “refurbished”
token.lower()	motorway	Current token is “motorway”
+1:token.lower()	advisories	Next token is “advisories”
token[-3:]	NER	Last 3 characters are “NER”, e.g. from “OWNER”

Appendix

Table 12: Top ten most significant CRF features for the Historic class.

Document Information Extraction

Document Information Extraction turns text, tables, forms, and images into structured, searchable data.

See all 290 papers

AutoSpecNER: A Fine-Grained Named Entity Recognition Dataset for Vehicle Specification Extraction
Jun 23, 2026Jordan Lee, Filippos Ventirozos, Abdirahman Abdullahm +3Named Entity Recognition Document Information Extraction
Department of Computing and Mathematics, Manchester Metropolitan University, UK · *Work conducted during an internship at Autotrader UK. · Autotrader Research Group, Autotrader UK
A $^3$ : Towards Advertising Aesthetic Assessment
Mar 25, 2026Kaiyuan Ji, Yixuan Gao, Lu Sun +7Image Quality Assessment Multimodal Large Language Models
Shanghai Artificial Intelligence Laboratory · School of Information and Electronic Engineering, East China Normal University · Institute of Image Communication and Network Engineering, Shanghai Jiao Tong University +1
On the Role of Language Representations in Auto-Bidding: Findings and Implications
May 7, 2026Guanyu Zhu, Jining Luan, Hanwen Du +11Automated Bidding Online Advertising
City University of Hong Kong · South China Agricultural University · University of Electronic Science and Technology of China +11

Jun 23, 2026J/K move · Enter open · S save

AutoSpecNER: A Fine-Grained Named Entity Recognition Dataset for Vehicle Specification Extraction

Jordan Lee, Filippos Ventirozos, Abdirahman Abdullahm, Ioanna Nteka+2

Department of Computing and Mathematics, Manchester Metropolitan University, UK · *Work conducted during an internship at Autotrader UK. · Autotrader Research Group, Autotrader UK

Named Entity Recognition Document Information Extraction

Open paper AskPDF arXiv

Vehicle advertisements contain rich specification information, but automotive NER resources remain limited. We introduce AutoSpecNER, an expert-annotated dataset for fine-grained entity recognition in vehicle listings. The dataset includes 659 advertisements from a popular car-selling website, with over 10,000 entities annotated across 15 categories, including MODEL, ENGINE_SPEC, and BATTERY_CAPACITY. Annotation quality was validated through inter-annotator agreement, achieving an average score of 91.5%. We benchmark rule-based extraction, fine-tuned transformer encoders, and large language models. DeBERTa achieves the best performance with a 90% micro-F1 score, outperforming the rule-based baseline (43%) and the strongest large language model (77.8%).

Abstract

Document Information Extraction

Explore similar work

AutoSpecNER: A Fine-Grained Named Entity Recognition Dataset for Vehicle Specification Extraction

A3^33: Towards Advertising Aesthetic Assessment

On the Role of Language Representations in Auto-Bidding: Findings and Implications

A $^3$ : Towards Advertising Aesthetic Assessment