Large language models encode extensive world knowledge valuable for zero-shot named entity recognition. However, their causal attention mechanism, where tokens attend only to preceding context, prevents effective token classification when disambiguation requires future context. Existing approaches use LLMs generatively, prompting them to list entities or produce structured outputs, but suffer from slow autoregressive decoding, hallucinated entities, and formatting errors. We propose Just Pass Twice (JPT), a simple yet effective method that enables causal LLMs to perform discriminative token classification with full bidirectional context. Our key insight is that concatenating the input to itself lets each token in the second pass attend to the complete sentence, requiring no architectural modifications. We combine these representations with definition-guided entity embeddings for flexible zero-shot generalization. Our approach achieves state-of-the-art results on zero-shot NER benchmarks, surpassing the previous best method by +7.9 F1 on average across CrossNER and MIT benchmarks, being over 20x faster than comparable generative methods. Interactive demo and access to the model are available at https://witness-ai-jpt-ner.hf.space/
Figures & tables
Figure 1: Just Pass Twice (JPT) enables bidirectional token classification in causal LLMs. (Top) Standard causal masking restricts the target token “Paris” from attending to future context like “album” (red barrier), leading to ambiguity. (Bottom) JPT duplicates the input sequence. In the second pass, the target token (green box) attends backwards to the entire original sequence (green arrows), resolving the entity type without architectural modifications.
Figure 2: Architecture of the proposed model. (Top) Entity definitions are encoded via a text encoder (dimension denc ) and projected into the shared space Rdp using the Entity Projection MLP, yielding entity embeddings pper,ploc,… (Bottom) The Causal LLM processes the duplicated input sequence. Hidden states from the second pass ( h5,…,h7 ) are projected to token embeddings t1,…,tn∈Rdp and scored against entity embeddings via a bilinear classifier.
Model
AI
Lit.
Music
Politics
Science
Movie
Rest.
Avg
Generative Baselines
UniNER-7B
62.9
64.9
70.6
66.9
70.8
61.2
35.2
61.8
GoLLIE
59.1
62.7
67.8
57.2
55.5
63.0
43.4
58.4
InstructUIE
49.0
47.2
53.2
48.2
49.3
63.0
21.0
47.8
SaM (MoE)
60.9
66.9
73.5
74.4
62.6
72.1
52.9
66.2
Discriminative Baselines
Table 1: Zero-shot F1 scores on CrossNER and MIT benchmarks. JPT-8B achieves the best overall average, improving by +7.9 F1 over the strongest baseline (SaM). Results for baselines are taken from prior work ( Ding et al., 2025 ) .
Dataset
UniNER-7B
GLiNER-L
JPT-4B
ACE05
36.9
27.3
44.6
AnatEM
25.1
33.3
37.2
bc2gm
46.2
47.9
54.8
bc4chemd
47.9
43.1
53.3
bc5cdr
68.0
66.4
70.4
Broad Twitter
67.9
61.2
71.2
Table 2: Extended zero-shot F1 results across 20 NER benchmarks spanning biomedical, social media, and multilingual domains. Results for UniNER-7B and GLiNER-L are reported from ( Zaratiana et al., 2024 ) .
Method
Cost()\downarrow$
Time (sec) ↓
F1 ↑
Generative Methods
UniNER-7B
2.77
1970.2
61.8
GNER
8.21
5831.5
75.8
GPT-5 †
0.49
579.6
67.1
Discriminative Methods
GLiNER-L
0.05
33.3
60.9
Table 3: Efficiency comparison on CrossNER-Politics. Cost: compute time × $5.07/hour (local) or API pricing (GPT). † GPT results vary with prompting strategy.
Configuration
Avg Micro F1
Input Duplication
Single Pass
55.7
Entity Definitions
No Definitions
58.3
Prompt-only definitions
63.3
Embedding-only definitions
65.2
Table 4: Ablation studies on JPT-4B. Avg. micro-F1 across CrossNER and MIT benchmarks. Input duplication provides +15.2 F1; dual-channel definitions provide +12.6 F1 over no definitions.
Figure 3: Attention weights averaged across all transformer layers from second-pass tokens (rows) to first-pass tokens (columns). The model attends to corresponding positions (diagonal) and semantically relevant context, suggesting effective bidirectional information flow.
Appendix figures & tables14 assets
Supplementary material from the paper’s appendix.
Appendix
Component
JPT-4B
JPT-8B
Base LLM (Frozen)
Model
Qwen3-4B
Qwen3-8B
Hidden dim ( dllm )
2560
4096
Layers / Attention heads
36 / 32
36 / 32
LoRA Adapters (Trained)
Rank ( r ) / Alpha ( α )
32 / 64
128 / 256
Appendix
Table 5: Detailed architecture configuration for JPT models. Only a small fraction of backbone parameters are trained via LoRA adapters and lightweight projection heads.
Hyperparameter
Value
Optimizer
AdamW
Learning rate
5×10−5
LR scheduler
Cosine with warmup
Warmup ratio
10% of total steps
Effective batch size
8
Gradient accumulation
2
Appendix
Table 6: Training hyperparameters used for both JPT-4B and JPT-8B.
Statistic
Value
Total sentences
17,489
Total tokens
3,391,899
Total entity mentions
374,705
Unique entity types
5,009
Avg. sentence length
194.9 tokens
Entity/non-entity ratio
1:1.5
Appendix
Table 7: Training dataset statistics.
Figure 4: Distribution of entity type frequencies in training data. The long-tail distribution (71% of types have ≤ 10 mentions, median=4) encourages the model to leverage definition semantics rather than memorizing patterns.
Example 1: Sports Domain
“the men’s foil competition \textsc[Competition] in fencing \textsc[Sport] at the 2016 olympic games \textsc[Event] in rio de janeiro \textsc[City] was held on 7 august \textsc[Date] at the carioca arena 3 \textsc[Venue] . the medals were presented by paul tergat \textsc[Person] , ioc \textsc[Org] member, kenya \textsc[Country] ”
Example 2: Natural Disaster Domain
“the 2014 mae lao earthquake \textsc[Earthquake] occurred at 18:08:43 \textsc[Time] indochina time \textsc[TimeZone] on may 5 \textsc[Date] . the epicenter was located 9 km \textsc[Distance] south of mae lao district \textsc[District] , 27 km \textsc[Distance] southwest of chiang rai \textsc[City] , thailand \textsc[Country] ”
Example 3: Biographical Domain
“ helga mühlberg-ulze \textsc[Athlete] is an east german \textsc[Nationality] sprint canoeist \textsc[SportDisc] who competed in the early to mid 1960s \textsc[TimePeriod] . she won gold \textsc[Medal] ( k-2 500m \textsc[RaceCat] : 1966 \textsc[Year] ) and two bronzes \textsc[Medal] ”
Appendix
Figure 5: Training examples showing fine-grained entity types across diverse domains. Entity spans are underlined in blue with type labels in subscript. The dataset contains 5,009 unique entity types including domain-specific categories like Earthquake , TimeZone , and RaceCategory .
Type
Generic Definition
Precise Definition
Δ F1
Location
“A geographical place”
“Any word indicating WHERE: explicit places (NYC, downtown), relative indicators (nearby, around, close by), directional phrases (east, south side). Tag the location word itself.”
“A feature, facility, or service offered by a restaurant. Includes: physical features (bar, parking), services (delivery, reservations, takeout), atmosphere descriptors (romantic, casual, family-friendly).”
+14.29
Appendix
Table 8: Generic vs. precise entity definitions and their impact. F1 computed per-type on MIT Restaurant. Precise definitions with boundary cases and examples dramatically improve recognition.
Model
Params
Token F1
Qwen3-1.7B
1.7B
92.4
Qwen3-4B
4B
93.9
Qwen3-8B
8B
94.1
Qwen3-14B
14B
94.5
Appendix
Table 9: Impact of base LLM size on token-level F1 (private evaluation set). Larger models yield consistent improvements, with diminishing returns beyond 8B parameters.
LoRA Rank
Token F1
r=0 (frozen)
82.0
r=16
93.5
r=32 (default)
93.9
r=64
94.2
r=128
94.5
r=256
94.7
Appendix
Table 10: Impact of LoRA rank on token-level F1 using JPT-4B (private evaluation set). Adaptation is essential ( r=0 drops 12 points), but returns diminish beyond r=32 .
Model
AI
Lit.
Music
Politics
Science
Avg.
Qwen3-4B
49.9
47.9
74.7
56.2
58.1
57.4
JPT-4B
68.3
73.7
84.1
76.4
69.5
74.4
Appendix
Table 11: Comparison between JPT-4B and its causal backbone, Qwen3-4B , on the CrossNER benchmark. JPT-4B consistently outperforms the backbone across all domains.
Entity at start of sentence
GT Type
Single-input prediction
Why future context matters
Type confusion at sentence start
"Autoencoders are trained to minimise…"
algorithm
product
Need “trained to minimise” to determine that it refers to an algorithm
"François Bayrou (…) is a French centrist politician…"
politician
person
Need “politician” later in the sentence to resolve the fine-grained type
Completely missed at sentence start
"The Color Purple is a 1982 epistolary novel…"
book
(missed)
Need “novel” to recognize it as a book title
"Young’s work on squid giant axons…"
scientist
(missed)
Need scientific context later in the sentence to identify the mention as a scientist
Appendix
Table 12: Representative errors made by the single-input JPT variant. When entity mentions appear early in the sentence, the absence of future context can lead to type confusion or missed predictions.
Figure 7: Additional attention visualizations. Rows: second-pass tokens. Columns: first-pass tokens. The diagonal pattern (self-attention across passes) combined with off-diagonal attention to context tokens demonstrates effective bidirectional information flow.
Dataset
Entity Type
Gold Span
Predicted Span
CrossNER-Science
ORGANIZATION
Virgo interferometer
Virgo interferometer collaboration
CrossNER-Science
ACADEMIC JOURNAL
the Springer Series Complexity
of the Springer Series Complexity
CrossNER-Politics
POLITICAL PARTY
Social Credit Party of Canada
national Social Credit Party of
CrossNER-Music
BAND
Collective Consciousness Society
The Collective Consciousness
CrossNER-Literature
LITERARY GENRE
Germany Romanticism
Romanticism era
MIT-Movie
TITLE
the reflecting skin
reflecting skin movie
Appendix
Table 13: Representative boundary detection errors across CrossNER and MIT. JPT often predicts the correct type and core mention but may over-extend or truncate spans when entities appear with modifiers, appositions, or colloquial phrasing.
Figure 8: Entity type confusion matrix aggregated across CrossNER and MIT. Most confusions occur between semantically adjacent types (e.g., PER vs. ORG , LOC vs. COUNTRY , and POL_PARTY vs. POLITICIAN ), suggesting that errors often stem from fine-grained boundary cases and overlapping semantic definitions rather than arbitrary label flips.
Error Type
Dataset
Text (excerpt)
Gold
Pred.
Analysis
Type Confusion
CrossNER-AI
“NIST also differs from Bilingual evaluation understudy in its calculation…”
metrics
conference
Acronym ambiguity
Type Confusion
CrossNER-Politics
“The provinces ceded to Augustus for that ten-year period…”
person
politician
Role vs. entity
Type Confusion
CrossNER-Music
“…on The Clash ’s London Calling …”
album
location
Title–location ambiguity
Missed Entity
CrossNER-AI
“Rethink Robotics introduced Baxter as an industrial robot …”
product
(none)
Descriptive form
Missed Entity
CrossNER-Politics
“growth in the Melanesian countries of Solomon Islands…”
location
(none)
Adjectival reference
Over-prediction
CrossNER-AI
“A frame language is a technology used for knowledge representation…”
(none)
prog. lang.
Over-generalization
Appendix
Table 14: Representative JPT-4B error instances with context excerpts, gold labels, predictions, and diagnostics.
Recent advancements of zero-shot Named Entity Recognition (NER) establish strong baselines by formulating sequence labeling into question answering where Large Language Models (LLMs) can be naturally adopted. However, existing LLM-based zero-shot NER methods suffer from the limitations of prompt and demonstration engineering. To address these issues with minimal human interventions, we introduce DE-NER, a dialogue elicitation framework which elicits the chatting ability of LLMs to fully extract the knowledge encoded in LLMs. Our experiments demonstrate that the proposed method outperform the competitive baselines in zero-shot settings across multiple benchmarks, with an average improvement of 3.75% F1 points. Codes are released in https://github.com/kkkenshi/DE-NER.
Zero-shot Named Entity Recognition (ZS-NER) remains brittle under domain and schema shifts, where unseen label definitions often misalign with a large language model's (LLM's) intrinsic semantic organization. As a result, directly mapping entity mentions to fine-grained target labels can induce systematic semantic drift, especially when target schemas are novel or semantically overlapping. We propose SAM-NER, a three-stage framework based on Semantic Archetype Mediation that stabilizes cross-domain transfer through an intermediate, domain-invariant archetype space. SAM-NER: (i) performs Entity Discovery via cooperative extraction and consensus-based denoising to obtain high-coverage, high-fidelity entity spans; (ii) conducts Abstract Mediation by projecting entities into a compact set of universal semantic archetypes distilled from high-level ontological abstractions; and (iii) applies Semantic Calibration to resolve archetype-level predictions into target-domain types through constrained, definition-aligned inference with a frozen LLM. Experiments on the CrossNER benchmark show that SAM-NER consistently outperforms strong prior ZS-NER baselines in cross-domain settings. Our implementation will be open-sourced at https://github.com/DMIRLAB-Group/SAM-NER.
Ruichu Cai, Juntao Gan, Miao Mai +2
School of Computer Science, Guangdong University of Technology · Peng Cheng Laboratory · Nanfang Media Group(Nanfang Daily) +1
Large language models (LLMs) have advanced information extraction (IE) by enabling zero-shot and few-shot named entity recognition (NER), yet their generative outputs still show persistent and systematic errors. Despite progress through instruction fine-tuning, zero-shot NER still lags far behind supervised systems. These recurring errors mirror inconsistencies observed in early-stage human annotation processes that resolve disagreements through pilot annotation. Motivated by this analogy, we introduce DiZiNER (Disagreement-guided Instruction Refinement via Pilot Annotation Simulation for Zero-shot Named Entity Recognition), a framework that simulates the pilot annotation process, employing LLMs to act as both annotators and supervisors. Multiple heterogeneous LLMs annotate shared texts, and a supervisor model analyzes inter-model disagreements to refine task instructions. Across 18 benchmarks, DiZiNER achieves zero-shot SOTA results on 14 datasets, improving prior bests by +8.0 F1 and reducing the zero-shot to supervised gap by over +11 points. It also consistently outperforms its supervisor, GPT-5 mini, indicating that improvements stem from disagreement-guided instruction refinement rather than model capacity. Pairwise agreement between models shows a strong correlation with NER performance, further supporting this finding.
Siun Kim, Hyung-Jin Yoon
Seltasquare Seoul, Korea · Biomedical Research Institute, Seoul National University Hospital · Seoul National University College of Medicine Seoul, Korea