EASformer: Environment-Action-State Alignment for Industrial Generative Ranking

Bohao Li; DingLun Cao; Pengcheng Huang; Yingda Tang; Zhiting Zhao; Yi Cao; Jilei Fu; Heng Chang

cs.AIOct 20, 2026

EASformer: Environment-Action-State Alignment for Industrial Generative Ranking

Authors: Bohao Li, DingLun Cao, Pengcheng Huang, Yingda Tang, Zhiting Zhao, Yi Cao, Jilei Fu, Heng Chang,

Organizations: Beihang University, Beijing, China · Harbin Institute of Technology, Shenzhen, China · Independent Researcher, Shanghai, China · Independent Researcher, Beijing, China

Abstract

Modern industrial ranking systems increasingly adopt generative recommendation (GR) paradigms, in which user behavior sequences and non-sequential (NS) features are tokenized and jointly modeled by Transformer-style backbones. This unified formulation provides a scalable interface for modeling long histories and heterogeneous features. However, sequence and NS representations often carry asymmetric semantics: historical representations may combine environment context with observed-action semantics, whereas candidate-side NS representations contain no observed action but incorporate richer supplementary context, such as user-item interaction features and recent behavior lists. Direct interaction between such representations can make query-key dot products compare different semantic objects, limiting meaningful cross-token interaction. To fill this gap, we propose EASformer, an Environment-Action-State disentangled architecture for industrial app ranking. EASformer constructs an explicitly aligned context environment space shared by historical and non-sequence tokens, while disentangling user actions from historical token and candidate-specific context form non-sequence tokens. Based on this aligned space, EASformer enables complementary historical action retrieval and state-conditioned action prediction. Furthermore, the disentangled design supports a Cloze-style self-supervised objective by masking historical actions and reusing the candidate prediction pathway. On an offline benchmark from a real-world industrial app marketplace, EASformer achieves an AUC of 0.89411, yielding a 0.88% relative improvement over the deployed baseline. Online A/B tests further support this advantage, with relative eCPM lifts of 1.26% and 3.94% across two real-world ranking scenarios.

Figures & tables

Figure 1. Semantic misalignment between sequence and NS representations. EASformer separates aligned context environment

E

, candidate-specific supplementary context

E^{\prime}_{c}

, and user action before interaction. A schematic comparison between existing tokenization and EASformer. Existing methods mix sequence and non-sequential features with unequal semantics, while EASformer aligns historical context tokens with candidate context tokens and separates actions as values.

Figure 2. Overview of the proposed framework.

Figure 3. Masked-action self-supervision in EASformer. Masked historical Action tokens are replaced with a learnable placeholder and processed normally by both the Temporal Retrieval and Spatial Mapping modules.

Sequence Modeling	Feature Interaction	AUC $\uparrow$	$\Delta$ AUC (%)
Two-Stage Models
LONGER	RankMixer	0.88895	$+0.30$
LONGER	DCNv2	0.88660	$+0.04$
LONGER	Wukong	0.88859	$+0.26$
SASRec	RankMixer	0.87410	$-1.37$
SASRec	DCNv2	0.86540	$-2.36$

Table 1. Overall performance.

Variant	AUC	$\Delta$ AUC (%)
EASformer (full)	0.89411	–
w/o SSL	0.89146	$-0.30$
w/o EAS	0.88832	$-0.65$
w/o Spatial Mapping	0.89262	$-0.17$
w/o Temporal Retrieval	0.89084	$-0.37$

Table 2. Ablation results.

\Delta

AUC denotes the relative change with respect to the full EASformer model.

Panel of Figure 4 — Figure 4. Qualitative comparison of history–candidate representation alignment.

Figure 5. Category-level candidate-to-history attention. Bold headers denote candidate categories; row labels denote history categories.

Figure 6. Parameter sensitivity of the masked-action self-supervised objective. (a) Masking ratio

\rho

. (b) Self-supervised loss weight

\lambda_{s}

$\bm{\mathcal{D}}$	History Length ( $T$ )	AUC $\uparrow$	Latency (ms) $\downarrow$
512	500	0.89411	$39.40$
512	375	$0.893\,45$	$39.23$
512	250	$0.893\,08$	$37.20$
512	125	$0.891\,35$	$35.14$
128	500	$0.893\,01$	27.71

Table 3. Efficiency analysis of EASformer.

\mathcal{D}

denotes the hidden dimension of each Environment, Action, and State stream, and

\mathcal{D}_{\mathrm{NS}}=512

for all settings.

Scenario	eCPM Lift $\uparrow$
Ranking Scenario 1	$+1.26\%$
Ranking Scenario 2	$+3.94\%$

Table 4. Relative eCPM lifts of EASformer - over the production HyFormer baseline during one online evaluation period.

Learning to Rank

Learning to rank develops models that order items by relevance or utility for search, recommendations, and other decisions.

See all 267 papers

From Language to Behavior: Scaling Sequence Transformers for Industrial Recommendation Ranking with Rec-Native Designs
Sep 1, 2026Jie Chen, Xiangqian Yu, Yanchao Lian +9Sequential Recommendation Learning to Rank
ByteDance
Beyond Interleaving: Causal Attention Reformulations for Generative Recommender Systems
Mar 11, 2026Hailing ChengCausal Attention Generative Recommendation
Linkedin Inc Mountain View, California, USA
Structuring and Tokenizing Distributed User Interest Context for Generative Recommendation
Jun 18, 2026Ruizhong Qiu, Yinglong Xia, Dongqi Fu +6Sequential Recommendation Tokenization
University of Illinois Urbana–Champaign · 2Meta MRS

Sep 1, 2026J/K move · Enter open · S save

From Language to Behavior: Scaling Sequence Transformers for Industrial Recommendation Ranking with Rec-Native Designs

Jie Chen, Xiangqian Yu, Yanchao Lian, Tan Lu+8

ByteDance

Sequential Recommendation Learning to Rank Recommender Systems

Open paper AskPDF arXiv

Scaling Transformers has driven large gains in language modeling, but transplanting this to behavior-sequence modeling in production ranking is challenging: recommendation differs in signal quality, where behavior sequences are noisy, temporally irregular, and sparsely supervised, and in computation asymmetry, where each request scores many candidates against one shared user history under tight latency budgets. We propose ReST, a recommendation-native Transformer scaling framework. For signal quality, it introduces a sequence encoder with dual-gated attention, rotary positional and temporal embedding, stabilized residual normalization, and training-only auxiliary objectives. For computation asymmetry, it factorizes ranking into a heavy reusable encoder and a lightweight cross decoder with projection-free KV attention and token-specific parameterization, coupling user-level shared-prefix training with shared-prefix serving for compute-once, decode-many-times ranking. Across industrial and public benchmarks, ReST achieves higher accuracy and scales more consistently along sequence length, depth, and width, where LLM-style Transformer blocks saturate. A one-week online A/B test on a production advertising platform improves online AUC by 1.31% and lifts a core revenue metric by 11.93% within a 50 ms P99 budget; ReST has since been fully deployed in production, showing that behavior-sequence scaling remains a promising, under-exploited axis for production ranking.

Abstract

Learning to Rank

Explore similar work

From Language to Behavior: Scaling Sequence Transformers for Industrial Recommendation Ranking with Rec-Native Designs

Beyond Interleaving: Causal Attention Reformulations for Generative Recommender Systems

Structuring and Tokenizing Distributed User Interest Context for Generative Recommendation