OPRO: Orthogonal Panel-Relative Operators for Panel-Aware In-Context Image Generation

Sanghyeon Lee; Minwoo Lee; Euijin Shin; Kangyeol Kim; Seunghwan Choi; Jaegul Choo

cs.CVMar 29, 2026

OPRO: Orthogonal Panel-Relative Operators for Panel-Aware In-Context Image Generation

Authors: Sanghyeon Lee, Minwoo Lee, Euijin Shin, Kangyeol Kim, Seunghwan Choi, Jaegul Choo

Organizations: Korea Advanced Institute of Science and Technology (KAIST), Daejeon, Korea

Abstract

We introduce a parameter-efficient adaptation method for panel-aware in-context image generation with pre-trained diffusion transformers. The key idea is to compose learnable, panel-specific orthogonal operators onto the backbone's frozen positional encodings. This design provides two desirable properties: (1) isometry, which preserves the geometry of internal features, and (2) same-panel invariance, which maintains the model's pre-trained intra-panel synthesis behavior. Through controlled experiments, we demonstrate that the effectiveness of our adaptation method is not tied to a specific positional encoding design but generalizes across diverse positional encoding regimes. By enabling effective panel-relative conditioning, the proposed method consistently improves in-context image-based instructional editing pipelines, including state-of-the-art approaches.

Figures & tables

Figure 1: Two positional regimes in tiled ICG and the role of OPRO in panelized attention. (a) Global-canvas encoding : Inpainting-based DiTs treat the tiled layout as a single image on a single global coordinate grid, so different panels become disjoint regions of a unified canvas. (b) Per-panel encoding : T2I-based methods encode each panel in its own local frame and then fuse context features into target generation through attention, reusing the same coordinate range across panels. (c) In panelized attention, diagonal blocks are intra-panel and off-diagonal blocks are inter-panel. OPRO preserves the intra-panel blocks while modulating the inter-panel blocks.

Figure 2: Overview of OPRO for tiled-panel in-context image generation. The proposed framework partitions a tiled canvas into $P$ panels and processes them as a single token sequence. Within each attention layer of a backbone, OPRO modulates the position-aware queries ( $\tilde{q}_{i}$ ) and keys ( $\tilde{k}_{j}$ ) via panel-specific orthogonal operators ( $U_{p(i)}$ and $U_{p(j)}$ ). This adaptation explicitly guides cross-panel interactions while preserving the original same-panel attention geometry. An example generated image is provided on the right.

Figure 3: Two-stage compositional reasoning. Stage 1 (single-panel pretext): classify the sum of two arrow orientations modulo $360^{\circ}$ (8-way) with distractors. Stage 2 (grid reasoning): on an $n{\times}n$ grid, each row provides context examples and a held-out query; the row-wise rule is either rotation by $k\!\cdot\!45^{\circ}$ or vertical mirror symmetry.

Figure 4: OPRO’s impact on parameter efficiency. Validation accuracy (%) plotted against the number of trainable adapter parameters (M) for 3×3 panels

Method	Isometry	SP-Inv	Accuracy
LoRA (Baseline)	-	-	36.20
+ APB	No	No	35.70
+ Asym-OPRO	Yes	No	39.70
+ OPRO (w/o Zero Init)	Yes	Yes	38.60
+ OPRO (Ours)	Yes	Yes	42.00

Table 2: Ablation on 2-stage compositional reasoning task (ViT-B,

3{\times}3

). We analyze the impact of component removal. APB lacks isometry, and Asym-OPRO violates SP-Inv. w/o Zero Init denotes OPRO with random initialization. OPRO (Ours) satisfies all properties.

Figure 5: Comparison with inpainting-based ICG baselines on MagicBrush [ 41 ] test set. Following ICEdit [ 43 ] , a diptych prompt is used: “A diptych with two side-by-side images …but { $\mathcal{P}$ } .” Red dotted boxes highlight regions where the baselines fail to preserve context from the input image, resulting in incorrect or incomplete edits.

Figure 6: Comparison with a T2I baseline on MagicBrush [ 41 ] test set. In UNO [ 37 ] , we use the following instruction: “Create a single image matching the reference, but with the following edit: { $\mathcal{P}$ } .”

Methods	Train. Pa	L1 ↓	CLIP-I ↑	DINO ↑
ACE++ [arXiv’25]	76.6M	0.1215	0.8658	0.7394
+ OPRO ( $\rho=32$ )	+0.93M	0.1114	0.8749	0.7767
InsertAnything [arXiv’25]	37.5M	0.1327	0.8722	0.7917
+ OPRO ( $\rho=32$ )	+0.93M	0.1269	0.8735	0.8009
ICEdit [NeurIPS’25]	22.4M	0.1189	0.8703	0.7706
+ OPRO ( $\rho=32$ )	+0.93M	0.0781	0.9002	0.8531

Table 3: Quantitative results on MagicBrush [ 41 ] test set. OPRO (

\rho=32

) adds only +0.93M learnable parameters and consistently improves.

Appendix figures & tables7 assets

Supplementary material from the paper’s appendix.

Appendix

Method	DINO ( $\uparrow$ )	CLIP-I ( $\uparrow$ )
ICEdit [ 43 ] (LoRA-only)	0.5828	0.7376
ICEdit [ 43 ] + OPRO	0.6192	0.7724

Appendix

Table 4: Quantitative comparison on subject-driven image generation. Results are reported on a subset of DreamBooth using a three-panel layout with two reference panels and one fully masked target panel. OPRO consistently improves ICEdit on both DINO and CLIP-I. Higher is better in all cases.

	Panel $2\times 2$		Panel $3\times 3$		Panel $4\times 4$
Type	+OPRO-BD	$\boldsymbol{\Delta}$	+OPRO-BD	$\boldsymbol{\Delta}$	+OPRO-BD	$\boldsymbol{\Delta}$
APE	37.10	$-0.90$	23.60	$-0.80$	19.00	$-0.50$
RoPE [ 32 ]	45.80	$-0.60$	38.70	$+2.50$	32.50	$+2.20$
LieRE [ 25 ]	58.70	$+0.60$	36.20	$+2.00$	23.30	$+0.40$
ComRoPE [ 39 ]	57.90	$-0.60$	40.90	$+3.10$	29.80	$+0.60$

Appendix

Table 5: Effect of the block-diagonal implementation of OPRO on top of LoRA (

r=8

). We report the accuracy (%) of LoRA+OPRO-BD and the absolute change

\Delta

(percentage points) compared to the LoRA baseline from Tab. 1 of the main manuscript.

Figure 8: Qualitative results on diverse instructional editing tasks. We demonstrate the versatility of OPRO across a broad spectrum of editing categories. The examples illustrate the model’s capability to precisely follow instructions for object replacement, attribute modification, text rendering, and global style transfer, all while maintaining high fidelity to the original image content.

Figure 9: Qualitative examples of multi-reference compositional generation. We demonstrate the capability to integrate attributes from multiple context panels. The model synthesizes a new image by combining the style from the first panel and the object from the second panel. Note that this compositional ability emerges without explicit training on multi-reference layouts.

Method	Isometry	SP-Inv	L1 $\downarrow$	CLIP-I $\uparrow$	DINO $\uparrow$
LoRA (Baseline)	-	-	0.1189	0.8703	0.7706
+ APB	No	No	0.0966	0.8893	0.8196
+ Asym-OPRO	Yes	No	0.0988	0.8880	0.8151
+ OPRO (w/o Zero Init)	Yes	Yes	0.0780	0.8989	0.8510
+ OPRO (Ours)	Yes	Yes	0.0781	0.9002	0.8531

Appendix

Table 6: Ablation Studies on MagicBrush. The table validates the design principles of OPRO on the ICEdit baseline (

r{=}16

). Relative to OPRO, breaking isometry (APB) or same-panel invariance (Asym-OPRO) reduces performance. Removing zero initialization preserves spatial alignment but lowers semantic consistency, as reflected by CLIP-I and DINO.

Method	Base Model	Pos. Encoding	LoRA Target Modules	LoRA Rank ( $r$ )	OPRO Rank ( $\rho$ )
ICEdit [ 43 ]	FluxFill Dev	Global-canvas	Attention ( q,k,v,out )	16	32
ACE++ [ 21 ]	FluxFill Dev	Global-canvas	Attn + MLP + Modulation	16	32
InsertAnything [ 31 ]	FluxFill Dev (+Redux)	Global-canvas	Attention Projections ( q,k,v,out )	16	32
UNO [ 37 ]	Flux Dev	Per-panel	Attention Projections + MLP	256	32

Appendix

Table 7: Hyperparameters for instructional image editing baselines. All models are trained for 5,000 steps using the AdamW optimizer (weight decay 0.01, learning rate

1\times 10^{-4}

) with a batch size of 8. We use bfloat16 precision and a constant learning rate schedule. Note that InsertAnything uses FluxPriorRedux for reference-image conditioning, while UNO adopts an in-context approach. We adapt OPRO in each self-attention layer.

Hyperparameter	Stage 1 (Pre-training)	Stage 2 (Fine-tuning)
Optimization	Adam ( $\beta_{1}=0.9,\beta_{2}=0.999$ )	Adam ( $\beta_{1}=0.9,\beta_{2}=0.999$ )
Batch Size	256	256
Learning Rate	$1\times 10^{-3}$ (Warmup+Cosine)	$5\times 10^{-4}$ (Constant)
Weight Decay	$0.05$	$0.05$
Training Steps	50k	2k
Architecture / Adapter

Appendix

Table 8: Detailed hyperparameters for two-stage compositional reasoning.

Image Generation

Image generation studies models that create or edit images from text, examples, layouts, and other structured controls.

See all 463 papers

Enhancing In-context Panoramic Generation via Geometric-aware Pretraining
Jul 9, 2026Haoran Feng, Ruiyang Zhang, Longyi Zhang +2Conditional Image Generation Image Generation
Insta360 Research · Tsinghua University · Beihang University +1
UNITY: Attention Flow Networks for Adaptive Conditioning in Diffusion
Jun 18, 2026Aryan Das, Koushik Biswas, Moloud Abdar +1Compositional Image Generation Conditional Diffusion Models
VIT Bhopal, India · IIIT Delhi, India · The University of Queensland, Australia +1
Beyond Attention Masks: Instruction Anchoring for Efficient In-Context Diffusion Generation
Aug 21, 2026Yangshuai Liu, Zheming Li, Jiaao Li +4Diffusion Transformer Diffusion Model Caching
Harbin Institute of Technology · KlingAI Research

Jul 9, 2026J/K move · Enter open · S save

Enhancing In-context Panoramic Generation via Geometric-aware Pretraining

Haoran Feng, Ruiyang Zhang, Longyi Zhang, Dizhe Zhang+1

Insta360 Research · Tsinghua University · Beihang University+1

Conditional Image Generation Image Generation Image Editing

Open paper AskPDF arXiv

In this work, we present Canvas360, a two-stage framework for in-context panoramic generation that combines geometry-aware pretraining with downstream task-specific fine-tuning. To address the lack of large-scale, high-quality training data tailored to in-context panoramic tasks, we propose Canvas360Dataset, a collection of 1M high-quality paired panoramic samples for style transfer, inpainting, outpainting, and editing, enabling effective supervision across diverse in-context generation scenarios. On the modeling side, Canvas360 enhances text-to-panorama generation through parallel depth generation, velocity circular padding, and similarity loss regularization, enabling the model to learn geometry-aware representations, capture object distortion details, and improve geometric consistency and global coherence. Furthermore, empowered by strong panoramic priors, Canvas360 enables a unified in-context panoramic generation framework that supports diverse downstream tasks via token-level concatenation, surpassing prior methods in both task coverage and modeling flexibility. Extensive experiments show that Canvas360 improves panoramic image fidelity, achieving particularly strong performance on the panorama-specific FAED metric and competitive or leading results across the reported quantitative evaluations. More information can be found on our project page: https://zry000.github.io/Canvas360/

Abstract

Image Generation

Explore similar work

Enhancing In-context Panoramic Generation via Geometric-aware Pretraining

UNITY: Attention Flow Networks for Adaptive Conditioning in Diffusion

Beyond Attention Masks: Instruction Anchoring for Efficient In-Context Diffusion Generation