Vision-language models can describe images fluently, but they often fail to provide actionable photographic critique because semantic content and aesthetic judgment remain entangled. We propose SE-MoLoRA, a modular parameter-efficient adaptation framework for domain-specific photographic assessment. The method separates general photographic knowledge from specialist residual judgments using an always-active shared LoRA expert and routed adapters for composition, lighting, and technical quality. A lightweight query router selects the relevant specialist, enabling targeted critique without training separate full models. A rank-64 shared adapter captures broad photographic vocabulary, while rank-32 specialists learn domain-specific residuals with an orthogonal regularization penalty that encourages disentangled representations. Training data is obtained by distilling the Reddit Photo Critique Dataset into domain-labeled critique samples. On held-out critique generation, SE-MoLoRA improves BERTScore-F1 from 0.2317 to 0.4215 over monolithic LoRA and is preferred in 84.6% of pairwise comparisons, while using fewer active parameters than separate specialist models. SVD-based ablation study shows that shared-specialist decomposition and orthogonal regularization reduce expert overlap. These results demonstrate that modular adaptation improves controllability and specificity in multimodal photographic critique.
Figures & tables
Figure 1: The SE-MoLoRA architecture. A shared adapter handles general photographic knowledge, while specialized adapters for composition, lighting, and technical aspects are conditionally activated based on the router’s decision.
Expert
Architecture
BERTScore F1 ( ↑ )
SBERT (Cos Sim) ( ↑ )
ROUGE-L F1 ( ↑ )
Shared
Base Model
0.0379 ± 0.0751
0.6105 ± 0.1140
0.0921 ± 0.0309
Monolithic
0.2317 ± 0.0831
0.6337 ± 0.1013
0.1787 ± 0.0629
SE-MoLoRA
0.4215 ± 0.1000
0.6851 ± 0.1061
0.3339 ± 0.1018
Composition
Base Model
-0.0211 ± 0.0478
0.5277 ± 0.1232
0.0810 ± 0.0270
Monolithic
0.2348 ± 0.0773
0.5362 ± 0.1265
0.1845 ± 0.0530
SE-MoLoRA
0.2999 ± 0.0745
0.5851 ± 0.1336
0.2145 ± 0.0573
Table 1: Automated metric evaluation comparing the zero-shot Base model, a Monolithic LoRA baseline, and our routed SE-MoLoRA architecture. Best results are highlighted in bold. A paired Student’s t-test confirmed statistical significance ( p<0.001 ).
Figure 2: LLM-as-a-Judge Evaluation Win Rates of the SE-MoLoRA architecture against the zero-shot base model (Top) and the monolithic adapter (Bottom).
Figure 3: Qualitative comparison between responses from each expert for the same image (Sample Taken from RPCD Nieto et al. (2022) ). Domain-specific vocabulary for each expert is highlighted in purple.
Figure 4: SVD-based expert disentanglement across different λortho constraints.
Figure 5: Confusion matrix for the text-only and multimodal router.
Benchmark
Text Router
Multimodal Router
Q-Bench
55.51%
57.65%
PhotoBench
50.66%
53.97%
Table 3: Router comparison on zero-shot aesthetic benchmarks.
Figure 6: Ablation results showing the impact of the shared expert on win rates with LLM-as-judge.
Model
Accuracy (%)
Random guess ∗
37.80
GPT-4o (2024-08-06) ∗
73.04
AesExpert ∗ Huang et al. (2024b)
64.15
Q-Instruct ∗ Wu et al. (2024)
67.09
PhotoEye ∗ Qi et al. (2025)
74.50
Qwen3-VL-8B
50.79
Table 4: Comparison of SE-MoLoRA with State-of-the-Art image aesthetic methods on Q-Bench (left) and PhotoBench (right). Results indicated by ∗ were reported in Qi et al. (2025) .
Figure 7: Qualitative comparison of SE-MoLoRA and base model on adversarial images (Sample Taken from RPCD Nieto et al. (2022) ).
Vision-language model adaptation tailors pretrained image-text and multimodal models to new tasks and domains while retaining transferable capabilities.
Low-Rank Adaptation (LoRA) has become a standard approach for parameter-efficient fine-tuning of image diffusion models, with over 200K models trained to replicate artistic styles and shared in online communities. One way to analyze styles of LoRA models is to generate image samples and apply image-based features like CLIP. However, this generation requires additional computation and prompt selection. In this work, we propose analyzing styles directly in the LoRA weight space, bypassing these limitations while achieving better performance even on unseen LoRAs. Since in-the-wild LoRAs lack ground truth model details, we design an experimental setup and show that LoRA weights alone serve as effective feature representations of adapted styles, robust to variations in fine-tuning seeds and image selection. We extensively evaluate using SD1.5 (the most popular base model within online communities) across data domains (fine-art paintings, architectural photos) and diverse prompt settings, with a small-scale SDXL experiment further verifying generalization. These evaluations confirm that our approach, under the selected aspects of variability, preserves fine-grained styles and outperforms image-based methods in clustering and retrieval of unseen LoRAs.
Chenxi Liu, Towaki Takikawa, Alec Jacobson
University of Toronto · Outerport · Adobe Research
Low-Rank Adaptation (LoRA) has become a widely adopted technique in text-to-image diffusion models, enabling the personalisation of visual concepts such as characters, styles, and objects. However, existing approaches struggle to effectively compose multiple LoRA adapters, particularly in open-ended settings where the number and nature of required skills are not known in advance. In this work, we present LoRAtorio, a novel train-free framework for multi-LoRA composition that leverages intrinsic model behaviour. Our method is motivated by two key observations: (1) LoRA adapters trained on narrow domains produce denoised outputs that diverge from the base model, and (2) when operating out-of-distribution, LoRA outputs show behaviour closer to the base model than when conditioned in distribution. The balance between these two observations allows for exceptional performance in the single LoRA scenario, which nevertheless deteriorates when multiple LoRAs are loaded. Our method operates in the latent space by dividing it into spatial patches and computing cosine similarity between each patch's predicted noise and that of the base model. These similarities are used to construct a spatially-aware weight matrix, which guides a weighted aggregation of LoRA outputs. To address domain drift, we further propose a modification to classifier-free guidance that incorporates the base model's unconditional score into the composition. We extend this formulation to a dynamic module selection setting, enabling inference-time selection of relevant LoRA adapters from a large pool. LoRAtorio achieves state-of-the-art performance, showing up to a 1.3% improvement in ClipScore and a 72.43% win rate in GPT-4V pairwise evaluations, and generalises effectively to multiple latent diffusion models. Project page: https://nickyfot.github.io/loratorio-skill-composer/
Niki Foteinopoulou, Ignas Budvytis, Stephan Liwicki
Cambridge Research Laboratory, Toshiba Europe · Independent Researcher
Mixture-of-Experts (MoE) architectures scale model capacity through sparse computation, routing each token through only a small subset of experts. In this work, we explore whether this sparsity gives rise to emergent intrinsic organization in multimodal MoEs. We find that experts develop strong semantic specialization across modalities and domains despite not being explicitly trained for modularity. Building on this structure, we introduce ExpertLens, a data-free method that identifies domain-specialized experts directly from pretrained model weights by decoding router weights into semantically meaningful vocabulary tokens. We leverage this specialization for efficient multimodal adaptation by selectively fine-tuning experts relevant to a target domain. Across math, medical, and remote sensing tasks, ExpertLens matches or surpasses full fine-tuning while updating only 21.7 - 47.0% of model parameters and achieving a 4.0x average training speedup, and outperforms LoRA in both adaptation performance and training efficiency. These results show that sparsity introduced for efficiency can give rise to semantic modularity that is directly useful for efficient adaptation.