Multimodal Optimization

The field covers both optimization of models that combine modalities such as text, images, audio, and video, and search over objectives with many distinct optima. For multimodal learning, methods tune data mixtures and model components, balance or prioritize modality-specific gradients, and use gating, distillation, or adaptive optimization; evaluation commonly spans multimodal understanding, generation, and domain-generalization benchmarks. For multimodal objective landscapes, population-based methods such as CMA-ES, Stein-based search, and learned proposal models aim to find high-quality solutions while preserving coverage of different peaks. A central tension is how to balance convergence speed and performance against robustness and diversity: equal treatment of modalities or solutions is not always best, but aggressive prioritization can neglect useful signals or optima.