Paper ID: 2410.24201 • Published Oct 31, 2024
P-Masking: Power Law Masking Improves Multi-attribute Controlled Generation
TL;DR
Get AI-generated summaries with premium
Get AI-generated summaries with premium
We introduce LingGen, a novel approach for controlled text generation that
offers precise control over a wide array of linguistic attributes, even as the
number of attributes varies. LingGen employs a dynamic P-MASKING strategy,
which samples masking rates from a power law distribution during training. This
innovative approach enables the model to develop robust representations and
adapt its attribute control capabilities across a variable number of
attributes, from a single attribute to multiple complex configurations. The
P-MASKING technique enhances LingGen's ability to manage different levels of
attribute visibility, resulting in superior performance in multi-attribute
generation tasks. Our experiments demonstrate that LingGen surpasses current
state-of-the-art models in both attribute control accuracy and text fluency,
particularly excelling in scenarios with varying attribute demands.
Additionally, our ablation studies highlight the effectiveness of P-MASKING and
the influence of different base language models on performance. These findings
demonstrate LingGen's potential for applications requiring precise and
adaptable control over multiple linguistic attributes in text generation.