Structured Output
Structured output in machine learning focuses on generating outputs with predefined formats and structures, improving the reliability and usability of AI systems. Current research emphasizes enhancing large language models (LLMs) to produce structured outputs like JSON, code, or tables, often employing techniques like retrieval-augmented generation (RAG) and constrained decoding to improve accuracy and efficiency. This area is crucial for deploying LLMs in real-world applications requiring precise and interpretable results, addressing challenges such as hallucination and bias while improving the overall reliability and trustworthiness of AI systems.
Papers
Distribution Learning with Valid Outputs Beyond the Worst-Case
Nick Rittler, Kamalika Chaudhuri
Modelling Structured Data Learning with Restricted Boltzmann Machines in the Teacher-Student Setting
Robin Thériault, Francesco Tosello, Daniele Tantari
Arithmetic Transformers Can Length-Generalize in Both Operand Length and Count
Hanseul Cho, Jaeyoung Cha, Srinadh Bhojanapalli, Chulhee Yun