LLM Prompting

LLM: Large Language Model

The field studies how the wording and structure of inputs affect what language models understand, generate, and reliably follow. Common approaches include zero- and few-shot in-context learning, chain-of-thought and task decomposition, retrieval-augmented context, and structured prompts using schemas, roles, or constraints. Researchers also develop automated prompt search, feedback-driven refinement, and methods that adapt examples or context to each input, testing them on tasks such as reasoning, classification, code generation, and information extraction. Prompting can improve performance without changing model weights, but its effects vary across models and tasks: added instructions can hurt, and gains in accuracy may trade off with reliability, diversity, safety, or cost.

Latest papers

431 papers34 papers in the last four weeks, down 13% on the four weeks before. 0.2% of all new papers.

Business as Rulesual: A Benchmark and Framework for Business Rule Flow Modeling with LLMs
May 24, 2025Chen Yang, Ruping Xu, Ruizhe Li +2LLM Grounding LLM Prompting
Zhejiang University of Technology, China1 · Zhejiang Key Laboratory of Visual Information Intelligent Processing, China 2 · University of Aberdeen, UK +1
Prompt Stability Matters: Evaluating and Optimizing Auto-Generated Prompt in General-Purpose Systems
May 19, 2025Ke Chen, Xucheng Yu, Yufei Zhou +1LLM Prompting Automatic Prompt Optimization
School of Information Sciences, University of Illinois Urbana-Champaign, Urbana, USA · Department of Economics, Duke University, Durham, USA
HoarePrompt: Structural Reasoning About Program Correctness in Natural Language
Mar 25, 2025Dimitrios Stamatios Bouras, Yihan Dai, Tairan Wang +2LLM Prompting Formal Verification
Key Laboratory of HCST (PKU), MoE, SCS, Peking University Beijing, China · University College London London, United Kingdom
Empirical Computation: Prompting versus Programming
Mar 13, 2025Eric Tang, Jing Liu, Marcel BöhmeLLM Prompting
CMU, USA · MPI-SP, Germany
Foundations of Large Language Models
Jan 16, 2025Tong Xiao, Jingbo ZhuLanguage Model Pretraining LLM Alignment
NLP Lab, Northeastern University & NiuTrans Research
Growing a Tail: Increasing Output Diversity in Large Language Models
Nov 5, 2024Michal Shur-Ofry, Bar Horowitz-Amsalem, Adir Rahamim +1LLM Evaluation LLM Prompting
Law Faculty, Hebrew University of Jerusalem; Jerusalem, Israel. · Faculty of Computer Science, Technion – Israel Institute of Technology; Haifa, Israel.
SG-FSM: A Self-Guiding Zero-Shot Prompting Paradigm for Multi-Hop Question Answering Based on Finite State Machine
Oct 22, 2024Xiaochen Wang, Liang Chen, Reza Haf Zhe Yang +5Multi-Hop QA LLM Self-Correction
School of Software & Microelectronics, Peking University · International Digital Economy Academy · National Key Laboratory for Multimedia Information Processing, School of Computer Science, Peking University +1
Which Demographics do LLMs Default to During Annotation?
Oct 11, 2024Johannes Schäfer, Aidan Combs, Christopher Bagdon +9LLM Prompting LLM-Assisted Annotation
Fundamentals of Natural Language Processing, University of Bamberg, Germany
Prompting the Unknown: Understanding Response Uncertainty in Large Language Models
Jul 20, 2024Ze Yu Zhang, Arun Verma, Finale Doshi-Velez +1LLM Prompting LLM Reliability
Department of Computer Science, National University of Singapore, Republic of Singapore · Singapore-MIT Alliance for Research and Technology Centre · Harvard University
Budgeted Prompt-Set Expansion and Refinement for Language Models via Simulation Optimization
Apr 12, 2024Haoting Zhang, Jinghai He, Rhonda Righter +1Prompt Optimization LLM Prompting
Department of Industrial Engineering and Operations Research, University of California Berkeley
An Incomplete Loop: Deductive, Inductive, and Abductive Reasoning in Language Models
Apr 3, 2024Emmy Liu, Graham Neubig, Jacob AndreasLLM Prompting Abductive Reasoning in Language Models
Language Technologies Institute Carnegie Mellon University · CSAIL Massachusetts Institute of Technology

May 24, 2025J/K move · Enter open · S save

Business as Rulesual: A Benchmark and Framework for Business Rule Flow Modeling with LLMs

Chen Yang, Ruping Xu, Ruizhe Li, Bin Cao+1

Zhejiang University of Technology, China1 · Zhejiang Key Laboratory of Visual Information Intelligent Processing, China 2 · University of Aberdeen, UK+1

LLM Grounding LLM Prompting Document Information Extraction Business Process Modeling

Open paper AskPDF arXiv

Extracting structured procedural knowledge from unstructured business documents is a critical yet unresolved bottleneck in process automation. While prior work has focused on extracting linear action flows from instructional texts, such as recipes, it has insufficiently addressed the complex logical structures, including conditional branching and parallel execution, that are pervasive in real-world regulatory and administrative documents. Furthermore, existing benchmarks are limited by simplistic schemas and shallow logical dependencies, restricting progress toward logic-aware large language models.To bridge this Logic Gap, we introduce BREX, a carefully curated benchmark comprising 409 real-world business documents and 2,855 expert-annotated rules. Unlike prior datasets centered on narrow service scenarios, BREX spans over 30 vertical domains, covering scientific, industrial, administrative, and financial regulations. We further propose ExIde, a structure-aware reasoning framework that investigates five distinct prompting strategies, ranging from implicit semantic alignment to executable grounding via pseudo-code generation. This enables explicit modeling of rule dependencies and provides an out-of-the-box framework for different business customers without finetuning their own large language models. We benchmark ExIde using 13 state-of-the-art large language models. Our extensive evaluation reveals that executable grounding serves as a superior inductive bias, significantly outperforming standard prompts in rule extraction. In addition, reasoning-optimized models demonstrate a distinct advantage in tracing long-range and non-linear rule dependencies compared to standard instruction-tuned models.