Language Model Generation Evaluation

The field studies how to tell whether generated text is correct, useful, faithful to its sources, well written, and appropriate for a task. Approaches range from lexical and embedding-based metrics to human ratings, rubric-based LLM judges, learned evaluators, and task-specific checks such as code execution or schema validation. Benchmarks increasingly test long-form and open-ended outputs, multiple samples, languages, and settings, while evaluation-cost methods aim to select informative examples efficiently. A central challenge is that scores can depend on prompts, judges, and task context: a fluent or highly rated answer may still omit facts, repeat familiar ideas, or fail to reflect human preferences.