Paper ID: 2405.03677

Towards A Human-in-the-Loop LLM Approach to Collaborative Discourse Analysis

Clayton Cohn, Caitlin Snyder, Justin Montenegro, Gautam Biswas

LLMs have demonstrated proficiency in contextualizing their outputs using human input, often matching or beating human-level performance on a variety of tasks. However, LLMs have not yet been used to characterize synergistic learning in students' collaborative discourse. In this exploratory work, we take a first step towards adopting a human-in-the-loop prompt engineering approach with GPT-4-Turbo to summarize and categorize students' synergistic learning during collaborative discourse. Our preliminary findings suggest GPT-4-Turbo may be able to characterize students' synergistic learning in a manner comparable to humans and that our approach warrants further investigation.

Submitted: May 6, 2024