HintMiner: Automatic Question Hints Mining From Q&A Web Posts with Language Model via Self-Supervised Learning

Zhenyu Zhang; JiuDong Yang

cs.LGSep 13, 2026

HintMiner: Automatic Question Hints Mining From Q&A Web Posts with Language Model via Self-Supervised Learning

Authors: Zhenyu Zhang, JiuDong Yang

Organizations: Independent Researcher

Abstract

Users often need ask questions and seek answers online. The Question - Answering (QA) forums such as Stack Overflow cannot always respond to the questions timely and properly. In this paper, we propose HintMiner, a novel automatic question hints mining tool for users to help them find answers. HintMiner leverages the machine comprehension and sequence generation techniques to automatically generate hints for users' questions. It firstly retrieve many web Q&A posts and then extract some hints from the posts using MiningNet that is built via a language model. Using the huge amount of online Q&A posts, we design a self-supervised objective to train the MiningNet that is a neural encoder-decoder model based on the transformer and copying mechanisms. We have evaluated HintMiner on 60,000 Stack Overflow questions. The experiment results show that the proposed approach is effective. For example, HintMiner achieves an average BLEU score of 36.17% and an average ROUGE-2 score of 36.29%. Our tool and experimental data are publicly available.

Figures & tables

Figure 1: An example of the posts in Stack Overflow

	50		100		150		200
	BLEU	ROU-2	BLEU	ROU-2	BLEU	ROU-2	BLEU	ROU-2
AnswerBot	14.22	16.81	15.63	19.63	17.58	21.27	17.33	21.28
SimCSE	16.53	18.05	16.72	19.62	20.98	22.13	20.11	22.66
PageRank++	19.25	20.02	19.28	22.55	21.84	22.76	19.67	22.84
GPT2	25.11	26.13	26.49	26.83	26.53	28.51	28.04	29.16
BART	29.35	29.57	30.75	31.29	31.52	32.45	33.11	33.17

Table 1: Evaluation of HintMiner and Compared Methods. “ROU-2” denotes ROUGE-2.

Retrieval Methods	BLEU	ROUGE-2
Linked Posts (User Marked)	36.63	36.85
Stack Exchange open API	36.22	36.25
Google Custom Search	36.19	36.37
Elastic Search (HintMiner)	36.17	36.29

Table 2: Comparison of HintMiner with Different Relevant Post Retrieval Methods

Question	Hints
how do I modify an existing a sheet in an excel workbook using openxlsx package in r? (https://stackoverflow.com/ questions/34172353)	you need to load the complete workbook, then modify its data and then save it to disk. with [CODE] you can also specify the starting row and column. and you could also modify other sections.
how do you save android emulator snapshot? (https://stackoverflow.com/ questions/4842612/)	start a telnet session to the android emulator then freezes for a few seconds while saving/loading a snaps. i found at google, simply closing the emhot window is the correct way to you.
is there a method to calculate something like general similarity score of a string? (https://stackoverflow.com/ questions/4323977/)	there are many such algorithms. keywords are fuzzy string matching. by it you can calculate the number of changes required to transform one string into another, so that gives you an estimate of how similar the strings are.

Table 3: Examples of Hints Generated by HintMiner

Question Answering

Question answering studies how AI systems interpret questions and produce accurate, evidence-grounded answers across text, dialogue, and multimodal data.

See all 260 papers

Hints Help But Do They Teach? Evaluating Skills Transfer in Code Generation
Sep 1, 2026Will BadrLanguage Model Generation Evaluation LLM Prompting
SearchSkill: Teaching LLMs to Use Search Tools with Evolving Skill Banks
May 9, 2026Jinchao Hu, Meizhi Zhong, Kehai Chen +1Tool-Augmented Language Model Agents Agentic Search
School of Computer Science and Technology, Harbin Institute of Technology, Shenzhen · Independent Researcher Shenzhen / Beijing, China
A Hybrid LTR-based System via Social Context Embedding for Recommending Solutions of Software Bugs in Developer Communities
Mar 7, 2026Fouzi Harrag, Mokdad KhemlicheRecommender Systems Software Engineering
Computer Sciences Department, College of Sciences, Ferhat Abbas University, Setif, Algeria

Sep 1, 2026J/K move · Enter open · S save

Hints Help But Do They Teach? Evaluating Skills Transfer in Code Generation

Will Badr

Language Model Generation Evaluation LLM Prompting Code Generation Evaluation Causal Interventions in Language Models Code Language Models

Open paper AskPDF arXiv

When a hint turns a failing generated program into a passing one, does it provide missing information or merely steer the model toward a solution it could already produce? We test these hypotheses on HumanEval+ and MBPP+ using executable evaluation. For Qwen2.5-3B-Instruct, adaptive relevant hints rescue 36 of 79 selected failures; an unrelated hint rescues 19, while eight unhinted samples solve 46 and recover 31 of the 36 relevant-hint rescues. Phi-3.5-mini shows the same pattern: relevant hints rescue 42 of 101 failures, an unrelated hint rescues 17, and unhinted sampling solves 57, including 36 of the 42 relevant-hint rescues. Because the hint conditions use different attempt budgets, these comparisons do not isolate a purely semantic effect. Mechanistic tests on Qwen identify a stable activation direction shared by relevant and unrelated hints. Persistently adding this direction yields 14 rescues and 18 regressions, with no detectable net accuracy gain; learned low-rank interventions have a positive but imprecise estimated effect. Full textual specifications solve 22 of 24 context-defined problems, versus 5-11 for tested virtual-KV prefixes. Post-generation hidden-state probes transfer across benchmarks, with pooled AUROC 0.806 and 0.780, but their top-one selection advantage over token confidence is statistically unresolved. Overall, relevant hints can rescue failures, but most rescued solutions are already reachable through ordinary sampling, and the internal interventions tested here do not establish task-general capability transfer.

Abstract

Question Answering

Explore similar work

Hints Help But Do They Teach? Evaluating Skills Transfer in Code Generation

SearchSkill: Teaching LLMs to Use Search Tools with Evolving Skill Banks

A Hybrid LTR-based System via Social Context Embedding for Recommending Solutions of Software Bugs in Developer Communities