TaxPraBen: A Scalable Benchmark for Structured Evaluation of LLMs in Chinese Real-World Tax Practice

Gang Hu; Yating Chen; Haiyan Ding; Wang Gao; Jiajia Huang; Min Peng; Qianqian Xie; Kun Yue

cs.CLApr 10, 2026

TaxPraBen: A Scalable Benchmark for Structured Evaluation of LLMs in Chinese Real-World Tax Practice

Authors: Gang Hu, Yating Chen, Haiyan Ding, Wang Gao, Jiajia Huang, Min Peng, Qianqian Xie, Kun Yue

Organizations: Yunnan University, Yunnan, China. · Jianghan University, Wuhan, China. · Nanjing Audit University, Nanjing, China. · Wuhan University, Wuhan, China.

Abstract

While Large Language Models (LLMs) excel in various general domains, they exhibit notable gaps in the highly specialized, knowledge-intensive, and legally regulated Chinese tax domain. Consequently, while tax-related benchmarks are gaining attention, many focus on isolated NLP tasks, neglecting real-world practical capabilities. To address this issue, we introduce TaxPraBen, the first dedicated benchmark for Chinese taxation practice. It combines 10 traditional application tasks, along with 3 pioneering real-world scenarios: tax risk prevention, tax inspection analysis, and tax strategy planning, sourced from 14 datasets totaling 7.3K instances. TaxPraBen features a scalable structured evaluation paradigm designed through process of "structured parsing-field alignment extraction-numerical and textual matching", enabling end-to-end tax practice assessment while being extensible to other domains. We evaluate 19 LLMs based on Bloom's taxonomy. The results indicate significant performance disparities: all closed-source large-parameter LLMs excel, and Chinese LLMs like Qwen2.5 generally exceed multilingual LLMs, while the YaYi2 LLM, fine-tuned with some tax data, shows only limited improvement. TaxPraBen serves as a vital resource for advancing evaluations of LLMs in practical applications.

Figures & tables

Figure 1: TaxPraBen’s data construction workflow uses 3 methods: (A) Book Data Collection, (B) Official Document Download and (C) Online Data Processing. All data is annotated via the manual "Availability Check".

Tax Ability-Hierarchy Task	Dataset Name	Specific Task	Data Scale	Evaluation Metric	NLP- Task Type	Method
Tax Ability-Hierarchy Task	Dataset Name	Specific Task	Data Scale	Evaluation Metric	NLP- Task Type	HG	MC
Tax Knowledge Memorization (KM)	TaxRecite	Tax Law Recitation	200	BERTScore, BARTScore	Generation (GEN)	$\checkmark$
Tax Knowledge Understanding (KU)	TaxSum	Tax News Summarization	1000	BERTScore, BARTScore	Generation (GEN)	$\checkmark$
	TaxTopic	Tax Topic Classification	1000	Accuracy, F1, Macro F1	Classification (CLS)		$\checkmark$
	TaxRead	Tax Reading Comprehension	1000	EM Accuracy	Generation (GEN)	$\checkmark$
Tax Knowledge Application (KA)	TaxCalc	Tax Payment Calculation	500	EM Accuracy	Reasoning (REA)		$\checkmark$

Table 1: Overview of tax tasks, datasets, specific tasks, data scale, evaluation metrics, NLP types, and creation methods in TaxPraBen. "MC" and "HC" refer to "Manually Created" and "Human-ChatGPT Collaborative".

Dataset	Real-World Scenarios	Application Value
TaxCalc	Pre-filing tax calculation for individuals and businesses.	Supports the underlying logic for intelligent tax filing tools.
TaxSCQ	Exams for tax system staff or entry qualifications.	Enables training/testing for tax Q&A systems or AI tutors.
TaxMCQ	Complex policy applicability and compliance judgment.	Trains models for regulatory perception and decision-making.
TaxQA	Inquiries to tax bureaus (e.g., 12366), tax advisory.	Builds intelligent tax consultation systems.
TaxBoard	Message boards and online government services.	Enhances automatic response and assistant capabilities in e-government.
TaxCrime	Legal article application in law enforcement.	Supports case legality judgment and transparent law enforcement.

Table 2: TaxPraBen subsets for Knowledge Application (KA) task: real-world scenarios and application value.

Panel of Figure 2 — Figure 2: A unified output format protocol for 3 typical cases of the tax practice scenarios.

Task	Prompt Description
TaxRisk	你会收到一段模型的输出内容，为了精准地匹配，你需要做的是判断模型有没有提取出内容。如果有，则从中提取模型输出的这两部分内容，如果模型本身就没有总结出来，你就留空，不需要替它总结，严格按照以下格式输出，留空的也需要保持格式：{"面临的风险": , "风险对应的解决方案": }。模型输出如下：{raw_output}。请只返回JSON对象，不要任何额外解释。
TaxInspect	你会收到一段模型的输出内容，为了精准地匹配，你需要做的是判断模型有没有提取出内容。如果有，则从中提取模型输出的这三部分内容，如果模型本身就没有总结出来，你就留空，不需要替它总结，严格按照以下格式输出，留空的也需要保持格式：{"犯罪行为": , "所犯罪名": , "处罚结果": }。模型输出如下：{raw_output}。请只返回JSON对象，不要任何额外解释。
TaxPlan	请你从模型输出中提取填空答案，若某一字段没有找到答案，则该字段留空，不要凭空生成答案，并严格按照以下格式输出：{"核心思路": , "会计利润": , "第一方案所得税": , "第二方案所得税": , "分年捐赠筹划后所得税": , "节税额": }，除核心思路外，其余字段都只返回数值，不要有任何符号和单位。模型输出如下：{raw_output}。请只返回JSON对象，不要任何额外解释。

Table 3: Structured output guidance prompts.

Figure 3: Structured evaluation pipeline for TaxPlan.

Model	P	Pre	Fin	Access	Base LLM	Release
Multilingual General LLMs
ChatGPT	175B	$\checkmark$	$\times$	API	–	11/2022
GPT-4o	200B	$\checkmark$	$\times$	API	–	05/2024
MistralV0.3	7B	$\checkmark$	$\times$	Weights	–	07/2024
Gemma	7B	$\checkmark$	$\times$	Weights	–	07/2024
LLaMA3	8B	$\checkmark$	$\times$	Weights	–	09/2024

Table 4: Various baseline LLMs, including parameters (P), pre-trained (Pre) or fine-tuned (Fin), access methods (public weights/API calls), base LLM, and release date.

Panel of Figure 4 — Figure 4: The zero-shot and one-shot overall performance of the 19 popular LLMs evaluated on TaxPraBen.

{ Id : [data_id] query : [prompt] text : [text] answer : [response]

(a) choices : […] gold : […] / (b) — / (c) — / (d) — }

Table 5: The JSON template for each task evaluation

Type	Settings	Name	ChatGPT	GPT-4o	MistralV0.3	Gemma	LLaMA3	Bayling2	Grok3	$\text{DeepSeek}_{\text{llm}}$	Baichuan2	Atom	Qwen2.5	ChnLLaMA3	ERNIE-3.5	ChatGLM3	Yi	GLM4	$\text{DeepSeek}_{\text{R1}}$	InternLM2.5	YaYi2
Tax Task	Zero- Shot	KM	0.488	0.478	0.400	0.244	0.412	0.466	0.519	0.494	0.470	0.474	0.499	0.445	0.667	0.487	0.482	0.480	0.461	0.505	0.485
		KU	0.602	0.637	0.277	0.085	0.236	0.409	0.579	0.341	0.243	0.370	0.538	0.312	0.599	0.427	0.449	0.515	0.455	0.273	0.307
		KA	0.415	0.472	0.114	0.127	0.183	0.202	0.482	0.258	0.188	0.225	0.375	0.247	0.475	0.206	0.295	0.404	0.324	0.349	0.239
	One- Shot	KM	0.497	0.499	0.413	0.239	0.407	0.475	0.527	0.474	0.480	0.467	0.494	0.426	0.619	0.372	0.476	0.225	0.461	0.165	0.350
		KU	0.611	0.653	0.429	0.393	0.182	0.350	0.591	0.488	0.491	0.439	0.453	0.460	0.633	0.341	0.485	0.257	0.520	0.348	0.257
		KA	0.425	0.464	0.217	0.188	0.197	0.271	0.468	0.297	0.286	0.269	0.386	0.281	0.477	0.186	0.293	0.265	0.340	0.172	0.203

Table 6: The performance of the 19 LLMs on Tax task (KM, KU, KA) and NLP main task (CLS, GEN, REA).

Appendix figures & tables27 assets

Supplementary material from the paper’s appendix.

Appendix

Dataset Source	URL
Tax House website	https://www.shui5.cn/
Tax Practice Cases	https://wxredian.com/art?id=ecf5c4f6a9986a6eda47f48ba7ed5e74
Certified Tax Agent Exam	http://www.canet.com.cn/sws/zhinan/kaoshijiaocai.html
Opinion Monitoring System	https://open-yuqing.stonedt.com/
Taxation Administration Website	https://www.chinatax.gov.cn/

Appendix

Table C2: The original source along with the URL of our used datasets.

Dataset	Zero-Shot (Prompt)			One-Shot (Prompt)			Input Text			Label
Dataset	Max	Min	Avg	Max	Min	Avg	Max	Min	Avg	Max	Min	Avg
TaxRecite	42	37	39.06	592	109	206.80	69	33	40.11	514	17	115.05
TaxSum	38	35	36.37	1469	273	573.60	1335	130	405.50	236	14	115.12
TaxTopic	130	125	128.02	207	120	150.50	100	6	37.26	1	1	1.00
TaxRead	56	49	53.82	1812	221	566.30	2066	125	521.85	286	1	15.44
TaxCalc	86	65	69.80	1383	327	636.40	1266	216	535.48	9	4	5.84

Appendix

Table C3: TaxPraBen includes 3 NLP and tax tasks across 14 datasets ranging from 200 to 1,000 entries, along with statistics on the maximum (Max), minimum (Min), and average (Avg) lengths of prompts and input/labels.

Metrics	Description
Accuracy (ACC)	Measure if the response meets the target answer provided by the query, avoiding errors and bias.
Naturalness (NAT)	Assesses whether the response description is fluent, natural, and consistent with human expression.
Informativeness (INF)	Evaluate if the response description includes enough key information and fully answers the question.

Appendix

Table C18: Briefly description of the three manual metrics (ACC,NAT,INF) for prompt quality evaluation.

\mathrm{MCC}=\frac{\sum_{i=1}^{K}\sum_{j=1}^{K}C_{ij}\left(\delta_{ij}-\frac{C_{i+}C_{+j}}{N^{2}}\right)}{\sqrt{\left(\sum_{i=1}^{K}C_{i+}\right)\left(\sum_{j=1}^{K}C_{+j}\right)\left(\sum_{i=1}^{K}\sum_{j=1}^{K}\frac{C_{i+}C_{+j}}{N^{2}}\right)}}

Appendix

Table 14

Task	Dataset	Definition	Collection Details
KM	TaxRecite	Tax Law Recitation: Provide the section number of the tax law, and require the model to accurately recite the original description.	Download legal and regulatory documents from the official website of the State Taxation Administration（国家税务总局）, extract a portion of the original text of the provisions, and process them into a question-and-answer format.
KU	TaxSum	Tax News Summarization: Provide the news report and require the model to extract key information and generate a summary.	Select tax-related news articles from the Tax House website, retain the manually annotated summaries from the site, and compress the original text while preserving its main points to optimize the input length.
	TaxTopic	Tax Topic Classification: Provide the news title and require the model to determine its corresponding thematic category.	Select tax-related articles from the Tax House website, extract their titles, and record the categories they belong to on the website as labels for thematic identification.
	TaxRead	Tax Reading Comprehension: Provide the article along with relevant questions, and require the model to extract relevant content for answer.	Select tax-related articles from the Tax House (税屋) website, compress the original text using ChatGPT while retaining its main points to optimize input length, and utilize ChatGPT to extract questions and corresponding answers from the compressed articles.
KA	TaxCalc	Tax Payment Calculation: Provide a corporate tax case and require the model to calculate and provide the accurate amount.	Carefully select calculation problems from the exercise book for the Certified Tax Agent exam, ensuring the questions cover multiple tax types to comprehensively address and encompass the tax domain.
	TaxSCQ	Tax Single-Choice Exam: Provide the question along with four options, requiring the model to identify one correct answer.	Carefully select single-choice questions from the exercise book for the Certified Tax Agent Exam (注册税务师考试）. To ensure broad coverage, we extract relevant exam questions from both real exam papers and mock exam papers, encompassing various knowledge areas and key exam topics.

Appendix

Table C1: Description of the data definitions and collection details for the three-level tax tasks in TaxPraBen.

Panel of Figure 16 — Figure C1: An example screenshot demonstrating manual adjustment of the TaxRead and TaxQA datasets.

Prompt: 根据问题需求，请提供法条的原文部分，不必对其含义做出解释，也不需要指明是第几条。( According to the question requirements, please provide the verbatim excerpt of the relevant legal provisions. Do not explain their meaning, and do not indicate article numbers. )

Zero-Shot

Query: {Prompt}

Text: 2007年12月6日通过的《中华人民共和国企业所得税法实施条例》中第一百一十一条的内容是什么？ ( What is the content of Article 111 of the Implementation Regulations of the Enterprise Income Tax Law of the People’s Republic of China, adopted on December 6, 2007? )

Answer: 企业所得税法第四十一条所称合理方法，包括：(一)可比非受控价格法，是指按照没有关联关系的交易各方进行相同或者类似业务往来的价格进行定价的方法；(二)再销售价格法，是指按照从关联方购进商品再销售给没有关联关系的交易方的价格，减除相同或者类似业务的销售毛利进行定价的方法；(三)成本加成法，是指按照成本加合理的费用和利润进行定价的方法；(四)交易净利润法，是指按照没有关联关系的交易各方进行相同或者类似业务往来取得的净利润水平确定利润的方法；(五)利润分割法，是指将企业与其关联方的合并利润或者亏损在各方之间采用合理标准进行分配的方法；(六)其他符合独立交易原则的方法。 ( The “reasonable methods” referred to in Article 41 of the Enterprise Income Tax Law include: (1) the Comparable Uncontrolled Price (CUP) method, i.e., determining the price by reference to the prices applied in identical or similar transactions between parties without an associated relationship; (2) the Resale Price method, i.e., determining the price by reference to the price at which goods purchased from an associated party are resold to an unrelated party, minus the gross margin for identical or similar transactions;(3) the Cost Plus method, i.e., determining the price based on cost plus reasonable expenses and profit; (4) the Transactional Net Margin method (TNMM), i.e., determining profit based on the net profit level realized in identical or similar transactions between unrelated parties; (5) the Profit Split method, i.e., allocating the combined profit or loss between an enterprise and its associated parties according to reasonable allocation keys; and (6) other methods consistent with the arm’s-length principle. )

One-Shot

Query: {Prompt}

Appendix

Table C4: An instance format with paired prompts for the TaxRecite dataset in the KM task.

Prompt: 请根据以下文本编写一个简短的概要，字数不超过300字，重点在于提炼核心内容。 ( Please write a concise summary of the following text in no more than 300 words, focusing on extracting the key points. )

Zero-Shot

Query: {Prompt}

Text: 为配合个人所得税专项附加扣除工作，依据相关法律法规，三部门联合发布通知，明确个人所得税住房贷款利息专项附加扣除相关信息归集事项。信息归集范围为1989年1月1日（含）后发放的商业性个人住房贷款，不包括个人商用房贷款。数据项涵盖借款人姓名、证件号码、贷款银行、合同编号、是否为首套住房贷款等11项内容。住房公积金贷款由主管部门负责归集。各商业银行于2019年3月1日前向征信系统报送信息。….。 ( To facilitate the work related to the special additional deductions for individual income tax, the three departments have jointly issued a notice clarifying the information collection matters related to the special additional deduction for housing loan interest. The scope of information collection includes commercial personal housing loans issued after January 1, 1989 (inclusive), excluding personal commercial property loans. The data items cover 11 contents, including the borrower’s name, identification number, loan bank, contract number, and whether it is a first home loan, among others. Housing provident fund loans are to be collected by the relevant management department. All commercial banks must report this information to the credit reporting system by March 1, 2019. … . )

Answer: 信息归集工作时间紧、任务重、要求高，各商业银行要高度重视，充分认识贯彻施行《个人所得税专项附加扣除暂行办法》的重要性和紧迫性，切实加强组织领导，务必在规定时间内做好各项工作。 ( Given the tight timelines, heavy workload, and stringent requirements associated with information consolidation, all commercial banks should attach great importance to this effort. They should fully recognize the significance and urgency of implementing the Interim Measures for Special Additional Deductions of Individual Income Tax , strengthen organizational leadership, and ensure that all related tasks are completed within the prescribed time frame. )

One-Shot

Query: {Prompt}

Appendix

Table C5: An instance format with paired prompts for the TaxSum dataset in the KU task.

Prompt: 阅读标题以了解文章主题，并在提供的选项中选择最适合表达这一主题的一个，选项有：A.财商资讯 B.财税表单 C.地方法规 D.法规解读 E.纳税调整 F.纳税评估 G.税收优惠 H.税收筹划 I.税务问答 J.中央法规。(R ead the title to identify the main topic of the article, and select the single option that best matches this topic from the following candidates: A. Financial literacy news; B. Tax and finance forms; C. Local regulations; D. Regulation interpretation; E. Taxable income adjustments; F. Tax assessment; G. Tax incentives; H. Tax planning; I. Tax Q&A; J. National regulations. )

Zero-Shot

Query: {Prompt}

Text: 浙江省国家税务局公告2014年第10号浙江省国家税务局关于企业资产损失税前扣除申报有关问题的公告 ( Announcement No. 10 (2014) of the Zhejiang Provincial Office of the State Administration of Taxation: On Issues Concerning the Pre-tax Deduction Declaration for Enterprise Asset Losses. )

Answer: C

Choices: [“A”, “B”, “C”, “D”, “E”, “F”, “G”, “H”, “I”, “J”]

Gold: 2

Appendix

Table C6: An instance format with paired prompts for the TaxTopic dataset in the KU task.

Prompt: 关于下面提出的问题，请仔细考虑四个可能的答案，并选出你认为正确的那个，仅需指出“A”、“B”、“C”或“D”，不需要解释。( Please carefully consider the four candidate answers to the question below and select the one you deem correct. Return only the letter “A”, “B”, “C”, or “D” without providing any explanation. )

Zero-Shot

Query: {Prompt}

Text: 关于退还关税，下列说法正确的是()。 A.纳税人自缴纳税款之日起3年内，可以申请退还关税 B.海关发现实际征收税款多于应收税款，可以不用通知纳税义务人办理退还手续 C.已征出口关税的货物，因故未装运出口申请退关的，可以申请退还关税 D.海关应当自受理退税申请之日起60日内查实并通知纳税义务人办理退还手续 ( Regarding the refund of customs duties, which of the following statements is correct()? A.Within three years from the date on which the taxpayer pays the duty, the taxpayer may apply for a refund of customs duties. B.If the Customs authorities discover that the duties actually collected exceed the duties payable, they may refund the overpaid amount without notifying the taxpayer or tax obligor to complete the refund procedures. C.For goods on which export duties have already been levied, if the goods are not shipped for certain reasons and a duty refund is requested, an application may be filed for a refund of the customs duties. D.The Customs authorities shall, within 60 days from the date of accepting the refund application, verify the relevant facts and notify the taxpayer or tax obligor to complete the refund procedures. )

Answer: C

Choices: [“A”, “B”, “C”, “D”]

Gold: 2

Appendix

Table C7: An instance format with paired prompts for the TaxSCQ dataset in the KA task.

Prompt: 在阅读完下面的文本之后，请根据其内容回答问题，确保回复是从文中摘录且言简意赅，不要重复问题，答案中不要出现句号。( After reading the following text, please answer the question based on its content. Ensure that your response is a concise excerpt from the text, without repeating the question, and do not include periods in your answer. )

Zero-Shot

Query: {Prompt}

Text: 为综合反映与科学评价资产评估机构整体实力和专业胜任能力，推动资产评估行业信用体系建设，提升行业公信力和影响力，促进高质量发展，中国资产评估协会修订并发布了《资产评估机构综合评价办法》。该办法旨在通过系统化的评估体系，全面衡量资产评估机构的业务水平、服务质量、风险管理等多方面的能力，从而引导行业健康发展。新办法已经第六届常务理事会第三次会议审议通过，于2025年2月18日正式发布。问题：《资产评估机构综合评价办法》何时正式发布？ ( To comprehensively reflect and rigorously assess the overall strength and professional competence of asset appraisal firms, promote the development of an industry-wide credit system, enhance credibility and influence, and foster high-quality growth, the China Appraisal Society (CAS) revised and released the Measures for the Comprehensive Evaluation of Asset Appraisal Firms. The revised measures aim to establish a systematic evaluation framework that holistically assesses firms’ capabilities across multiple dimensions, including operational performance, service quality, and risk management, thereby guiding the healthy development of the industry. The new measures were reviewed and approved at the Third Meeting of the Third Executive Council of the Sixth Standing Council and were formally released on February 18, 2025. Question: When was the Measures for the Comprehensive Evaluation of Asset Appraisal Firms formally released? )

Answer: 2025年2月18日 ( February 18, 2025 )

One-Shot

Query: {Prompt}

Appendix

Table C8: An instance format with paired prompts for the TaxRead dataset in the KA task.

Prompt: 针对题目，请直接给出xx处的答案，并将答案带入到问题中复述一遍，例如："业务（1）应缴纳消费税为xx万元"，保证输出字符在30个字符以内。( For the question, please directly provide the answer for xx , and restate it in the question, such as: ’For transaction (1), the consumption tax payable is xx yuan,’ ensuring the output is within 30 characters. )

Zero-Shot

Query: {Prompt}

Text: 位于某市区国家重点扶持的高新技术企业为增值税一般纳税人（制造业企业），2023年销售产品取得不含税收入6500万元，另外取得投资收益320万元，全年发生产品销售成本和相关费用共计5300万元。缴纳的税金及附加339万元，发生的营业外支出420万元，12月末企业自行计算的全年会计利润总额761万元，预缴企业所得税96万元。2024年1月经聘请的税务师事务所审核，发现以下问题：（1）8月中旬以预收款方式销售一批产品，预收含税价款234万元，并收存银行。12月下旬将该批产品发出，但未将预收款转作收入。（不考虑相关成本）… 问题为：新产品研发费应调整的应纳税所得额为xx万元？ ( A nationally supported key high-tech enterprise located in the urban district of a certain city is a general VAT taxpayer (manufacturing enterprise). In 2023, it generated RMB 65.00 million in product sales revenue (VAT-exclusive) and additionally earned investment income of RMB 3.20 million. For the year, the total cost of goods sold and related expenses amounted to RMB 53.00 million. Taxes and surcharges paid were RMB 3.39 million, and non-operating expenses were RMB 4.20 million. As of the end of December, the enterprise calculated an annual accounting profit before tax of RMB 7.61 million, and it had prepaid corporate income tax (CIT) of RMB 0.96 million. In January 2024, after an audit by a hired tax agent firm, the following issues were identified:(1) In mid-August, the enterprise sold a batch of products under an advance-receipt arrangement, receiving an advance payment of RMB 2.34 million (VAT-inclusive), which was deposited in the bank. In late December, the batch of products was delivered, but the advance receipt was not reclassified as revenue. (Relevant costs are ignored.) … Question: The adjustment to taxable income attributable to the new product R&D expenses should be xx 10,000 RMB. )

Answer: -72.75

One-Shot

Query: {Prompt}

Appendix

Table C9: An instance format with paired prompts for the TaxCalc dataset in the KA task.

Prompt: 针对以下问题，请分析五个选项，并选择最适合的两个或更多，然后按字母顺序提供“A”、“B”、“C”、“D”或“E”作为你的答案，输出格式为字母组合，字母之间不要有任何其他字符，且必须是顺序的，例如：“AB”或“ABC”。 ( For the following question, analyze the five options and select the two or more most appropriate ones. Then provide your answer as a sequence of letters chosen from “A”, “B”, “C”, “D”, and “E”, listed in alphabetical order (e.g., “AB”, or “ABC”). )

Zero-Shot

Query: {Prompt}

Text: 关于不动产租赁服务的增值税处理，正确的有()。 A.个体工商户异地出租不动产，在不动产所在地预缴的增值税款，可在当期增值税税款中抵减 B.以经营租赁方式将土地出租给他人使用，按不动产经营租赁服务缴纳增值税 C.一般纳税人出租其2016年5月1日前取得的不动产可选择简易计税方法计税 D.其他个人异地出租不动产，向不动产所在地预缴税款，向居住所在地申报纳税 E.纳税人向其他个人出租不动产，可以开具增值税专用发票 ( Regarding the VAT treatment of real estate leasing services, which of the following statements are correct? A. Individual businesses leasing real estate in a different location can offset the prepaid VAT at the property’s location against their current VAT payable. B. Leasing land to others under an operating lease is subject to VAT as real estate leasing services. C. General taxpayers renting out real estate acquired before May 1, 2016, can choose the simplified tax calculation method. D. Individuals leasing real estate in a different location should prepay VAT at the property’s location and declare taxes at their place of residence. E. Taxpayers leasing real estate to individuals can issue a VAT special invoice. )

Answer: ABC

One-Shot

Query: {Prompt}

Appendix

Table C10: An instance format with paired prompts for the TaxMCQ dataset in the KA task.

Prompt: 对于下面提出的税务问题，请准备简要的答案，注意每项答复应保持在300字之内。 ( For the tax-related question below, please provide concise answers. Each response should be limited to no more than 300 words. )

Zero-Shot

Query: {Prompt}

Text: 转让债权如何确定其应予减除的财产原值和合理费用？ ( How should the original value of the property to be deducted and the reasonable expenses be determined when transferring a debt? )

Answer: 采用“加权平均法”确定其应予减除的财产原值和合理费用。即以纳税人购进的同一种类债券买入价和买进过程中缴纳的税费总和，除以纳税人购进的该种类债券数量之和，乘以纳税人卖出的该种类债券数量，再加上卖出的该种类债券过程中缴纳的税费。 ( he "weighted average method" is used to determine the deductible original value and reasonable expenses of the asset. Specifically, the total purchase cost and related taxes and fees for the same type of bonds are divided by the total quantity purchased to obtain the average cost per bond. This average is then multiplied by the quantity sold, and any taxes and fees paid during the sale are added. )

One-Shot

Query: {Prompt}

Appendix

Table C11: An instance format with paired prompts for the TaxQA dataset in the KA task.

Prompt: 对于由群众提出的税务问题，请提供确切的解答，并指出相关的法律依据，确保每个回答都在300字以内。 ( For tax-related inquiries raised by the general public, please provide precise answers and cite the relevant legal basis. Ensure that each response is no more than 300 Chinese characters in length. )

Zero-Shot

Query: {Prompt}

Text: 标题：文化事业建设费今年还有减半优惠吗？内容：文化事业建设费今年还有减半优惠吗？ ( Title: Is there still a 50% reduction policy for the Cultural Undertaking Construction Fee this year? Content: Is there still a 50% reduction policy for the Cultural Undertaking Construction Fee this year? )

Answer: 根据《财政部关于延续实施文化事业建设费优惠政策的通知》(财税〔2025〕7号)规定，自2025年1月1日至2027年12月31日，对归属中央收入的文化事业建设费，将按应缴费额的50%减征；对归属地方收入的文化事业建设费，各省可以结合当地经济发展和文化事业需求，在50%的幅度内减征。若已征收的费用符合减免条件，可予以退还或抵减以后应缴费用。 ( According to the Notice of the Ministry of Finance on Extending Preferential Policies for the Cultural Undertakings Construction Fee (Caishui [2025] No. 7), from January 1, 2025 to December 31, 2027, the cultural undertakings construction fee attributable to central government revenue shall be reduced by 50% of the payable amount. For the portion attributable to local government revenue, each province may, in light of local economic development and cultural-sector needs, grant a reduction of up to 50%. If fees already collected meet the eligibility criteria for such reductions or exemptions, they may be refunded or credited against future payable amounts. )

One-Shot

Query: {Prompt}

Appendix

Table C12: An instance format with paired prompts for the TaxBoard dataset in the KA task.

Prompt: 阅读下面的犯罪事实，请判断其主要违反了以下哪一法律条款：A. 刑法第二百零一条 B. 刑法第二百零三条 C. 刑法第二百零四条 D. 刑法第二百零五条 E. 刑法第二百零六条 F. 刑法第二百零七条 G. 刑法第二百零八条 H. 刑法第二百零九条 I. 刑法第二百一十条 J. 刑法第二百一十一条 K. 刑法第二百一十二条 L. 刑法第二百二十七条 M. 刑法第二百六十四条。只需输出对应字母，不需要解释原因。 ( Read the following statement of criminal facts and determine which of the following legal provisions is primarily violated: A. Article 201 of the Criminal Law; B. Article 203 of the Criminal Law; C. Article 204 of the Criminal Law; D. Article 205 of the Criminal Law; E. Article 206 of the Criminal Law; F. Article 207 of the Criminal Law; G. Article 208 of the Criminal Law; H. Article 209 of the Criminal Law; I. Article 210 of the Criminal Law; J. Article 211 of the Criminal Law; K. Article 212 of the Criminal Law; L. Article 227 of the Criminal Law; M. Article 264 of the Criminal Law. Output only the corresponding letter; no explanation is required. )

Zero-Shot

Query: {Prompt}

Text: 被告人叶俊在2017年4月11日10时许，在北京市丰台区七里庄“速八酒店”门前向樊某出售北京市增值税普通发票，现场被北京市公安局丰台分局东铁匠营派出所民警当场查获，起获增值税普通发票19张，票面金额共计人民币1875921.23元，且经鉴定均为真发票。 ( The defendant, Ye Jun, was apprehended at approximately 10:00 on April 11, 2017, while selling Beijing value-added tax ordinary invoices to an individual surnamed Fan in front of the Super 8 Hotel in Qilizhuang, Fengtai District, Beijing. Officers from the Dongtiejianying Police Station of the Fengtai Branch of the Beijing Municipal Public Security Bureau seized 19 VAT ordinary invoices with a total face value of RMB 1,875,921.23; an appraisal confirmed that all of the invoices were genuine. )

Answer: H

Choices: [“A”, “B”, “C”, “D”, “E”, “F”, “G”, “H”, “I”, “J”, “K”, “L”, “M”]

Gold: 7

Appendix

Table C13: An instance format with paired prompts for the TaxCrime dataset in the KA task.

Prompt: 请将下述税务舆情概括为一段约300字的摘要，要求突出主要事件、观点或问题核心。 ( Please summarize the following tax-related public opinion into an approximately 300-word abstract, highlighting the main events and the core viewpoints or issues. )

Zero-Shot

Query: {Prompt}

Text: 近日，网红二驴涉嫌商业欺诈及逃税事件引发舆论关注。多名受害人和合作方纷纷揭露其不当行为，特别是一位女老板称自己被二驴骗走3000万元。…。更有消息称，二驴的主持人华少也曾为其花费40万元包机，整场活动的费用累计超过3000万元。在这一事件的背后，二驴的税务问题也受到关注。有报道称，二驴通过其父亲并玉坤的账户进行大额收款，意图规避税务责任。这一做法引发了外界对其是否存在逃税行为的质疑。值得注意的是，二驴的妻子曾因涉嫌偷税漏税而被处罚，因此人们对二驴的税务问题更加关注。此举是否涉及逃税，以及这些收入是否依法缴纳税款，目前尚无官方回应。…。 ( Recently, the influencer Er Lu has come under public scrutiny due to allegations of commercial fraud and tax evasion. Numerous victims and partners have come forward to expose his misconduct, particularly a female business owner who claims to have been defrauded of 30 million yuan. … Furthermore, reports indicate that Er Lu’s host, Hua Shao, spent 400,000 yuan on a charter flight for him, with the total expenses for the event exceeding 30 million yuan. Behind this incident, Er Lu’s tax issues have also attracted attention. It has been reported that he received large sums through his father Bing Yukun’s account in an apparent attempt to evade tax responsibilities. This practice has raised questions about whether he is engaging in tax evasion. Notably, Er Lu’s wife has previously been penalized for tax evasion, which has drawn even more scrutiny to his tax matters. It remains unclear whether these actions involve tax evasion or if the income has been properly taxed, as there has been no official response yet. … )

Answer: 近日，网红二驴因涉嫌商业欺诈及逃税引发广泛关注。一名女老板曝光称，与二驴合作举办演唱会时投入了3000万元，但二驴未完成合同约定的销售目标，且未投入任何资金。女老板还提供了聊天记录和转账截图，揭露二驴虚假承诺，甚至收取了其他商家的抗位费。…。 ( Recently, Chinese livestream celebrity Erlv has attracted widespread attention following allegations of commercial fraud and tax evasion. A female business owner claimed that she invested RMB 30 million to co-organize a concert with Erlv; however, Erlv allegedly failed to meet the contractual sales targets and did not contribute any capital. The business owner further provided purported chat logs and bank-transfer screenshots, alleging that Erlv made false promises and even collected so-called booth (slotting) fees from other merchants. … .)

One-Shot

Query: {Prompt}

Appendix

Table C14: An instance format with paired prompts for the TaxOpinion dataset in the KA task.

Prompt: 阅读下列财税风险控制相关内容，提炼出“面临的风险”及“风险对应的解决方案”，只需直接输出如下格式内容：{"面临的风险": , "风险对应的解决方案": } ( Read the following content related to fiscal and tax risk control, and extract (i) the “Risks Encountered” and (ii) the “Corresponding Mitigation Measures.” Output only the following JSON format: { "Risks Encountered": , "Corresponding Mitigation Measures": })

Zero-Shot

Query: {Prompt}

Text: 建筑企业存在较大的税收风险，主要涉及错误适用税目税率、简易计税进项税额未转出、合同金额与结算金额不一致。针对这些风险，建议企业要加强对税务风险的防控。对于总承包业务未分别核算的风险，建议按照不同税率核算销售额，避免高税率缴纳；对于简易计税进项税额未转出的风险，要注意用于简易计税项目的进项税额不可抵扣；对于合同金额与结算金额不一致的风险，应根据实际情况及时进行印花税补缴。企业在实际操作中需谨慎遵守政策规定，避免风险的发生。 ( Construction enterprises face substantial tax risks, which mainly stem from (i) misapplication of taxable items and applicable tax rates, (ii) failure to reverse input VAT credits when adopting the simplified taxation method, and (iii) inconsistencies between contract amounts and settlement amounts. To mitigate these risks, enterprises should strengthen tax risk management and internal controls. Regarding the risk that general contracting activities are not accounted for separately, firms are advised to segregate sales by the corresponding tax rates to avoid being taxed at a higher rate. With respect to the failure to reverse input VAT credits under the simplified method, firms should note that input VAT attributable to simplified-taxation projects is not creditable. For discrepancies between contract and settlement amounts, enterprises should, where appropriate, promptly pay any additional stamp duty based on the actual circumstances. In practice, enterprises should adhere strictly to relevant tax policies and regulations to minimize the likelihood of non-compliance. )

Answer: {"面临的风险": "错误适用税目税率、简易计税进项税额未转出、合同金额与结算金额不一致。", "对应的解决措施": "加强税务风险防控、按照不同税率核算销售额、注意简易计税项目进项税额不可抵扣、及时进行印花税补缴。"} ({ "Risks Encountered": "Misapplication of tax items and rates; failure to reverse input VAT under the simplified taxation regime; inconsistencies between the contract amount and the settlement amount.", "Corresponding Mitigation Measures": "Strengthen tax risk control; account for sales revenue by applicable tax rates; note that input VAT for simplified taxation items is non-creditable; promptly settle any outstanding stamp duty through supplementary payment." }.)

One-Shot

Query: {Prompt}

Appendix

Table C15: An instance format with paired prompts for the TaxRisk dataset in the KA task.

Prompt: 请提取以下税务稽查文章中的关键信息，内容包括：犯罪行为、所犯罪名与处罚结果。所犯罪名仅可在以下罪名中选择其一：持有伪造的发票罪，非法出售增值税专用发票罪，非法购买增值税专用发票罪，购买伪造的增值税专用发票罪，骗取出口退税罪，逃税罪，虚开发票罪。直接按照以下格式作答，无需添加解释说明：{"犯罪行为": , "所犯罪名": , "处罚结果": } ( Please extract the key information from the following tax audit case report, including: criminal act, the charged offense, and the penalty outcome. The charged offense must be selected from exactly one of the following options: Crime of holding forged invoices, Crime of illegally selling special VAT invoices, Crime of illegally purchasing special VAT invoices, Crime of purchasing forged special VAT invoices, Crime of fraudulently obtaining export tax refunds, Crime of tax evasion, Crime of falsely issuing invoices. Provide the answer strictly in the following format without any additional explanation: { "Criminal Act": , "Charged Offense": , "Penalty Outcome": })

Zero-Shot

Query: {Prompt}

Text: 税务局检查发现，A石油化工有限公司在2017年至2019年期间存在多起偷税行为。具体违法事实包括将购进的非应征消费税产品变为应征消费税产品进行销售，涉及煤油和普柴的数量分别为2017年590.7吨和2018年1927.6吨，以及2019年将有机化工原料变为汽油的6700.842吨。根据相关法律规定，该行为构成偷税行为，导致公司未缴纳的消费税总额达到1769.1万元。对此，税务局依法对A公司分别处以总计8845.5万元的罚款。 ( During a tax inspection, the tax authority found that Company A (Petrochemical) committed multiple acts of tax evasion between 2017 and 2019. The specific violations included converting purchased non–excise-taxable products into excise-taxable products for sale. The quantities involved were 590.7 tons of kerosene and 1,927.6 tons of diesel in 2017 and 2018, respectively; additionally, in 2019, 6,700.842 tons of organic chemical feedstock were converted into gasoline. Under the relevant statutory provisions, these actions constitute tax evasion and resulted in unpaid excise tax totaling RMB 17.691 million. Accordingly, the tax authority imposed a total fine of RMB 88.455 million on Company A in accordance with law. )

Answer: {"犯罪行为": "A石油化工有限公司将购进的非应征消费税产品变为应征消费税产品进行销售，涉及煤油、普柴及有机化工原料，未缴纳消费税总额达到1769.1万元。", "所犯罪名": "逃税罪", "处罚结果": "对A公司分别处以总计8845.5万元的罚款。"} ({ "Criminal Act": "Company A (a petroleum and petrochemical enterprise) sold products initially purchased as non-excise-taxable goods after converting them into excise-taxable products, involving kerosene, ordinary diesel, and organic chemical raw materials, and evaded a total of RMB 17.691 million in excise tax.", "Charged Offense": "Tax Evasion", "Penalty Outcome": "Company A was fined a total of RMB 88.455 million." }.)

One-Shot

Query: {Prompt}

Appendix

Table C16: An instance format with paired prompts for the TaxInspect dataset in the KA task.

Prompt: 请补全下方空项，完成后请按题目原格式返回填写结果。核心思路为文字回答，其余字段统一填写数字，若为百分比项，请勿添加百分号，仅填数字。{"核心思路": , "原年收入": , "筹划后年收入": , "节增收入": , "原应纳增值税": , "筹划后应纳增值税": } ( Please complete the missing fields below. After completion, return the filled results in the original format. The “Core Rationale” should be answered in text, and all other fields must be filled with numbers only. For percentage fields, do not include the percent sign; provide only the numeric value. { "Core Rationale": , "Original Annual Income": , "Post-Planning Annual Income": , "Incremental Income": , "Original VAT Payable": , "Post-Planning VAT Payable": })

Zero-Shot

Query: {Prompt}

Text: 某个体工商户销售水果、杂货，每月含税销售额为20600元左右，当地财政厅和国家税务局规定的增值税起征点为20000元。请计算该个体工商户全年应纳增值税额，并提出纳税筹划方案。（不考虑小微企业免征增值税优惠） ( A sole proprietorship engages in the retail sale of fruits and general groceries, with an average monthly tax-inclusive sales revenue of approximately RMB 20,600. The VAT threshold (minimum taxable turnover) stipulated by the local Department of Finance and the State Taxation Administration is RMB 20,000. Please calculate the annual value-added tax (VAT) payable by this taxpayer and propose an appropriate tax planning strategy. (Assume that the VAT exemption for small and micro enterprises does not apply.) )

Answer: {"核心思路": "通过将每月含税销售额降低到略低于起征点，个体工商户可以避免增值税的缴纳，从而实现全年收入的增加。", "原年收入": 24, "筹划后年收入": 24.6, "节增收入": 0.6, "原应纳增值税": 0.72, "筹划后应纳增值税": 0} ({ "Core Rationale": "By reducing the monthly VAT-inclusive sales revenue to just below the VAT exemption threshold, a self-employed household business can avoid paying VAT, thereby increasing annual income.", "Original Annual Income": 24, "Post-Planning Annual Income": 24.6, "Incremental Income": 0.6, "Original VAT Payable": 0.72, "Post-Planning VAT Payable": 0 }.)

One-Shot

Query: {Prompt}

Appendix

Table C17: An instance format with paired prompts for the TaxPlan dataset in the KA task.

Data	Prompt	Input	Target	Output	Metric
					AC	NA	IN	AVG
TaxRecite	根据问题需求，请提供法条的原文部分，不必对其含义做出解释，也不需要指明是第几条。	2007年12月6日通过的《中华人民共和国企业所得税法实施条例》中第一百一十一条的内容是什么？	企业所得税法第四十一条所称合理方法，包括：(一)可比非受控价格法；(二)再销售…(六)其他符合独立交易原则的方法。	企业所得税法第四十一条所称合理方法，包括：(一)可比非受控价格法；(二)再销售…(六)其他符合独立交易原则的方法。	3	3	3	3
TaxSum	根请根据以下文本编写一个简短的概要，字数不超过300字，重点在于提炼核心内容。	为配合个人所得税专项附加扣除工作，依据相关法律法规，三部门联合发布通知，明确个人所得税住房贷款利息专项附加扣除相关信息归集事项。信息归集范围为1989年1月1日（含）后发放的商业性个人住房贷款，不包括个人商用房贷款。…	信息归集工作时间紧、任务重、要求高，各商业银行要高度重视，充分认识贯彻施行《个人所得税专项附加扣除暂行办法》的重要性和紧迫性，切实加强组织领导，务必在规定时间内做好各项工作。	本次信息归集工作期限紧、工作量大、标准要求严格。各商业银行应提高站位，充分把握《个人所得税专项附加扣除暂行办法》贯彻实施的重要意义与紧迫要求，强化统筹协调和责任落实，完善工作机制与保障措施，确保在规定时限内高质量完成各项工作任务。	3	2	2	2.3
TaxTopic	阅读标题以了解文章主题，并在提供的选项中选择最适合表达这一主题的一个，选项有：A.财商资讯 B.财税表单 C.地方法规 D.法规解读 E.纳税调整 F.纳税评估 G.税收优惠 H.税收筹划 I.税务问答 J.中央法规。	浙江省国家税务局公告2014年第10号浙江省国家税务局关于企业资产损失税前扣除申报有关问题的公告	C	C	3	3	3	3
TaxRead	通过阅读以下文本，然后回答以下问题。	为综合反映与科学评价资产评估机构整体实力和专业胜任能力，…问题：《资产评估机构综合评价办法》何时正式发布？	2025年2月18日	2024年7月6日。《资产评估机构综合评价办法》的发布时间为2024年7月6日。	1	1	3	1.7
TaxCalc	针对题目，请直接给出xx处的答案，并将答案带入到问题中复述一遍，例如："业务（1）应缴纳消费税为xx万元"，保证输出字符在30个字符以内。	位于某市区国家重点扶持的高新技术企业为增值税一般纳税人（制造业企业），2023年销售产品取得不含税收入6500万元… 问题为：新产品研发费应调整的应纳税所得额为xx万元？	-72.75	-72.75	3	3	3	3

Appendix

Table C19: Examples of human evaluation with metrics like accuracy(AC), naturalness(NA), informativeness(IN) and average(AVG) for human written prompts in various tax datasets. We retain the prompts with AVG above 2.

Data	Prompt	Input	Target	Output	Metric
					AC	NA	IN	AVG
TaxMCQ	针对以下问题，请分析五个选项，并选择最适合的两个或更多，然后按字母顺序提供A、B、C、D或E作为你的答案，输出格式为字母组合，字母之间不要有任何其他字符，且必须是顺序的。	关于不动产租赁服务的增值税处理，正确的有()。 A.个体工商户异地出租不动产，在不动产所在地预缴的增值税款，可在当期增值税税款中抵减 B.以经营租赁方式将土地出租给他人使用… E.纳税人向其他个人出租不动产，可以开具增值税专用发票	ABC	A	1	3	1	1.7
TaxQA	对于下面提出的税务问题，请准备简要的答案，注意每项答复应保持在300字之内。	转让债权如何确定其应予减除的财产原值和合理费用？	采用“加权平均法”确定其应予减除的财产原值和合理费用。即以纳税人购进的同一种类债券买入价和买进过程中缴纳的税费总和，除以纳税人购进的该种类债券数量之和，乘以纳税人卖出的该种类债券数量，再加上卖出的该种类债券过程中缴纳的税费。	转让债权时，一般可采用加权平均法来确定应予减除的财产原值和合理费用：将纳税人购入同一类别债券的累计买入金额汇总后，除以累计购入数量，计算出单位成本；再用该单位成本乘以本次卖出的数量，作为可减除的财产原值，相关合理费用按实际发生情况一并计入扣除。	2	3	2	2.3
TaxBoard	对于由群众提出的税务问题，请提供确切的解答，并指出相关的法律依据，确保每个回答都在300字以内。	标题：文化事业建设费今年还有减半优惠吗？内容：文化事业建设费今年还有减半优惠吗？	根据《财政部关于延续实施文化事业建设费优惠政策的通知》(财税〔2025〕7号)规定，自2025年1月1日至2027年12月31日，对归属中央收入的文化事业建设费，将按应缴费额的50%减征；对归属地方收入…。	有。依据《财政部关于延续实施文化事业建设费优惠政策的通知》（财税〔2025〕7号）：（1）归属中央收入…。	2	2	3	2.3
TaxCrime	阅读下面的犯罪事实，请判断其主要违反了以下哪一法律条款：A. 刑法第二百零一条 B. 刑法第二百零三条 C. 刑法第二百零四条 D. 刑法第二百零五条 E. 刑法第二百零六条 F. 刑法第二百零七条 G. 刑法第二百零八条 H. 刑法第二百零九条 I. 刑法第二百一十条 J. 刑法第二百一十一条 K. 刑法第二百一十二条 L. 刑法第二百二十七条 M. 刑法第二百六十四条。只需输出对应字母，不需要解释原因。	被告人叶俊在2017年4月11日10时许，在北京市丰台区七里庄“速八酒店”门前向樊某出售北京市增值税普通发票，现场被北京市公安局丰台分局东铁匠营派出所民警当场查获，起获增值税普通发票19张，票面金额共计人民币1875921.23元，且经鉴定均为真发票。	H	H	3	3	3	3

Appendix

Table C20: Examples of human evaluation with metrics like accuracy(AC), naturalness(NA), informativeness(IN) and average(AVG) for human written prompts in various tax datasets. We retain the prompts with AVG above 2.

Data	Prompt	Input	Target	Output	Metric
					AC	NA	IN	AVG
TaxOpinion	请将下述税务舆情概括为一段约300字的摘要，要求突出主要事件、观点或问题核心。	记者从市市场监管委获悉，为期一年的加油机作弊专项治理行动已接近尾声。…。下一步，市市场监管委将继续加大执法力度，将专项治理与常态化执法检查相结合，推动建立健全加油机计量监管的长效机制，进一步规范成品油零售市场秩序。	自去年8月起，市市场监管委联合多部门开展为期一年的加油机作弊专项治理行动，已基本完成。…接下来，市市场监管委将加强执法力度，推动建立长期有效的加油机计量监管机制，进一步规范市场秩序。	市市场监管委通报，自去年8月起，会同公安、商务、税务等部门在全市开展为期一年的加油机作弊专项治理行动，目前已接近尾声并完成对加油站的全面检查。…下一步将把专项治理与常态化检查结合，健全长效监管机制。	3	1	3	2.3
TaxRisk	阅读下列财税风险控制相关内容，提炼出“面临的风险”及“风险对应的解决方案”，只需直接输出如下格式内容：{"面临的风险": , "风险对应的解决方案": }	建筑企业存在较大的税收风险，主要涉及错误适用税目税率、简易计税进项税额未转出、合同金额与结算金额不一致。针对这些风险，建议企业要加强对税务风险的防控。…企业在实际操作中需谨慎遵守政策规定，避免风险的发生。	{"面临的风险": "错误适用税目税率、简易计税进项税额未转出、合同金额与结算金额不一致。", "对应的解决措施": "加强税务风险防控、按照不同税率核算销售额、注意简易计税项目进项税额不可抵扣、及时进行印花税补缴。"}	{"面临的风险":"税目税率适用错误、简易计税进项未转出、合同金额与结算金额不一致。","风险对应的解决方案":"加强税务风控；按不同税率分别核算；简易计税进项不得抵扣并转出；按实际补缴印花税。"}	2	3	3	2.7
TaxInspect	请提取以下税务稽查文章中的关键信息，内容包括：犯罪行为、所犯罪名与处罚结果。所犯罪名仅可在以下罪名中选择其一：持有伪造的发票罪，非法出售增值税专用发票罪，非法购买增值税专用发票罪，购买伪造的增值税专用发票罪，骗取出口退税罪，逃税罪，虚开发票罪。直接按照以下格式作答，无需添加解释说明：{"犯罪行为": , "所犯罪名": , "处罚结果": }	税务局检查发现，A石油化工有限公司在2017年至2019年期间存在多起偷税行为。…根据相关法律规定，该行为构成偷税行为，导致公司未缴纳的消费税总额达到1769.1万元。对此，税务局依法对A公司分别处以总计8845.5万元的罚款。	{"犯罪行为": "A石油化工有限公司将购进的非应征消费税产品变为应征消费税产品进行销售，涉及煤油、普柴及有机化工原料，未缴纳消费税总额达到1769.1万元。", "所犯罪名": "逃税罪", "处罚结果": "对A公司分别处以总计8845.5万元的罚款。"}	{"犯罪行为":"A石油化工有限公司将购进的非应征消费税产品转换为应征消费税产品后销售，涉及煤油、普柴及有机化工原料，少缴消费税1769.1万元。","所犯罪名":"逃税罪","处罚结果":"被税务机关处罚款8845.5万元。"}	3	3	2	2.7
TaxPlan	请补全下方空项，完成后请按题目原格式返回填写结果。核心思路为文字回答，其余字段统一填写数字，若为百分比项，请勿添加百分号，仅填数字。{"核心思路": , "原年收入": , "筹划后年收入": , "节增收入": , "原应纳增值税": , "筹划后应纳增值税": }	某个体工商户销售水果、杂货，每月含税销售额为20600元左右，当地财政厅和国家税务局规定的增值税起征点为20000元。请计算该个体工商户全年应纳增值税额，并提出纳税筹划方案。（不考虑小微企业免征增值税优惠）	{"核心思路": "通过将每月含税销售额降低到略低于起征点，个体工商户可以避免增值税的缴纳，从而实现全年收入的增加。", "原年收入": 24, "筹划后年收入": 24.6, "节增收入": 0.6, "原应纳增值税": 0.72, "筹划后应纳增值税": 0}	{"核心思路": "个体工商户如果把每月含税销售额控制在起征点以下，就可以免缴增值税，从而提高全年到手收入。", "原年收入": 24, "筹划后年收入": 27, "节增收入": 3, "原应纳增值税": 0.72, "筹划后应纳增值税": 0}	1	3	3	2.3

Appendix

Table C21: Examples of human evaluation with metrics like accuracy(AC), naturalness(NA), informativeness(IN) and average(AVG) for human written prompts in various tax datasets. We retain the prompts with AVG above 2.

Task	Dataset	Prompt	English Translation
KM	TaxRecite	根据问题需求，请提供法条的原文部分，不必对其含义做出解释，也不需要指明是第几条。	Please provide the original text of the legal provision based on the question requirements, without explaining its meaning or indicating which article it is from.
KU	TaxSum	请根据以下文本编写一个简短的概要，字数不超过300字，重点在于提炼核心内容。	Please provide a brief summary based on the following text, keeping it under 300 words and focusing on extracting the core content.
	TaxTopic	阅读标题以了解文章主题，并在提供的选项中选择最适合表达的一个选项，选项有 A.财商资讯 B.财税表单 C.地方法规 D.法规解读 E.纳税调整 F.纳税评估 G.税收优惠 H.税收筹划 I.税务问答 J.中央法规	Read the title to grasp the article’s theme and select the option that best reflects it. The options are: A. Financial Information B. Tax Forms C. Local Regulations D. Regulatory Interpretation E. Tax Adjustments F. Tax Assessment G. Tax Incentives H. Tax Planning I. Tax Q&A J. Central Regulations.
	TaxRead	在阅读完下面的文本后，请根据其内容回答问题，确保回复是从文中摘录且言简意赅，不要重复问题，答案中不要出现句号。	After reading the text, please provide concise excerpts for the questions without repeating them or using periods in your answers.
KA	TaxCalc	针对题目，请直接给出xx处的答案，并将答案带入到问题中复述一遍，例如："业务（1）应缴纳消费税为xx万元"，保证输出字符在30个字符以内。	For the question, please provide the answer from the designated section, and incorporate it into the question. For example: "The consumption tax payable by the business (1) is xx million yuan." Ensure the output is within 30 characters.
	TaxSCQ	关于下面提出的问题，请仔细考虑四个可能的答案，并选出你认为正确的那个，仅需指出“A”、“B”、“C”或“D”，不需要解释。	Regarding the question posed below, please carefully consider the four possible answers and select the one you believe is correct. Simply indicate "A", "B", "C", or "D" without any explanation.

Appendix

Table C23: Examples of human-annotated prompts and their corresponding English translations in TaxPraBen.

Domain	Case Description	Raw Text	Structured JSON	Evaluation Metrics
Legal	刑事案件裁判要旨与量刑结果生成：给定案情与法条要点，让模型输出“裁判理由 + 刑期/罚金等可核对数值”。( Case Ruling Summary and Sentencing Generation: Given the circumstances of the case and legal provisions, the model is asked to output "ruling reasons + verifiable numerical values such as sentence length/fines" )	被告人张某入户盗窃财物价值约人民币50,000元，到案后如实供述并退赔部分损失，依法可从轻处罚。综合犯罪情节与认罪认罚情况，建议判处有期徒刑一年六个月，缓刑二年，并处罚金人民币10,000元。( The defendant, Zhang Mou, stole property worth approximately 50,000 yuan by entering a house, and truthfully confessed to the crime and partially compensated for the losses after being apprehended. He can be given a lighter punishment according to the law. Considering the criminal circumstances and his guilty plea, it is recommended to sentence him to one year and six months in prison, with a two-year probation period, and a fine of 10,000 yuan. )	{" 核心思路 ":"基于入户盗窃的法定构成与认罪认罚、退赔等从轻情节，依法确定从轻量刑并适用缓刑。", " 刑期_月 ":18, " 缓刑_月 ":24, " 罚金_元 ":10000} { " Core Idea ": "Based on the legal composition of housebreaking and mitigating circumstances such as a guilty plea and compensation, a lighter sentence is determined and probation is applied." , " Sentence_Months ": 18 , " Probation_Months ": 24 , " Fine_Yuan ": 10000 }	语义维度: BERTScore(核心思路) 数值维度: EM Accuracy(刑期_月, 缓刑_月, 罚金_元) Semantic Dimension: BERTScore (Core_Reasoning) Numerical Dimension: EM Accuracy (Sentence_Months, Probation_Months, Fine_Yuan)
Finance	审计风险说明 + 差异率核对：给定对账摘要与抽样结果，让模型输出“风险结论与原因 + 交易金额/差异率等关键数字”。( Audit Risk explanation + discrepancy rate verification: given reconciliation summaries and sampling results, the model outputs "risk conclusion with reasons + key figures such as transaction amount/discrepancy rate". )	本期收入确认存在较高错报风险：部分合同在未完成履约义务前提前确认收入，且回款与发货节点不一致。抽样核对显示异常交易金额约1,250,000元，占抽样总金额12,500,000元的10.0%。建议对相关收入进行重分类或调减，并将该类交易纳入重点复核。( There is a high risk of misstatement in revenue recognition: Some contracts recognized revenue prematurely before fulfilling obligations, and payment/receipt nodes are inconsistent with delivery. Sampling verification shows abnormal transaction amount of approximately RMB 1,250,000, accounting for 10.0% of the total sampled amount (RMB 12,500,000). It is recommended to reclassify or reduce relevant revenue and include such transactions in key review. )	{" 风险结论 ":"收入确认与履约/回款节点不一致导致潜在提前确认，形成重大错报风险，需调整并重点复核。", " 异常交易金额 ":1250000, " 抽样总金额 ":12500000, " 差异率 ":0.10} { " Risk_Conclusion ": "Inconsistency between revenue recognition and performance/payment nodes leads to potential premature recognition, creating a material misstatement risk requiring adjustment and key review" ., " Abnormal_Transaction_Amount ": 1250000 , " Total_Sampled_Amount ": 12500000 , "Discrepancy_Rate": 0.10 }	语义维度: BERTScore(风险结论) 数值维度: EM Accuracy(异常交易金额, 抽样总金额, 差异率) Semantic Dimension: BERTScore (Risk_Conclusion) Numerical Dimension: EM Accuracy (Abnormal_Transaction_Amount, Total_Sampled_Amount, Discrepancy_Rate)
Medicine	出院小结自动生成：给定病程与化验单，模型输出“诊断与处置叙述 + 生命体征/化验值等可核对数值”。( Discharge summary auto-generation: given medical records and test reports, the model outputs "diagnosis and treatment narrative + verifiable values such as vital signs/laboratory results". )	患者以发热、咳嗽咳痰入院，影像提示右下肺感染，诊断为肺炎；合并2型糖尿病血糖控制欠佳。入院体温 $38.5\,^{\circ}\mathrm{C}$ ，WBC $12.3\times 10^{9}/\mathrm{L}$ ，空腹血糖 11.2 mmol/L。予以头孢类抗感染、雾化祛痰及胰岛素强化控制血糖后症状改善，体温恢复正常，建议规律随访复查血常规与血糖。( The patient was admitted with fever and cough; imaging indicated right lower lung infection, diagnosed as pneumonia; complicated with poorly controlled type 2 diabetes. Admission Temperature: $38.5\,^{\circ}\mathrm{C}$ ，WBC $12.3\times 10^{9}/\mathrm{L}$ , Fasting Blood Glucose: 11.2mmol/L. After cephalosporin anti-infection, aerosol expectoration, and insulin-intensive glucose control, symptoms improved with normalized temperature. Regular follow-up for blood routine and glucose recheck is recommended. )	{" 诊断叙述 ":"依据症状与影像诊断肺炎，结合白细胞升高支持感染；同时强化控糖并随访复查关键指标。", " 体温( ${}^{\circ}\mathrm{C}$ ) ":38.5, " WBC(10 9 /L) ":12.3, " 空腹血糖(mmol/L) ":11.2} { " Diagnosis_Narrative ": "Pneumonia diagnosed based on symptoms and imaging, supported by elevated white blood cells; concurrent intensive glucose control with follow-up for key indicators "., " Temperature( ${}^{\circ}\mathrm{C}$ ) ": 38.5 , " WBC( $10^{9}$ /L) ": 12.3 , " Fasting_Glucose(mmol/L) ": 11.2 }	语义维度: BERTScore(诊断叙述) 数值维度: EM Accuracy(体温_ ${}^{\circ}\mathrm{C}$ , WBC_10 9 /L, 空腹血糖_mmol/L) Semantic Dimension: BERTScore (Diagnosis_Narrative) Numerical Dimension: EM Accuracy (Temperature_ ${}^{\circ}\mathrm{C}$ , WBC_ $10^{9}$ /L, Fasting_Glucose(mmol/L))

Appendix

Table D: Examples of our structured evaluation approach applied in legal, financial, and medical domains.

Panel of Figure 36 — Figure E1: Overview of the TaxPraBen open leaderboard and its representative interface screenshot.

Dataset	LLM Raw Response	Aligned Structured Output
TaxRisk	{"风险": "企业存在“不应享而享”的税收优惠问题，可能被追缴税款并取消高新技术企业资格。","措施": "企业需重视税务风险，确保合规经营，精准把握资格认定的关键要素，提升知识产权质量，加强研发费用核算，确保申报材料的真实性、准确性和完整性，严格执行后续管理措施。"} { "Risk": "The enterprise may have issues of improperly claiming tax incentives to which it is not entitled, and may be required to repay taxes and have its high and new technology enterprise qualification revoked.", "Measures": "The enterprise should attach importance to tax-related risks and ensure compliant operations; accurately grasp the key elements of qualification accreditation; improve the quality of intellectual property; strengthen the accounting of R&D expenses; ensure the authenticity, accuracy, and completeness of the application materials; and strictly implement subsequent management measures." }	{" 面临的风险 ": "企业存在“不应享而享”的税收优惠问题，可能被追缴税款并取消高新技术企业资格。", " 对应的解决措施 ": "企业需重视税务风险，确保合规经营，精准把握资格认定的关键要素，提升知识产权质量，加强研发费用核算，确保申报材料的真实性、准确性和完整性，严格执行后续管理措施。"} { " Risks Encountered ": "The enterprise may have issues of improperly claiming tax incentives to which it is not entitled, and may be required to repay taxes and have its high and new technology enterprise qualification revoked.", " Corresponding Mitigation Measures ": "The enterprise should attach importance to tax-related risks and ensure compliant operations; accurately grasp the key elements of qualification accreditation; improve the quality of intellectual property; strengthen the accounting of R&D expenses; ensure the authenticity, accuracy, and completeness of the application materials; and strictly implement subsequent management measures." }
TaxInspect	从案情看，公司让财务人员黄某用其私人账户代收货款，导致账外收入无法完整入账，并在申报时通过虚假申报的方式少缴税款，属于典型的隐匿收入逃避纳税义务行为。按照刑法相关规定，可能构成逃税罪。处罚一般包括补缴税款、滞纳金及罚款，情节严重还可能追究刑责。答案：犯罪行为：通过财务人员黄某的私人账户收取货款并虚假申报，隐匿销售收入逃避税款。所犯罪名：逃税罪, 处罚结果：补缴税款、加收滞纳金，并处罚款共计1411.65万元。 From the case, the company instructed its finance employee, Mr.Huang, to collect sales proceeds through his personal account. As a result, off-book income could not be fully recorded, and the company underpaid taxes by making false declarations when filing. This constitutes a typical act of concealing income to evade tax obligations. Under the relevant provisions of the Criminal Law, it may constitute the crime of tax evasion. Penalties generally include paying back the underpaid taxes, late payment surcharges, and fines; in serious cases, criminal liability may also be pursued. Answer: Criminal Act: Collecting sales payments through the personal account of the finance employee, Mr. Huang, and making false tax filings to conceal sales revenue and evade taxes. Charged Offense: Tax evasion. Penalty Outcome: Payment of back taxes, imposition of late payment surcharges, and fines totaling RMB 14.1165 million.	{" 犯罪行为 ": "通过财务人员黄某的私人账户收取货款并虚假申报，隐匿销售收入逃避税款。"," 所犯罪名 ": "逃税罪", " 处罚结果 ": "补缴税款、加收滞纳金，并处罚款共计1411.65万元。"} { " Criminal Act ": "Collecting sales payments through the personal account of the finance employee, Mr. Huang, and making false tax filings to conceal sales revenue and evade taxes."," Charged Offense ": "Tax evasion", " Penalty Outcome ": "Payment of back taxes, imposition of late payment surcharges, and fines totaling RMB 14.1165 million." }
TaxPlan	其中包含的计算过程：0.548-0.524 = 0.024(万元/人/年), 若10000人则合计240万元。 {"核心思路": "通过为全体员工统一购买符合税法规定的商业健康保险，员工缴费可税前扣除，从而减少个人所得税负担。","人均年应纳个人所得税_购买健康保险": 0.524, "人均年应纳个人所得税_未购买健康保险": 0.548, "甲公司全体员工年节税": 240, "备注": "按题设人数推算"} The calculation involved is as follows: 0.548-0.524=0.024 (ten-thousand yuan per person per year). For 10,000 people, the total is 2.4 million RMB. { "Core Rationale": "By purchasing commercial health insurance for all employees in compliance with tax law, employees’ premiums can be deducted pre-tax, thereby reducing their individual income tax burden.","Per-capita annual individual income tax payable_with health insurance": 0.524, "Per-capita annual individual income tax payable_without health insurance": 0.548, "Annual tax savings for all employees of Company A": 240, "Note": "Estimated based on the headcount given in the problem." }	{" 核心思路 ": "通过为全体员工统一购买符合税法规定的商业健康保险，员工缴费可税前扣除，从而减少个人所得税负担。", " 人均年应纳个人所得税_购买健康保险 ": 0.524, " 人均年应纳个人所得税_未购买健康保险 ": 0.548, " 人均节税 ": 0.024, " 甲公司全体员工年节税 ": 240} { " Core Rationale ": "By purchasing commercial health insurance for all employees in compliance with tax law, employees’ premiums can be deducted pre-tax, thereby reducing their individual income tax burden.", " Per-capita annual individual income tax payable_with health insurance ": 0.524, " Per-capita annual individual income tax payable_without health insurance ": 0.548, " Per-capita tax savings ": 0.024, " Annual tax savings for all employees of Company A ": 240 }

Appendix

Table F: LLM raw responses with redundant descriptions are converted into aligned structured outputs by ChatGPT.

LLM Evaluation

Large language model evaluation measures what models can do, how reliably they do it, and whether benchmark scores predict real-world performance.

See all 2,408 papers

Mar 27, 2026

Xpertbench: Expert Level Tasks with Rubrics-Based Evaluation

As Large Language Models (LLMs) exhibit plateauing performance on conventional benchmarks, a pivotal challenge persists: evaluating their proficiency in complex, open-ended tasks characterizing genuine expert-level cognition. Existing frameworks suffer from narrow domain coverage, reliance on generalist tasks, or self-evaluation biases. To bridge this gap, we present XpertBench, a high-fidelity benchmark engineered to assess LLMs across authentic professional domains. XpertBench consists of 1,346 meticulously curated tasks across 80 categories, spanning finance, healthcare, legal services, education, and dual-track research (STEM and Humanities). These tasks are derived from over 1,000 submissions by domain experts--including researchers from elite institutions and practitioners with extensive clinical or industrial experience--ensuring superior ecological validity. Each task uses detailed rubrics with mostly 15-40 weighted checkpoints to assess professional rigor. To facilitate scalable yet human-aligned assessment, we introduce ShotJudge, a novel evaluation paradigm that employs LLM judges calibrated with expert few-shot exemplars to mitigate self-rewarding biases. Our empirical evaluation of state-of-the-art LLMs reveals a pronounced performance ceiling: even leading models achieve a peak success rate of only ~66%, with a mean score around 55%. Models also exhibit domain-specific divergence, showing non-overlapping strengths in quantitative reasoning versus linguistic synthesis.. These findings underscore a significant "expert-gap" in current AI systems and establish XpertBench as a critical instrument for navigating the transition from general-purpose assistants to specialized professional collaborators.

Xue Liu, Xin Ma, Yuxin Ma +36

ByteDance Seed

LLM Evaluation Rubric-Based Evaluation LLM-as-a-Judge

Ask PDF

May 15, 2026

Reasoners or Translators? Contamination-aware Evaluation and Neuro-Symbolic Robustness in Tax Law

Recent advances in large language models (LLMs) have significantly enhanced automated legal reasoning. Yet, it remains unclear whether their performance reflects genuine legal reasoning ability or artifacts of data contamination. We present a comprehensive empirical study of tax law reasoning approaches and implement a contamination detection protocol to rigorously assess LLM reliability. We show that performance can be inflated by contamination. Building on this analysis, we conduct a systematic evaluation, comparing monolithic LLMs with hybrid systems that translate statutory text into formal representations and delegate inference to symbolic solvers. We build a novel test suite designed to probe generalization to unseen documents via case and rule variations. Our findings indicate that legal reasoning is inherently compositional and that neuro-symbolic frameworks offer a more reliable and robust foundation for legal AI, as well as improved generalization to unobserved situations.

Parisa Kordjamshidi, Samer Aslan, Madhavan Seshadri +2

Bloomberg · Michigan State University

Data Contamination in Language Models Neuro-Symbolic Reasoning Language Model Robustness Legal Reasoning

Ask PDF

Oct 20, 2026

ABench: Adversarily Benchmarking Large Language Models across Taxonomized Values

Ensuring the alignment of Large Language Models (LLMs) with diverse cultural values is critical for their safe global deployment. However, existing alignment benchmarks are predominantly English-centric or confined to simple discriminative tasks. Moreover, they often fail to effectively evaluate model robustness against complex, adversarial inductions within specific cultural contexts. To address these limitations, we propose ABench, a fine-grained, handcrafted benchmark specifically tailored to taxonomized values in Chinese. Grounded in a rigorous hierarchical taxonomy comprising 158 derived value subcategories, ABench consists of 2,895 open-ended questions. These questions employ techniques such as role-playing and logical traps to probe the stability of models' stances in adversarially induced contexts. Furthermore, we propose a weighted automated evaluation framework that utilizes question-specific scoring criteria for nuanced assessment. Extensive experiments on 16 mainstream LLMs reveal that most models exhibit significant vulnerabilities when subjected to sophisticated adversary attacks on values (scoring below 60). This underscores the need for more robust, culturally specific alignment strategies. Our data will be publicly released to promote further research on taxonomized value alignment.

Liping Yang, Yongqi Leng, Jingting Zheng +3

TJUNLP Lab, School of Computer Science and Technology, Tianjin University, Tianjin, China · Yuanhui AI

LLM Safety Benchmarks Language Model Safety Evaluation Cultural Alignment Adversarial Robustness of Language Models Cross-Cultural Language Model Evaluation

Ask PDF

Abstract

LLM Evaluation

Explore similar work

Xpertbench: Expert Level Tasks with Rubrics-Based Evaluation

Reasoners or Translators? Contamination-aware Evaluation and Neuro-Symbolic Robustness in Tax Law

ABench: Adversarily Benchmarking Large Language Models across Taxonomized Values