OpenTWBench (Open Taiwan Benchmark): Taiwan-Domain Knowledge Evaluation for Large Language Models, and How the Measurement Apparatus Shapes the Score
我们提出OpenTWBench，这是一个用于评估大语言模型台湾领域知识的开放评估套件，围绕两个基准进行构建。tw-exam-bench包含255,854个四选一问题，覆盖148个学术学科，来自台湾教育部发布的十五年全国考试开放数据；答案一致在99.91%个重复问题组中，所有27个分歧已予考虑。
来自arXiv AI的《OpenTWBench (Open Taiwan Benchmark): Taiwan-Domain Knowledge Evaluation for Large Language Models, and How the Measurement Apparatus Shapes the Score》。重点看模型能力与工程、评测与基准。摘要提到：这条英文动态主要涉及模型能力与工程、评测与基准。原文要点：We present OpenTWBench, an open evaluation suite for Taiwan-domain knowledge in large language models, built around two benchmarks. tw-exam-bench contains 255,854 four-option questions across 148 academic subjects, from fifteen years of Taiwan's national examinations published as open data by the Ministry of Examination; answer keys agree across 31,312 duplicated question groups at 9
本站只提供摘要与原文入口。完整内容请阅读原文。
来源：arXiv AI · arxiv.org