How do you build an expert-grade research QA benchmark without hand-authoring a single question?Mine the survey articles. Their section structure already encodes what a field asks and what a complete answer must contain, so a new benchmark distills 21K queries and grading rubrics from surveys across 75 fields. It doubles as a stress test: even the best system tops out at 75% rubric coverage, fully addressing under 11% of needed citations.https://benjaminhan.net/posts/20260625-researchqa/?utm_source=mastodon&utm_medium=social#LLMs #Evaluation #AI
Related
I tested my USB-C cables with this free Mac app - some weren't what they claimedWhatCable is a free Mac app that gives y...
I tested my USB-C cables with this free Mac app - some weren't what they claimedWhatCable is a free Mac app that gives you all the information you could need for your USB-C cables....
Jedesmal wenn ich Bilder vom Inneren eines Rechenzentrums (KI oder sonstiges) sehe, denke ich sofort an einem Borg-Kubus...
Jedesmal wenn ich Bilder vom Inneren eines Rechenzentrums (KI oder sonstiges) sehe, denke ich sofort an einem Borg-Kubus aus Star Trek. Und andersrum auch.#KI #AI #Borg
紙ですか。紙のことはマリアさんに聞いてみないとPFU、ScanSnapの新たな核となるクラウドサービス「ScanSnap Cloud+」を発表 https://www.macotakara.jp/etc/report/entry-51451...
紙ですか。紙のことはマリアさんに聞いてみないとPFU、ScanSnapの新たな核となるクラウドサービス「ScanSnap Cloud+」を発表 https://www.macotakara.jp/etc/report/entry-51451.html#Apple #LLM #news #bot