704 articles tagged with Benchmark
📰 NARS-Reasoning-v0.1: The First Executable Narsese Benchmark for Neuro-Symbolic AI in 2026A new benchmark called NARS-Reasoning-v0.1 translates natural language into executable Na...
Stop evaluating LLMs with vibes. Here's a practical framework for benchmarking open-source models against your API provider using real production data.
Full code, aggregated numbers (n=10 across 5 tasks and 5 transports), and a curated selection of 8...
【QIMMA قِمّة ⛰: 品質第一のアラビア語LLMリーダーボード】https://huggingface.co/blog/tiiuae/qimma-arabic-leaderboard※AI生成の自動投稿（見出し＋リンク）#AI #生成AI #LLM #AIGenerated
At present, executable visual workflows have emerged as a mainstream paradigm in real-world industrial deployments, offering strong reliability and controllability. However, in cur...
PRL-Bench: LLMs Score Below 50% on End-to-End Physics Research TasksResearchers introduced PRL-Bench, a benchmark built from 100 recent Physical Review Letters papers, testing LLMs...
SocialGrid Benchmark Shows LLMs Fail at Deception, Score Below 60% on PlanningResearchers introduced SocialGrid, a multi-agent benchmark inspired by Among Us. It shows state-of-the...
Introducing the Open Ko-LLM Leaderboard: Leading the Korean LLM Evaluation Ecosystem #AINews #Shorts.
Mike Saleme — 2026-04-20 — views my own This week OpenAI released GPT-5.4-Cyber, positioned as the...
BEIJING: Robots outpace humans in half marathon, Bloomberg reports new AI benchmark. China’s robotic surge reshapes urban mobility and beyond.🚩 #Robotik #AI
📰 KWBench 2026: The First Benchmark for AI’s Unprompted Problem Recognition in Knowledge WorkKWBench, a groundbreaking benchmark for unprompted problem recognition in knowledge wor...
📢 Benchmark de LLMs auto-hébergés pour la sécurité offensive : résultats et observations📝 ## 🔍 ContextePublié le 14 avril 2026 sur le blog de TrustedSec par Brandon McGrath, cet ar...
Mathematical problem solving remains a challenging test of reasoning for large language and multimodal models, yet existing benchmarks are limited in size, language coverage, and t...
Multimodal Large Language Models (MLLMs) have been increasingly used as automatic evaluators-a paradigm known as MLLM-as-a-Judge. However, their reliability and vulnerabilities to ...
Claude's Opus 4.7 reclaims coding benchmark leadership with 87.6% on SWE-bench Verified, while Anthropic narrows OpenAI's enterprise lead to just 4.6 percentage points. Meanwhile, ...
Unlike code completion, debugging requires localizing faults and applying targeted edits. We observe that frontier LLMs often regenerate correct but over-edited solutions during de...
Delve faked compliance certificates for 494 companies. Now agents are faking benchmark scores. Same pattern, new layer. The only thing that catches both is behavioral telemetry.
GeoAgentBench: New Dynamic Benchmark Tests LLM Agents on 117 GIS ToolsA new benchmark, GeoAgentBench, evaluates LLM-based GIS agents in a dynamic sandbox with 117 tools. It introdu...
GPU server rental 2026 — H100 from $1.85/hr, B200 access via CoreWeave. Full benchmark + price comparison: https://server-rental-guide.pages.dev/ #AI #infrastructure
Ivan Fioravanti ᯅ (@ivanfioravanti)평가(Eval)에서 thinking tokens가 결과와 소요 시간에 큰 영향을 주며, 모델이 과도하게 생각하는 경우를 벌점화하기 위해 평가별 최대 토큰 제한이 필요할 수 있다는 제안이다.https://x.com/ivanfioravanti/status/2045...
Gökdeniz Gülmez (@ActuallyIsaak)Apple의 MLX 프레임워크에서 LLM 성능을 평가하기 위한 첫 번째 종합 벤치마크인 MLX-Benchmark Suite가 소개되었습니다. 코드 이해, 작성, 디버깅 능력을 측정하는 CLI 도구와 데이터셋을 포함한 개발자용 평가 도구입니다.https://x.com...
As AI-assisted video creation becomes increasingly practical, instruction-guided video editing has become essential for refining generated or captured footage to meet professional ...
We present SemanticQA, an evaluation suite designed to assess language models (LMs) in semantic phrase processing tasks. The benchmark consolidates existing multiword expression (M...