Visual Aesthetic Benchmark: Can Frontier Models Judge Beauty?
Multimodal large language models (MLLMs) are now routinely deployed for visual understanding, generation, and curation. A substantial fraction of these applications require an expl...
#Benchmark
698 articles tagged with Benchmark
ShapeCodeBench: A Renewable Benchmark for Perception-to-Program Reconstruction of Synthetic Shape Scenes
We introduce ShapeCodeBench, a synthetic benchmark for perception-to-program reconstruction: given a rendered raster image, a model must emit an executable drawing program that a d...
PreScam: A Benchmark for Predicting Scam Progression from Early Conversations
Conversational scams, such as romance and investment scams, are emerging as a major form of online fraud. Unlike one-shot scam lures such as fake lottery or unpaid toll messages, t...
A new Microsoft Research benchmark called DELEGATE-52 found something enterprise teams need to know: even the best model...
A new Microsoft Research benchmark called DELEGATE-52 found something enterprise teams need to know: even the best models (Gemini 3.1 Pro, Claude 4.6 Opus, GPT 5.4) corrupted 25% o...
STRADVISION Sets New Global Benchmark for ADAS with Launch of Production-Ready Integrated Perception Platform
Strategic collaboration with leading Physical AI developer delivers a turnkey solution for accelerated Level 2+ vehicle integration SEOUL, South Korea, May 12, 2026 /PRNewswire/ --...
π§ This article ranks and reviews artificial intelligence tools designed for social media use in 2026. The tools are eval...
π§ This article ranks and reviews artificial intelligence tools designed for social media use in 2026. The tools are evaluated across various categories to help users select options...
oh-my-agent: 9 new skills, cursor as first-class vendor, 80/100 benchmark
When you tell an agent to scaffold a Next.js app, it picks the wrong version, ignores your lint...
Agentick Benchmark: GPT-5 Mini Tops at 0.309, No Agent Paradigm DominatesAgentick benchmark evaluates RL, LLM, VLM, and ...
Agentick Benchmark: GPT-5 Mini Tops at 0.309, No Agent Paradigm DominatesAgentick benchmark evaluates RL, LLM, VLM, and hybrid agents on 37 tasks. GPT-5 mini leads at 0.309 ONS, bu...
RoboMemArena: A Comprehensive and Challenging Robotic Memory Benchmark
Memory is a critical component of robotic intelligence, as robots must rely on past observations and actions to accomplish long-horizon tasks in partially observable environments. ...
PlantMarkerBench: A Multi-Species Benchmark for Evidence-Grounded Plant Marker Reasoning
Cell-type-specific marker genes are fundamental to plant biology, yet existing resources primarily rely on curated databases or high-throughput studies without explicitly modeling ...
Agent-ValueBench: A Comprehensive Benchmark for Evaluating Agent Values
Autonomous agents have rapidly matured as task executors and seen widespread deployment via harnesses such as OpenClaw. Safety concerns have rightly drawn growing research attentio...
SleepWalk: A Three-Tier Benchmark for Stress-Testing Instruction-Guided Vision-Language Navigation
Vision-Language Models (VLMs) have advanced rapidly in multimodal perception and language understanding, yet it remains unclear whether they can reliably ground language into spati...
WildClawBench: A Benchmark for Real-World, Long-Horizon Agent Evaluation
Large language and vision-language models increasingly power agents that act on a user's behalf through command-line interface (CLI) harnesses. However, most agent benchmarks still...
AnthonyXu109/FairPrivacySignal: Synthetic benchmark for privacy-preserving and fairness-aware ranking under signal loss
Synthetic benchmark for privacy-preserving and fairness-aware ranking under signal loss
6 months solo on a multi-agent PR reviewer. 10.93 vs 3.80 blockers/PR (claude alone) on my benchmark β please test on real PRs and tell me where it's wrong
TL;DR: I built a 3-LLM code reviewer (Claude + GPT-5 + Gemini that deliberate). My synthetic-bug...
I open-sourced a 3-agent blind eval team. Any agent runtime can call it for pre-commitment review of its own plans.
Shipped this weekend: a 3-agent blind cross-lab evaluation workflow on heym, MIT licensed, callable...
Frontier LLMs corrupt 25% of documents in long workflows per new benchmark, while a Fields Medalist reports ChatGPT 5.5 ...
Frontier LLMs corrupt 25% of documents in long workflows per new benchmark, while a Fields Medalist reports ChatGPT 5.5 Pro solving PhD-level math. Mayo Clinic AI detects pancreati...
Metal-Sci: A Scientific Compute Benchmark for Evolutionary LLM Kernel Search on Apple Silicon
We present Metal-Sci, a 10-task benchmark of scientific Apple Silicon Metal compute kernels spanning six optimization regimes (stencils, all-pairs in n-body problems, multi-field B...
π° StepAudio 2.5 TTS Ranks #1 Chinese Voice Model on Artificial Analysis Leaderboard 2026StepAudio 2.5 TTS has emerged as...
π° StepAudio 2.5 TTS Ranks #1 Chinese Voice Model on Artificial Analysis Leaderboard 2026StepAudio 2.5 TTS has emerged as China's leading voice model, ranking among the top three gl...
Show HN: Nexa-gauge β Cache/cost-aware graph-based eval for LLM and RAGNexa-gaugeλ LLM, RAG, μμ΄μ νΈ μμ€ν μ μμ± κ²°κ³Όλ₯Ό νκ°νκΈ° μν νμ΄...
Show HN: Nexa-gauge β Cache/cost-aware graph-based eval for LLM and RAGNexa-gaugeλ LLM, RAG, μμ΄μ νΈ μμ€ν μ μμ± κ²°κ³Όλ₯Ό νκ°νκΈ° μν νμ΄μ¬ ν¨ν€μ§μ΄μ CLI λꡬλ‘, κ·Έλν κΈ°λ° νκ° νμ΄νλΌμΈμ ν΅ν΄ λ°λ³΅ κ°λ₯ν νμ§ μ§νμ λΉμ© μΆμ , μΊμ...
How a $0.02/Call Model Scored 78.2% on SWE-bench Verified β Beating Every Model on the Leaderboard
TL;DR We added architectural context to AI coding agents via MCP and tested on SWE-bench...
Soohak: A Mathematician-Curated Benchmark for Evaluating Research-level Math Capabilities of LLMs
Following the recent achievement of gold-medal performance on the IMO by frontier LLMs, the community is searching for the next meaningful and challenging target for measuring LLM ...
MuSS: A Large-Scale Dataset and Cinematic Narrative Benchmark for Multi-Shot Subject-to-Video Generation
While video foundation models excel at single-shot generation, real-world cinematic storytelling inherently relies on complex multi-shot sequencing. Further progress is constrained...
DiagnosticIQ: A Benchmark for LLM-Based Industrial Maintenance Action Recommendation from Symbolic Rules
Monitoring complex industrial assets relies on engineer-authored symbolic rules that trigger based on sensor conditions and prompt technicians to perform corrective actions. The bo...