703 articles tagged with Benchmark
A week after shipping a flattering tokens-saved number for my AI context tool, I noticed it was apples-to-oranges. Here's the workload-matched redo, the smaller honest number, and ...
📰 Tencent Open-Source Translation Model Beats Google Translate – 2026 Benchmark VictoryTencent has released two open-weight AI translation models that outperform Google Translate i...
📰 Claude Biyoinformatikte İnsan Uzmanlarla Eşleşti: Yeni Benchmark BioMysteryBenchAnthropic'in yeni BioMysteryBench testinde Claude, biyoinformatik problemlerinde insan uzmanlarla ...
📰 2026 Human-in-the-Loop LLM Benchmark: Why Smaller Models Outperform Giants in Math AssessmentA new study benchmarks large language models in automating secondary math competency ...
【VAKRAの内部構造：エージェントの推論、ツールの使用、および障害モード】https://huggingface.co/blog/ibm-research/vakra-benchmark-analysis※AI生成の自動投稿（見出し＋リンク）#AI #生成AI #LLM #AIGenerated
LLM agents are expected to complete end-to-end units of work across software tools, business services, and local workspaces. Yet many agent benchmarks freeze a curated task set at ...
Surgical resection remains the only potentially curative treatment for pancreatic ductal adenocarcinoma (PDAC), and eligibility depends on accurate assessment of vascular invasion ...
While GUI agents have shown impressive capabilities in common computer-use tasks such as OSWorld, current benchmarks mainly focus on isolated and single-application tasks. This ove...
GPT-5.5 Pro Leapfrogs on Epoch Benchmark; Base Model Beats Prior ProA tweet from @kimmonismus reveals GPT-5.5 Pro shows significant Epoch benchmark gains, and the non-Pro GPT-5.5 s...
【AI評価が新たな計算上のボトルネックになりつつある】https://huggingface.co/blog/evaleval/eval-costs-bottleneck※AI生成の自動投稿（見出し＋リンク）#AI #生成AI #LLM #AIGenerated
Tool-use agents fail silently when a prompt change rewires which tool gets called. 90 lines of Python, 3 judges in a ladder, runnable on a small golden set for a few dollars.
Your eval library calls the judge once per test case and prints a number. The judge flips its verdict on 5-15% of reruns. That number is noise wearing a suit.
Show HN: A new benchmark for testing LLMs for deterministic outputshttps://interfaze.ai/blog/introducing-structured-output-benchmark#HackerNews #Tech #AI
I'm not a professional developer. I learned by doing — vibe-coding with AI assistance — and a few...
【Open ASRリーダーボード：新しい多言語トラックと長尺トラックによるトレンドとインサイト】https://huggingface.co/blog/open-asr-leaderboard※AI生成の自動投稿（見出し＋リンク）#AI #生成AI #LLM #AIGenerated
Hace un tiempo empecé a sentir una incomodidad rara construyendo aplicaciones con LLMs en Go. Go...
Frontier coding agents solve complex tasks when given complete context but collapse when specifications are incomplete or ambiguous. The bottleneck is not raw capability, but judgm...
Current status: attempting to run my scraping/reverse-engineering benchmark prompt against DeepSeek 4 Pro via Ollama, but their servers are melting, as one might expect. So I'm hav...
Why isn't AMD's MI300X competitive?https://newsletter.semianalysis.com/p/mi300x-vs-h100-vs-h200-benchmark-part-1-training#HackerNews #Tech #AI
Standard text-to-speech (TTS) evaluation measures intelligibility (WER, CER) and overall naturalness (MOS, UTMOS) but does not quantify accent. A synthesiser may score well on all ...
📰 HappyHorse 1.0: AI Anime Breakthrough for Perfect Character Consistency (2026)HappyHorse 1.0 has set a new benchmark in AI-generated anime animation by maintaining perfect charac...
Autonomous agents capable of navigating Graphical User Interfaces (GUIs) hold the potential to revolutionize digital productivity. However, achieving true digital autonomy extends ...
Mikeysee (@mikeysee)@convex의 평가(eval) 비용이 역대 최고로 비싸졌다는 언급과 함께, 지나치게 비싼 모델 호출 빈도를 줄이기 위해 변경을 했다고 합니다. AI 개발 도구 사용 비용과 운영 최적화와 관련된 실무적 업데이트입니다.https://x.com/mikeysee/status/204846097...
Three ways eval sets go wrong in production: leakage, drift, judge bias. Plus a 40-line drift detector you can ship today.