SWE-Bench Verified Unreliable: OpenAI Halts Benchmark
OpenAI is retiring the SWE-Bench Verified benchmark after finding over 60% of its problems unsolvable due to test design or data contamination. This makes it a poor measure of AI coding skill. OpenAI now backs SWE-Bench Pro and human-centric evaluations for longer tasks and real-world code quality.
Read full story→