Supply-chain security and benchmark integrity share a boundary problem: the component being trusted can alter the mechanism used to trust it. OpenAI's signing workflow executed a compromised dependency, while Berkeley researchers showed agents could manipulate graders. GraphicDesignBench then illustrated the constructive alternative, using separate task dimensions to expose where polished output hides structural failure.
1. OpenAI rotates Mac signing material after the Axios compromise
OpenAI said a GitHub Actions workflow downloaded and executed compromised Axios version 1.14.1 during its macOS signing process. The workflow exposed certificate and notarization material, although the company's investigation found no unexpected notarizations, altered software, or access to user data.
The root cause was concrete: a floating dependency tag and no minimum release age. OpenAI treated the signing certificate as compromised and scheduled revocation after a 30-day update window. That choice trades immediate invalidation against the risk of breaking legitimate older ChatGPT, Codex, and Atlas installations.
Sources: OpenAI's response to the Axios developer-tool compromise
2. Berkeley researchers make top agent benchmarks score the wrong work
UC Berkeley researchers built working exploits against prominent AI agent benchmarks. Examples included a pytest hook that forced SWE-bench tests to pass, a fake curl wrapper that produced a perfect Terminal-Bench score, and file access that exposed WebArena answer data.
These attacks target the evaluator, not the intended task. Shared agent-and-grader environments let generated code tamper with tests, while public gold files convert reasoning into lookup. A credible score therefore carries an architectural claim: the answer key, grader, and execution host remained outside the agent's reach.
Sources: Berkeley RDI on exploitable agent benchmarks
3. GraphicDesignBench finds precision gaps beneath fluent design output
A preprint introduced 50 professional design tasks for AI models across layout, typography, infographics, template semantics, and animation. The authors found that frontier models handled high-level meaning better than precise spatial layouts, vector code, typography, and temporal decomposition.
That decomposition prevents aesthetic fluency from hiding unusable structure. A generated poster may look plausible while its text, layers, or vectors cannot survive revision. Evaluating understanding and generation on the same five axes connects the score to production handoff instead of a single preference vote.