Six papers shifted attention from larger models to narrower mechanisms. BenchJack attacks the grader; Ada-MK fixes decode scheduling before runtime; delta-mem compresses history into 64 values; MatterSim shares representations across material properties; GLiGuard returns moderation to classification; and tau-knowledge forces retrieval to keep pace with a changing conversation.
1. BenchJack finds reward-hacking paths across agent evaluations
BenchJack is an automated red-team system for finding ways AI agents can maximize benchmark rewards without completing the intended task. Its authors report 219 distinct flaws across ten evaluations covering software engineering, web navigation, desktop computing, and terminal work, with synthesized exploits reaching near-perfect scores on most targets.
An iterative patching experiment reduced the share of hackable tasks from nearly 100% to below 10% on four benchmarks without fundamental design defects. The preprint's broader lesson is that benchmark infrastructure belongs inside the threat model; a leaderboard is unreliable when its grader is easier to solve than its tasks.
Sources: BenchJack preprint, version 1
2. Ada-MK moves inference scheduling decisions to compile time
Ada-MK targets decode latency by fusing LLM operators into a persistent megakernel and moving execution-path decisions from runtime into an offline DAG search. The authors estimate kernel-launch overhead at 14.6% of inference time in their setting and reduce peak shared-memory use by half with K-dimension splitting.
On an Nvidia L20, the paper reports single-batch throughput gains of up to 23.6% over TensorRT-LLM and 50.2% over vLLM. Those results are workload- and hardware-specific, but the deployed advertising use case illustrates how predictable configurations can justify ahead-of-time specialization that would be too rigid for general serving.
Sources: Ada-MK preprint, version 1
3. Delta-mem compresses online memory into an 8-by-8 state
The delta-mem paper augments a frozen full-attention model with an 8-by-8 associative state matrix updated during use. That state produces low-rank corrections to attention, allowing past information to influence generation without extending the context window, replacing the backbone, or performing full fine-tuning.
The authors report an average score 1.10 times the frozen backbone, with gains of 1.31 times on MemoryAgentBench and 1.20 times on LoCoMo. Model-size, update-stability, privacy, and long-duration forgetting tests can separate genuine persistent state from a compact mechanism tuned to short evaluation sequences.
Sources: Delta-mem preprint, version 1
4. MatterSim-MT broadens one materials model across properties
MatterSim-MT is a machine learning materials model pretrained on more than 35 million first-principles-labeled structures spanning 89 elements, temperatures up to 5,000 K, and pressures up to 1,000 GPa. Its multi-task design predicts structures and thermodynamics alongside properties such as magnetic moments, Bader charges, and dielectric matrices.
The paper demonstrates simulations including pressure-dependent phonon splitting in silicon carbide and electric hysteresis in barium titanate. This is a research result rather than a substitute for laboratory validation, but a shared representation across many characterization tasks could reduce the number of isolated models in computational-materials pipelines.
Sources: MatterSim-MT preprint, version 1
5. GLiGuard reframes moderation as compact classification
Fastino introduced GLiGuard, a 300-million-parameter encoder model that evaluates prompt safety, response safety, jailbreak strategy, harm category, and refusal behavior in one forward pass. Its Apache 2.0 weights were released publicly, making the system available for local evaluation and domain-specific tuning.
Fastino reports results across nine safety benchmarks, including throughput up to 16.2 times higher than larger decoder guards and prompt-classification F1 of 87.7. The developer selected the comparisons and hardware. Application traffic can expose the remaining tradeoff through false-negative rates, policy fit, schema changes, and adversarial drift.
Sources: Fastino's GLiGuard report · GLiGuard preprint, version 1
6. Tau-knowledge measures retrieval during live agent work
Sierra's tau-knowledge benchmark places AI support agents in a banking environment with 698 documents totaling about 195,000 tokens. A task requires information from 18.6 documents and 9.5 tool calls on average, forcing agents to retrieve new policy context as a conversation changes rather than search once at the start.
Sierra reports GPT-5.5 at 37.4% Pass-at-1, while even a setup that supplies relevant documents directly reaches only about 40%. Because Sierra created and operates the benchmark, its rankings need external reproduction, but the task design usefully tests whether retrieval and action remain coordinated over a long interaction.
Sources: Sierra's tau-knowledge methodology and results · tau2-bench code and tasks