AI agent changes altered 17% of task outcomes in platform test

De Wang and Peng Qi reported in an arXiv preprint published Oct. 21 that temporary changes to an AI agent altered 17% of task outcomes beyond normal run-to-run variation.

The result matters because the authors argue unattended self-improvement should depend on two measured safeguards, after an LLM judge failed silently in 9 of 37 tested settings.

The paper addresses agent platforms that can inspect execution traces, derive software changes, seek automated approval and release updates without a human author. The authors propose using verifier strength and recoverability to decide when that loop can run on its own.

Verifier strength, denoted σ in the paper, measures how closely an available verifier agrees with a trusted reference verifier. Recoverability measures how cheaply a bad change can be reversed, including whether rollback needs a new approval and how much prior behaviour is restored.

“A step should run unattended only if both are adequate,” the authors wrote in the abstract. On their platform, they said that rule allowed prompt, tool and skill edits to run without human intervention, while model-weight updates still required human sign-off.

The researchers said they built a platform applying the rule and tested it in two settings: the platform’s own approval gate and controlled experiments on the τ2-bench agent benchmark. An LLM judge, meaning a large language model used to assess proposed changes, sometimes returned the same score regardless of what it was shown.

That failure occurred in 9 of 37 settings and did not produce an error report, according to the abstract. The authors wrote that “a verifier can look healthy while approving anything,” and said two simple tests caught the failure.

The recoverability results point to a separate constraint. Restoring a saved copy returned the agent to its original behaviour, while a regenerated undo did not, according to the preprint. The authors said this means only changes whose old version is retained can be reversed without a human.

The paper’s limits are material. The arXiv posting is a preprint, and the abstract describes results from the authors’ own platform and benchmark experiments rather than an industrywide deployment.

The source page says the six-page paper plus appendix was accepted at the NeurIPS 2026 Workshop on AgenticOS and lists the subjects as artificial intelligence, computation and language, and software engineering.

Source: arXiv cs.AI preprint, Oct. 21, 2026