Prompt-Induced Waste in Coding Agents: Reasoning, Effort, Harness Design, and End-to-End Cost

  • 2026-08-21 15:45:21
  • Sarel Weinberger, Amir Hozez
  • 0

Abstract

Coding-agent efficiency cannot be characterized by token count or model price alone. We study how end-to-end cost and task success depend jointly on prompt semantics, inference effort, harness policy, model, task difficulty, tool use, context management, and provider accounting. Controlled prompt experiments show that wording can change reasoning and verification behavior without changing the task. A separate SWE-bench Verified study shows that additional inference effort can improve difficult tasks for some models but can also add cost without benefit. A DeepSeek Harness extension shows that the effect of an effort-control intervention changes substantially when the harness changes, even when the model, tasks, prompts, and controller logic are held fixed. These results show that prompt, effort, and harness are interacting experimental factors rather than independent efficiency controls. We model efficiency as cost per successful task induced by the agent trajectory. Token and cache counts are measurements of that trajectory, not sufficient optimization targets. Agent evaluations should therefore measure success and end-to-end cost while controlling the system variables that determine how the trajectory is produced

 

Quick Read (beta)

loading the full paper ...