cs.SEApr 10, 2026

Can Coding Agents Be General Agents?

Authors: Maksim Ivanov, Abhijay Rana, Gokul Prabhakaran

Organizations: Agentic Labs

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

As coding agents have seen rapid capability and adoption gains, users are applying them to general tasks beyond software engineering. In this post, we investigate whether coding agents can successfully generalize to end-to-end business process automation. We identify gaps in current evaluations, and conduct a case study to evaluate a coding agent on practical business tasks in an open-core Enterprise Resource Planning system. We find that the agent reliably completes simple tasks but exhibits characteristic failures on complex tasks, suggesting that bridging domain logic and code execution is a key bottleneck to generalizability.

Figures & tables

Figure 1: The coding agent is an emergent agent architecture, which allows a model to create its own code tools at runtime and iterate with the help of code outputs and error logs.
Figure 2: The critical evaluation gap: Code evaluations lack business context; business evaluations lack code interaction.
Figure 3: An example of a process in an ERP, involving an order-to-delivery workflow.
Figure 4: A visualization of the decisions present in a simplified scenario. The solution to this task requires multiple choices and execution steps within the ERP.
Figure 5: We test Claude Sonnet 4.5 and GPT-5 on 20 scenarios across a gradient of difficulty from Easy to Hardest. We observe that increasing the complexity of the scenarios by increasing the number of constraints results in a breakdown of performance. We separate our evaluation of each scenario into 4 dimensions: Constraint Resolution, Resource Optimization, Traceability, and Policy Adherence. Noticeably, GPT-5 tended to come up with comparable or even better-quality business plans as Claude 4.5 Sonnet, but GPT-5 struggled more with correct API calls, which caused lower scores across the board.
Figure 6: The agent hallucinates that damaged LED boards must be stored in the fridge, without any supporting evidence in the system, leading to functional-but-misleading code.
Figure 7: The agent disregards the policy requirement to only schedule consecutive days off, and incorrectly claims successful completion.
More in this topic

Coding Agents

Coding agents use language models, tools, and feedback to modify software and complete engineering tasks.

See all 140 papers

Explore similar work

CardsList

  1. AgentSWE: Can Coding Agents Build the Agent You Actually Want?

    Oct 9, 2026Jiahao Wang, Hongjin Qian, Yuyang Hu +2Coding AgentsSoftware Engineering Agents

    Beijing Academy of Artificial Intelligence · Institute of Automation, Chinese Academy of Sciences · Renmin University of China

  2. Code Understanding is a Bottleneck for Coding Agents

    Oct 7, 2026Nishant Balepur, Kiran Tomlinson, Tobias SchnabelCoding AgentsLLM Agent Evaluation

    Microsoft Research · University of Maryland

  3. Scaling Coding Agents via Atomic Skills

    Apr 6, 2026Yue LiuCoding AgentsLLM Agent Skill Learning

    NUS