Terminal Agents Suffice for Enterprise Automation
Organizations: ServiceNow · Mila – Quebec AI Institute · Université de Montréal
Abstract
There has been growing interest in building agents that can interact with digital platforms to execute meaningful enterprise tasks autonomously. Among the approaches explored are tool-augmented agents built on abstractions such as Model Context Protocol (MCP) and web agents that operate through graphical interfaces. Yet, it remains unclear whether such complex agentic systems are necessary given their cost and operational overhead. We argue that a coding agent equipped only with a terminal and a filesystem can solve many enterprise tasks more effectively by interacting directly with platform APIs. We evaluate this hypothesis across diverse real-world systems and show that these low-level terminal agents match or outperform more complex agent architectures at a fraction of the cost. Our findings suggest that simple, flexible programmatic interfaces combined with strong foundation models should be the backbone of enterprise automation.
Figures & tables
| ServiceNow | GitLab | ERPNext | |
| Samples | 330 | 192 | 207 |
| MCP Tools | 83 | 81 | 7 |
| Doc. Pages | 61k | 2.65k | 5.41k |
| ServiceNow | GitLab | ERPNext | Overall | |||||||||
| Agent | (330) | (192) | (207) | (729) | ||||||||
| SR (%) | Cost ($) | SR (%) | Cost ($) | SR (%) | Cost ($) | SR (%) | Cost ($) | |||||
| Claude Sonnet 4.6 | ||||||||||||
| Tool-use | 11.5 | 0.76 | 45.2 | 0.48 | 55.6 | 0.14 | 32.9 | 0.51 | ||||
| Web | 72.4 | 4.49 | 82.9 | 0.88 | 61.8 | 3.63 | 72.2 | 3.29 | ||||
| Terminal | 73.6 | 0.78 | 76.5 | 0.28 | 67.6 | 0.46 | 72.7 | 0.56 | ||||
| ServiceNow | GitLab | ERPNext | Overall | |||||||||
| Agent | (330) | (192) | (207) | (729) | ||||||||
| SR (%) | Cost ($) | SR (%) | Cost ($) | SR (%) | Cost ($) | SR (%) | Cost ($) | |||||
| Claude Sonnet 4.6 | ||||||||||||
| Web | 72.4 | 4.49 | 82.9 | 0.88 | 61.8 | 3.63 | 72.2 | 3.29 | ||||
| Terminal | 73.6 | 0.78 | 76.5 | 0.28 | 67.6 | 0.46 | 72.7 | 0.56 | ||||
| Hybrid | 72.1 | 3.02 | 77.7 | 0.64 | 66.7 | 1.44 | 72.0 | 1.94 | ||||
Appendix figures & tables20 assets
Supplementary material from the paper’s appendix.
Appendix
| ServiceNow | GitLab | ERPNext | Overall | |||||||||
| Agent | (120) | (127) | (197) | (444) | ||||||||
| SR (%) | Cost ($) | SR (%) | Cost ($) | SR (%) | Cost ($) | SR (%) | Cost ($) | |||||
| Claude Sonnet 4.6 | ||||||||||||
| Tool-use | 24.2 | 1.30 | 66.9 | 0.52 | 58.4 | 0.14 | 51.6 | 0.56 | ||||
| Web | 66.7 | 4.57 | 86.4 | 0.91 | 60.7 | 3.82 | 69.6 | 3.20 | ||||
| Terminal | 75.8 | 0.49 | 80.2 | 0.44 | 66.0 | 0.48 | 72.7 | 0.47 | ||||
| ServiceNow | GitLab | ERPNext | Overall | |||||||||
| Agent | (330) | (192) | (207) | (729) | ||||||||
| SR (%) | Cost ($) | SR (%) | Cost ($) | SR (%) | Cost ($) | SR (%) | Cost ($) | |||||
| Claude Sonnet 4.6 | ||||||||||||
| Tool-use | 11.5 | 0.76 | 45.2 | 0.48 | 55.6 | 0.14 | 32.9 | 0.51 | ||||
| Web | 72.4 | 4.49 | 82.9 | 0.88 | 61.8 | 3.63 | 72.2 | 3.29 | ||||
| Terminal | 73.6 | 0.78 | 76.5 | 0.28 | 67.6 | 0.46 | 72.7 | 0.56 | ||||
| ServiceNow | GitLab | ERPNext | Overall | |||||||||
| Agent | (330) | (192) | (207) | (729) | ||||||||
| SR (%) | Cost ($) | SR (%) | Cost ($) | SR (%) | Cost ($) | SR (%) | Cost ($) | |||||
| Gemma-4-31B-it | ||||||||||||
| Tool-use | 9.4 | 0.005 | 39.9 | 0.012 | 30.4 | 0.007 | 23.4 | 0.007 | ||||
| Web | 20.3 | 0.148 | 35.6 | 0.235 | 22.7 | 0.247 | 25.0 | 0.199 | ||||
| Terminal | 48.5 | 0.024 | 52.1 | 0.016 | 36.7 | 0.091 | 46.1 | 0.041 | ||||
| ServiceNow | GitLab | ERPNext | Overall | |||||||||
| Agent | (330) | (192) | (207) | (729) | ||||||||
| SR (%) | Cost ($) | SR (%) | Cost ($) | SR (%) | Cost ($) | SR (%) | Cost ($) | |||||
| Claude Sonnet 4.6 | ||||||||||||
| No Docs | 73.6 | 0.78 | 76.5 | 0.28 | 67.6 | 0.46 | 72.7 | 0.56 | ||||
| With Docs | 67.3 | 1.10 | 79.1 | 0.48 | 72.3 | 0.47 | 71.8 | 0.76 | ||||
| Claude Opus 4.6 | ||||||||||||
| ServiceNow | GitLab | ERPNext | Overall | |||||||||
| Agent | (330) | (192) | (207) | (729) | ||||||||
| SR (%) | Cost ($) | SR (%) | Cost ($) | SR (%) | Cost ($) | SR (%) | Cost ($) | |||||
| Claude Sonnet 4.6 | ||||||||||||
| Single Agent | 73.6 | 0.78 | 76.5 | 0.28 | 67.6 | 0.46 | 72.7 | 0.56 | ||||
| Planner/Executor | 71.5 | 1.13 | 80.8 | 0.36 | 74.4 | 0.68 | 74.8 | 0.80 | ||||
| Claude Opus 4.6 | ||||||||||||
| Success rate (%) | Tokens per task (K) | |||||||
| Category | n | p | Tool-use | Web | Term. | Tool-use | Web | Term. |
| ServiceNow | ||||||||
| Form Creation | 50 | 0.001 | 2.0 | 67.0 | 89.0 | 59 | 513 | 42 |
| Dashboard | 40 | 0.302 | 11.9 | 36.2 | 39.1 | 107 | 599 | 186 |
| List Filtering | 60 | 0.072 | 20.8 | 70.0 | 76.7 | 50 | 397 | 34 |
| List Sorting | 60 | 0.003 | 35.8 | 62.5 | 75.4 | 87 | 174 | 249 |
| pass^ (all instances succeed) | pass@ | |||||||
| Agent | ||||||||
| Claude Sonnet 4.6 | ||||||||
| Terminal | 73.3 | 43.9 | 27.6 | 9.1 | 94.8 | 96.7 | 97.0 | |
| Web | 72.7 | 44.5 | 30.9 | 15.6 | 95.7 | 99.3 | 100.0 | |
| Tool-use | 11.8 | 2.0 | 0.7 | 0.0 | 25.1 | 32.8 | 42.4 | |
| Claude Opus 4.6 | ||||||||
| Category | Description | Example output |
| Success categories | ||
| Success | Valid JSON response with expected data. | {"result": {"sys_id": "a1b2...", "number": "INC001"}} |
| Success (trunc.) | Valid response cut off by output length limit. | {"result": [{"sys_id": ...} ...] [OUTPUT TRUNCATED] |
| Non-JSON success | Output piped through jq , python3 , or awk ; non-JSON but correct. | number: INC0000039 state: 6 |
| Failure categories | ||
| API error | Server returned a JSON error response (invalid payload, bad field, wrong HTTP method). | {"error": {"detail": "The payload is not valid JSON."}, "status": "failure"} |
| ServiceNow | GitLab | ERPNext | Overall | |||||||||||||||||
| Agent | (330) | (192) | (207) | (729) | ||||||||||||||||
| SR | Tools | Time | Cost | SR | Tools | Time | Cost | SR | Tools | Time | Cost | SR | Tools | Time | Cost | |||||
| Claude Sonnet 4.6 | ||||||||||||||||||||
| Tool-use | 11.5 | 4.3 | 35 | 0.76 | 45.2 | 4.0 | 22 | 0.48 | 55.6 | 5.7 | 17 | 0.14 | 32.9 | 4.6 | 26 | 0.51 | ||||
| Web | 72.4 | 26.0 | 153 | 4.49 | 82.9 | 10.4 | 55 | 0.88 | 61.8 | 19.9 | 138 | 3.63 | 72.2 | 20.2 | 123 | 3.29 | ||||
| Terminal | 73.6 | 10.5 | 67 | 0.78 | 76.5 | 8.8 | 42 | 0.28 | 67.6 | 13.0 | 42 | 0.46 | 72.7 | 10.8 | 53 | 0.56 | ||||
| ServiceNow | GitLab | ERPNext | Overall | |||||||||||||||||
| Agent | (330) | (192) | (207) | (729) | ||||||||||||||||
| SR | Tools | Time | Cost | SR | Tools | Time | Cost | SR | Tools | Time | Cost | SR | Tools | Time | Cost | |||||
| Claude Sonnet 4.6 | ||||||||||||||||||||
| No Docs | 73.6 | 10.5 | 67 | 0.78 | 76.5 | 8.8 | 42 | 0.28 | 67.6 | 13.0 | 42 | 0.46 | 72.7 | 10.8 | 53 | 0.56 | ||||
| With Docs | 67.3 | 18.0 | 94 | 1.10 | 79.1 | 12.2 | 45 | 0.48 | 72.3 | 15.3 | 52 | 0.47 | 71.8 | 15.7 | 69 | 0.76 | ||||
| Claude Opus 4.6 | ||||||||||||||||||||
| ServiceNow | GitLab | ERPNext | Overall | |||||||||||||||||
| Agent | (330) | (192) | (207) | (729) | ||||||||||||||||
| SR | Tools | Time | Cost | SR | Tools | Time | Cost | SR | Tools | Time | Cost | SR | Tools | Time | Cost | |||||
| Claude Sonnet 4.6 | ||||||||||||||||||||
| No Skills | 73.6 | 10.5 | 67 | 0.78 | 76.5 | 8.8 | 42 | 0.28 | 67.6 | 13.0 | 42 | 0.46 | 72.7 | 10.8 | 53 | 0.56 | ||||
| With Skills | 77.0 | 8.7 | 67 | 0.44 | 78.1 | 9.9 | 50 | 0.31 | 72.9 | 11.7 | 62 | 0.35 | 76.1 | 9.9 | 61 | 0.38 | ||||
| ServiceNow | GitLab | ERPNext | Overall | |||||||||||||||||
| Agent | (330) | (192) | (207) | (729) | ||||||||||||||||
| SR | Tools | Time | Cost | SR | Tools | Time | Cost | SR | Tools | Time | Cost | SR | Tools | Time | Cost | |||||
| Claude Sonnet 4.6 | ||||||||||||||||||||
| Single Agent | 73.6 | 10.5 | 67 | 0.78 | 76.5 | 8.8 | 42 | 0.28 | 67.6 | 13.0 | 42 | 0.46 | 72.7 | 10.8 | 53 | 0.56 | ||||
| Planner/Executor | 71.5 | 16.6 | 103 | 1.13 | 80.8 | 10.9 | 54 | 0.36 | 74.4 | 20.9 | 109 | 0.68 | 74.8 | 16.3 | 92 | 0.80 | ||||
| Claude Opus 4.6 | ||||||||||||||||||||
| ServiceNow | GitLab | ERPNext | Overall | |||||||||||||||||
| Agent | (330) | (192) | (207) | (729) | ||||||||||||||||
| SR | Tools | Time | Cost | SR | Tools | Time | Cost | SR | Tools | Time | Cost | SR | Tools | Time | Cost | |||||
| Claude Sonnet 4.6 | ||||||||||||||||||||
| Web | 72.4 | 26.0 | 153 | 4.49 | 82.9 | 10.4 | 55 | 0.88 | 61.8 | 19.9 | 138 | 3.63 | 72.2 | 20.2 | 123 | 3.29 | ||||
| Terminal | 73.6 | 10.5 | 67 | 0.78 | 76.5 | 8.8 | 42 | 0.28 | 67.6 | 13.0 | 42 | 0.46 | 72.7 | 10.8 | 53 | 0.56 | ||||
| Hybrid | 72.1 | 21.0 | 127 | 3.02 | 77.7 | 10.5 | 49 | 0.64 | 66.7 | 16.5 | 89 | 1.44 | 72.0 | 17.0 | 96 | 1.94 | ||||
| Category | Task Name | # Tasks |
| Form Creation | create-change-request | 10 |
| create-hardware-asset | 10 | |
| create-incident | 10 | |
| create-problem | 10 | |
| create-user | 10 | |
| Dashboard | single-chart-value-retrieval | 10 |
| Category | Task Name | # Tasks |
|---|---|---|
| User Management | assign-role | 2 |
| create-user-email-first-last | 2 | |
| create-user-email-first-last-username-phone | 3 | |
| create-user-group-with-new-users | 3 | |
| create-user-with-email-account-dependency | 3 | |
| disable-user-account | 2 |
| Category | Tool Name | # Tools | |
|---|---|---|---|
| Incident Management | create_incident | resolve_incident | 6 |
| update_incident | list_incidents | ||
| add_comment | get_incident_by_number | ||
| Service Catalog | list_catalog_items | move_catalog_items | 11 |
| get_catalog_item | get_optimization_recommendations | ||
| update_catalog_item | create_catalog_item_variable |
| Category | Tool Name | # Tools | |
|---|---|---|---|
| Authentication | authenticate_erpnext | 1 | |
| Document Management | get_documents | update_document | 3 |
| create_document | |||
| Schema | get_doctypes | get_doctype_fields | 2 |
| Reporting | run_report | 1 | |
| Total | 7 |
| Category | Tool Name | # Tools | |
|---|---|---|---|
| Merge Requests | create_merge_request | update_merge_request_note | 31 |
| get_merge_request | delete_merge_request_note | ||
| update_merge_request | create_merge_request_thread | ||
| merge_merge_request | resolve_merge_request_thread | ||
| list_merge_requests | mr_discussions | ||
| get_merge_request_diffs | create_merge_request_discussion_note |
| Category | Tool Name | # Tools |
|---|---|---|
| Core Automation | browser_click | 20 |
| browser_close | ||
| browser_console_messages | ||
| browser_drag | ||
| browser_evaluate | ||
| browser_file_upload |
Terminal Agents
Terminal agents use command-line interfaces and filesystems to carry out multi-step tasks in software and other digital environments.
See all 31 papers