GHAgentCI: A Dataset of Coding-Agent Invocations in CI Workflow Histories
Organizations: University of Nevada Las Vegas Las Vegas, USA
Abstract
Coding agents are increasingly integrated into GitHub Actions to review pull requests, triage issues and modify code. Identifying these integrations is challenging because agentic workflows often resemble ordinary CI files, with evidence appearing in action references, shell commands and workflow conventions. We present GHAgentCI, a dataset of 15K+ workflow histories from 6K+ of 196K+ mined GitHub repositories, comprising 159K+ revisions and 205K+ agent-evidence records linked to workflow triggers and declared execution constraints. Our open-source extractor, cai-miner, detects agent-related evidence using an extensible catalogue. Manual validation of 576 stratified records confirmed 96.1% of dedicated-action matches and 98.8% of GitHub Agentic Workflow matches, with lower rates for other evidence kinds. Exploratory analyses reveal a shift toward command-line and GitHub Agentic Workflow integrations, differences in declared safeguards, and frequent removal of detected agentic workflows. GHAgentCI supports longitudinal studies of coding-agent integration into CI.
Figures & tables
| kind | matched in | example | tier | records |
| uses | step uses: | anthropics/claude-code-action | strong | 69,104 |
| run | step run: | claude -p | strong | 64,822 |
| path | file name | copilot-setup-steps.yml | strong | 8,805 |
| ghaw | lock-file header | engine declared by gh-aw | strong | 36,657 |
| job | job identifier | copilot-setup-steps | weak | – * |
| url | step content | api.anthropic.com | weak | 1,969 |
| Kind | Yes | Maybe | No | Confirmed (%) | 95% CI | |
|---|---|---|---|---|---|---|
| gh-aw | 80 | 79 | 0 | 1 | 98.8 | [93.3,99.8] |
| uses | 128 | 123 | 3 | 2 | 96.1 | [91.2,98.3] |
| run | 116 | 95 | 4 | 17 | 81.9 | [73.9,87.8] |
| path | 30 | 26 | 0 | 4 | 86.7 | [70.3,94.7] |
| parse-failed | 30 | 14 | 1 | 15 | 46.7 | [30.2,63.9] |
| weak-only | 96 | 41 | 19 | 36 | 42.7 | [33.3,52.7] |
| file | description |
|---|---|
| revisions.csv.gz | Compressed CSV file with one row per touch of a workflow file in an agentic history. |
| steps.csv.gz | Compressed CSV file with one row per agent-evidence record per revision. |
| ghaw_manifests.csv.gz | Compressed CSV file with one row per revision of a gh-aw artifact: engine, compiler and schema versions, model, tools, MCP tools, safe outputs, secrets, sandboxing, network policy and permissions. |
| repositories.csv.gz | Compressed CSV file with one row per mined repository: commit count, workflow and agentic file counts, agents used, first and last agentic dates, blob accounting and status. |
| content.tar.gz | Compressed archive of every distinct workflow revision. |
| characteristic | all snapshots | last snapshot |
|---|---|---|
| # repositories | 6,345 | 4,598 (72.5%) |
| with gh-aw workflows | 506 | 377 (74.5%) |
| # histories | 15,928 | 9,942 (62.4%) |
| # commits | 104,545 | 6,923 (6.6%) |
| # workflow revisions | 159,878 | 9,942 (6.2%) |
| agentic | 125,266 | 9,265 (7.4%) |
Agentic Workflows
Agentic workflows coordinate AI agents, tools, and humans across multi-step tasks, emphasizing reliable execution, efficiency, and oversight.
See all 230 papers