GHA-SmellFix: Taking Action with Agentic LLMs for Detecting and Refactoring GitHub Actions Workflow Smells
Organizations: Federal University of Ceará, Brazil · North Carolina State University, USA · Federal University of Campina Grande, Brazil
Abstract
Continuous integration and continuous delivery workflows are key artifacts in software projects. In GitHub Actions, they encode build, test, release, deployment, and security automation in YAML. These workflows contain configuration smells affecting security, reliability, maintainability, or operational stability. Existing approaches are mostly rule-based, with rules hard to create and evolve, and typically report findings without fixes. Agentic LLM pipelines can combine contextual detection, transformation selection, and patch generation. Specialized agents can gate refactoring on positive detection verdicts, potentially avoiding unnecessary changes, but it is unclear whether decomposition improves effectiveness or mainly changes overhead. We present GHA-SmellFix, an agentic LLM pipeline for detecting and refactoring GitHub Actions workflow smells. We compare two designs: Joint, where one agent detects and refactors, and Decomposed, where separate agents detect and then refactor. We operationalize 19 smells from an existing catalog with explicit detection criteria, refactoring expectations, exclusion rules, and safety checks, and curate a ground truth of 1,710 (workflow, smell) pairs. We evaluate the designs with four LLMs (GPT-5.4-nano, Gemini 3.5 Flash, Claude Haiku 4.5, and DeepSeek V4 Flash) on 570 sampled pairs. Results show that decomposition does not universally improve patch quality; its main benefit is controlling when patch generation is attempted. It helps models prone to missing smells and can reduce overhead by gating patches. Once a smell is correctly detected, clean refactoring is generally effective, making detection the main end-to-end bottleneck. However, false positives can trigger unnecessary edits to execution-relevant constructs. Practical CI/CD agents thus need reliable abstention and construct-aware validation alongside patch generation.
Figures & tables
| Type | Configuration smell | Core operational criterion | P/N |
| Localized | MissingPermExternalActions | A job uses an external step-level GitHub action and has no effective workflow- or job-level permissions block. | 50/50 |
| UnpinnedActionVersion | A step-level GitHub action reference uses owner/repo@ref , but ref is not a full 40-character commit SHA. | 36/36 | |
| MissingTokenPermissions | A job uses GITHUB_TOKEN or authenticated GitHub operations without an effective permissions block. | 50/50 | |
| MissingJobTimeout | A job defined in the current workflow file does not declare job-level timeout-minutes . | 48/48 | |
| UnsafeArtifactUploadFork | A PR-triggered workflow uploads retained GitHub Actions artifacts without an effective fork guard. | 33/33 | |
| MissingUploadCondition | A retained artifact-upload step lacks an explicit step-level or job-level if condition. | 38/38 |
| Type | Smell | Ground-truth-positive outputs | Ground-truth-negative outputs | |||||||
| SRR | CR | UX | BR | FN | Assessable/Expected Pos. Outputs | CA | UE | Assessable/Expected Negative Outputs | ||
| Localized | MissingPermExternalActions | 99% (117) | 98% (116) | 1% (1) | 0% (0) | 1% (1) | 118/120 | 100% (120) | 0% (0) | 120/120 |
| UnpinnedActionVersion | 66% (78) | 40% (47) | 0% (0) | 26% (31) | 34% (40) | 118/120 | 100% (120) | 0% (0) | 120/120 | |
| MissingTokenPermissions | 99% (119) | 99% (119) | 0% (0) | 0% (0) | 1% (1) | 120/120 | 100% (120) | 0% (0) | 120/120 | |
| MissingJobTimeout | 93% (105) | 91% (103) | 0% (0) | 2% (2) | 7% (8) | 113/120 | 56% (65) | 44% (52) | 117/120 | |
| UnsafeArtifactUploadFork | 95% (110) | 95% (110) | 0% (0) | 0% (0) | 5% (6) | 116/120 | 100% (120) | 0% (0) | 120/120 | |
| Perspective | Design | Out. | Det. miss | Patch fail | SRR | CR |
| E2E | D1 | 1,092 | 238 | 21 | 76.3% (833) | 71.8% (784) |
| D2 | 1,089 | 201 | 14 | 80.3% (874) | 76.1% (829) | |
| All | 2,181 | 439 | 35 | 78.3% (1,707) | 74.0% (1,613) | |
| Cond. | D1 | 854 | – | 21 | 97.5% (833) | 91.8% (784) |
| D2 | 888 | – | 14 | 98.4% (874) | 93.4% (829) | |
| All | 1,742 | – | 35 | 98% ( 1,707 ) | 92.6% ( 1,613 ) |