DevOps programming (e.g., using CLI/API scripts or IaC frameworks) is key to cloud infrastructure management. Unlike traditional programming tasks, DevOps program testing needs provisioning and execution against actual cloud resources, which is often time-consuming, unsafe, and costly. Cloud emulators have gained popularity for easing DevOps program testing; they are generally API-level mocks that can execute DevOps programs in a local environment. Still, building these emulators remains challenging: developers must manually interpret extensive cloud documentation and handcraft logic for each service, API, and their interaction. This does not scale to the complexity of the cloud, which is further a moving target as the services and APIs evolve. CloudEmu is an automated approach that constructs emulators based on cloud documentation via neurosymbolic code synthesis. The key idea is to combine LLMs' general strengths in documentation understanding and code generation with cloud-specific symbolic abstractions that suppress hallucinations and enforce precision at scale, while using the real cloud as an oracle for automated testing, repair, and alignment. Our evaluation shows the effectiveness of CloudEmu on major cloud provider (AWS and GCP) services in both coverage and accuracy. CloudEmu outperforms the existing leading tool LocalStack, which was manually developed by a large team of engineers over a decade.
Figures & tables
Services
APIs
Emulated
Coverage
Compute (EC2)
756
244
32%
Firewall (WAF)
55
30
54%
DNS (Route 53)
68
51
75%
CDN (CloudFront)
167
44
26%
Overall (subset)
1046
369
~35%
Table 1. API coverage of LocalStack ( 33 ) , the leading cloud emulator; even on popular services coverage is under 40%.
Figure 1. Cloud is a hierarchical state machine allowing inter-SM transitions, resource nodes maintain per-resource state space, with create/destroy API calls to change the state space.
Figure 2. Overview of CloudEmu .
Figure 3. Documentation wrangling builds a cloud resource index. Nodes are resources and edges are (strict or optional) dependencies.
Figure 4. LocalStack and CloudEmu scored against real AWS traces (50 CLI + 50 Terraform cases).
Figure 6
Mismatch class
Example in CloudEmu -LS Alignment
Example in CloudEmu Alignment
MISSING_FIELD
CreateLaunchTemplate : added missing nested response fields that were expected by the LocalStack execution trace.
CreateVpc : added missing fields such as CidrBlockAssociationSet , IPv6 state, and owner ID to match AWS responses.
STATUS_MISMATCH
CreateDhcpOptions : corrected whether the call should return success or an error.
DescribeSecurityGroups : corrected the call to return success, matching AWS behavior for a newly created security group.
VALUE_MISMATCH
CreateSecurityGroup : corrected response values echoed back to the caller to match LocalStack behavior.
DescribeKeyPairs : corrected the returned KeyFingerprint to match the fingerprint computed by AWS.
EXTRA_FIELD
CreateSubnet : removed or reshaped extra response fields so that the returned object matched LocalStack more closely.
DescribeInstances : removed internal state fields that appeared in the emulator output but are not present in AWS responses.
Table 2. Examples of behavioral corrections introduced by LocalStack-guided alignment in CloudEmu -LS and by AWS-guided alignment in CloudEmu .
Mismatch type
CloudEmu -LS
CloudEmu
MISSING_FIELD
16.3%
35.0%
STATUS_MISMATCH
13.5%
30.0%
VALUE_MISMATCH
32.0%
22.5%
EXTRA_FIELD
38.2%
12.5%
Table 3. Distribution of patch attempts by mismatch type in the LocalStack- and AWS-guided alignment stages.
Stage
Time (s)
Tokens (k)
Synthesis
52 / resource
478 / resource
CloudEmu -LS Alignment
21 / case
3.4 / case
CloudEmu Alignment
126 / case
4.1 / case
Table 4. Normalized cost of CloudEmu code synthesis and automated alignment on AWS EC2.
Figure 7. CDFs of patch size (left) and latency (right) for GPT patching attempts; verified OpenHands escalations are included in the latency panel only.
Figure 8. Cumulative unique APIs exercised and discrepancies found for scraped vs. coverage-guided cases
CloudEmu -GCP
Error-class
Resp. Fidelity
State Consist.
before alignment
0.644
0.571
0.545
after alignment
0.956
0.857
0.734
Table 5. Accuracy on Terraform cases over GCP Compute resource cases