We present NormCode Canvas (v1.1.3), a deployed system realizing Case-Based Reasoning at two levels for multi-step LLM workflows. The foundation is NormCode, a semi-formal planning language whose compiler-verified scope rule ensures every execution checkpoint is a genuinely self-contained case -- eliminating the implicit shared state that makes retrieval unreliable and failure non-localizable in standard orchestration frameworks. Level 1 treats each checkpoint as a concrete case (suspended runtime); Fork implements retrieve-and-reuse, Value Override implements revision with automatic stale-boundary propagation. Level 2 treats each compiled plan as an abstract case; the compilation pipeline is itself a NormCode plan, enabling recursive case learning. Three structural properties follow: (C1) direct checkpoint inspection; (C2) pre-execution review via compiler-generated narrative; (C3) scope-bounded selective re-execution. Four deployed plans serve as structured evidence: PPT Generation produces presentation decks at ~40s per slide on commercial APIs; Code Assistant carries out multi-step software-engineering tasks spanning up to ten reasoning cycles; NC Compilations converts natural-language specifications into executable NormCode plans; and Canvas Assistant, when connected to an external AI code editor, automates plan debugging. Together these plans form a self-sustaining ecosystem in which plans produce, debug, and refine one another -- realizing cumulative case-based learning at system scale.
Figures & tables
Figure 1: NormCode syntax and scope rule. Left: annotated .ncds ; right: isolation boundary at runtime. Each node’s tensor is self-contained (C1).
Figure 2: Two-level CBR architecture. Level 2 (top): abstract cases (plans); Level 1 (bottom): concrete cases (suspended runtimes). Route A distills experience into plans; Route B instantiates plans to generate Level 1 cases.
Figure 3: NormCode Canvas (v1.1.3). The Graph Canvas (center) renders the full inference graph; the Detail Panel (right) inspects any node’s bounded inputs — failure localization (C1).
Figure 4: Three trace views in NormCode Canvas. (a) Agent-centric: LLM call detail with prompt input and structured output. (b) Graph-centric: scoped reference data between selected node indices (C1). (c) Orchestration-centric: execution log with error chain and cycle-level breakdown.
Figure 5: Core excerpts from three production NormCode plans as Level 2 cases ( .ncds ); actual production plans are larger and more complex. Left: NC Compilations — sequential compilation pipeline (Sect. 5.3 ). Center: PPT Generation — nested dual-output loop (HTML + PPTX per slide). Right: Code Assistant — ReAct outer loop [ 25 ] with inner explore, plan, edit, and verify stages. <- Value; <= Functional; <* Context. All three plans are reviewable as explicit plans before any run begins (C2).
Figure 6: PPT Generation plan output. (a) Six slides from a Chinese-language run; each slide is a Level 1 checkpoint. (b) Client interface showing step-by-step progress and downloadable .pptx / .html output.
Runs on qwen-turbo / qwen-plus commercial APIs with negligible cost; produces a full presentation at ∼40 s per slide.
Code Assistant
Software-engineering workflow; ∼40 inferences
ReAct outer loop with parallel Explore
Executes up to 10-step reasoning cycles to carry out complex code editing tasks with scoped explore, plan, implement, and verify stages.
NC Compilations
Self-hosted compilation; ∼25 inferences
Sequential pipeline (Derivation → Activation)
With qwen-plus / GLM-4.7 , produces within ∼15 steps executable plans for report writing, paper drafting, and contract review that run directly as new NormCode plans.
Canvas Assistant
Built-in chat controller; ∼20 inferences
Interactive command-dispatch loop
Handles a wide range of canvas tasks; when connected to an AI code editor (Cursor, Claude Code), enables automated plan debugging by overtaking the canvas interface on the user’s behalf.
Table 1: Four deployed NormCode plans: workflow class, scale, and operational status.
Figure 7: The .ncn narrative compiled from a .ncds source. Left: three-marker source. Right: deterministic plain-prose rendering used for pre-execution review (C2). A domain expert reads the narrative to verify that the plan’s logic matches intent before any LLM call.
Every major LLM agent framework gives the LLM the role of orchestrator; the model decides what to do next, when to call tools, and when to stop. We argue that token explosion, control-flow hallucination, and unreliable completion are not implementation bugs but architectural consequences of assigning the deterministic work of looping, branching, and sequencing to a probabilistic system. A better prompt or a stronger model cannot guarantee the reliability of the LLM agent. We therefore propose Agentic Programming, in which the program governs all control flow, and the LLM is itself part of it, an adaptive component we call LLM-as-Code and invoke only where a task calls for reasoning or generation. Within each call the model keeps full flexibility, but it cannot alter the program's execution path. With control in the program, the LLM's context is built from the execution history's call tree and forms a directed acyclic graph (DAG). Each call's context length is then determined by its call depth rather than by accumulation over steps. A case study of computer-use agents shows that the design is practical, not just a theoretical stance, substantially improving the stability of long visual operation sequences.
Humans solve problems by executing targeted plans, yet large language models (LLMs) remain unreliable for structured workflow execution. We propose RunAgent, a multi-agent plan execution platform that interprets natural-language plans while enforcing stepwise execution through constraints and rubrics. RunAgent bridges the expressiveness of natural language with the determinism of programming via an agentic language with explicit control constructs (e.g., IF, GOTO, FORALL). Beyond verifying syntactic and semantic verification of the step output, which is performed based on the specific instruction of each step, RunAgent autonomously derives and validates constraints based on the description of the task and its instance at each step. RunAgent also dynamically selects among LLM-based reasoning, tool usage, and code generation and execution (e.g., in Python), and incorporates error correction mechanisms to ensure correctness. Finally, RunAgent filters the context history by retaining only relevant information during the execution of each step. Evaluations on Natural-plan and SciBench Datasets demonstrate that RunAgent outperforms baseline LLMs and state-of-the-art PlanGEN methods.
Arunabh Srivastava, Mohammad A., Khojastepour +2
Amir · University of Maryland, College Park, MD · 2NEC Laboratories America, Inc.
Can LLM agents explore codebases and reason about code semantics without executing the code? We study this capability, which we call agentic code reasoning, and introduce semi-formal reasoning: a structured prompting methodology that requires agents to construct explicit premises, trace execution paths, and derive formal conclusions. Unlike unstructured chain-of-thought, semi-formal reasoning acts as a certificate: the agent cannot skip cases or make unsupported claims. We evaluate across three tasks (patch equivalence verification, fault localization, and code question answering) and show that semi-formal reasoning consistently improves accuracy on all of them. For patch equivalence, accuracy improves from 78% to 88% on curated examples and reaches 93% on real-world agent-generated patches, approaching the reliability needed for execution-free RL reward signals. For code question answering on RubberDuckBench Mohammad et al. (2026), semi-formal reasoning achieves 87% accuracy. For fault localization on Defects4J Just et al. (2014), semi-formal reasoning improves Top-5 accuracy by 5 percentage points over standard reasoning. These results demonstrate that structured agentic reasoning enables meaningful semantic code analysis without execution, opening practical applications in RL training pipelines, code review, and static program analysis.