cs.SESep 24, 2026
SaveRequirement-Bound Verified Commissioning: A Frozen Four-Billion-Parameter Local Model as a Candidate Generator under an External Acceptance Layer with Verification and Release Authority
Organizations: PythaLab, Yıldız Technical University Istanbul, Turkey
Abstract
An acceptance protocol is developed for sensor-coordinate and polarity binding in mechatronic commissioning. Candidate generation is separated from release authority. Requirements unsupported by a deterministic parser are routed to a frozen local language model with four billion parameters. Plans are released only when both facts can be derived by an external gate under a sealed grammar. One canonical answer is requested from a gold-standard user when eligible. The protocol was evaluated once under a criterion fixed before benchmark construction, on 144 tasks written by isolated agent contexts without access to the gate, grammar, or experimental plan. Three contributions are established. First, candidate generation and release decisions were measured separately. Fabricated ready plans were committed on 21 of 22 routed unanswerable tasks, and all were rejected. The same 83 releases were reproduced without model calls. Second, no false release was observed among 83 releases. A one-sided 95% Clopper-Pearson upper bound of 0.0354 was obtained as a diagnostic under an independent-and-identically-distributed assumption, below the sealed 5% threshold. However, one false release was subsequently recorded among 146 releases outside the benchmark at seed 0. Third, protection against incorrect user answers was characterized. Both facts were bound from the original text on 13 of 96 answerable tasks. Incorrect answers were released in 169 of 431 pairings on the remaining tasks, including failures involving coordinate exclusion. A deployable questioning policy was not tested because eligibility was determined from the answer key. Gate sensitivity and real user behavior were not measured.
Figures & tables
Figure 1: Two-turn protocol : candidate generation, user questioning, and release. Candidates are produced by the deterministic parser or the frozen model. Release is decided by the external acceptance layer. Post-run scoring against the gold answer key is omitted from the diagram, as no feedback from this scoring is supplied to the release decision.
| (a) Compositions by physical domain | |
|---|---|
| Physical domain | Compositions |
| Motion, actuation, and power transmission | Belt gantry tension, flywheel centrifugal brake, SMA wire tensioner, magnetic gear coupling, ball screw preloaded stage |
| Fluid power and hydrostatic bearings | Pump line accumulator, hydrostatic bearing pad, servo valve spool chamber |
| Thermal, process, and vacuum systems | Thermoelectric load plate, battery pack cold plate, recycle loop reactor, pressurised headbox, boiler drum natural circulation, vacuum chamber outgassing pumpdown |
| Vehicle, marine, flight, and hydropower systems | Wheel slip brake line, ship rudder yaw axis, short period pitch axis, penstock turbine governor |
| Microelectromechanical systems (MEMS) and precision transducers | Coriolis rate gyroscope, comb drive electrostatic actuator, inkjet piezo chamber |
Table 1: Benchmark coverage : (a) physical domains of the compositions, grouped editorially rather than used as experimental strata, and (b) stage-level counts. Tasks for which both facts are bound by a plan-free gate call on the unmodified requirement are denoted by . Gate pairings were counted in the specificity sweep. Shape memory alloy (SMA) is abbreviated in panel (a).
| Component | Recorded value |
|---|---|
| Model | Frozen Qwen3, four billion parameters, 4-bit quantized weights |
| Local model server | Ollama 0.32.15 |
| Execution environment | Python 3.12.3 |
| Numerical libraries | NumPy 2.2.6, SciPy 1.14.0, and SymPy 1.13.3 |
| Generation settings | Temperature 0.6, nucleus sampling threshold (top-p) 0.95, candidate-token limit (top-k) 20 |
| Context / output limit | 12288 / 4096 tokens, reasoning mode enabled |
Table 2: Experimental environment : Recorded settings for the primary run of the main experiment. Previously documented hardware is reported separately in Table A6 , as its unchanged use during the main experiment could not be established from the records.
Figure 2: Experimental and evaluation workflow : From the blind benchmark to the single sealed decision. The protocol and criterion fixed before measurement are represented by the upper seals. Scoring of released pairs against the gold answer key is represented by the main flow. Supplementary analyses excluded from the primary verdict are grouped in the lower band.
| Question | Experiment and arm | Table and figure |
|---|---|---|
| Question 1. Can a valid candidate be generated by the frozen model, and can its subsequent commitment as a ready plan be made correctly? | Main experiment, seed 17, second model | Table 5 , Figure 4 |
| Question 2. Is the false-release rate among released plans bounded under the sealed acceptance criterion? | Main experiment, two ungated arms | Table 4 , Figure 3 |
| Question 3. What shares of the delivered plans can be attributed to the model and the deterministic components? | Main experiment, model-free replay | Table 4 , Figure 5 |
| Question 4. Where is protection by the acceptance layer lost when an incorrect fact is supplied by the user, and how is that textual trust boundary defined? | Main experiment: wrong-answer and specificity sweeps. Earlier development experiment: selected strata | Table 7 , Figure 6 |
Table 3: Preregistered research questions and the corresponding experiments, arms, tables, and figures. A sealed tier verdict was assigned only for Question 2. The other questions were addressed through descriptive or secondary analyses.
| Arm | Delivered | False | Correct | Upper bound | Lower bound | Model calls | Asked |
|---|---|---|---|---|---|---|---|
| Two-turn protocol, main experiment, seed 0 | — | ||||||
| Two-turn protocol, seed 17 | — | — | |||||
| Model-free replay | — | — | |||||
| Ungated shell, separate run | — | — | |||||
| In-run ungated terminal, seed 0 | — | — | |||||
| In-run ungated terminal, seed 17 | — | — |
Table 4: Comparison across arms : Delivered plans, false releases, correct deliveries among the 96 answerable tasks, one-sided 95% Clopper–Pearson bounds (Equations ( 8 ) and ( 9 )), model calls, and tasks on which a question is asked. Separate arms and runs are reported in separate rows. Their counts must not be pooled, and their bounds must not be compared as estimates for a common population. Each bound is one-sided. Together, they do not form a two-sided 95% interval. A five-call budget was used by the ungated shell and a three-call budget by the other model-using arms. The 13 in-run terminal deliveries were counted over 144 turn-1 records, with 6 outside the answerable set, so they cannot be assigned the denominator 96. Seed 17 was used only for sensitivity. The second model was development-labeled, and no confirmatory bound was reported. Its 68 questioned tasks form , with 60 also in the release set. A dash denotes an unreported or undefined value. Bounds are displayed at the precision retained in the source records (Table A5 ).
Figure 3: Stage-level outcomes : Answerable tasks, releases, and release turns by stage, totaling 96 answerable tasks, 83 releases, 9 turn-1 releases, and 74 turn-2 releases. The ceiling is labeled “Model-free ceiling with added answer” and coincides with . Tasks released at turn 2 for which ready status was retained after the appended fact was removed are labeled “Accepted after answer removal.”
| Qwen3, seed 0 | Qwen3, seed 17 | Second model | |
| 25 routed answerable tasks | |||
| Ready plan committed | 25 | 23 | 15 |
| Plan equal to gold | 24 | 22 | 11 |
| Validity among committed plans | 96.0% | — | 73.3% |
| Lower bound for equality to gold | 0.824 | 0.810 | — |
| Episode without a plan | 0 | 2 | 10 |
Table 5: Candidate generation : Outputs on tasks routed to the model-calling branch. The same tasks were routed in all three columns because routing is determined by the parser. Seed 17 was used only for sensitivity. The second model was development-labeled. Columns must not be pooled. One-sided 95% Clopper–Pearson lower bounds are reported. A call is counted as returning no visible content only when termination at the length limit, zero visible content, and exhaustion of the fixed 4096-token limit are all recorded. Second-model episodes without plans are detailed in Appendix E . Different denominators are used in the two fabrication rows for the second model, and neither row may be substituted for the other. A dash denotes an unreported or undefined value.
| Item | Count | Share or composition |
| (a) Turn-1 routing | ||
| Tasks executed at turn 1 | ||
| Deterministic branch | ||
| Model-calling branch | ||
| Answerable | ||
| Contradictory gold status, unanswerable | ||
Table 6: Turn-1 routing and acceptance-layer decisions : Main experiment at seed 0. (a) Routing to deterministic and model-calling branches. (b) Model-committed plans and turn-1 decisions. Routing is determined by the parser. Unanswerable tasks parsed deterministically are not routed to the model, so 22 rather than 48 is used as the fabrication-rate denominator. These 48 unanswerable tasks are distinct from the 48 answerable tasks of S2. In the single seed-0 miss, T-1, a noncontact magnetic gear, input-rotor speed was proposed instead of the gold pole angle with reversed sign. The two gold-equal plans never released, the seed-17 miss, and two episodes without plans are identified in Appendix C .
Figure 4: Two mechatronic examples : Requirement-to-release traces from the sealed records. In (A), missing-fact status was assigned to the deterministic plan in both turns. The pair (liquid level, ) was released by the gate after the canonical answer was appended (Equation ( 7 )). In (B), the initial gold-equal model plan was rejected. The same pair was released after the canonical answer was appended.
Figure 5: Process contribution of the model : (A) Model exposure within the release set and (B) the two measured process-event types. Different populations are used in the panels, so bar lengths must not be compared across panels. The 83 releases in are partitioned in (A). Delivery without a question on 4 of 83 releases and a one-fact question on 11 of 96 answerable tasks are shown in (B), totaling 15 distinct process events. Without the model, a question was asked in 15 of the replay’s 80 questioned tasks where no question, or a narrower question, had been used in the live run.
| Regime | Measurement | Population | Upper bound |
| Rejected answer | 0 of 65 pairings | 13 protected tasks | |
| Trusted answer | 64 of 66 (97.0 %) | Protocol, silent substratum | — |
| 64 of 74 (86.5 %) | Gate, silent substratum | — | |
| 64 of 80 (80.0 %) | Protocol, all cell S | — | |
| Incorrect pair released | 4 of 290 pairings | Gate, cell C refusing | |
| 4 of 226 pairings | Protocol, cell C refusing |
Table 7: Textual trust boundary : Wrong-fact regimes and two selected-stratum measurements. A separate measurement, population, and one-sided 95% Clopper–Pearson upper bound are specified in each row. Counts must not be summed across these different stratifications. Each nongold coordinate is paired with both signs in cell C. The gold coordinate is paired with its reversed sign in cell S. Under the preregistered stratum-A classification, refusing and silent substrata are assigned from the reason returned for the disputed fact on the unmodified text before T1 is appended. Value mismatch, explicit exclusion without a supporting mention, and multiple remaining valid values are classified as refusing. Absence of a binding mention is classified as silent. Bounds were computed at the pairing level and are diagnostic under the i.i.d. assumption, as pairings from the same task are not independent. A dash denotes an unreported bound. Trusted-answer rows were compared with preregistered lower thresholds (Table A3 ), not upper bounds. The incorrect-pair rows concern the same 2 protected tasks with explicit coordinate exclusions. Task counts were not reported for trusted-answer rows. Selected strata were drawn from the earlier development experiment and were used as existence demonstrations, not protection-rate estimates. The preregistered pass rule was not met in the first selected-stratum measurement.
Figure 6: Bindability and release : Answerable tasks partitioned by textual support and decision. Binding of both facts by a plan-free gate call on the unmodified requirement is labeled “Text-bound.” Its complement, , is labeled “text-open.” From left to right, 13 text-bound and released tasks, 70 text-open and released tasks, and 13 text-open tasks with abstention are shown. The 83 tasks in are marked by the upper bracket, and the 83 tasks in by the lower bracket. The sets overlapped on 70 tasks.
| (a) Post-seal campaign, 252 tasks | ||
| Measurement | Seed 0 | Seed 17 (sensitivity) |
| Released plans | 146 | 143 |
| False releases | 1 | 0 |
| Upper bound | ||
| Lower bound | — | |
| Raw label, both tiers | supported | — |
Table 8: Post-seal confirmation campaign : (a) Results for the unchanged sealed method on 252 tasks outside the main benchmark and (b) measurements of incorrect ready decisions and clauses from which no polarity cue was extracted. The campaign was conducted after the main experiment had closed and was excluded from the sealed final look. Its results did not alter the primary tier verdict. Counts must not be pooled with Table 4 , and bounds must not be compared across those populations. One-sided 95% Clopper–Pearson bounds are reported. The guarded label is defined by the guard in Equation ( 11 ). A single false release is necessarily contained in one composition, so inconclusive is assigned whenever , regardless of . Seed 17 was used only for sensitivity and was not pooled with seed 0. In the first row of (b), requirements are evaluated by the gate alone, without model calls or fact augmentation. Tasks with incorrect ready status are counted, not all tasks containing the mechanism. Polarity cues are extracted from rise/fall clauses by grammar V1’s monotonic pattern only when recognized head nouns are used. The denominator in the second row of (b) is all tasks. Ready gold status was assigned to every identified task, but no cue was extracted from the relevant clauses, and those tasks remained unbindable. Contradictions expressed without a rise/fall relation cannot be detected by this lexical sweep, so the mechanism’s occurrence may have been undercounted. A dash denotes an unreported value.
| Approach | Location of authority | Basis of the decision |
|---|---|---|
| Simplex and runtime assurance ( Seto et al., 1998a ; Sha, 2001a ; Bak et al., 2009a ) | Verified decision module | Conditions on physical dynamics |
| Shielding ( Alshiekh et al., 2018a ) | Action-constraining shield | Formal shield synthesis |
| Candidate generator with an external critic ( Kambhampati et al., 2024a ) | External sound critic | Verification by the critic |
| Self-verification ( Stechly et al., 2025a ) | Candidate-generating model | Model-side checking of its own output |
| Present protocol | External deterministic entailment gate | Derivability of both facts from the requirement and user fact under grammar V1 |
Table 9: Decision authority : Location and basis of decisions in the compared approaches. No ranking under a common performance metric is implied.
Appendix figures & tables8 assets
Supplementary material from the paper’s appendix.
Appendix
| Arm | Design | Population | Measurement | Location |
|---|---|---|---|---|
| 1 | Seed-17 sensitivity run | Protected benchmark, seed 17 | Seed sensitivity of the sealed result | § 3.2 |
| 2 | Ungated shell, separate run | Protected benchmark, separate run | Ungated delivery and false release | § 3.2 |
| 3 | Specificity sweep | 4645 gate pairings | Ready status on mismatched requirements | § 3.4 |
| 4 | Wrong-fact instrument | Selected stratum from the earlier development experiment | Rejection of a text-contradicting fact | § 3.4 |
| 4b | Wrong-polarity instrument | Selected complementary stratum | Acceptance of a fact left open by text | § 3.4 |
| 5 | Model-free replay | Protected benchmark, deterministic | Measured model contribution | § 4 |
Table A1: Experimental arms : Population, intervention, and reporting location for each comparison. Different quantities were measured when the questioning, budget, or user-answer conditions were changed. Counts are therefore not pooled, seed 17 is never combined with seed 0, and no common total or single verdict is implied. Arm 1 was executed under the second preregistration layer after the sealed final look.
| Label | Composition / physical system | Role |
|---|---|---|
| T-1 | Magnetic gear coupling, a noncontact gear assembly | Only candidate-generation miss at seed 0. Proposed input speed , gold pole angle |
| T-2 | Thermoelectric load plate with a Peltier module | Seed-17 miss |
| T-3 | MEMS comb drive electrostatic actuator | Episode without a plan at seed 17 |
| T-4 | Wheel slip brake line in a vehicle | Release despite explicit exclusion |
| T-5 | Penstock and turbine governor | Release despite explicit exclusion |
| T-6 | Pressurised paper-machine headbox | Release without a question |
Table A2: Task examples : Labels, physical systems, and experimental roles for tasks named in the main text or used to illustrate a count. Task identity and role are recorded without stage or seed columns. Multiple rows may be associated with the same composition under different requirements and different release outcomes.
| Prediction | Measurement | Verdict |
| Arm 6, in-run ungated terminal | ||
| At least 1 false delivery and | 6 false of 13 deliveries over 144 turn-1 records, | Met |
| At least 6 false deliveries | 6 of 13 deliveries | Met at the threshold |
| At most 9 correct deliveries | 7 of 96 answerable tasks | Met |
| Arm 7, wrong-answer sweep | ||
| P1 0 releases in cell C refusing | 4 of 290 gate, 4 of 226 protocol | Falsified |
Table A3: Preregistration outcomes : Every closing-arm prediction, measurement, and verdict, including failures. Each verdict is restricted to its own row. Unmet predictions are marked in bold (§ 3.5 ). The counting level is specified in each measurement, so no separate population column is used. P2b and false-refusal results were measured after campaign closure because no verdict was reported in the arm outputs. The single verdict under the primary sealed criterion is not replaced by this table.
| Field | Sealed primary model | Second model, development-labeled |
| Model | Qwen3 with four billion parameters and 4-bit quantized weights | 4-bit quantized SmolLM3 model in the three-billion-parameter (3B) class, distributed through Hugging Face |
| Model fingerprint | Fingerprint retained in the sealed record for matching weights to the archive | Distinct fingerprint retained in the sealed record |
| Local runtime version | 0.32.15 | 0.32.15 |
| Sampling temperature | 0.6 | 0.6 |
| Nucleus sampling threshold | 0.95 | 0.95 |
| Candidate-token limit | 20 | 20 |
Table A4: Model configurations : Generation settings for the two frozen models, as retained in the sealed records. All inference settings other than model identity and model fingerprint were identical. This equality does not extend to parameter counts or measured model-call counts. No exact parameter count was recorded for the primary model, so a dash is used. The second arm’s development label applies to every result in its column.
| Quantity | Recorded value | Displayed form | Source |
| Clopper–Pearson bounds | |||
| Sealed final look | |||
| Arm 6, seed 0 | |||
| Arm 6, seed 0 | |||
| Arm 6, seed 17 | |||
| Arm 6, seed 17 | |||
Table A5: Numerical presentation : Values retained in the source records for Clopper–Pearson bounds and release rates, alongside their displayed forms. No new calculation is introduced. Bounds in the main text are displayed to at most four decimal places. Each row must be interpreted with its arm and seed. The rate values were retained at the precision recorded in the source and must not be interpreted as exact fractions.
| Component | Recorded value | Record date |
|---|---|---|
| Processor | Intel Core i7-12700H, 20 logical central processing units (CPUs) | 12 September 2026 |
| System memory | 31.2 GiB total reported to the server | 1 September 2026 |
| Graphics processing unit (GPU) | NVIDIA GeForce RTX 3060 Laptop GPU | 13 September 2026 |
| GPU memory / driver | 6144 MiB / 591.74 | 13 September 2026 |
| Platform | Linux 6.18.33.2, Windows Subsystem for Linux 2 (WSL2), x86_64, glibc 2.39 | 12 September 2026 |
Table A6: Hardware environment : Dated hardware inventory supplementing the main-experiment environment. These records were obtained before, and separately from, the main runs. A record date is specified for each row. System memory is the total reported to the server, not a nominal installed-memory capacity. Unchanged use of this hardware during the main experiment could not be established through a persistent machine identity. Memory units are gibibytes (GiB) and mebibytes (MiB). The GNU (GNU’s Not Unix) C Library is denoted by glibc.
| Research line | Architecture | Evidence setting | Present addition |
|---|---|---|---|
| Simplex and runtime assurance ( Seto et al., 1998a ; Sha, 2001a ; Bak et al., 2009a ) | Untrusted component behind a verified decision module | Control systems without a language-model component | Language-model component and measured false-release upper bound |
| Safety for learned components ( Phan et al., 2020a ; Alshiekh et al., 2018a ) | Neural Simplex: transfer to a safe controller. Shielding: action restriction | Runtime assurance and formal shield synthesis | Entailment from textual requirements, without a physical safety result |
| Candidate generator and external verifier ( Kambhampati et al., 2024a ; Stechly et al., 2025a ) | Model-generated candidates checked by an external sound critic | Reasoning and planning tasks | Sealed criterion, blind benchmark, false-release upper bound |
| Language model and classical planner ( Liu et al., 2023a ) | Model-generated PDDL description solved by a classical planner | Plan generation and planning benchmarks | Release conditional on candidate derivability from text |
| Rails and control planes ( Rebedea et al., 2023a ; Reddy et al., 2026a ; Madatha, 2026a ) | Programmable checks, with some NeMo rails based on LLMs | Policy and safety checks | Deterministic entailment and a measured boundary for trust in user facts |
| Abstention ( Rajpurkar et al., 2018a ; Kirichenko et al., 2025a ; Wen et al., 2025a ) | Model-side abstention | Question answering, language-model benchmarks, and a method survey | Candidate generation and commitment measured on the same routed set |
Table A7: Positioning in the literature : Decision authority, evidence type, and the present scope. No ranking under a common performance metric is implied. Separation of candidate generation from release authority is the shared architectural feature. A sealed evaluation of that division and a measured textual trust boundary are provided by the present study.
| Symbol | Meaning |
|---|---|
| Tasks, texts, and sets | |
| A task | |
| Protected task set | |
| Natural-language requirement of task | |
| Sensor coordinates named in the task’s sensor cards | |
| Text evaluated by the gate | |
Table A8: Symbols : Notation for tasks, plans, the gate, the protocol, and statistical decisions, grouped by object. Definitions are supplied below the corresponding equations or in the associated sections. Status, branch, and decision symbols are included in the formal definitions and referred to by name in the prose. A single quantity is assigned to each symbol in the equations and algorithms. Placement U is a rule name.