cs.CVAug 19, 2026
SaveBIMScript: Material-Aware Structured Scene Programs for BIM Ingestion
Organizations: AI:Xpertise Lab, Aalborg University, Denmark · Visual Analysis and Perception Laboratory, Aalborg University, Denmark · Pioneer Centre for Artificial Intelligence, Denmark
Abstract
Structured-language models such as SceneScript reconstruct a scene as a short program of parametric commands, an inherently editable and semantically explicit representation. We ask three questions that stand between such models and their most compelling application, automated ingestion of existing buildings into BIM tools, studied here on synthetic scans: what is the scene made of, how fast can it be produced, and exactly where is each element. BIMScript answers all three within one grammar. First, we extend the layout language with per-element material and condition attributes, supervised by a vision-language-model material-passport corpus we build over 100k synthetic scenes (1.9M pseudo-labeled elements), and route image appearance to the material tokens through a lifted-feature point encoder. Second, we show that autoregressive decoding of these programs is dominated not by compute but by kernel-launch and host-synchronization overhead, and remove it with an output-exact CUDA-graph decoder (1.9 vs 6.4,ms/step, ) plus a grammar-parallel, tolerance-verified draft-and-verify scheme that exploits the deterministic entity schema. Third, we address the model's 5cm token-grid granularity with training-free geometric snapping and a hybrid discrete--continuous decoder head that regresses a sub-bin offset, and measure how much of the residual error each recovers. Because each command maps one-to-one onto a native Revit object, we validate direct ingestion into a BIM authoring tool end to end with a working add-in and its IFC4 export, and the same program's language form is designed to support LLM-driven, sustainability-aware reasoning over the built asset.
Figures & tables
Figure 1: BIMScript turns a scan into a material- and condition-aware BIM program. A point cloud and its egocentric keyframes are encoded jointly; the decoder emits a structured program in which every element carries geometry, material, and condition, and each command maps one-to-one onto a native Revit/IFC object.
| Model | Params (M) | F1@5cm | AvgF1 | AvgF1 (local-FSS) | ||
|---|---|---|---|---|---|---|
| Enc. | Dec. | Total | ||||
| Public ckpt | 10.47 | 15.38 | 25.85 | 0.581 | 0.667 | 0.931 |
| Ours (640-bin grid) | 20.15 | 36.42 | 56.57 | 0.554 | 0.658 | 0.930 |
| Ours (2048-bin grid) | 20.15 | 37.86 | 58.01 | 0.570 | 0.672 | 0.896 |
| SceneScript Table 2 | 0.848 | 0.784 | – | |||
| FSS Table 1 (SSD, ) | – | 15.05 | – | – | – | 0.912 |
Table 1: Reproducing the SceneScript point-only encoder–decoder from public artifacts for layout estimation. Top : 1,000 held-out test scenes, same evaluator, one greedy decode per model, so F1@5cm and both AvgF1 columns come from a single set of predictions. “Local-FSS” is our reimplementation of Fast SceneScript’s 20-threshold average ( Section 3.3 ); parameters are measured from the instantiated models. Bottom : numbers as published, on each paper’s own split and evaluator, neither of them public.
Figure 2: BIMScript overview. A lifted-feature encoder routes image appearance into the point encoder, so the decoder can condition each material and condition token on the element’s own geometry tokens. Decoding is accelerated by GraphStep and SchemaDraft and refined below the token grid by SubBin ( Fig. 4 ), and every emitted command maps one-to-one onto a native Revit/IFC object.
Figure 3: A decoded BIMScript program over the input point cloud, wall faces colored by predicted material and openings by role. On this 18-element, three-room validation scene every ground-truth element is recovered (coverage , F1@5cm ), and of carry the correct material, at condition accuracy. The one visible attribute error, the tile wall at the rear predicted wallpaper , is the frequent-class collapse that Table 4 quantifies at corpus scale.
Figure 4: The three FastBIMScript mechanisms. GraphStep collapses a decode step whose cost is kernel-launch and host-synchronization overhead into a single captured CUDA graph, emitting identical tokens. SchemaDraft exploits the fact that a command token fixes the types of all remaining parameter slots: it drafts them in one forward pass with a learned [INFILL] embedding ( [I] ) and accepts the longest verified prefix. SubBin keeps the coarse token for structure and adds a bounded within-bin offset, so a coordinate is no longer pinned to the 5cm grid.
| Decode step variant | ms/step | Whole-scene decode | speedup |
|---|---|---|---|
| reference loop (recompute) | 6.40 | reference (1.31 s/scene) | |
| KV cache (growing buffers) | 5.43 | KV cache | |
| KV cache (static buffers) | 5.49 | + GraphStep (graph) | |
| + GraphStep (graph) | 1.91 | + SchemaDraft ( ) |
Table 2: Decode acceleration, all measured on A100, batch 1, real scenes. Left : per-step decode cost; per-step time is flat in sequence length, so the eager graph gap is overhead, not compute. Right : whole-scene wall-clock factors (64 scenes, greedy), smaller than per-step factors because sequence assembly and grammar bookkeeping run outside the graph.
| Encoder | Appearance source | Coverage | Material | Condition | ||
|---|---|---|---|---|---|---|
| Accuracy | Macro-F1 | Accuracy | Macro-F1 | |||
| point-only | none (geometry) | 0.269 | 0.326 | 0.259 | 0.715 | 0.322 |
| lifted | raw per-point RGB | 0.276 | 0.305 | 0.220 | 0.701 | 0.324 |
| lifted | ResNet-50 layer2 | 0.260 | 0.347 | 0.239 | 0.734 | 0.337 |
| lifted | ResNet-50 layer3 | 0.259 | 0.341 | 0.256 | 0.729 | 0.348 |
| lifted | DINOv3 ViT-S/16 | 0.253 | 0.370 | 0.245 | 0.742 | 0.356 |
Table 3: Appearance-source ablation for material and condition. All six arms share one 2,250-scene training subset, one 20k-step budget, and the same 250 validation scenes, so the comparison isolates the appearance source. Condition macro-F1 is over the three classes every arm’s matched population realizes ( good , new , worn ). Single-seed throughout, so we read the ordering as evidence rather than a settled ranking.
| F1@5cm | AvgF1 | Material | Condition | Model | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Model | wall | door | window | mean | ours | FSS | Cov. | Acc. | F1 | Acc. | F1 | Par. | s/sc. ∗ |
| Public ckpt | 0.621 | 0.720 | 0.400 | 0.581 | 0.667 | 0.931 | – | – | – | – | – | 25.9M | 1.47 |
| BIMScript (40k) | 0.509 | 0.752 | 0.469 | 0.577 | 0.676 | 0.907 | 0.701 | 0.629 | 0.285 | 0.842 | 0.352 | 58.4M | 1.14 |
| BIMScript + (60k) | 0.513 | 0.752 | 0.462 | 0.576 | 0.669 | 0.897 | 0.699 | 0.639 | 0.291 | 0.840 | 0.350 | 58.5M | 1.12 |
Table 4: The full BIMScript grammar against its reproduction anchor, on the -scene test split. Public ckpt (anchor) emits layout only, so its attribute cells are empty by construction. BIMScript + continues the 40k model for 20k steps with the SchemaDraft and SubBin objectives ( Section 5 , Section 6 ). Layout is scored over all ground-truth elements, attributes only over elements matched within 10 cm ( Section 7 ). Latency is batch 1 with the GraphStep decoder on one L40S, the production path; all other metrics come from one greedy decode per scene at batch 4. ( ∗ Scene latency tracks emitted program length: the public checkpoint’s batch-1 decode emits more tokens per scene than BIMScript on the same scenes, while per token its 4-layer decoder is about faster than our 8-layer one.)
Figure 5: A decoded BIMScript program ingested into Autodesk Revit by our pyRevit add-in. The imported plan consists of native Revit walls, here an instance of the auto-created type BIMScript - concrete , and every element carries its predicted material, condition, and source id as shared parameters.
| BIMScript | Revit API target | IFC entity / property set |
| make_wall | Wall.Create (curve, level, height) | IfcWall |
| make_door | hosted FamilyInstance on host wall | IfcDoor + IfcRelFillsElement |
| make_window | hosted FamilyInstance on host wall | IfcWindow + IfcRelFillsElement |
| material=… | wall/family-type material parameter | IfcMaterial / IfcMaterialLayerSet |
| condition=… | shared parameter (custom) | custom Pset ( e.g . Pset_Condition ) |
Table 5: BIMScript-to-BIM mapping. Both Revit add-in ( Fig. 5 ), and the IFC exporter through Revit’s own IFC4 exporter, whose output we validate independently for element classes, hosting relations, and the BIMScript property set on the evaluated scenes.
Appendix figures & tables6 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 6: Material-passport extraction on one scene. Surface points are associated to elements, the observing keyframes are projected onto each element to cut per-element crops, and the fused per-element label is written back as a material- and condition-annotated floor plan. The crops are what the vision-language model actually sees, which is why occlusion filtering ( Section 4.3 ) matters: an unfiltered crop can show the wrong surface entirely.
| Corpus | Materials | Conditions | |||
|---|---|---|---|---|---|
| Scenes (total) | 99,990 | aluminum | 20.5% | good | 83.0% |
| skipped/missing | 10 | wallpaper | 19.1% | new | 12.9% |
| train split | 97,990 | painted_plaster | 14.0% | worn | 3.7% |
| val split | 1,000 | wood_paneling | 13.1% | damaged | 0.05% |
| test split | 1,000 | wood | 10.3% | unknown | 0.4% |
| Elements | 1,900,908 | composite | 9.1% | ||
Table 6: The BIMScript dataset and material-passport corpus extracted from ASE dataset. Cross-cue agreement indicates label uncertainty ( Section 4.2 ).
| MAE (cm) | mean F1 | ||||
|---|---|---|---|---|---|
| Method | wall | door | window | F1@1cm | F1@2cm |
| no refinement (5cm grid) | 8.74 | 1.74 | 2.77 | 0.001 | 0.034 |
| + geometric snapping | 8.37 | 1.74 | 2.75 | 0.001 | 0.034 |
| + SubBin offset head | 8.61 | 1.56 | 2.68 | 0.004 | 0.071 |
Table 7: Sub-bin refinement, conditional on correct detection (64 scenes, shared detections, matched elements). Per-type coordinate MAE (cm) and mean F1 at tight thresholds.
Figure 7: Validation F1@5cm over training for the two language grids (in-run 64-scene slice, identical data, recipe, and step count). The finer grid starts slower, since each coordinate needs more optimization, and then overtakes once the model’s residual token error shrinks to a few bins, which is exactly where grid pitch decides whether that error passes or fails the 5cm test. The two curves end within the slice’s own eval-to-eval noise, consistent with the small margin in Table 1 .
Figure 8: Qualitative layout results on the first three validation scenes: ground truth, BIMScript prediction, and a zoomed corner inset (GT solid, prediction dashed); doors are drawn as swing arcs, windows as blue sill glyphs. Rows 1–2 show typical centimeter-level corner agreement; row 3 shows a residual corner offset, and row 1 a missed wall segment, the failure modes behind the strict-F1 numbers in Table 1 .
Figure 9: Decoded programs rendered in 3D over the input point cloud. Geometry tracks the ground truth closely, and openings are placed on the correct host walls. The materials show both regimes behind the numbers in Tables 3 and 4 : scene 1 recovers exactly the five materials present in its ground truth, whereas scene 2 collapses tile and wood paneling into the corpus-frequent wallpaper , which is what a material accuracy of alongside a macro-F1 of looks like element by element. Door and window materials ( composite , aluminum ) are recovered essentially always, since they are near-deterministic given the element class.