Learning to Edit IFC Building Models: Verifiable Checkers as the Training Signal for Dependency-Aware Editing Agents
Organizations: Architecture and Design College, Nanchang University, Nanchang, China · School of Architecture and Urban Planning, Chongqing University, Chongqing, China · Key Laboratory of New Technology for Construction of Cities in Mountain Area (Ministry of Education), Chongqing University, China · Department of Intelligent Media, SANKEN, The University of Osaka, Osaka, Japan
Abstract
Editing a building information model in the Industry Foundation Classes (IFC) schema requires propagating changes to dependent elements while preserving unrelated entities. Existing language-model agents for IFC editing use prompted models without learning from edit outcomes. Target-file comparisons do not specify which alternative edits are allowed or which unrelated entities must remain unchanged. We present IfcGym, a CodeAct environment in which a language model edits IFC through executable code and four checkers verify validity relative to the input model, intent, dependency and preservation against a task specification. Their conjunction is strict success. IfcEdit distils verified teacher episodes into a Qwen3.5-9B student through supervised fine-tuning (SFT), then applies Group Relative Policy Optimization (GRPO) with strict success as the reward. On IfcEditBench, 1,990 editing tasks on 13 held-out real projects, the final student reaches 86.78% strict success, from 79.15% after SFT. The paired confidence interval for the difference from its teacher includes zero; the same recipe on a 27B model reaches 91.76% and exceeds the teacher. On the public benchmarks BIM-Edit and BIBIMBAP the final student scores 55.19 and 54.50. On IfcEditBench, the students outperform all prompted models except the teacher under matched tool access; both evaluated frameworks also score lower. Rewarding all four checkers yields 4.83 points more strict success than rewarding intent and validity alone.
Figures & tables
| Group | Helpers |
|---|---|
| Inspection (read-only) | inspect , find , unit_scale , world_matrix , wall_axis_world , profile_world_polygon , wall_length , space_area , check_model |
| Element-local edits | translate , move_wall_end , stretch_wall_end , shift_vertices_on_plane , shift_profile_vertices , shift_boundary_curve , set_extrusion_depth , set_wall_thickness , rotate , set_property , set_attribute , delete , connect_walls , private |
| Creation | add_wall , add_slab , add_column , add_space , add_opening_fill |
| Dependency tier What must follow | Reference How the target is named | ||
| Tier 0 | attribute or property of one element; only the target may change | Direct | by its GlobalId |
| Tier 1 | translate a wall with openings, delete a door, change a wall height, create an element; openings and fills translate with the wall and stay hosted; by the task’s policy a deleted door’s opening is removed and its host wall stays | Spatial | by geometry relative to a named entity: the nearest parallel wall north of a wall, the window on the storey above |
| Tier 2 | translate a wall that bounds spaces; as tier 1, and each bounded space’s face on the wall side meets the moved wall, its space boundary moving with it | Topological | through model relationships: the space bounded by a given set of walls, the wall that hosts a given door |
| Tier 3 | translate a wall joined to other walls; as tier 2, and each joined wall stays joined to the moved wall | Path | along a route through the building: after entering a space through a given door, the wall on your left |
| Viewpoint | as seen from a standpoint: the wall facing you when looking in through a given window | ||
| Operation What the task asks for | Target element What is targeted | ||
| Instruction reference | Operation | |||||||||
| Model and training | All | Direct | Spatial | Topol. | Path | Create | Update | Delete | Retr. | Comp. |
| 1,990 | 1,503 | 217 | 193 | 70 | 355 | 1,019 | 331 | 200 | 85 | |
| Prompted models | ||||||||||
| DeepSeek-V4-Flash [ 5 ] | 61.11 | 56.9 | 76.0 | 81.9 | 50.0 | 61.1 | 62.4 | 51.7 | 82.0 | 32.9 |
| Gemma 4 31B [ 12 ] | 67.99 | 66.1 | 74.7 | 83.4 | 44.3 | 60.8 | 64.7 | 88.2 | 75.0 | 42.4 |
| Kimi K3 [ 27 ] | 88.04 | 86.6 | 94.0 | 92.7 | 85.7 | 89.9 | 88.1 | 88.5 | 89.5 | 74.1 |
| Model and training | Valid. | Intent | Dep. | Pres. |
|---|---|---|---|---|
| Prompted models | ||||
| DeepSeek-V4-Flash [ 5 ] | 96.0 | 68.3 | 80.4 | 96.4 |
| Gemma 4 31B [ 12 ] | 96.5 | 77.5 | 90.3 | 93.0 |
| Kimi K3 [ 27 ] | 99.4 | 91.8 | 95.7 | 96.4 |
| Qwen3.5-9B [ 48 ] | 97.8 | 19.5 | 56.7 | 97.7 |
| Qwen3.5-27B [ 48 ] | 98.8 | 65.6 | 80.9 | 95.7 |
| Category (20 tasks each) | Operation | |||||||||
| Model | Direct | Spatial | Topol. | Numeric | Concept. | Create | Read | Update | Delete | All |
| Prompted models | ||||||||||
| Claude Opus 4.6 [ 1 ] † | 65.83 | 63.89 | 53.06 | 38.33 | 30.00 | 81.1 | 47.3 | 61.5 | 18.9 | 50.22 |
| GPT-5.4 [ 47 ] † | 62.15 | 42.08 | 20.56 | 26.67 | 5.00 | 56.0 | 22.4 | 50.4 | 20.0 | 31.29 |
| Gemini 3 Flash [ 15 ] † | 65.42 | 40.97 | 21.11 | 20.00 | 6.67 | 45.6 | 20.6 | 46.7 | 37.8 | 30.83 |
| GPT-5.2 [ 45 ] † | 52.50 | 34.58 | 35.42 | 5.00 | 25.00 | 58.9 | 18.2 | 51.1 | 26.7 | 30.50 |
| Model | Final | Geom. | Sem. | Topo. |
|---|---|---|---|---|
| Prompted models | ||||
| Gemini 3 Flash [ 15 ] † | 49.48 | 68.87 | 41.81 | 37.77 |
| Qwen 3.6 Plus [ 49 ] † | 47.63 | 59.21 | 35.93 | 47.77 |
| Claude Sonnet 4.6 [ 2 ] † | 45.31 | 55.90 | 33.85 | 46.19 |
| GPT-5.4 Pro [ 47 ] † | 43.94 | 50.14 | 38.69 | 42.97 |
| DeepSeek V3.2 [ 4 ] † | 43.21 | 50.97 | 34.80 | 43.86 |
| Project | Schema | Unit | Authoring tool | Products | Walls | Doors | Windows | Openings | Slabs | Columns | Spaces | Storeys |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 4351 | IFC2X3 | ft | Revit Architecture 2011 | 441 | 43 | 17 | 3 | 32 | 6 | 0 | 6 | 5 |
| city_house_munich | IFC4X3 | m | Revit 2025 | 293 | 83 | 16 | 13 | 33 | 19 | 0 | 19 | 15 |
| dental_clinic | IFC2X3 | m | Revit Architecture 2011 | 3,298 | 1,080 | 254 | 58 | 403 | 3 | 0 | 269 | 4 |
| digital_hub | IFC4 | m | Revit 2019 | 1,026 | 178 | 64 | 47 | 244 | 23 | 62 | 64 | 3 |
| duplex | IFC2X3 | m | Revit Architecture 2011 | 295 | 57 | 14 | 24 | 50 | 21 | 0 | 21 | 4 |
| fantasy_hotel_1 | IFC4 | m | Revit 2024 | 1,082 | 53 | 25 | 73 | 138 | 49 | 39 | 26 | 4 |
| Project | Wall bodies | Connections | Boundaries | Voids | Fills | Tasks T0/T1/T2/T3 | Total |
|---|---|---|---|---|---|---|---|
| 4351 | S 58, C 37, M 5 | 65 | 142 | 32 | 20 | 22/153/16/16 | 207 |
| city_house_munich | S 53, C 47 | 93 | 0 | 33 | 26 | 22/174/0/37 | 233 |
| dental_clinic | S 96, C 2, M 1 | 2,103 | 3,124 | 403 | 302 | 23/184/32/16 | 255 |
| digital_hub | S 81, F 19 | 0 | 0 | 244 | 109 | 14/69/0/0 | 83 |
| duplex | S 91, C 7, M 2 | 82 | 265 | 50 | 38 | 18/166/15/14 | 213 |
| fantasy_hotel_1 | S 64, F 36 | 0 | 0 | 138 | 98 | 15/68/0/0 | 83 |
| Target element | Create | Update | Delete | Retrieve | Total |
|---|---|---|---|---|---|
| Wall | 336 | 4,581 | 296 | – | 5,213 |
| Slab | 663 | 1,204 | 382 | – | 2,249 |
| Column | 503 | 609 | 452 | – | 1,564 |
| Door | 530 | 253 | 740 | – | 1,523 |
| Window | 551 | 755 | 625 | – | 1,931 |
| Opening | – | 216 | 206 | – | 422 |
| Intent | Test | Tolerance |
|---|---|---|
| translate | centroid displacement equals the requested vector | 0.02 m |
| set_attribute, set_property | the attribute, or the property in the named property set, equals the value (numbers: relative tolerance) | exact; 0 |
| delete, delete_all | the product (every listed product) no longer exists | – |
| resize_height | mesh height equals the requested height; where given, the top or bottom elevation as well | 0.02 m |
| wall_axis (shorten, rotate) | axis end points match the requested points in either order, and the body follows the axis: both ends moved like the nearest axis end (parallel case) or the body’s principal direction aligned with the new axis with its length kept | 0.03 m per end; 0.1 m body |
| set_thickness, set_thickness_side | profile and body thickness across the axis; for the one-sided variant the fixed face stays in place | 0.02 m |
| Rule | Test | Uses |
|---|---|---|
| translate_with | the dependent’s centroid displacement equals the target’s move vector (0.02 m) | 958 |
| stays_hosted | the opening still voids a wall and, if it had a fill in , still has one | 614 |
| removed, present | the dependent no longer exists; still exists | 487, 32 |
| space_meets_wall | along the move axis, the nearest face of the space mesh (restricted to the wall’s extent) lies within 0.02 m of a face of the moved wall, as it did in | 374 |
| boundary_on_wall | every space boundary between the space and the wall with connection geometry, transformed through the space’s placement, lies within 0.02 m of the wall’s centre or faces along the move axis, as it did in | 374 |
| unchanged | the dependent is neither removed nor modified (no attribute, property, placement or geometry change) | 353 |
| Change | How it happens | Tested by |
|---|---|---|
| openings and fills placed relative to the moved wall | follow through the placement chain when translate or rotate edits the wall’s own placement; inspect reports whether an opening or fill is placed relative to its wall | translate_with |
| openings and fills with an absolute placement | the agent translates each | translate_with |
| relationships of a deleted product | delete removes the product with its voids, fills, containment, aggregation, connection, boundary, type, material and property relationships and repairs geometry shared with survivors; it does not delete the opening of a deleted door or the openings of a deleted wall | removed, stays_hosted |
| space footprint on the moved wall face | the agent moves the vertices on that face ( shift_profile_vertices , shift_vertices_on_plane ) | space_meets_wall |
| space-boundary surface on the moved wall | the agent moves the boundary curve ( shift_boundary_curve ); it is placed relative to the space and never follows the wall | boundary_on_wall |
| joined wall ends | the agent lengthens or shortens the touching end along its own axis ( move_wall_end , stretch_wall_end ) | joined_to |
| Gate | Passes | Fails |
|---|---|---|
| Validity | the wall’s placement is edited; the validator logs the same lines as before and the wall, its opening and door rebuild | a predefined type set to a value outside its enumeration (new validator line); a profile replaced by a two-point polyline (the wall no longer rebuilds); a storey aggregated into itself (cycle) |
| Intent | the wall’s centroid moved by within 0.02 m; exactly one new door, hosted through an opening of the named wall, m within 0.03 m, inside the wall’s length | the wall moved by m; two doors created; a property written to a set whose name carries a trailing space, so the lookup finds nothing |
| Dependency | opening and door moved by the same vector and still hosted; the joined wall’s end stretched so that its surface gap to the moved wall is no larger than before | the joined wall left in place (gap 0.25 m); the door deleted instead of moved; at tier two, the space footprint or boundary curve left 0.25 m from the new wall face |
| Preservation | only the wall, its opening and door, the listed spaces and joined walls appear in the diff, and every changed relationship has an end among them | a profile shared with a second wall edited in place, so that wall’s geometry hash changes; the storey placement moved, so every element counts as modified; a new space created in a wall-creation task; a containment relationship of an unrelated element removed |
| Helper | Arguments | Returns or effect |
|---|---|---|
| Inspection read-only functions of the model | ||
| inspect | entity, [psets], [related_to] | placement chain, representation summary, openings and fills, connected walls and connection types, space boundaries and bounded spaces, shared sub-entities, property sets |
| find | [class], [storey], [name], [near, radius], [type] | the matching elements |
| unit_scale , world_matrix , wall_axis_world , profile_world_polygon | model or product | metres per model unit; world placement; wall axis and footprint profile in world coordinates |
| wall_length , space_area | wall or space | axis length; net floor area |
| check_model | path of the input model | new schema errors; entities added, removed and modified against the input |
| Stage | Setting | Value |
|---|---|---|
| SFT | adapters | LoRA on all linear layers, rank 64, scaling 128 |
| SFT | optimiser | learning rate , cosine decay, 3 epochs, batch size 16 |
| SFT | sequence length | up to 32,768 tokens; final checkpoint used without selection |
| GRPO | pool | synthetic tasks, prefiltered offline with 4 attempts of the SFT student at temperature 1; unsolved tasks capped at one fifth |
| GRPO | sampling | 64 prompts per step, rollouts each, evaluation protocol (20 rounds, no checker feedback) |
| GRPO | filtering | groups with eight equal rewards dropped and resampled until 64 mixed groups |
| Row | Weights | Evaluation | Settings beyond the protocol | Strict |
| Qwen3.5-9B (untrained) | public Qwen3.5-9B | III | – | 16.08 |
| Qwen3.5-27B (untrained) | public Qwen3.5-27B | I | – | 59.35 |
| Qwen3.5-9B, SFT | SFT, final checkpoint (3,588 steps) | I | – | 79.15 |
| Qwen3.5-9B, SFT + GRPO | GRPO step 100 from the SFT weights | I | – | 86.78 |
| Qwen3.5-27B, SFT | SFT, final checkpoint | I | – | 88.29 |
| Qwen3.5-27B, SFT + GRPO | GRPO step 100 from the 27B SFT weights | IV | – | 91.76 |
| Run | Start | Data | Compute | Served checkpoint |
|---|---|---|---|---|
| 9B SFT | Qwen3.5-9B | 19,141 episodes (3 epochs, 3,588 steps) | 8 H100 | final step, adapters merged |
| 27B SFT | Qwen3.5-27B | same | 8 H100 | final step, merged |
| 9B SFT on 25 / 50 / 75 % | Qwen3.5-9B | nested subsets of 4,785 / 9,570 / 14,356 episodes | 8 H100 | final step, merged |
| 9B GRPO (final student) | 9B SFT | pool of 5,895 tasks | 32 H100, 18 h; restarted once from step 65 | step 100, merged; steps 25, 50, 75 exported |
| 9B GRPO, intent-and-validity reward | 9B SFT | same pool | 16 H100, 23 h | step 100, merged |
| 27B GRPO | 27B SFT | same pool | 16 H100, 49 h | step 100, merged; steps 10–75 exported |
| Run | Role | MAE final | MAE geom. | MAE sem. | MAE topo. | Within 5 | Mean ours / official | Bias |
|---|---|---|---|---|---|---|---|---|
| Claude Sonnet 4.6 | calibration | 18.4 | 26.2 | 27.0 | 20.4 | 0.46 | 42.06 / 45.31 | |
| Gemini 3.0 Flash | calibration | 23.2 | 27.6 | 33.9 | 38.4 | 0.20 | 50.48 / 49.48 | |
| GPT-5.4 Pro | held-out | 20.3 | 29.9 | 28.5 | 23.8 | 0.35 | 45.26 / 43.94 | |
| GPT-5.4 Mini | held-out | 20.1 | 27.5 | 27.4 | 21.5 | 0.41 | 37.57 / 39.78 | |
| All four | 20.5 | 27.8 | 29.2 | 26.0 | 0.36 | 43.84 / 44.63 |
| Final score | Bias per axis | ||||||
|---|---|---|---|---|---|---|---|
| Operation | Official | Ours | MAE | Geom. | Sem. | Topo. | |
| Create | 25.97 | 27.14 | 14.3 | 0.75–0.78 | |||
| Delete | 50.83 | 49.59 | 8.6 | –0.79 | |||
| Update | 57.09 | 54.80 | 38.6 | –0.24 | |||
| Gemma 4 31B, prompted CodeAct | Ours | Published |
|---|---|---|
| Final score | 30.55 | 37.54 |
| Create | 0.9 | 0.6 |
| Delete | 46.8 | 54.0 |
| Update | 44.0 | 58.1 |
| Episodes executed (%) | 84.0 | 85.19 |
| Budget exhausted (%) | 10.2 | 8.3 |
| Model | Tier 0 | Tier 1 | Tier 2 | Tier 3 | All |
|---|---|---|---|---|---|
| DeepSeek-V4-Flash | 85.8 | 62.0 | 13.0 | 27.0 | 61.11 |
| Gemma 4 31B | 80.4 | 68.2 | 38.0 | 60.0 | 67.99 |
| Kimi K3 | 92.5 | 89.3 | 74.0 | 70.0 | 88.04 |
| Qwen3.5-9B | 42.0 | 12.5 | 1.0 | 13.0 | 16.08 |
| Qwen3.5-27B | 76.5 | 60.6 | 25.0 | 27.0 | 59.35 |
| MCP4IFC | 75.4 | 36.8 | 2.0 | 15.0 | 39.40 |
| Group | 27B SFT 9B SFT | 9B SFT + GRPO 9B SFT | 27B SFT 9B SFT + GRPO | 27B SFT + GRPO 27B SFT |
|---|---|---|---|---|
| All tasks | [7.42, 11.05] (1989) | [5.79, 9.47] (1988) | [-0.05, 3.48] (1987) | (1,989) |
| Wall, slab, column | [7.68, 13.92] (740) | [4.48, 11.04] (739) | [0.13, 5.93] (739) | (740) |
| Door, window, opening | [5.38, 11.18] (771) | [6.00, 11.46] (772) | [-2.99, 2.41] (771) | (771) |
| Space | [3.06, 17.86] (103) | [3.00, 18.00] (103) | [-5.71, 5.88] (103) | (103) |
| Property, quantity | [-1.16, 7.14] (171) | [-5.68, 3.47] (170) | [0.59, 8.88] (170) | (171) |
| GRPO step | IfcEditBench | BIM-Edit | BIBIMBAP |
|---|---|---|---|
| 0 | 88.29 | 54.76 | 59.83 |
| 10 | 90.15 | 58.00 | 65.83 |
| 20 | 90.25 | 58.68 | 59.17 |
| 25 | 90.20 | 58.78 | 63.50 |
| 30 | 90.35 | 57.05 | 63.00 |
| 40 | 90.80 | 59.51 | 64.50 |
| Model | Executed | Budget | Crash | Other | Under-edit | Calls / task |
|---|---|---|---|---|---|---|
| DeepSeek-V4-Flash | 75.2 | 20.4 | 4.3 | 0.2 | 22.3 | 13.67 |
| Gemma 4 31B | 95.5 | 4.2 | 0.3 | 0.1 | 2.7 | 7.37 |
| Kimi K3 | 95.0 | 2.3 | 1.4 | 1.4 | 3.9 | 9.10 |
| Qwen3.5-9B | 53.5 | 44.6 | 1.4 | 0.5 | 70.3 | 15.22 |
| Qwen3.5-27B | 76.9 | 20.8 | 2.3 | 0.0 | 26.3 | 12.06 |
| Qwen3.5-9B, SFT | 91.6 | 7.1 | 1.0 | 0.3 | 7.9 | 8.35 |
| Item | Project | Template (tier) | Checkers | Rater | Conf. | Reason |
|---|---|---|---|---|---|---|
| A01 | fantasy_hotel_2 | compound of three edits (1) | fail: intent (slab elevation off by 0.1 m) | cannot judge | low | elevations after the edit not shown |
| A02 | city_house_munich | set wall height, spatial reference (1) | pass | yes | high | |
| A03 | fantasy_hotel_2 | delete slab and its containment (1) | fail: validity, preservation (5,314 side effects) | no | high | side effects, dependents, invalid: 679 elements changed |
| A04 | fantasy_office_building_1 | set wall height (1) | pass | yes | medium | |
| A05 | smiley_west | create door, path reference (1) | pass | no | medium | door placed in the wall on the other side of the route |
| A06 | molio | re-host window (1) | fail: intent (file unchanged) | no | high | requested change missing |
| Pair | Benchmark | Task | Other | Score student / other | Preference | Acceptable (student / other) and note |
|---|---|---|---|---|---|---|
| B31 | BIM-Edit | create door between two rooms | GPT-5.4 Pro | 0.65 / 0.00 | student | no / no; door overshoots the wall end |
| B32 | BIBIMBAP | rotate wall by a quarter turn | 9B SFT | 0.67 / 0.33 | tie | yes / yes; different pivots, both legitimate |
| B33 | BIBIMBAP | delete columns unrelated to a slab | 9B SFT | 0.50 / 0.00 | student | yes / unsure |
| B34 | BIBIMBAP | raise window by 0.5 m | 27B SFT | 0.33 / 0.00 | 27B SFT | no / yes; student raised the opening only |
| B35 | BIM-Edit | delete wall, spatial reference | 9B SFT | 0.58 / 0.58 | tie | yes / yes; identical results |
| B36 | BIBIMBAP | move column by 3 m | 9B SFT | 1.00 / 1.00 | tie | yes / yes |
Building Information Modeling
Building Information Modeling (BIM) represents buildings as structured digital models linking geometry, materials, components, and project data.
See all 17 papers