W-ISA: Transactional Execution of Weight-Space Intervention Programs
Abstract
We present W-ISA, a software instruction set and transactional runtime for executing weight-space intervention programs on pretrained language models. Shape-checked instructions express parameter updates, projections, ablations, and rank-one transformations. The runtime applies these programs within request-scoped execution contexts and restores saved parameters after completion or interruption. Activation-based match-action rules connect measured model states to intervention selection. We evaluate restoration and replay on SmolLM-135M-Instruct and Qwen2.5-1.5B-Instruct, and compare materialized rank-one updates with equivalent PyTorch hooks on the latter. In 45 timed requests, both intervention implementations produce identical greedy sequences at 32, 128, and 512 tokens; their maximum prompt-logit difference is 0.000184. On the complete WikiText-2 raw test split, the combined SmolLM attention-head ablation and MLP projection changes perplexity from 22.2353 to 22.3188, a relative increase of 0.3754%. W-ISA separates intervention specification, execution, and measurement, making parameter-level experiments reproducible and composable.
Figures & tables
| Instruction | Operation |
|---|---|
| ADD | |
| SCALE | |
| RANK1 | |
| PROJECT_OUT | , where |
| WRITE_KEY | |
| PRUNE | where |
| Condition | Perplexity | Relative change |
|---|---|---|
| Original | 22.23531898 | — |
| Head ablation | 22.31904706 | +0.37655% |
| MLP projection | 22.23131240 | -0.01802% |
| Combined | 22.31878257 | +0.37536% |
| Trigger result | Calibration | Held-out evaluation |
|---|---|---|
| True positives | 5/5 | 4/6 |
| False positives | 0/5 | 0/6 |
| False negatives | 0 | 2 |
| Balanced accuracy | 1.0 | 0.83333 |
| Request timing component | Mean |
|---|---|
| Activation profiling | 13.288 ms |
| Apply phase, matched requests | 10.635 ms |
| Control overhead across twelve requests | 17.493 ms |
| Entire request including inference callback | 442.535 ms |
| Generated tokens | Mode | Median request (s) | Median TTFT (ms) | Median decode (ms/token) |
|---|---|---|---|---|
| 32 | Original | 1.1858 | 50.644 | 36.579 |
| 32 | W-ISA | 1.2312 | 99.210 | 36.426 |
| 32 | Hook | 1.1868 | 50.776 | 36.626 |
| 128 | Original | 4.7445 | 50.576 | 36.941 |
| 128 | W-ISA | 4.8527 | 108.252 | 36.894 |
| 128 | Hook | 4.7873 | 52.161 | 37.268 |
| Mode | Maximum sampled RSS (MiB) | Maximum MPS active allocation (MiB) | Maximum RSS increase over request start (MiB) |
|---|---|---|---|
| Original | 1446.53 | 5975.53 | 1.67 |
| W-ISA | 1446.84 | 5975.52 | 498.94 |
| Hook | 1445.91 | 5975.60 | 1.64 |