Tensor computing application programming interface: An application-oriented, lightweight API for portable high-performance tensor network computations

Rong-Yang Sun; Tomonori Shirakawa; Hidehiko Kohshiro; D. N. Sheng; Seiji Yunoki

quant-phDec 30, 2025

Tensor computing application programming interface: An application-oriented, lightweight API for portable high-performance tensor network computations

Authors: Rong-Yang Sun, Tomonori Shirakawa, Hidehiko Kohshiro, D. N. Sheng, Seiji Yunoki

Organizations: Department of Physics and Astronomy, California State University Northridge, Northridge, California 91330, USA · RIKEN Interdisciplinary Theoretical and Mathematical Sciences Program (iTHEMS), Wako, Saitama 351-0198, Japan · Computational Materials Science Research Team, RIKEN Center for Computational Science (R-CCS), Kobe, Hyogo, 650-0047, Japan · Quantum Computational Science Research Team, RIKEN Center for Quantum Computing (RQC), Wako, Saitama, 351-0198, Japan · Computational Condensed Matter Physics Laboratory, RIKEN Cluster for Pioneering Research (CPR), Saitama 351-0198, Japan · Quantum Computing Simulation Unit, Quantum-HPC Hybrid Platform Division, RIKEN Center for Computational Science (R-CCS), Kobe, Hyogo, 650-0047, Japan · Next-Generation HPC Application Development Unit, Next-Generation HPC Infrastructure Development Division, RIKEN Center for Computational Science (R-CCS), Kobe, Hyogo, 650-0047, Japan · Computational Quantum Matter Research Team, RIKEN Center for Emergent Matter Science (CEMS), Wako, Saitama 351-0198, Japan

Abstract

Tensor networks (TNs) are a central computational tool in quantum science and artificial intelligence. However, the lack of a unified software interface across tensor-computing frameworks severely limits the portability of TN applications, coupling algorithmic development to specific hardware and software back ends. To address this challenge, we introduce the Tensor Computing Application Programming Interface (TCAPI)---an application-oriented, lightweight API designed to enable framework-independent, high-performance TN applications, currently specified for C++ and Python. TCAPI provides well-defined type-system conventions that abstract tensor objects, together with a minimal yet expressive set of core functions for essential tensor manipulations and tensor linear-algebra operations, with operation semantics shared across language specifications. Through numerical demonstrations of representative tensor-network applications implemented in C++ using TCAPI, we show that code written against TCAPI can be migrated seamlessly across heterogeneous hardware and software platforms while achieving performance comparable to native framework implementations. We further provide open-source implementations of TCAPI based on Cytnx and ITensor in C++, and on NumPy in Python, demonstrating its practicality and ease of integration with existing tensor-computing frameworks.

Figures & tables

Figure 1: System architectures and development workflows for tensor-network (TN) applications. Arrows indicate dependency relations, with arrowheads pointing to the dependent components. (a) Typical software architecture of a TN application, in which the application logic and TN algorithms are directly tied to a specific tensor-computing framework (TCF). (b) Development workflow without TCAPI: TN applications must be reimplemented separately for each TCF, leading to duplicated development effort and limited portability. (c) Development workflow with TCAPI: application logic and TN algorithms are decoupled from back-end hardware and software through a uniform interface, enabling portable development across heterogeneous TCFs.

Figure 2: Graphic representations of tensors. (a) Array representation of a third-order tensor

A_{ijk}

, where the entry at coordinates

(i,j,k)

corresponds to the tensor element

A_{i,j,k}

. (b) Penrose diagram of a fourth-order tensor

A_{ijkl}

, in which the node represents the tensor and each incident leg corresponds to a tensor mode (bond).

Figure 3: Typical tensor-manipulation operations. (a) Transpose, which reorders tensor bonds. (b) Concatenation, which combines multiple tensors along a specific bond. (c) Broadcasting, which applies a scalar function elementwise while preserving tensor order and shape.

Figure 4: Representative tensor linear-algebra operations. (a) Tensor contraction over shared bonds. (b) Tensor decomposition, illustrated by the SVD as a representative example.

Figure 5: Configuration used in Application B. (a) Heavy-hex lattice geometry corresponding to the 156-qubit superconducting chips, e.g., ibm_marrakesh and ibm_kobe , in IBM Quantum System Two. This connectivity is mapped directly onto the 2dTNS used in the simulation. (b) Representative portion of the quantum circuit implementing a single Floquet cycle of the KIM. Rectangles denote single-qubit

R^{X}

gates, while wires connecting pairs of qubits represent two-qubit

R^{ZZ}

gates. Gray vertical bands indicate groups of

R^{ZZ}

gates executed concurrently by the parallel algorithm (see Appendix B ).

Figure 6: Wall-clock time per iTEBD iteration as a function of the bond dimension

\chi

for the ground-state calculation of 1D TFIM at

g=1

(Application A). Blue dots correspond to TCF I executed on an Intel Xeon 6240R CPU, while orange triangles correspond to TCF II executed on an NVIDIA H100 GPU. Marks indicate mean values computed over the final ten iterations, and shaded bands represent the corresponding minimum–maximum ranges.

Figure 7: Wall-clock time per BP iteration as a function of the bond dimension

\chi

for the KIM simulation on a heavy-hex lattice (Application B). Blue dots correspond to TCF III executed on an Apple M2 Ultra CPU, while green squares and orange triangles correspond to TCF I executed on 4 and 156 nodes of supercomputer Fugaku, respectively. Marks indicate mean values over ten BP iterations, and shaded regions represent the corresponding minimum–maximum ranges.

Figure 8: Comparison between TCAPI-based implementations and native implementations using TCF I, analogous to Fig. 6 and Fig. 7 . Blue dots correspond to TN applications developed using TCAPI, while orange triangles correspond to implementations written directly against the native API of TCF I. Insets show the relative timing difference,

(t_{\rm TCAPI}-t_{\rm native})/t_{\rm native}

, quantifying the abstraction overhead introduced by TCAPI. (a) Application A. (b) Application B.

Appendix figures & tables5 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 9: (a) Schematic of a quantum circuit composed of single-qubit (1q) and two-qubit (2q) gates. (b) Construction of the base TNO representing the identity operator together with the first gate layer. White circles denote tensors of bond dimension one on the virtual bonds, corresponding to the operator

1\otimes 1\otimes\cdots\otimes 1

. (c) Decomposition of a two-qubit unitary gate into two site tensors—each acting on one qubit and a shared virtual bond—obtained via SVD. Non-nearest-neighbor gates are handled by inserting identity operators along the intermediate path connecting the two target qubits. (d) Assembly of the full TNO by attaching the decomposed site tensors to the base network. Tensors updated through gate contractions are highlighted in color, and thicker lines indicate enlarged bond dimensions.

Figure 10: Workflow for updating a TNS by applying a TNO using BP, followed by low-rank approximation of the resulting tensors. (a) Local contraction of TNO site tensors

O_{i}

with TNS site tensors

V_{i}

, producing intermediate tensors

\tilde{V}_{i}

. (b) Fixed-point relations for the BP message tensors. (c) Consistency constraints for the square-root and inverse square-root message tensors,

M_{i\to j}^{1/2}

and

M_{i\to j}^{-1/2}

. (d) SVD-based truncation of the message tensors

M_{i\to j}

. With

D_{ij}

denoting the diagonal matrix of singular values, the largest

\chi

singular values are retained to reconstruct the updated site tensors

V_{i}

within the prescribed maximum bond dimension. Dashed boxes highlight the subnetwork in which the updated tensors

V_{i}

are assembled.

Member type	Description
ten_t	Concrete tensor type corresponding to TenT .
order_t	Integral type for the tensor order.
shape_t	Shape type in bond-index order; List<bond_dim_t> .
bond_dim_t	Integral type for a bond dimension.
bond_idx_t	Integral type for a bond index (zero-based).
bond_label_t	Integral type for a user-defined bond label.

Appendix

Table 1: Member types defined in tcapi::tensor_traits<TenT> and their intended roles.

Auxiliary type	Description
List<T>	Sequence container of values of type T ; alias of std::vector<T> .
CRef<T>	Non-owning const reference to an object of type T ; alias of std::reference_wrapper<const T> .
Pair<T, U>	Pair of values of types T and U ; alias of std::pair<T,U> .
Map<T, U>	Associative container mapping T $\to$ U ; alias of std::unordered_map<T,U> .

Appendix

Table 2: Auxiliary types defined in TCAPI and their intended roles.

Metadata	Description
ten_type	Raw backend tensor class returned by TCAPI functions.
dtype	Backend-native dtype of elements; Python analogue of tcapi::elem_t<TenT> .
real_dtype	Backend-native real dtype; Python analogue of tcapi::real_t<TenT> .
cplx_dtype	Backend-native complex dtype; Python analogue of tcapi::cplx_t<TenT> .
real_ten_kind	Tensor kind corresponding to tcapi::real_ten_t<TenT> .
cplx_ten_kind	Tensor kind corresponding to tcapi::cplx_ten_t<TenT> .

Appendix

Table 3: Metadata associated with a Python TenKind and their intended roles.

Tensor Networks

Tensor networks represent high-dimensional data and quantum states as interconnected low-rank tensors to enable efficient computation, learning, and simulation.

See all 50 papers

Oct 16, 2026

TorchTPU - A Hybrid Eager/Compiled Architecture for Native PyTorch Execution on TPU

TorchTPU is an open-source collaboration between Google and Meta that introduces a native and principled PyTorch backend for Google TPUs. It enables seamless out-of-the-box model execution and TPU migration while delivering industry-competitive inference and training performance through eager execution optimizations, torch.compile integration, and native hardware acceleration.

Claudio Basile, Michael Voznesensky, Gunhyun Park +50

Neural Network Hardware Acceleration Efficient Neural Network Inference

Ask PDF

Jul 30, 2026

It's All Just Vectorization: einx, a Universal Notation for Tensor Operations

Tensor operations represent a cornerstone of modern scientific computing. However, the Numpy-like notation adopted by predominant tensor frameworks is often difficult to read and write and prone to so-called shape errors, i.a., due to following inconsistent rules across a large, complex collection of operations. Alternatives like einsum and einops have gained popularity, but are inherently restricted to few operations and lack the generality required for a universal model of tensor programming. To derive a better paradigm, we revisit vectorization as a function for transforming tensor operations, and use it to both lift lower-order operations to higher-order operations, and conceptually decompose higher-order operations to lower-order operations and their vectorization. Building on the universal nature of vectorization, we introduce einx, a universal notation for tensor operations. It uses declarative, pointful expressions that are defined by analogy with loop notation and represent the vectorization of tensor operations. The notation reduces the large APIs of existing frameworks to a small set of elementary operations, applies consistent rules across all operations, and enables a clean, readable and writable representation in code. We provide an implementation of einx that is embedded in Python and integrates seamlessly with existing tensor frameworks: https://github.com/fferflo/einx

Florian Fervers, Sebastian Bullinger, Christoph Bodensteiner +1

Fraunhofer IOSB

Topic pending

Ask PDF

Apr 27, 2026

TACO: Efficient Communication Compression of Intermediate Tensors for Scalable Tensor-Parallel LLM Training

Handling communication overhead in large-scale tensor-parallel training remains a critical challenge due to the dense, near-zero distributions of intermediate tensors, which exacerbate errors under frequent communication and introduce significant computational overhead during compression. To this end, we propose TACO (Tensor-parallel Adaptive COmmunication compression), a robust FP8-based framework for compressing TP intermediate tensors. First, we employ a data-driven reshaping strategy combined with an Adaptive Scale-Hadamard Transform to enable high-fidelity FP8 quantization, while its Dual-Scale Quantization mechanism ensures numerical stability throughout training. Second, we design a highly fused compression operator to reduce memory traffic and kernel launch overhead, allowing efficient overlap with communication. Finally, we integrate TACO with existing state-of-the-art methods for Data and Pipeline Parallelism to develop a compression-enabled 3D-parallel training framework. Detailed experiments on GPT models and Qwen model demonstrate up to 1.87X end-to-end throughput improvement while maintaining near-lossless accuracy, validating the effectiveness and efficiency of TACO in large-scale training.

Man Liu, Xingchen Liu, Xingjian Tian +8

Hangzhou Institute for Advanced Study, University of Chinese Academy of Sciences · Institute of Computing Technology, Chinese Academy of Sciences · University of Science and Technology of China

Communication-Efficient Distributed Training Tensor Parallelism Distributed Training

Ask PDF

Abstract

Tensor Networks

Explore similar work

TorchTPU - A Hybrid Eager/Compiled Architecture for Native PyTorch Execution on TPU

It's All Just Vectorization: einx, a Universal Notation for Tensor Operations

TACO: Efficient Communication Compression of Intermediate Tensors for Scalable Tensor-Parallel LLM Training