Deep Neural Networks are rapidly growing in complexity, placing increasing demand on the performance and efficiency of Neural Processing Units (NPUs). Analytical models are useful for early-stage design exploration, but a cycle-accurate simulator is required to study real execution scenarios and system bottlenecks. However, existing NPU simulators suffer from several fundamental limitations — they lack proper compiler integration, do not support multi-model or multi-core execution, have limited ISA expressiveness, and often support only inference. As a result, they fail to provide a practical environment for full-stack design space exploration.

​

PyTorchSim addresses these challenges as a PyTorch-integrated NPU simulation framework that bridges machine learning frameworks, compiler flows, and cycle-accurate architecture simulation. It introduces a custom RISC-V-based ISA to support various acceleration units such as systolic arrays, and uses PyTorch 2’s compilation pipeline to lower DNN models from native PyTorch code through MLIR and LLVM into executable machine code. Our extended Gem5 and Spike simulators are used to model both functional behavior and timing characteristics of NPUs with high fidelity.

​

To overcome the performance bottleneck of conventional Instruction-Level Simulation (ILS), PyTorchSim additionally provides Tile-Level Simulation (TLS), which achieves far higher simulation speed by reusing offline tile-level latencies while still modeling DRAM and interconnect with cycle accuracy. TLS also extends naturally to sparse tensor operations by incorporating auxiliary per-tile latency obtained offline. Through this approach, PyTorchSim enables fast and accurate simulation of realistic workloads, making it a practical platform for NPU architecture research, HW/SW co-design, compiler optimization, and beyond.

​

You can find out more in our paper!