1: \begin{abstract}
2:
3: Operator learning for Partial Differential Equations (PDEs) is rapidly emerging as a promising approach for surrogate modeling of intricate systems. Transformers with the self-attention mechanism---a powerful tool originally designed for natural language processing---have recently been adapted for operator learning. However, they confront challenges, including high computational demands and limited interpretability. This raises a critical question: {\em Is there a more efficient attention mechanism for Transformer-based operator learning?} This paper proposes the Position-induced Transformer (PiT), built on an innovative position-attention mechanism, which demonstrates significant advantages over the classical self-attention in operator learning. Position-attention draws inspiration from numerical methods for PDEs.
4: Different from self-attention, position-attention is induced by only the spatial interrelations of sampling positions for input functions of the operators, and
5: does not rely on the input function values themselves, thereby greatly boosting efficiency. PiT exhibits superior performance over current state-of-the-art neural operators in a variety of complex operator learning tasks across diverse PDE benchmarks. Additionally, PiT possesses an enhanced discretization convergence feature, compared to the widely-used Fourier neural operator.
6:
7:
8: \end{abstract}
9: