Journal article · May 2026 Scientific machine learning
Diagnosing Failure Modes of Neural Operators Across Diverse PDE Families
Good benchmark accuracy is only the beginning. What happens when the equation, boundary conditions, resolution, or rollout changes?
Georgia Institute of Technology
Transactions on Machine Learning Research (TMLR)
Neural-operator robustness is a profile across shifts, not a single accuracy score.
Abstract
Neural PDE solvers are increasingly used as learned surrogates for families of partial differential equations, where the key machine learning challenge is not only interpolation on a fixed benchmark distribution but generalization under structured shifts in coefficients, boundary conditions, discretization, and rollout horizon. Yet evaluation is still often dominated by in-distribution test error, making robustness difficult to assess. We introduce a standardized stress-testing framework for neural PDE solvers under deployment-relevant shift. We instantiate it on three representative architectures—Fourier Neural Operators (FNOs), a DeepONet-style model, and convolutional neural operators (CNOs)—across five qualitatively different PDE families: dispersive, elliptic, multi-scale fluid, financial, and chaotic systems. Across 750 trained models, we measure robustness using baseline-normalized degradation factors together with spectral and rollout diagnostics. The resulting comparisons reveal that strong in-distribution accuracy does not reliably predict robustness, and that failure patterns depend jointly on architecture and PDE family. Our results provide a clearer basis for evaluating robustness claims in neural PDE solvers and suggest that function-space generalization under structured shift should be treated as a first-class evaluation target.
The method
The framework compares FNO, a DeepONet-style model, and CNO under a shared data-generation pipeline and fixed stress protocols within each PDE family. Fifty independently trained models per architecture–PDE pair separate reproducible behavior from a favorable random seed.
Interpret relative degradation alongside absolute prediction error.
- 01
Establish the baseline
Train each architecture on its in-distribution regime and record absolute relative L₂ error. The study covers nonlinear Schrödinger, Poisson, Navier–Stokes, Black–Scholes, and Kuramoto–Sivashinsky systems.
- 02
Change the deployment conditions
Apply coefficient or parameter shifts, boundary or terminal-condition shifts, resolution extrapolation, longer autoregressive rollouts, and input perturbations. Applicable stressors are shared across architectures, with no fine-tuning on shifted data.
- 03
Read the whole failure profile
Report worst-case stress error relative to baseline, absolute errors, spectral diagnostics, rollout growth, and seed-level uncertainty. A large normalized degradation and a large absolute error answer different questions.
What changes at deployment?
Qualitative illustrations of the selected stress category. These synthetic fields are not paper data, learned-model outputs, or measured failure rates.
Change the physical regime
Move coefficients or physical parameters beyond the training range: lower viscosity in Navier–Stokes, higher volatility in Black–Scholes, or rougher Poisson coefficients.
Does accuracy survive a different physical regime?
Protocol descriptions from the paper. Applicable stressors are held fixed across architectures within each PDE family.
Experiments & results
FNO achieves the lowest in-distribution error across all five PDE families in this comparison. That advantage does not give a stable robustness ranking: the architecture that looks strongest on the baseline can become the most fragile under a particular shift.
Baseline accuracy does not determine robustness
The cross-PDE degradation matrix and baseline errors tell different stories. FNO is strongest at baseline in this study, while robustness depends on the architecture, equation, and stress condition.
Different equations expose different failures
Degradation factors for nonlinear Schrödinger, Poisson, Navier–Stokes, and Black–Scholes. Poisson reveals a striking baseline-versus-robustness reversal; fluid rollout reveals a difficulty shared across architectures. Original panel annotations are retained.
The best baseline and rollout model can differ
Kuramoto–Sivashinsky rollout and perturbation degradation. The architecture ranking changes when the learned map is used repeatedly, rather than evaluated only on its nominal input distribution.
Five deployment shifts, five different questions
| Stress test | What changes | What it probes |
|---|---|---|
| Parameter / coefficient | Physical regime or coefficient field | Generalization beyond training parameters |
| Boundary / terminal | Boundary values or payoff family | Dependence on the learned operator family |
| Resolution | Spatial discretization | Transfer across grids and spectral scales |
| Rollout | Number of repeated prediction steps | Error accumulation and dynamical stability |
| Perturbation | Small changes to state inputs | Local sensitivity under corrupted observations |
Stress categories from Section 3.2. Not every stressor applies to every equation; within each PDE family, applicable protocols are fixed and shared across architectures.
What the results mean
Poisson: a reversal hidden by baseline testing
FNO has the lowest baseline error but is least robust under perturbation and resolution shift in the tested Poisson setting. CNO shows the opposite pattern: weaker baseline accuracy with greater stability under those shifts.
Dynamics: rollout is a separate requirement
Navier–Stokes exposes substantial rollout degradation across all three architectures. A method’s ability to predict one step does not establish stable repeated composition. Spectral and rollout diagnostics help explain where the failure occurs.
Scope and limitations
The results characterize the tested architectures, PDE families, and shift protocols. They do not establish a universal robustness ranking across all neural operators or deployment conditions. Baseline-normalized degradation should be interpreted alongside absolute prediction error.
Relevant to neural operator benchmarking, out-of-distribution generalization, robustness under boundary and coefficient shifts, resolution transfer, and evaluation of learned PDE surrogates beyond matched-distribution error.
- Neural operator robustness
- Distribution shift
- PDE benchmarks
- Spectral diagnostics
- Out-of-distribution generalization
Citation & resources
Lennon J. Shikhman. Diagnosing Failure Modes of Neural Operators Across Diverse PDE Families. Transactions on Machine Learning Research (TMLR), 2026.
OpenReview 0S1LWZHQYn · arXiv:2601.11428
@article{shikhman2026diagnosing,
title={Diagnosing Failure Modes of Neural Operators Across Diverse {PDE} Families},
author={Shikhman, Lennon J.},
journal={Transactions on Machine Learning Research},
year={2026},
url={https://openreview.net/forum?id=0S1LWZHQYn}
}Figures and reported numbers are reproduced from the linked paper. Interactive illustrations and conceptual diagrams are labeled separately.