Perpendis Array tools
Royalty tools Sign in

Array tools › Blog

ONNX vs PyTorch output mismatch: divergence and tolerance

By Perpendis · Published 19 Sep 2026 · Reviewed 19 Sep 2026

You ran torch.onnx.export, loaded the graph in ONNX Runtime, compared the outputs against PyTorch, and np.testing.assert_allclose failed. Before you assume the export is broken: two correct implementations of the same network will almost never produce bit-identical outputs. The question is not whether they differ, but where, by how much, and whether the shape of the difference looks like rounding or like a fault.

This page is about telling those two apart. We look at why the numbers legitimately diverge, how to compare tensors so the answer is evidence rather than a guess, what tolerances are a reasonable starting point for fp32 and fp16, and how to bisect a genuine op-level bug by layer.

Same input, two runtimes, one differing valueinputtensorPyTorchONNX Runtime0.4180.3170.9020.1630.6640.2810.4180.3170.9020.1640.6640.281first divergence
The same input tensor runs through PyTorch and ONNX Runtime; every output value matches except one — the first divergence.

First, rule out the boring failures

Before any talk of tolerance, check the two things no tolerance can rescue: shape and dtype. A transposed axis, an NHWC-versus-NCHW layout, a dynamic axis you forgot to declare, or one side returning fp16 while the other returns fp32 all present as a "mismatch" that a looser atol only hides. torch.testing.assert_close checks dtype and shape by default and names which one failed, so run it before you reach for numbers.

Non-numerical causes to eliminate first

Why the numbers legitimately differ

Non-associative addition changes the last digita = 0.1 b = 0.2 c = 0.3(a + b) + c0.1 + 0.2+ 0.30.6000000000000001a + (b + c)0.1 +0.2 + 0.30.6000000000000000last bit
Reduction order changes the last bit: (a+b)+c and a+(b+c) use the same inputs but land on a different final digit.

The root cause is that floating-point addition is not associative. (a + b) + c and a + (b + c) can round to different results. Any reduction — the sum inside a matmul, a softmax denominator, the mean in a norm layer — has no single canonical order. PyTorch's kernel (cuDNN, oneDNN, or its own) and ONNX Runtime's or TensorRT's kernel tile and accumulate that sum differently, so the last bit or two legitimately disagree. Nothing is wrong. On top of that root cause, several specific triggers widen the gap:

Compare properly: find where they diverge

ULP distance on the float32 number lineULP distance = 2A = 1.0000000B = 1.0000002representable float32 values
On the float32 number line, A and B sit two representable steps apart — a ULP distance of 2.

Do not compare two totals or a mean. A mean hides one catastrophic element among thousands of good ones, and inflates when a thousand elements are each off by a bit. Localise instead. Upcast to float64 before subtracting so the difference itself is exact, then find the worst element, its absolute and relative diff, and the ULP gap.

import numpy as np
import onnxruntime as ort
import torch

model.eval()
x = torch.randn(1, 3, 224, 224)

with torch.no_grad():
    ref = model(x).cpu().numpy()

sess = ort.InferenceSession(
    "model.onnx",
    providers=["CPUExecutionProvider"],
)
got = sess.run(None, {"input": x.numpy()})[0]

# 1. dtype and shape before anything else
assert ref.shape == got.shape, (ref.shape, got.shape)
assert ref.dtype == got.dtype, (ref.dtype, got.dtype)

# 2. upcast so the diff is not itself rounded
a = ref.astype(np.float64)
b = got.astype(np.float64)
diff = np.abs(a - b)

# 3. the worst element, not an average
i = np.unravel_index(diff.argmax(), diff.shape)
rel = diff / np.maximum(np.abs(a), 1e-12)

print("max abs", diff.max(), "at", i)
print("max rel", rel.max())
print("A", a[i], "B", b[i])

The ULP distance — the count of representable floats between two values — is the measurement that tells rounding from corruption. Zero means bit-identical; one or two means the values are a step or two apart, which is pure rounding; a large ULP gap on a handful of elements is still fine if their absolute and relative diffs are small.

def ulp_diff(a, b):
    ia = np.float32(a).view(np.int32).astype(np.int64)
    ib = np.float32(b).view(np.int32).astype(np.int64)
    lo = -(1 << 31)
    ka = np.where(ia < 0, lo - ia, ia)
    kb = np.where(ib < 0, lo - ib, ib)
    return np.abs(ka - kb)

# pass the original fp32 values, not the upcast
print(ulp_diff(ref[i], got[i]))

If you would rather not hand-roll the ULP step, paste both arrays — a tensor(...) wrapper is fine — or load two .npy files into Perpendis diff and read the first differing index, both bit patterns, and the ULP distance straight off.

Test at batch=1 and at a realistic batch. At batch=1 the runtime may pick a tiling that happens to match PyTorch; at batch=128 it selects a different algorithm, accumulates a longer sum, and shows more rounding. A comparison that passes at batch=1 and fails at batch=128 is usually still rounding — but it is exactly the regime where a real broadcasting or reshape bug also hides, so find the first divergent index before you conclude either way.

What tolerance is actually normal

These are starting points, not guarantees. The right tolerance depends on network depth, output dynamic range, and hardware. Both np.testing.assert_allclose and torch.testing.assert_close test |actual - reference| <= atol + rtol * |reference|, so rtol scales with the reference magnitude and atol is the floor near zero — you almost always want both. One trap worth naming: numpy.testing.assert_allclose defaults to rtol=1e-7 and atol=0. With atol=0, any element whose reference sits near zero is held to a pure relative bar it can almost never meet, so the assertion fails on legitimate rounding until you set a real atol. (np.allclose is a separate function with different defaults: rtol=1e-5, atol=1e-8.)

Scenarioatol (start)rtol (start)Notes
fp32, CPU vs ONNX Runtime CPU1e-51e-3Tightest case; many models pass at 1e-5 to 1e-4.
fp32 with TF32 (Ampere+ GPU)1e-31e-2Disable TF32 to get a clean fp32-vs-fp32 read.
fp161e-21e-2Abs diffs of 1e-3 to 1e-2 are common; lean on rtol.
bf161e-22e-2Only 7 mantissa bits, so wider still.
High-dynamic-range outputnear 01e-3 to 1e-2Logits and unnormalised scores need relative, not absolute.

For reference, torch.testing.assert_close's own dtype-dependent defaults are atol=1e-5, rtol=1.3e-6 for float32 and atol=1e-5, rtol=1e-3 for float16. A high-dynamic-range output is where absolute tolerance breaks: a single atol is either too strict on the large values or too loose on the small ones, so switch to relative and keep atol only as a near-zero floor.

Rounding, or a real bug?

Once you have the worst element localised, the pattern of the error is diagnostic.

Looks like rounding (expected)Looks like a bug (investigate)
A handful of elements off by 1 to 2 ULP.A whole contiguous region wrong — a channel, a row, an attention head — while the rest matches.
Diffs that scale with each element's magnitude (roughly constant relative error).A NaN or Inf on one side only.
Error that grows smoothly with depth and batch size.A diff that jumps at one specific op and stays large downstream.
Max relative diff near the dtype's unit roundoff times network depth.Error that does not shrink when you force fp32 everywhere; a constant offset or a sign flip.

When it looks like a bug, bisect by layer. Register PyTorch forward hooks to capture intermediates, or mark the ONNX graph's internal tensors as outputs, and compare each. NVIDIA Polygraphy automates this: it runs the same model through ONNX Runtime and TensorRT and compares every marked layer.

polygraphy run model.onnx \
  --trt --onnxrt \
  --atol 1e-3 --rtol 1e-3 \
  --onnx-outputs mark all \
  --trt-outputs mark all

The first layer whose output exceeds tolerance is where to look. From there it is usually a single op: an attribute silently dropped on export, an opset version that changed a default, a norm epsilon in the wrong place, or an unsupported operator that the converter approximated.

Find where two arrays first diverge. Perpendis diff takes your two arrays — paste them from a REPL with brackets and commas or a tensor(...) wrapper, or load two .npy files — and reports the first differing index, both bit patterns at that index, and the ULP distance between them. Equality is bit equality, not ==, so +0 and -0 are treated as differing, two NaNs with matching bits are not, and infinity against a finite value is never reported as a distance. It works on float32 and float64, runs in the tab with nothing uploaded and no account, and it will not guess which side is correct — that judgement is what this page is about. Open Perpendis diff.

Frequently asked questions

What atol/rtol should I use for ONNX export?

For an fp32 export compared on CPU, start at atol=1e-5 and rtol=1e-3, then tighten if it passes; many models are fine at atol=1e-4. On an Ampere or later GPU with TF32 enabled, or with fused kernels in play, loosen to about atol=1e-3, rtol=1e-2, or disable TF32 to get a clean fp32-versus-fp32 read. Treat these as starting points: the correct value depends on depth and dynamic range, so localise the worst element rather than trusting one global threshold.

Why does the mismatch appear only at larger batch sizes?

Kernel selection and reduction order depend on tensor shape. At batch=1 the runtime may pick a tiling that happens to match PyTorch; at batch=128 it selects a different algorithm and accumulates a longer sum, so more rounding surfaces. If the per-element relative error stays near the dtype's unit roundoff and no single region is wrong, that growth is expected. It is also the regime where a genuine reshape or broadcasting bug hides, so find the first divergent index before deciding.

Is rtol=1e-3 too loose?

For fp32 it is loose: torch.testing.assert_close defaults to rtol=1.3e-6 for float32, so 1e-3 will pass differences you might want to see. For fp16 it is about right, since per-op relative error is already around 5e-4. Loose is not wrong in itself; it just has to match the precision you are actually running. Pair rtol with a small atol so near-zero values are not held to an impossible relative bar.

How much difference is acceptable for an fp16 model?

fp16 resolves about three significant decimal digits, so per-op relative error near 1e-3 is normal and accumulates with depth. Output absolute diffs of 1e-3 to 1e-2 against an fp32 reference are common and usually fine. Judge by the task metric, not the raw tensor: if top-1 class, detection boxes, or your downstream loss are unchanged, the model is behaving. Watch for NaN or Inf, or a whole region wrong — those are bugs, not precision.

Sources

Keep reading