You changed a preprocessing step, moved a model to a second machine, or refactored a hot loop, and now you have two saved arrays: a reference and a fresh run. They should be identical. Are they? And if not, where do they first part company?
Saved as .npy files, the answer is a few lines of NumPy, provided you check the right things in the right order. The most common surprise is not a numeric drift at all. It is a silent change in dtype or shape that no tolerance check would have caught, because a shape mismatch fails equality outright and a float32-versus-float64 change quietly upcasts before you ever compare a value.
Load both arrays with np.load, then print dtype and shape before you compare a single element. This one habit catches the majority of real-world differences.
import numpy as np
a = np.load("reference.npy")
b = np.load("candidate.npy")
print(a.dtype, a.shape)
print(b.dtype, b.shape)
If the shapes differ, stop here: the arrays are not comparable element-for-element, and you should not let a downstream check paper over it. Be especially wary of a trailing or leading unit axis, such as (3,) against (3, 1). NumPy will happily broadcast those two shapes in an arithmetic comparison, turning a genuine mismatch into a plausible-looking result. If the dtypes differ, note it: comparing a float32 save against a float64 reference will report differences that are really just the rounding you baked in when you saved at lower precision.
There are two honest questions, and they need different tools. "Are these the same bytes?" is not the same as "are these the same to within floating-point noise?" That noise is often legitimate: the same computation on a different machine, BLAS build, or GPU can reorder a non-associative sum, and reduced-precision paths such as fp16 or TF32 change the low-order bits by design.
| Check | Returns | Use when |
|---|---|---|
a.tobytes() == b.tobytes() | scalar bool, true byte identity | You need bit-for-bit sameness, including signed zero and NaN payloads |
np.array_equal(a, b) | scalar bool, value equality | Elements should match exactly, but -0.0 may equal 0.0 |
np.allclose(a, b) | scalar bool, tolerant | Floating-point drift is expected and acceptable |
np.isclose(a, b) | bool array, tolerant | You need to know which elements differ |
Two subtleties matter here. First, np.array_equal treats -0.0 and 0.0 as equal and, by default, treats NaN as not equal to NaN; pass equal_nan=True to change that. For genuine byte identity, compare a.tobytes() to b.tobytes() after confirming dtype and shape match. Second, the tolerant checks are not symmetric.
The tolerance formula. np.allclose and np.isclose test |a - b| <= atol + rtol * |b|, with defaults rtol=1e-05 and atol=1e-08. The relative term is scaled by the second argument, so the order of your arrays can change the verdict at the margin. More importantly, when you compare against zero the relative term vanishes and only atol governs, so rtol does nothing for values near zero. Set atol to the smallest magnitude you actually care about.
For automated tests, numpy.testing.assert_allclose raises an AssertionError with a readable report instead of returning a bool, and its defaults are stricter than np.allclose: rtol=1e-7 and atol=0. Because atol is zero, it demands near-exact agreement for values at or near zero, so pass an explicit atol when your data straddles zero.
A boolean "they differ" is rarely enough. You want the index of the first divergence and the two values there, so you can look at them. Build a mask of differing elements, confirm any are set, then use argmax to get the first True. On a boolean array, argmax returns the flat index of the first True; np.unravel_index turns that into array coordinates.
mask = ~np.isclose(a, b, equal_nan=True)
if mask.any():
idx = np.unravel_index(mask.argmax(), a.shape)
print("first diff at", idx)
print("A =", a[idx], " B =", b[idx])
The mask.any() guard is not optional. On an all-False mask, argmax returns 0, which would point you at index zero as if it differed. For a bit-exact diff instead, build the mask with a != b, but remember that NaN != NaN is True, so any NaN will be flagged as a difference regardless of the other array.
Wrap the ordered checks into one function you can drop into a test or a scratch session. It reports the first divergence, the worst absolute difference, and how many elements differ.
import numpy as np
def diff_npy(path_a, path_b,
rtol=1e-5, atol=1e-8):
a = np.load(path_a)
b = np.load(path_b)
if a.dtype != b.dtype:
print(f"dtype: {a.dtype} vs {b.dtype}")
if a.shape != b.shape:
print(f"shape: {a.shape} vs {b.shape}")
return
if a.tobytes() == b.tobytes():
print("bit-identical")
return
mask = ~np.isclose(a, b, rtol=rtol,
atol=atol, equal_nan=True)
if not mask.any():
print(f"equal within atol={atol}, "
f"rtol={rtol}")
return
idx = np.unravel_index(mask.argmax(),
a.shape)
d = np.abs(a - b)
print(f"first diff at {idx}")
print(f" A={a[idx]!r} B={b[idx]!r}")
print(f" max abs={np.nanmax(d)} "
f"n={int(mask.sum())}/{a.size}")
Note that np.nanmax ignores NaN entries when reporting the worst difference; the count from the mask still records them as differing. If an array mixes NaN or inf with real values, read both numbers together rather than trusting either alone.
An .npy file holds a single array. An .npz file is a zip archive of several named arrays, and np.load returns a lazy, dictionary-like NpzFile rather than an array. You iterate its keys through the .files attribute and compare each array in turn.
with np.load("run_a.npz") as A, \
np.load("run_b.npz") as B:
keys = set(A.files) | set(B.files)
for k in sorted(keys):
if k not in A.files or k not in B.files:
print(f"{k}: missing in one file")
continue
same = np.array_equal(A[k], B[k])
print(f"{k}: {'ok' if same else 'DIFF'}")
Compare the key sets first. A missing or renamed array is another silent difference that a value comparison alone will not surface, and it is easy to introduce when you change what a checkpoint saves.
If an array will not fit comfortably in memory, load it with mmap_mode="r". NumPy maps the file from disk and reads pages on demand, returning a memmap you can index like any array.
a = np.load("big.npy", mmap_mode="r")
b = np.load("big_ref.npy", mmap_mode="r")
The catch is that a whole-array expression such as a - b still materialises a full-size result, which defeats the point. For genuinely large arrays, walk them in chunks along the first axis and short-circuit on the first block that diverges, so you never hold more than one slice of differences at a time.
step = 1_000_000
for i in range(0, a.shape[0], step):
sa = np.asarray(a[i:i+step])
sb = np.asarray(b[i:i+step])
if not np.allclose(sa, sb):
print("diverges in block at", i)
break
Model outputs, embeddings, and intermediate tensors are not always safe to hand around. They can encode training data, user inputs, or proprietary behaviour, and "it is only a numeric array" is not a security review. Most online "compare two files" tools work by uploading your files to a server you do not control, which is the wrong default for anything you would not paste into a public issue.
If you want the same first-difference readout without a Python session, and without your data leaving the machine, a client-side diff is the honest option. Perpendis array diff runs entirely in your browser: you load two arrays and it computes the comparison locally, with nothing sent anywhere.
Diff two arrays in the browser. Paste two float arrays, or load two .npy files, into perpendis.com/diff. It tells you whether they are bit-identical, the first differing index, the A and B values there, the max absolute and max relative difference, the p95 absolute difference, the ULP distance, and the raw float32/float64 bit patterns of both values, plus a divergence-by-class breakdown. It runs entirely client-side in WebAssembly. Nothing is uploaded, and there is no account.
The ULP distance is worth the attention: a max absolute difference of 1e-7 means little on its own, because it is enormous near zero and negligible near a billion. ULP counts the representable floats between the two values, which is scale-aware, and the bit patterns tell you at a glance whether you are looking at a last-bit rounding difference or something structural.
Load both with np.load, then check dtype and shape before anything else. If they match, use np.array_equal (or a.tobytes() == b.tobytes() for true byte identity) for an exact check, or np.allclose for a tolerant one. To find where they differ, build a mask with ~np.isclose(a, b, equal_nan=True), confirm mask.any(), then use np.unravel_index(mask.argmax(), a.shape) to get the coordinates of the first differing element.
An .npy file stores a single array. An .npz file is a zip archive of several named arrays, so np.load returns a lazy, dictionary-like NpzFile rather than an array. Iterate its .files attribute to get the array names, and compare the key sets between two files first, since a missing or renamed array is a difference that a value-by-value comparison will not reveal.
np.allclose and np.isclose test |a - b| <= atol + rtol * |b| with defaults rtol=1e-5 and atol=1e-8, and return a bool (a scalar for allclose, an array for isclose). numpy.testing.assert_allclose, meant for tests, instead raises an AssertionError and uses stricter defaults, rtol=1e-7 and atol=0. Because its atol is zero, values at or near zero need an explicit atol, or they will fail.
Yes. A client-side browser tool such as perpendis.com/diff lets you load two .npy files and see whether they are bit-identical, the first differing index, the values there, max absolute and relative difference, p95, ULP distance, and the raw float bit patterns. It runs entirely in the browser in WebAssembly, so nothing is uploaded, which matters when the arrays are model outputs or other sensitive data.
Load each file with mmap_mode="r" so NumPy maps it from disk instead of reading it all into memory. Avoid whole-array expressions like a - b, which still allocate a full-size result. Instead, walk the arrays in chunks along the first axis, compare each block with np.allclose or np.isclose, and stop at the first block that diverges so you never hold more than one slice of differences at once.