ONNX LpNormalization loses finite vectors through intermediate range

Finding

LpNormalization is scale invariant for every finite nonzero vector: multiplying a vector by a positive finite constant must not change its normalized output. The released ONNX Python ReferenceEvaluator and ONNX Runtime CPU provider violate that invariant because both form the L1 or L2 norm directly in the input's floating-point range.

Minimal finite counterexamples

p = 1, float32
input       [3e38, 3e38]
exact       [0.5, 0.5]
reference   [0, 0]
runtime CPU [0, 0]

p = 2, float32
input       [1e30, 1e30]
exact       [0.70710677, 0.70710677]
reference   [0, 0]
runtime CPU [0, 0]

p = 2, float32
input       [1e-30, 1e-30]
exact       [0.70710677, 0.70710677]
reference   [0, 0]
runtime CPU [0, 0]

The first case overflows the L1 sum. The second overflows the sum of squares. The third underflows both squares to zero. Every input component and every expected output is representable; the failure is confined to an avoidable intermediate calculation.

Why the scale-first oracle is exact here

Let s = max(abs(x)) on one normalization slice. For finite s > 0,

x / ||x||p = (x / s) / ||x / s||p

The scaled components lie in [-1, 1]. The calculation therefore avoids both the original magnitude and any need to represent a norm larger than the format itself. For each equal-pair counterexample the analytical answer is simply [1/2, 1/2] for L1 or [1/sqrt(2), 1/sqrt(2)] for L2.

Independent verification

• The public executable checks seven magnitude/type combinations with both axis=1 and axis=-1.
• It compares an analytical equal-pair answer, an independent scale-first NumPy oracle, ONNX ReferenceEvaluator 1.24.0 and ONNX Runtime CPU 1.29.0.
• The result is 28 implementation mismatches: fourteen in the reference evaluator and fourteen in Runtime. This is a check count, not a claim of 28 distinct defects.
• The same construction reproduces in float64. The smallest positive float32 subnormal also becomes zero for L2 instead of 1/sqrt(2).
• Ordinary [3, 4] and all-zero controls pass in both implementations.
• The source paths remained unchanged in inspected ONNX main commit c9f169adac34bd690bf0d628e9aae7fde3d4be85 and ONNX Runtime main commit a7df32cf6087a11884042a2a95526d72100e3b95.

Bounded searches for the operator name together with overflow, underflow, numerical stability and extreme-range terms found no direct duplicate in the reviewed ONNX and ONNX Runtime results. This is not an exhaustive priority claim.

Corrections and falsification tests

The ONNX correction applies the scale-first identity in the Python reference evaluator, retains the specified all-zero result, supports zero-extent axes and preserves prior non-finite behavior. The complete module reported 465 passes and 4 skips. Restoring the old implementation makes all three new finite-extreme tests fail. The change is submitted as onnx/onnx#8454.

The ONNX Runtime correction uses the output vector as the scale-free workspace, avoiding a temporary tensor. A macOS ARM64 source build completed, and all 13 selected LpNormalizationTest.* cases passed or produced the three expected CUDA-only skips: 10 passed, 3 skipped. Replacing the kernel with the prior direct-norm path makes the six-case finite-range test fail. The change is submitted as microsoft/onnxruntime#32574.

Impact and boundary

A reference evaluator acts as a conformance oracle while a runtime kernel evaluates deployed graphs. Agreement between the two on ordinary magnitudes does not protect either from a shared avoidable intermediate-range failure. For the finite cases above, the output direction is erased completely.

This report covers ONNX ReferenceEvaluator and the ONNX Runtime CPU provider only. Other execution providers, gradients, performance effects and production-model impact were not tested. No definition is proposed here for vectors containing infinities or NaNs. This is an ordinary numerical-correctness report, not a security claim.