ONNX ReduceL2 loses finite norms at extreme scales

Finding

ReduceL2 returns the Euclidean norm sqrt(sum(x²)). The released ONNX Python ReferenceEvaluator and ONNX Runtime CPU provider both form the squares directly in the input floating-point range. The intermediate sum can therefore overflow or underflow even when every input and the final norm are finite and representable.

Minimal finite counterexamples

dtype     input                  exact norm          reference   runtime CPU
float32   [1e30,  1e30]         1.4142135e30        inf         inf
float32   [1e-30, 1e-30]        1.4142136e-30       0           0
float64   [1e200, 1e200]        1.41421356e200      inf         inf
float64   [1e-200, 1e-200]      1.41421356e-200     0           0

The large cases overflow while forming x². The small cases round both squares to zero before the square root. The ordinary control [3, 4] still returns 5, so agreement at routine scales does not expose the defect.

Independent oracle

For s = max(abs(x)) and finite s > 0, the same norm can be evaluated as:

||x||₂ = s · sqrt(sum((x / s)²))

The normalized values have magnitude at most one. For each equal pair in the table, the exact result is also available analytically as sqrt(2) · abs(x₀). The reproducer uses this scale-first calculation, not either implementation under test.

Verification

• The executable checks float32 and float64 at both overflow and underflow scales.
• It runs operator versions 17 and 18 against ONNX ReferenceEvaluator 1.24.0 and ONNX Runtime CPU 1.29.0.
• All 16 implementation comparisons mismatch the independent oracle. This is a comparison count, not a claim of 16 independent defects.
• Ordinary [3, 4] and all-zero controls pass in both implementations.
• Bounded GitHub searches combining ReduceL2 with overflow, underflow, finite range and numerical stability found no direct duplicate in the reviewed results. This is not an exhaustive priority claim.

Corrections and falsification

The ONNX correction scales each reduction slice before summing squares and restores the output scale afterwards. Tests cover float32 and float64, both operator versions and both magnitude directions. The full reference-evaluator test module reported 468 passes and 4 skips. Replacing the helper with the former direct-square expression makes all eight new version/type cases fail. The change is submitted as onnx/onnx#8455.

The ONNX Runtime CPU correction uses a scaled sum-of-squares accumulator for both the contiguous all-axis path and the incremental partial-axis path. All 27 local ReductionOpTest.ReduceL2* tests pass. Restoring the prior direct-square code makes both new tests fail with infinity on large values and zero on small values; rebuilding the correction restores all 27 passes. The change is submitted as microsoft/onnxruntime#32575.

Impact and boundary

ReduceL2 can appear directly in exported graphs and also expresses a basic building block for norms and distance calculations. In the demonstrated range, its finite output is erased completely. The same avoidable intermediate pattern affected both a conformance reference and a deployed CPU runtime.

This report covers only the released ONNX ReferenceEvaluator and ONNX Runtime CPU provider on the stated cases. Other execution providers, gradients, performance and production-model prevalence were not measured. The submitted Runtime change intentionally leaves integer and other element-type paths unchanged. This is an ordinary numerical-correctness report, not a security claim.