One finite FP16 batch can corrupt BatchNorm's running variance.

In recorded Apple MLX 0.32.2 CPU tests, [-256,256] makes the default float32 running variance infinite. With momentum 0.125 it should become 16384.875. Later small batches do not restore the state, and evaluation produces zeros. Small-batch training can look normal again while the evaluation state stays damaged.

A seven-line research patch computes FP16/BF16 statistics in float32. The main suite covers 192 layer scenarios and 12 gradient scenarios: 2760 assertions per variant, including 1020 numerical comparisons. Failures: 102 before, zero after. Another 33 compatibility checks pass, including save/load and explicit buffer-dtype behavior.

The patch prevents new corruption in the tested range; it does not restore already infinite statistics. Variance beyond float32 range still fails. GPU, compiled execution, full-model performance and model-quality effects remain untested. Related InstanceNorm overflow and the earlier BatchNorm variance-estimator fix are credited. No upstream acceptance or absolute novelty claim.

Report, reproduction, patch and preserved original logs:
https://www.gero.uz/research/articles/mlx-batchnorm-low-precision-state.html

Independent, AI-assisted research and publication preparation. CPU tests ran sequentially with one numerical thread; publication preparation did not rerun them.

#MLX #MachineLearning #NumericalComputing

Code: https://github.com/kadyrbekovhamit-cyber/gero-numerical-observatory/tree/main/audits/2026-09-10-mlx-batchnorm-low-precision-state/

Synthetic narration: macOS Samantha, fictional Alex Vector. Original evidence diagrams. AI-assisted research, writing and production. Technology Product / Xamit Kadirbekov. No sponsor or paid promotion.
