WEBVTT

00:00:00.000 --> 00:00:03.763
Two equal inputs should split the gradient equally. But in

00:00:03.763 --> 00:00:07.208
MLX, a large common offset changes one half into one.

00:00:07.733 --> 00:00:11.652
The inputs are one hundred million and one hundred million.

00:00:11.652 --> 00:00:15.309
For the final log cumulative sum exponential, the exact

00:00:15.309 --> 00:00:19.097
gradient is point five, point five. MLX returns one, one.

00:00:19.600 --> 00:00:23.536
The equivalent log sum exponential returns the correct halves.

00:00:23.536 --> 00:00:27.409
The scan backward pass reuses a rounded forward value, losing

00:00:27.409 --> 00:00:28.596
the normalization.

00:00:29.100 --> 00:00:32.667
Subtracting one global maximum is not a general repair. A

00:00:32.667 --> 00:00:36.173
later large value can corrupt the gradient of an earlier

00:00:36.173 --> 00:00:39.987
prefix. The prototype instead tracks normalized mass for each

00:00:39.987 --> 00:00:40.479
prefix.

00:00:41.000 --> 00:00:44.477
Together with a separate repair for an already reported double

00:00:44.477 --> 00:00:47.898
precision exponential issue, the new recurrence passes eleven

00:00:47.898 --> 00:00:51.209
hundred twenty two gradient checks and four hundred seventy

00:00:51.209 --> 00:00:53.251
seven higher derivative comparisons.

00:00:53.767 --> 00:00:57.702
These are bounded CPU tests, not a production benchmark. GPU

00:00:57.702 --> 00:01:01.445
and long sequence performance remain untested. The report

00:01:01.445 --> 00:01:05.122
separates both causes and includes the code and all four

00:01:05.122 --> 00:01:06.477
comparison variants.