WEBVTT

00:00:00.000 --> 00:00:03.870
A correct forward result can hide a wrong gradient. In M L X,

00:00:03.870 --> 00:00:07.677
non transposed gathered quantized multiplication miscomputes

00:00:07.677 --> 00:00:09.674
the scale and bias derivatives.

00:00:10.200 --> 00:00:14.131
Only the first weight row contributes. The output sum needs

00:00:14.131 --> 00:00:18.128
gradient thirty two, followed by zeros. The original returns

00:00:18.128 --> 00:00:21.535
all ones. Finite differences confirm the reference.

00:00:22.067 --> 00:00:25.890
With a quadratic objective, an actual update raises loss from

00:00:25.890 --> 00:00:29.653
sixteen to sixteen point nine five three one two five. After

00:00:29.653 --> 00:00:33.168
the repair, it falls to one point nine three seven five.

00:00:33.700 --> 00:00:37.175
The backward helper returns the weight gradient transposed.

00:00:37.175 --> 00:00:40.709
The patch restores the physical orientation before grouping.

00:00:40.709 --> 00:00:44.242
Rectangular weights also expose a wrong gradient shape and a

00:00:44.242 --> 00:00:45.226
scale exception.

00:00:45.733 --> 00:00:49.437
The actual C plus plus patch passes two hundred five scenarios

00:00:49.437 --> 00:00:52.905
and three hundred seventy nine checks. Original exceptions

00:00:52.905 --> 00:00:54.903
prevent some checks from running.

00:00:55.433 --> 00:00:59.268
This is separate from the sorted index defect. Tests are

00:00:59.268 --> 00:01:03.034
local, single precision, on C P U with a partial native

00:01:03.034 --> 00:01:06.263
rebuild. G P U and full models remain untested.