WEBVTT

00:00:00.000 --> 00:00:04.052
A trainable mask starts at zero, and learning gets stuck. In M

00:00:04.052 --> 00:00:07.911
L X, block masked matrix multiplication returns a zero mask

00:00:07.911 --> 00:00:10.162
gradient when it should be twelve.

00:00:10.667 --> 00:00:14.912
Two small all ones matrices produce four threes. Multiply by a

00:00:14.912 --> 00:00:19.023
real mask, m. Their sum is twelve times m, so the derivative

00:00:19.023 --> 00:00:20.707
is twelve, even at zero.

00:00:21.233 --> 00:00:24.928
For a quadratic target, loss starts at eighteen. The original

00:00:24.928 --> 00:00:28.622
gradient leaves it there. With the corrected gradient and the

00:00:28.622 --> 00:00:32.197
same step size, the mask becomes one and loss reaches zero.

00:00:32.700 --> 00:00:36.214
The output mask is applied again inside its own backward

00:00:36.214 --> 00:00:40.097
calculation. Removing that extra mask fixes the derivative. No

00:00:40.097 --> 00:00:41.823
division by zero is needed.

00:00:42.333 --> 00:00:46.203
The actual C plus plus repair passes six hundred four checks

00:00:46.203 --> 00:00:50.199
across three hundred seventy seven scenarios. The original has

00:00:50.199 --> 00:00:53.117
seventy seven mismatches. This is one defect.

00:00:53.633 --> 00:00:57.753
These are local, single precision C P U tests with a partial

00:00:57.753 --> 00:01:01.671
rebuild. G P U and complete models were not tested. Code,

00:01:01.671 --> 00:01:04.373
formulas and execution logs are linked.