WEBVTT

00:00:00.103 --> 00:00:02.178
If a function always returns one,

00:00:02.603 --> 00:00:04.755
its input gradient should be zero.

00:00:05.503 --> 00:00:07.346
In N N Trainer, a zero

00:00:07.359 --> 00:00:09.459
exponent gives that constant output.

00:00:09.897 --> 00:00:12.178
But at zero input, backward returns

00:00:12.191 --> 00:00:12.939
not a number.

00:00:13.364 --> 00:00:15.426
Tiny nonzero inputs can fail too.

00:00:16.183 --> 00:00:18.760
The derivative code evaluates a reciprocal

00:00:18.863 --> 00:00:20.590
before multiplying by zero.

00:00:21.028 --> 00:00:23.167
Handling the constant case first avoids

00:00:23.180 --> 00:00:24.610
that invalid intermediate.

00:00:25.370 --> 00:00:27.548
After the correction, all three hundred

00:00:27.689 --> 00:00:30.009
eighty-five cases and twenty-five Pow

00:00:30.022 --> 00:00:30.988
tests passed.

00:00:31.414 --> 00:00:33.411
Restoring the original code brings the

00:00:33.424 --> 00:00:34.377
failures back.

00:00:34.816 --> 00:00:36.916
Production-model impact was not measured.