Skip to content

1.14.4: Fix L1-norm case in `weights_norm`

Compare
Choose a tag to compare
@OverLordGoldDragon OverLordGoldDragon released this 10 Jun 06:37
· 16 commits to master since this release
41a114b

norm_fn=np.abs would compute L1 norm as: np.sqrt(np.sum(np.abs(x))), which is incorrect; the sqrt is redundant. norm_fn=np.abs will now work correctly. L2-norm case always worked correctly.

For L2-norm, set norm_fn = (np.sqrt, np.square) = (outer_fn, inner_fn), which will compute outer_fn(sum(inner_fn(x))). Note that norm_fn=np.square will no longer compute L2-norm correctly.