NormalizeL2¶
Versioned name : NormalizeL2-1
Category : Normalization
Short description : NormalizeL2 operation performs L2 normalization of the 1st input tensor in slices specified by the 2nd input.
Attributes
eps
Description : eps is the number to be added/maximized to/with the variance to avoid division by zero when normalizing the value. For example, eps equal to 0.001 means that 0.001 is used if all the values in normalization are equal to zero.
Range of values : a positive floating-point number
Type :
float
Default value : None
Required : yes
eps_mode
Description : Specifies how eps is combined with L2 value calculated before division.
Range of values :
add
,max
Type :
string
Default value : None
Required : yes
Inputs
1 :
data
- input tensor to be normalized. Type of elements is any floating point type. Required.2 :
axes
- scalar or 1D tensor with axis indices for thedata
input along which L2 reduction is calculated. Required.
Outputs
1 : Tensor of the same shape and type as the
data
input and normalized slices defined byaxes
input.
Detailed Description
Each element in the output is the result of division of corresponding element from the data
input tensor by the result of L2 reduction along dimensions specified by the axes
input:
output[i0, i1, ..., iN] = x[i0, i1, ..., iN] / sqrt(eps_mode(sum[j0,..., jN](x[j0, ..., jN]**2), eps))
Where indices i0, ..., iN
run through all valid indices for the 1st input and summation sum[j0, ..., jN]
have jk = ik
for those dimensions k
that are not in the set of indices specified by the axes
input of the operation. One of the corner cases is when axes
is an empty list, then we divide each input element by itself resulting value 1 for all non-zero elements. Another corner case is where axes
input contains all dimensions from data
tensor, which means that a single L2 reduction value is calculated for entire input tensor and each input element is divided by that value.
eps_mode
selects how the reduction value and eps
are combined. It can be max
or add
depending on eps_mode
attribute value.
Example
<layer id="1" type="NormalizeL2" ...>
<data eps="1e-8" eps_mode="add"/>
<input>
<port id="0">
<dim>6</dim>
<dim>12</dim>
<dim>10</dim>
<dim>24</dim>
</port>
<port id="1">
<dim>2</dim> <!-- value is [2, 3] that means independent normalization in each channel -->
</port>
</input>
<output>
<port id="2">
<dim>6</dim>
<dim>12</dim>
<dim>10</dim>
<dim>24</dim>
</port>
</output>
</layer>