SoftSign

Versioned name: SoftSign-9

Category: Activation function

Short description: SoftSign performs element-wise activation on a given input tensor.

Detailed description:

SoftSign operation is introduced in this article.

SoftSign Activation Function is a neuron activation function based on the mathematical function:

\[SoftSign(x) = \frac{x}{1+|x|}\]

Inputs:

• 1: data. Input tensor of type T

Outputs:

• 1: The resulting tensor of the same shape and type as the input tensor.

Types:

• T: Arbitrary supported floating-point type.

Example

 <layer ... type="SoftSign">
     <input>
         <port id="0">
             <dim>256</dim>
             <dim>56</dim>
         </port>
     </input>
     <output>
         <port id="1">
             <dim>256</dim>
             <dim>56</dim>
         </port>
     </output>
 </layer>