SoftSign

Versioned name: SoftSign-9

Category: Activation function

Short description: SoftSign performs element-wise activation on a given input tensor.

Detailed description:

SoftSign operation is introduced in this article.

SoftSign Activation Function is a neuron activation function based on the mathematical function:

\[SoftSign(x) = \frac{x}{1+|x|}\]

Inputs:

• 1: data. Input tensor of type T

Outputs:

• 1: The resulting tensor of the same shape and type as the input tensor.

Types:

• T: Arbitrary supported floating-point type.

Example

<layer ... type="SoftSign">
    <input>
        <port id="0">
            <dim>256</dim>
            <dim>56</dim>
        </port>
    </input>
    <output>
        <port id="1">
            <dim>256</dim>
            <dim>56</dim>
        </port>
    </output>
</layer>