Skip to main content
`Ctrl`+`K`

# Activation

MIOpen: Activation
Enumerations | Functions
Activation

## Enumerations

enum | miopenActivationMode_t { miopenActivationPASTHRU = 0 , miopenActivationLOGISTIC = 1 , miopenActivationTANH = 2 , miopenActivationRELU = 3 , miopenActivationSOFTRELU = 4 , miopenActivationABS = 5 , miopenActivationPOWER = 6 , miopenActivationCLIPPEDRELU , miopenActivationLEAKYRELU , miopenActivationELU , miopenActivationCLAMP = 10 }

## Functions

| MIOPEN_DECLARE_OBJECT (miopenActivationDescriptor)
| Creates the miopenActivationDescriptor_t type.
miopenStatus_t | miopenCreateActivationDescriptor (miopenActivationDescriptor_t *activDesc)
| Creates the Activation descriptor object.
miopenStatus_t | miopenSetActivationDescriptor (const miopenActivationDescriptor_t activDesc, miopenActivationMode_t mode, double activAlpha, double activBeta, double activGamma)
| Sets the activation layer descriptor details.
miopenStatus_t | miopenGetActivationDescriptor (const miopenActivationDescriptor_t activDesc, miopenActivationMode_t *mode, double *activAlpha, double *activBeta, double *activGamma)
| Gets the activation layer descriptor details.
miopenStatus_t | miopenActivationForward (miopenHandle_t handle, const miopenActivationDescriptor_t activDesc, const void *alpha, const miopenTensorDescriptor_t xDesc, const void *x, const void *beta, const miopenTensorDescriptor_t yDesc, void *y)
| Execute an activation forward layer.
miopenStatus_t | miopenActivationBackward (miopenHandle_t handle, const miopenActivationDescriptor_t activDesc, const void *alpha, const miopenTensorDescriptor_t yDesc, const void *y, const miopenTensorDescriptor_t dyDesc, const void *dy, const miopenTensorDescriptor_t xDesc, const void *x, const void *beta, const miopenTensorDescriptor_t dxDesc, void *dx)
| Execute a activation backwards layer.
miopenStatus_t | miopenDestroyActivationDescriptor (miopenActivationDescriptor_t activDesc)
| Destroys the activation descriptor object.
miopenStatus_t | miopenGLUForward (miopenHandle_t handle, const miopenTensorDescriptor_t inputDesc, const void *input, const miopenTensorDescriptor_t outputDesc, void *output, const uint32_t dim)
| Execute a GLU forward layer.
miopenStatus_t | miopenGLUBackward (miopenHandle_t handle, const miopenTensorDescriptor_t inputDesc, const void *input, const miopenTensorDescriptor_t outputGradDesc, const void *outputGrad, const miopenTensorDescriptor_t inputGradDesc, void *inputGrad, const uint32_t dim)
| Execute a GLU backward layer.

## Detailed Description

## Enumeration Type Documentation

## ◆ miopenActivationMode_t

enum miopenActivationMode_t

Activation layer modes

Enumerator
miopenActivationPASTHRU | No activation, pass through the data
miopenActivationLOGISTIC | Sigmoid function: \(1 / (1 + e^{-x})\)
miopenActivationTANH | Tanh activation \( \beta * tanh( \alpha * x) \)
miopenActivationRELU | Rectified Linear Unit \( max(0, x) \)
miopenActivationSOFTRELU | \(log(1 + e^x)\)
miopenActivationABS | Absolute value \(abs(x)\)
miopenActivationPOWER | Scaled and shifted power \((\alpha + \beta * x)^{gamma}\)
miopenActivationCLIPPEDRELU | Clipped Rectified Linear Unit \( max(0, min(\alpha, x)) \)
miopenActivationLEAKYRELU | Leaky Rectified Linear Unit \( \alpha * x | x <= 0; x | x > 0 \)
miopenActivationELU | Exponential Rectified Linear Unit \( \alpha * (e^{x} - 1) | x <= 0; x | x > 0 \)
miopenActivationCLAMP | Clamp \( max(\alpha, min(\beta, x)) \)

## Function Documentation

## ◆ MIOPEN_DECLARE_OBJECT()

MIOPEN_DECLARE_OBJECT | ( | miopenActivationDescriptor |  | ) |

Creates the miopenActivationDescriptor_t type.

Activation descriptor is an object that allows the user to specify the activation mode.

## ◆ miopenActivationBackward()

miopenStatus_t miopenActivationBackward | ( | miopenHandle_t | handle,
|  | const miopenActivationDescriptor_t | activDesc,
|  | const void * | alpha,
|  | const miopenTensorDescriptor_t | yDesc,
|  | const void * | y,
|  | const miopenTensorDescriptor_t | dyDesc,
|  | const void * | dy,
|  | const miopenTensorDescriptor_t | xDesc,
|  | const void * | x,
|  | const void * | beta,
|  | const miopenTensorDescriptor_t | dxDesc,
|  | void * | dx )

Execute a activation backwards layer.

Parameters

  handle | MIOpen handle (input)
  activDesc | Descriptor for activation layer (input)
  alpha | Floating point scaling factor, allocated on the host (input)
  yDesc | Tensor descriptor for input data tensor y (input)
  y | Data tensor y (input)
  dyDesc | Tensor descriptor for input data tensor dy (input)
  dy | Data delta tensor dy (input)
  xDesc | Tensor descriptor for data input tensor x (input)
  x | Data tensor x (input)
  beta | Floating point shift factor, allocated on the host (input)
  dxDesc | Tensor descriptor for data output tensor dx (input)
  dx | Output data delta tensor dx (output)

Returns
  miopenStatus_t

## ◆ miopenActivationForward()

miopenStatus_t miopenActivationForward | ( | miopenHandle_t | handle,
|  | const miopenActivationDescriptor_t | activDesc,
|  | const void * | alpha,
|  | const miopenTensorDescriptor_t | xDesc,
|  | const void * | x,
|  | const void * | beta,
|  | const miopenTensorDescriptor_t | yDesc,
|  | void * | y )

Execute an activation forward layer.

Parameters

  handle | MIOpen handle (input)
  activDesc | Descriptor for activation layer (input)
  alpha | Floating point scaling factor, allocated on the host (input)
  xDesc | Tensor descriptor for data input tensor x (input)
  x | Data tensor x (input)
  beta | Floating point shift factor, allocated on the host (input)
  yDesc | Tensor descriptor for output data tensor y (input)
  y | Data tensor y (output)

Returns
  miopenStatus_t

## ◆ miopenCreateActivationDescriptor()

miopenStatus_t miopenCreateActivationDescriptor | ( | miopenActivationDescriptor_t * | activDesc | ) |

Creates the Activation descriptor object.

Parameters

  activDesc | Pointer to an activation tensor descriptor type

Returns
  miopenStatus_t

## ◆ miopenDestroyActivationDescriptor()

miopenStatus_t miopenDestroyActivationDescriptor | ( | miopenActivationDescriptor_t | activDesc | ) |

Destroys the activation descriptor object.

Parameters

  activDesc | Activation tensor descriptor type (input)

Returns
  miopenStatus_t

## ◆ miopenGetActivationDescriptor()

miopenStatus_t miopenGetActivationDescriptor | ( | const miopenActivationDescriptor_t | activDesc,
|  | miopenActivationMode_t * | mode,
|  | double * | activAlpha,
|  | double * | activBeta,
|  | double * | activGamma )

Gets the activation layer descriptor details.

Retrieves all of the descriptor details for the activation layer.

Parameters

  activDesc | Pointer to a activation layer descriptor (input)
  mode | Activation mode enum (output)
  activAlpha | Alpha value for some activation modes (output)
  activBeta | Beta value for some activation modes (output)
  activGamma | Gamma value for some activation modes (output)

Returns
  miopenStatus_t

## ◆ miopenGLUBackward()

miopenStatus_t miopenGLUBackward | ( | miopenHandle_t | handle,
|  | const miopenTensorDescriptor_t | inputDesc,
|  | const void * | input,
|  | const miopenTensorDescriptor_t | outputGradDesc,
|  | const void * | outputGrad,
|  | const miopenTensorDescriptor_t | inputGradDesc,
|  | void * | inputGrad,
|  | const uint32_t | dim )

Execute a GLU backward layer.

Parameters

  handle | MIOpen handle (input)
  inputDesc | Tensor descriptor for input tensor (input)
  input | Input tensor (input)
  outputGradDesc | Tensor descriptor for delta output tensor (input)
  outputGrad | Delta output tensor (input)
  inputGradDesc | Tensor descriptor for delta input tensor (input)
  inputGrad | Delta input tensor (output)
  dim | Dimension to split the input (input)

Returns
  miopenStatus_t

## ◆ miopenGLUForward()

miopenStatus_t miopenGLUForward | ( | miopenHandle_t | handle,
|  | const miopenTensorDescriptor_t | inputDesc,
|  | const void * | input,
|  | const miopenTensorDescriptor_t | outputDesc,
|  | void * | output,
|  | const uint32_t | dim )

Execute a GLU forward layer.

Parameters

  handle | MIOpen handle (input)
  inputDesc | Tensor descriptor for input tensor (input)
  input | Input tensor (input)
  outputDesc | Tensor descriptor for output tensor (input)
  output | Output tensor (output)
  dim | Dimension to split the input (input)

Returns
  miopenStatus_t

## ◆ miopenSetActivationDescriptor()

miopenStatus_t miopenSetActivationDescriptor | ( | const miopenActivationDescriptor_t | activDesc,
|  | miopenActivationMode_t | mode,
|  | double | activAlpha,
|  | double | activBeta,
|  | double | activGamma )

Sets the activation layer descriptor details.

Sets all of the descriptor details for the activation layer

Parameters

  activDesc | Pointer to a activation layer descriptor (output)
  mode | Activation mode enum (input)
  activAlpha | Alpha value for some activation modes (input)
  activBeta | Beta value for some activation modes (input)
  activGamma | Gamma value for some activation modes (input)

Returns
  miopenStatus_t