Loading...
Searching...
No Matches
itk::AdaptiveStochasticVarianceReducedGradientOptimizer Class Reference
Components » Optimizers

`#include <itkAdaptiveStochasticVarianceReducedGradientOptimizer.h>`

## Detailed Description

This class implements a gradient descent optimizer with adaptive gain.

If is a cost function that has to be minimized, the following iterative algorithm is used to find the optimal parameters :

The gain at each iteration is defined by:

.

And the time is updated according to:

where equals at iteration . For the InitialTime is used, which is defined in the the superclass (StandardGradientDescentOptimizer). Whereas in the superclass this parameter is superfluous, in this class it makes sense.

This method is described in the following references:

[1] P. Cruz, "Almost sure convergence and asymptotical normality of a generalization of Kesten's stochastic approximation algorithm for multidimensional case." Technical Report, 2005. http://hdl.handle.net/2052/74

[2] S. Klein, J.P.W. Pluim, and M. Staring, M.A. Viergever, "Adaptive stochastic gradient descent optimisation for image registration," International Journal of Computer Vision, vol. 81, no. 3, pp. 227-239, 2009. http://dx.doi.org/10.1007/s11263-008-0168-y It is very suitable to be used in combination with a stochastic estimate of the gradient . For example, in image registration problems it is often advantageous to compute the metric derivative ( ) on a new set of randomly selected image samples in each iteration. You may set the parameter `NewSamplesEveryIteration` to `"true"` to achieve this effect. For more information on this strategy, you may have a look at:

See also
  AdaptiveStochasticVarianceReducedGradient, StandardGradientDescentOptimizer

If is a costfunction that has to be minimised, the following iterative algorithm is used to find the optimal parameters :

The gain at each iteration is defined by:

.

And the time is updated according to:

where equals at iteration . For the InitialTime is used, which is defined in the the superclass (StandardGradientDescentOptimizer). Whereas in the superclass this parameter is superfluous, in this class it makes sense.

This method is described in the following references:

[1] P. Cruz, "Almost sure convergence and asymptotical normality of a generalization of Kesten's stochastic approximation algorithm for multidimensional case." Technical Report, 2005. http://hdl.handle.net/2052/74

[2] S. Klein, J.P.W. Pluim, and M. Staring, M.A. Viergever, "Adaptive stochastic gradient descent optimisation for image registration," International Journal of Computer Vision, vol. 81, no. 3, pp. 227-239, 2009. http://dx.doi.org/10.1007/s11263-008-0168-y

It is very suitable to be used in combination with a stochastic estimate of the gradient . For example, in image registration problems it is often advantageous to compute the metric derivative ( ) on a new set of randomly selected image samples in each iteration. You may set the parameter `NewSamplesEveryIteration` to `"true"` to achieve this effect. For more information on this strategy, you may have a look at:

See also
  AdaptiveStochasticVarianceReducedGradient, StandardGradientDescentOptimizer

Definition at line 70 of file itkAdaptiveStochasticVarianceReducedGradientOptimizer.h.

Inheritance diagram for itk::AdaptiveStochasticVarianceReducedGradientOptimizer:
[legend]

## Public Types

using | ConstPointer = SmartPointer<const Self>
using | Pointer = SmartPointer<Self>
using | Self = AdaptiveStochasticVarianceReducedGradientOptimizer
using | Superclass = StandardStochasticVarianceReducedGradientOptimizer
Public Types inherited from itk::StandardStochasticVarianceReducedGradientOptimizer
using | ConstPointer = SmartPointer<const Self>
using | Pointer = SmartPointer<Self>
using | Self = StandardStochasticVarianceReducedGradientOptimizer
enum | StopConditionType
using | Superclass = StochasticVarianceReducedGradientDescentOptimizer
Public Types inherited from itk::StochasticVarianceReducedGradientDescentOptimizer
using | ConstPointer = SmartPointer<const Self>
using | Pointer = SmartPointer<Self>
using | ScaledCostFunctionPointer
using | ScaledCostFunctionType
using | ScalesType
using | Self = StochasticVarianceReducedGradientDescentOptimizer
enum | StopConditionType { MaximumNumberOfIterations , MetricError , MinimumStepSize , InvalidDiagonalMatrix , GradientMagnitudeTolerance , LineSearchError }
using | Superclass = ScaledSingleValuedNonLinearOptimizer
Public Types inherited from itk::ScaledSingleValuedNonLinearOptimizer
using | ConstPointer = SmartPointer<const Self>
using | Pointer = SmartPointer<Self>
using | ScaledCostFunctionPointer = ScaledCostFunctionType::Pointer
using | ScaledCostFunctionType = ScaledSingleValuedCostFunction
using | ScalesType = NonLinearOptimizer::ScalesType
using | Self = ScaledSingleValuedNonLinearOptimizer
using | Superclass = SingleValuedNonLinearOptimizer

## Public Member Functions

virtual const char * | GetClassName () const
virtual double | GetSigmoidMax () const
virtual double | GetSigmoidMin () const
virtual double | GetSigmoidScale () const
virtual bool | GetUseAdaptiveStepSizes () const
| ITK_DISALLOW_COPY_AND_MOVE (AdaptiveStochasticVarianceReducedGradientOptimizer)
virtual void | SetSigmoidMax (double _arg)
virtual void | SetSigmoidMin (double _arg)
virtual void | SetSigmoidScale (double _arg)
virtual void | SetUseAdaptiveStepSizes (bool _arg)
Public Member Functions inherited from itk::StandardStochasticVarianceReducedGradientOptimizer
void | AdvanceOneStep () override
virtual double | GetCurrentTime () const
virtual double | GetInitialTime () const
virtual double | GetParam_A () const
virtual double | GetParam_a () const
virtual double | GetParam_alpha () const
virtual double | GetParam_beta () const
| ITK_DISALLOW_COPY_AND_MOVE (StandardStochasticVarianceReducedGradientOptimizer)
virtual void | ResetCurrentTimeToInitialTime ()
virtual void | SetInitialTime (double _arg)
virtual void | SetParam_A (double _arg)
virtual void | SetParam_a (double _arg)
virtual void | SetParam_alpha (double _arg)
virtual void | SetParam_beta (double _arg)
void | StartOptimization () override
Public Member Functions inherited from itk::StochasticVarianceReducedGradientDescentOptimizer
virtual unsigned int | GetCurrentInnerIteration () const
virtual unsigned int | GetCurrentIteration () const
virtual const DerivativeType & | GetGradient ()
virtual unsigned int | GetLBFGSMemory () const
virtual const double & | GetLearningRate ()
virtual const unsigned long & | GetNumberOfInnerIterations ()
virtual const unsigned long & | GetNumberOfIterations ()
virtual const DerivativeType & | GetPreviousGradient ()
virtual const ParametersType & | GetPreviousPosition ()
virtual const DerivativeType & | GetSearchDir ()
virtual const StopConditionType & | GetStopCondition ()
virtual const double & | GetValue ()
| ITK_DISALLOW_COPY_AND_MOVE (StochasticVarianceReducedGradientDescentOptimizer)
virtual void | MetricErrorResponse (ExceptionObject &err)
virtual void | ResumeOptimization ()
virtual void | SetLearningRate (double _arg)
virtual void | SetNumberOfIterations (unsigned long _arg)
void | SetNumberOfWorkUnits (ThreadIdType numberOfThreads)
virtual void | SetPreviousGradient (DerivativeType _arg)
virtual void | SetPreviousPosition (ParametersType _arg)
virtual void | SetUseEigen (bool _arg)
virtual void | SetUseMultiThread (bool _arg)
void | StartOptimization () override
virtual void | StopOptimization ()
Public Member Functions inherited from itk::ScaledSingleValuedNonLinearOptimizer
const ParametersType & | GetCurrentPosition () const override
virtual bool | GetMaximize () const
virtual const ScaledCostFunctionType * | GetScaledCostFunction ()
virtual const ParametersType & | GetScaledCurrentPosition ()
bool | GetUseScales () const
virtual void | InitializeScales ()
| ITK_DISALLOW_COPY_AND_MOVE (ScaledSingleValuedNonLinearOptimizer)
virtual void | MaximizeOff ()
virtual void | MaximizeOn ()
void | SetCostFunction (CostFunctionType *costFunction) override
virtual void | SetMaximize (bool _arg)
virtual void | SetUseScales (bool arg)

## Static Public Member Functions

static Pointer | New ()
Static Public Member Functions inherited from itk::StandardStochasticVarianceReducedGradientOptimizer
static Pointer | New ()
Static Public Member Functions inherited from itk::StochasticVarianceReducedGradientDescentOptimizer
static Pointer | New ()
Static Public Member Functions inherited from itk::ScaledSingleValuedNonLinearOptimizer
static Pointer | New ()

## Protected Member Functions

| AdaptiveStochasticVarianceReducedGradientOptimizer ()
void | UpdateCurrentTime () override
| ~AdaptiveStochasticVarianceReducedGradientOptimizer () override=default
Protected Member Functions inherited from itk::StandardStochasticVarianceReducedGradientOptimizer
virtual double | Compute_a (double k) const
virtual double | Compute_beta (double k) const
| StandardStochasticVarianceReducedGradientOptimizer ()
| ~StandardStochasticVarianceReducedGradientOptimizer () override=default
Protected Member Functions inherited from itk::StochasticVarianceReducedGradientDescentOptimizer
void | PrintSelf (std::ostream &os, Indent indent) const override
| StochasticVarianceReducedGradientDescentOptimizer ()
| ~StochasticVarianceReducedGradientDescentOptimizer () override=default
Protected Member Functions inherited from itk::ScaledSingleValuedNonLinearOptimizer
virtual void | GetScaledDerivative (const ParametersType &parameters, DerivativeType &derivative) const
virtual MeasureType | GetScaledValue (const ParametersType &parameters) const
virtual void | GetScaledValueAndDerivative (const ParametersType &parameters, MeasureType &value, DerivativeType &derivative) const
void | PrintSelf (std::ostream &os, Indent indent) const override
| ScaledSingleValuedNonLinearOptimizer ()
void | SetCurrentPosition (const ParametersType &param) override
virtual void | SetScaledCurrentPosition (const ParametersType &parameters)
| ~ScaledSingleValuedNonLinearOptimizer () override=default

## Protected Attributes

DerivativeType | m_PreviousGradient {}
Protected Attributes inherited from itk::StandardStochasticVarianceReducedGradientOptimizer
double | m_CurrentTime { 0.0 }
bool | m_UseConstantStep {}
Protected Attributes inherited from itk::StochasticVarianceReducedGradientDescentOptimizer
unsigned long | m_CurrentInnerIteration {}
unsigned long | m_CurrentIteration { 0 }
DerivativeType | m_Gradient {}
unsigned long | m_LBFGSMemory { 0 }
double | m_LearningRate { 1.0 }
ParametersType | m_MeanSearchDir {}
unsigned long | m_NumberOfInnerIterations {}
unsigned long | m_NumberOfIterations { 100 }
DerivativeType | m_PreviousGradient {}
ParametersType | m_PreviousPosition {}
ParametersType | m_PreviousSearchDir {}
ParametersType | m_SearchDir {}
bool | m_Stop { false }
StopConditionType | m_StopCondition { MaximumNumberOfIterations }
MultiThreaderBase::Pointer | m_Threader { MultiThreaderBase::New() }
double | m_Value { 0.0 }
Protected Attributes inherited from itk::ScaledSingleValuedNonLinearOptimizer
ScaledCostFunctionPointer | m_ScaledCostFunction {}
ParametersType | m_ScaledCurrentPosition {}

## Private Attributes

double | m_SigmoidMax { 1.0 }
double | m_SigmoidMin { -0.8 }
double | m_SigmoidScale { 1e-8 }
bool | m_UseAdaptiveStepSizes { true }

## Additional Inherited Members

Protected Types inherited from itk::StochasticVarianceReducedGradientDescentOptimizer
using | ThreadInfoType = MultiThreaderBase::WorkUnitInfo

## Member Typedef Documentation

## ConstPointer

using itk::AdaptiveStochasticVarianceReducedGradientOptimizer::ConstPointer = SmartPointer<const Self>

Definition at line 80 of file itkAdaptiveStochasticVarianceReducedGradientOptimizer.h.

## Pointer

using itk::AdaptiveStochasticVarianceReducedGradientOptimizer::Pointer = SmartPointer<Self>

Definition at line 79 of file itkAdaptiveStochasticVarianceReducedGradientOptimizer.h.

## Self

using itk::AdaptiveStochasticVarianceReducedGradientOptimizer::Self = AdaptiveStochasticVarianceReducedGradientOptimizer

Standard ITK.

Definition at line 76 of file itkAdaptiveStochasticVarianceReducedGradientOptimizer.h.

## Superclass

using itk::AdaptiveStochasticVarianceReducedGradientOptimizer::Superclass = StandardStochasticVarianceReducedGradientOptimizer

Definition at line 77 of file itkAdaptiveStochasticVarianceReducedGradientOptimizer.h.

## Constructor & Destructor Documentation

## AdaptiveStochasticVarianceReducedGradientOptimizer()

itk::AdaptiveStochasticVarianceReducedGradientOptimizer::AdaptiveStochasticVarianceReducedGradientOptimizer | ( |  | ) |

protected

## ~AdaptiveStochasticVarianceReducedGradientOptimizer()

itk::AdaptiveStochasticVarianceReducedGradientOptimizer::~AdaptiveStochasticVarianceReducedGradientOptimizer | ( |  | ) |

override protected default

## Member Function Documentation

## GetClassName()

virtual const char * itk::AdaptiveStochasticVarianceReducedGradientOptimizer::GetClassName | ( |  | ) | const

virtual

Run-time type information (and related methods).

Reimplemented from itk::StandardStochasticVarianceReducedGradientOptimizer.

Reimplemented in elastix::AdaptiveStochasticVarianceReducedGradient< TElastix >.

## GetSigmoidMax()

virtual double itk::AdaptiveStochasticVarianceReducedGradientOptimizer::GetSigmoidMax | ( |  | ) | const

virtual

## GetSigmoidMin()

virtual double itk::AdaptiveStochasticVarianceReducedGradientOptimizer::GetSigmoidMin | ( |  | ) | const

virtual

## GetSigmoidScale()

virtual double itk::AdaptiveStochasticVarianceReducedGradientOptimizer::GetSigmoidScale | ( |  | ) | const

virtual

## GetUseAdaptiveStepSizes()

virtual bool itk::AdaptiveStochasticVarianceReducedGradientOptimizer::GetUseAdaptiveStepSizes | ( |  | ) | const

virtual

## ITK_DISALLOW_COPY_AND_MOVE()

itk::AdaptiveStochasticVarianceReducedGradientOptimizer::ITK_DISALLOW_COPY_AND_MOVE | ( | AdaptiveStochasticVarianceReducedGradientOptimizer |  | ) |

## New()

Pointer itk::AdaptiveStochasticVarianceReducedGradientOptimizer::New | ( |  | ) |

static

Method for creation through the object factory.

## SetSigmoidMax()

virtual void itk::AdaptiveStochasticVarianceReducedGradientOptimizer::SetSigmoidMax | ( | double | _arg | ) |

virtual

Set/Get the maximum of the sigmoid. Should be >0. Default: 1.0

## SetSigmoidMin()

virtual void itk::AdaptiveStochasticVarianceReducedGradientOptimizer::SetSigmoidMin | ( | double | _arg | ) |

virtual

Set/Get the maximum of the sigmoid. Should be <0. Default: -0.8

## SetSigmoidScale()

virtual void itk::AdaptiveStochasticVarianceReducedGradientOptimizer::SetSigmoidScale | ( | double | _arg | ) |

virtual

Set/Get the scaling of the sigmoid width. Large values cause a more wide sigmoid. Default: 1e-8. Should be >0.

## SetUseAdaptiveStepSizes()

virtual void itk::AdaptiveStochasticVarianceReducedGradientOptimizer::SetUseAdaptiveStepSizes | ( | bool | _arg | ) |

virtual

Set/Get whether the adaptive step size mechanism is desired. Default: true

## UpdateCurrentTime()

void itk::AdaptiveStochasticVarianceReducedGradientOptimizer::UpdateCurrentTime | ( |  | ) |

override protected virtual

Function to update the current time If UseAdaptiveStepSizes is false this function just increments the CurrentTime by . Else, the CurrentTime is updated according to:
time = max[ 0, time + sigmoid( -gradient*previousgradient) ]
In that case, also the m_PreviousGradient is updated.

Reimplemented from itk::StandardStochasticVarianceReducedGradientOptimizer.

## Field Documentation

## m_PreviousGradient

DerivativeType itk::AdaptiveStochasticVarianceReducedGradientOptimizer::m_PreviousGradient {}

protected

The PreviousGradient, necessary for the CruzAcceleration

Definition at line 132 of file itkAdaptiveStochasticVarianceReducedGradientOptimizer.h.

## m_SigmoidMax

double itk::AdaptiveStochasticVarianceReducedGradientOptimizer::m_SigmoidMax { 1.0 }

private

Definition at line 137 of file itkAdaptiveStochasticVarianceReducedGradientOptimizer.h.

## m_SigmoidMin

double itk::AdaptiveStochasticVarianceReducedGradientOptimizer::m_SigmoidMin { -0.8 }

private

Definition at line 138 of file itkAdaptiveStochasticVarianceReducedGradientOptimizer.h.

## m_SigmoidScale

double itk::AdaptiveStochasticVarianceReducedGradientOptimizer::m_SigmoidScale { 1e-8 }

private

Definition at line 139 of file itkAdaptiveStochasticVarianceReducedGradientOptimizer.h.

## m_UseAdaptiveStepSizes

bool itk::AdaptiveStochasticVarianceReducedGradientOptimizer::m_UseAdaptiveStepSizes { true }

private

Settings

Definition at line 136 of file itkAdaptiveStochasticVarianceReducedGradientOptimizer.h.

Generated on 1768596610 for elastix by 1.15.0 |