General Adam optimizer struct. More...

#include <aiopti_adam.h>

Data Fields
aiopti_t	base
	Inherited field members from general optimizer struct.

Optimizer configuration
Required configuration parameters for the optimizer These fields have to be configured by the user before calling the initializer function.
void *	beta1
	aiscalar: Exponential decay rate for the first moment estimates \( \beta_1 \)

void *	beta2
	aiscalar: Exponential decay rate for the second moment estimates \( \beta_2 \)

void *	eps
	aiscalar: Small positive number for numerical stability \( \hat{\epsilon} \) (avoid dividing by 0)

Variables for internal computation
These fields are automatically configured in the initializer function.
void *	beta1t
	aiscalar: Auxiliary variable to calculate \( \beta_1^t \)

void *	beta2t
	aiscalar: Auxiliary variable to calculate \( \beta_2^t \)

void *	one_minus_beta1
	aiscalar: Auxiliary variable to calculate \( (1 - \beta_1) \)

void *	one_minus_beta2
	aiscalar: Auxiliary variable to calculate \( (1 - \beta_2) \)

void *	lrt
	aiscalar: Auxiliary variable to calculate \( lr_t = lr \cdot \frac{\sqrt{1-\beta_2^t}}{(1-\beta_1^t)} \)

Math functions
Required data type specific math functions
void(*	multiply )(const aitensor_t a, const aitensor_t b, aitensor_t *result)
	Required math function: Element wise tensor multiplication. More...

void(*	divide )(const aitensor_t a, const aitensor_t b, aitensor_t *result)
	Required math function: Element wise tensor dividation. More...

void(*	tensor_add )(const aitensor_t a, const aitensor_t b, aitensor_t *result)
	Required math function: Element wise tensor addition. More...

void(*	tensor_sub )(const aitensor_t a, const aitensor_t b, aitensor_t *result)
	Required math function: Element wise tensor subtraction. More...

void(*	scalar_mul )(const void scalar, const aitensor_t a, aitensor_t *result)
	Required math function: Multiplication of a scalar with a tensor. More...

void(*	scalar_add )(const void scalar, const aitensor_t a, aitensor_t *result)
	Required math function: Element wise addition of a scalar to a tensor. More...

void(*	sqrt )(const aitensor_t x, aitensor_t result)
	Required math function: Square root. More...

void(*	zero_tensor )(aitensor_t *tensor)
	Required math function: Sets the elements of a tensor to zero. More...

Detailed Description

General Adam optimizer struct.

Field Documentation

◆ divide

void(* divide) (const aitensor_t *a, const aitensor_t *b, aitensor_t *result)

Required math function: Element wise tensor dividation.

Requires a math function that devides two tensors element wise:

\[ result_i = a_i \div b_i \]

◆ multiply

void(* multiply) (const aitensor_t *a, const aitensor_t *b, aitensor_t *result)

Required math function: Element wise tensor multiplication.

Requires a math function that multiplies two tensors element wise:

\[ result = a \odot b \]

◆ scalar_add

void(* scalar_add) (const void *scalar, const aitensor_t *a, aitensor_t *result)

Required math function: Element wise addition of a scalar to a tensor.

Requires a math function that adds a scalar to every element of a tensor:

\[ result = scalar \cdot \left( \begin{array}{c} 1 & \cdots & 1 \\ \vdots & \ddots & \vdots \\ 1 & \cdots & 1 \\ \end{array}\right) + a \]

◆ scalar_mul

void(* scalar_mul) (const void *scalar, const aitensor_t *a, aitensor_t *result)

Required math function: Multiplication of a scalar with a tensor.

Requires a math function that multiplies a scalar with a tensor:

\[ result = scalar \cdot a \]

◆ sqrt

void(* sqrt) (const aitensor_t *x, aitensor_t *result)

Required math function: Square root.

Requires a math function that calculates the element wise square root of a tensor:

\[ result_{i} = \sqrt{x_i} \]

◆ tensor_add

void(* tensor_add) (const aitensor_t *a, const aitensor_t *b, aitensor_t *result)

Required math function: Element wise tensor addition.

Requires a math function that adds two tensors element wise:

\[ result = a + b \]

◆ tensor_sub

void(* tensor_sub) (const aitensor_t *a, const aitensor_t *b, aitensor_t *result)

Required math function: Element wise tensor subtraction.

Requires a math function that subtracts two tensors element wise:

\[ result = a - b \]

◆ zero_tensor

void(* zero_tensor) (aitensor_t *tensor)

Required math function: Sets the elements of a tensor to zero.

Requires a math function that sets the elements of the given tensor to zero (or to the zero_point):

\[ tensor_i = 0 \]

The documentation for this struct was generated from the following file:

aiopti_adam.h

Data Fields