aiopti Struct Reference

AIfES optimizer interface. More...

#include <aifes_core.h>

Data Fields
const aicore_optitype_t *	optimizer_type
	Type of the optimizer (for example aiopti_sgd_type)
void *	optimizer_configuration
	Optimizer specific configurations (back-link from abstract aiopti class to implementation)
const aimath_dtype_t *	dtype
	The data-type of the parameter that the optimizer can optimize and the learning rate.
void *	learning_rate
	The learning rate configures the training speed. More...
uint32_t(*	sizeof_optimem )(aiopti_t *self, const aitensor_t *params)
	Calculates the optimization memory size for a trainable parameter tensor. More...
void(*	init_optimem )(aiopti_t *self, const aitensor_t *params, const aitensor_t *gradients, void *optimem)
	Initialize the optimization memory for a trainable parameter tensor. More...
void(*	zero_gradients )(aiopti_t *self, aitensor_t *gradients)
	Set the gradient tensor to zero. More...
void(*	begin_step )(aiopti_t *self)
	Called in the beginning of every model optimization step for parameter initialization. More...
void(*	update_params )(aiopti_t *self, aitensor_t *params, const aitensor_t *gradients, void *optimem)
	Performs an optimization step on the given tensor. More...
void(*	end_step )(aiopti_t *self)
	Called in the end of every model optimization step. More...

Detailed Description

AIfES optimizer interface.

The interface contains the necessary functions and parameters for parameter optimizers in backpropagation training. (Refer to aifes_core.h for a structural overview)
Optimizers are responsible for updating the trainable parameters of the model with the gradients calculated in the backward pass.

The call order of the functions is:

allocate memory of size sizeof_optimem()
init_optimem()
 
for batch in dataset
    for each trainable parameter tensor in the model
        zero_gradients()
    endfor
 
    forward_model(batch)
    backward_model(batch)
 
    begin_step()
    for each trainable parameter tensor in the model
        update_params()
    endfor
    end_step()
endfor

Field Documentation

begin_step

void(* begin_step) (aiopti_t *self)

Called in the beginning of every model optimization step for parameter initialization.

Parameters

self

The layer

end_step

void(* end_step) (aiopti_t *self)

Called in the end of every model optimization step.

Parameters

self

The layer

init_optimem

void(* init_optimem) (aiopti_t *self, const aitensor_t *params, const aitensor_t *gradients, void *optimem)

Initialize the optimization memory for a trainable parameter tensor.

Parameters

self	The layer
params	The trainable parameter tensor
gradients	The associated gradients tensor
optimem	The associated optimization memory to initialize

learning_rate

void* learning_rate

The learning rate configures the training speed.

The learning rate is an aiscalar_t value of given dtype.

sizeof_optimem

uint32_t(* sizeof_optimem) (aiopti_t *self, const aitensor_t *params)

Calculates the optimization memory size for a trainable parameter tensor.

Parameters

self	The layer
params	The trainable parameter tensor

update_params

void(* update_params) (aiopti_t *self, aitensor_t *params, const aitensor_t *gradients, void *optimem)

Performs an optimization step on the given tensor.

Parameters

self	The layer
params	The trainable parameter tensor
gradients	The associated gradients tensor
optimem	The associated optimization memory to initialize

zero_gradients

void(* zero_gradients) (aiopti_t *self, aitensor_t *gradients)

Set the gradient tensor to zero.

Parameters

self	The layer
params	The gradient tensor

---

The documentation for this struct was generated from the following file:

• aifes_core.h