ailayer Struct Reference

AIfES layer interface. More...

#include <aifes_core.h>

Data Fields
const aicore_layertype_t *	layer_type
	Type of the layer (for example ailayer_dense_type)
void *	layer_configuration
	Layer specific configurations (back-link from abstract layer class to implementation)
uint32_t	settings
	General layer settings like freezing weights or switching between training and evaluation mode. More...
aitensor_t	result
	The result of the forward function is stored here.
void(*	calc_result_shape )(ailayer_t *self)
	Calculate and write the shape to the result tensor. More...
void(*	calc_result_tensor_params )(ailayer_t *self)
	If available, calculate and set the tensor_params of the result tensor. More...
void(*	forward )(ailayer_t *self)
	Calculate the forward pass and write the result to the result tensor. More...
aitensor_t	deltas
	The result of the backward function is stored here.
void(*	backward )(ailayer_t *self)
	Calculate the backward pass and write the result to the deltas tensor. More...
Layer connections
Defines the model graph.
ailayer_t *	input_layer
ailayer_t *	output_layer
Inference and training scheduling order (Not in use yet)
The scheduler can execute the layers along this path.
ailayer_t *	next_scheduled
ailayer_t *	prev_scheduled
Training memory API
Makes the memory of the trainable params, the gradients and optimizer stuff accessible. This is used, for example, for the optimiser or debugging purposes.
uint8_t	trainable_params_count
	Number of trainable parameter tensors.
aitensor_t **	trainable_params
	Array of tensor pointers with length trainable_params_count.
aitensor_t **	gradients
	Array of tensor pointers with length trainable_params_count.
void **	optimem
	Array of memory pointers with length trainable_params_count.
Parameter memory
Calculate the size and set the memory for the parameter. This memory (for example for weights, bias, ...) will last through all the lifetime of the model. This is only intended for training when no initial weights are available. If the parameters are already known, set the parameter directly to the layer.
uint32_t(*	sizeof_paramem )(const ailayer_t *self)
	Size of required memory (in bytes).
void(*	set_paramem )(ailayer_t *self, void *memory_ptr)
	Set and distribute the memory block internally.
void(*	init_params )(ailayer_t *self)
	Initialize the (trainable and not trainable) parameters of the layer with default initializers.
Temporary memory for forward and backward pass
Calculate the size of the required memory for temporary result in forward and backward pass
uint32_t(*	sizeof_fwdmem )(const ailayer_t *self)
	Size of required memory for the forward pass (in bytes).
uint32_t(*	sizeof_bwdmem )(const ailayer_t *self)
	Size of required memory for the backward pass (in bytes).
void *	tempmem
	Pointer to the memory for the forward pass, backward pass and the optimizer.
Training memory
Calculate the size and set the working memory for the training This memory (for example for gradients, momentums, ...) is needed during the whole training process. If the training is finished, it can be deleted.
uint32_t(*	sizeof_trainmem )(const ailayer_t *self)
	Size of required memory (in bytes).
void(*	set_trainmem )(ailayer_t *self, void *memory_ptr)
	Set and distribute the memory block internally.

Detailed Description

AIfES layer interface.

The interface contains the necessary functions and parameters for inference and training on the model. (Refer to aifes_core.h for a structural overview)

The call order of the functions for inference:

for each layer in the model
    calc_result_shape()
endfor
 
for each layer in the model
    forward()
endfor
// The result of the inference is now in output_layer.result tensor

The call order of the functions for training:

for each layer in the model
    calc_result_shape()
endfor
 
// If the parameters are not already pretrained, a new parameter memory block can be created
for each layer in the model
    allocate memory of size sizeof_paramem()
    set_paramem()
endfor
init_trainable_parameters_model() // Do some weight initialization
 
for each layer in the model
    allocate memory of size sizeof_trainmem()
    set_trainmem()
endfor
 
init_trainmem_model() // For example set the optimem to zero
for iterations
    for each batch in dataset
        zero_gradients_model()
 
        for each sample in the batch
            for each layer in the model
                forward()
            endfor
 
            calc_delta()
            for each layer in the model (reverse)
                backward()
            endfor
        endfor
 
        update_params_model()
    endfor
endfor

Field Documentation

backward

void(* backward) (ailayer_t *self)

Calculate the backward pass and write the result to the deltas tensor.

Parameters

self

The layer

calc_result_shape

void(* calc_result_shape) (ailayer_t *self)

Calculate and write the shape to the result tensor.

Made for easy creation of the model (layers can be connected to each other without worrying about the shapes).

Parameters

self

The layer

calc_result_tensor_params

void(* calc_result_tensor_params) (ailayer_t *self)

If available, calculate and set the tensor_params of the result tensor.

Some layers (like some activation functions) have pre-defined tensor_params that can be set to the result tensor with this function. Set this function to 0 if the tensor_params calculation is not trivial (e.g. dependent on input data).

Parameters

self

The layer

forward

void(* forward) (ailayer_t *self)

Calculate the forward pass and write the result to the result tensor.

Parameters

self

The layer

settings

uint32_t settings

General layer settings like freezing weights or switching between training and evaluation mode.

Example: Read a value from the settings

if(AILAYER_SETTINGS_IS(layer->settings, 0b1, AILAYER_SETTINGS_TRAINABLE)){
  ...
}

Example: Write a value to the settings

AILAYER_SETTINGS_SET(layer->settings, 0b1, AILAYER_SETTINGS_TRAINABLE, FALSE);

General layer settings like freezing weights or switching between training and evaluation mode.

---

The documentation for this struct was generated from the following file:

• aifes_core.h