aialgo_sequential_training.h File Reference

Functions required for the training of models. More...

Go to the source code of this file.

Functions
uint32_t	aialgo_sizeof_training_memory (aimodel_t *model, aiopti_t *optimizer)
	Calculate the memory requirements for model training. More...
uint8_t	aialgo_schedule_training_memory (aimodel_t *model, aiopti_t *optimizer, void *memory_ptr, uint32_t memory_size)
	Assign the memory for model training. More...
void	aialgo_init_model_for_training (aimodel_t *model, aiopti_t *optimizer)
	Initialize the optimization memory of the model layers. More...
void	aialgo_backward_model (aimodel_t *model, aitensor_t *target_data)
	Perform the backward pass. More...
uint8_t	aialgo_train_model (aimodel_t *model, aitensor_t *input_tensor, aitensor_t *target_tensor, aiopti_t *optimizer, uint32_t batch_size)
	Perform one training epoch on all data batches of the dataset using backpropagation. More...
uint8_t	aialgo_calc_loss_model_f32 (aimodel_t *model, aitensor_t *input_data, aitensor_t *target_data, float *result)
	Calculate the loss in F32 data type. More...
uint8_t	aialgo_calc_loss_model_q31 (aimodel_t *model, aitensor_t *input_data, aitensor_t *target_data, aiscalar_q31_t *result)
	Calculate the loss in Q31 data type. More...
void	aialgo_zero_gradients_model (aimodel_t *model, aiopti_t *optimizer)
	Set the gradients to zero. More...
void	aialgo_update_params_model (aimodel_t *model, aiopti_t *optimizer)
	Perform the optimization step on the model parameters. More...
void	aialgo_print_loss_specs (ailoss_t *loss)
	Print the loss specs. More...
void	aialgo_print_optimizer_specs (aiopti_t *opti)
	Print the optimizer specs. More...
void	aialgo_initialize_parameters_model (aimodel_t *model)
	Initialize the parameters of the given model with their default initialization method. More...

Detailed Description

Functions required for the training of models.

Version

2.2.0

Copyright

Copyright (C) 2020-2023 Fraunhofer Institute for Microelectronic Circuits and Systems. All rights reserved.

AIfES is free software: you can redistribute it and/or modify it under the terms of the GNU Affero General Public License as published by the Free Software Foundation, either version 3 of the License, or (at your option) any later version.

This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU Affero General Public License for more details.

You should have received a copy of the GNU Affero General Public License along with this program. If not, see https://www.gnu.org/licenses/.

The functions target memory allocation/scheduling and the backpropagation for model training

Function Documentation

aialgo_backward_model()

void aialgo_backward_model	(	aimodel_t *	model,
		aitensor_t *	target_data
	)

Perform the backward pass.

Parameters

*model	The model
*target_data	The tensor containing the target data / labels

aialgo_calc_loss_model_f32()

uint8_t aialgo_calc_loss_model_f32	(	aimodel_t *	model,
		aitensor_t *	input_data,
		aitensor_t *	target_data,
		float *	result
	)

Calculate the loss in F32 data type.

Parameters

*model	The model
*input_data	Tensor containing the input data
*target_data	Tensor containing the target data / labels
*result	The calculated loss will be written here

aialgo_calc_loss_model_q31()

uint8_t aialgo_calc_loss_model_q31	(	aimodel_t *	model,
		aitensor_t *	input_data,
		aitensor_t *	target_data,
		aiscalar_q31_t *	result
	)

Calculate the loss in Q31 data type.

Parameters

*model	The model
*input_data	Tensor containing the input data
*target_data	Tensor containing the target data / labels
*result	The calculated loss will be written here. The zero_point and the scale should be set to proper values.

aialgo_init_model_for_training()

void aialgo_init_model_for_training	(	aimodel_t *	model,
		aiopti_t *	optimizer
	)

Initialize the optimization memory of the model layers.

Parameters

*model	The model
*optimizer	The optimizer that is used for training

aialgo_initialize_parameters_model()

void aialgo_initialize_parameters_model

(

aimodel_t *

model

)

Initialize the parameters of the given model with their default initialization method.

Initialize the parameters of all layers that have a default initialization function (ailayer.init_params) configured and that are set to trainable (ailayer.settings[AILAYER_SETTINGS_TRAINABLE] = TRUE).

Parameters

*model

The model to initialize

aialgo_print_loss_specs()

void aialgo_print_loss_specs

(

ailoss_t *

loss

)

Print the loss specs.

Prints information like type, data type and constants to the console.

Parameters

*loss

The loss

aialgo_print_optimizer_specs()

void aialgo_print_optimizer_specs

(

aiopti_t *

opti

)

Print the optimizer specs.

Prints information like type, data type and constants to the console.

Parameters

*opti

The optimizer

aialgo_schedule_training_memory()

uint8_t aialgo_schedule_training_memory	(	aimodel_t *	model,
		aiopti_t *	optimizer,
		void *	memory_ptr,
		uint32_t	memory_size
	)

Assign the memory for model training.

This memory is used for intermediate results, gradients and momentums.

The required memory size can be calculated with aialgo_sizeof_training_memory().

Parameters

*model	The model
*optimizer	The optimizer that is used for training
*memory_ptr	Pointer to the memory block
memory_size	Size of the memory block (for error checking)

Returns

0 if successful

aialgo_sizeof_training_memory()

uint32_t aialgo_sizeof_training_memory	(	aimodel_t *	model,
		aiopti_t *	optimizer
	)

Calculate the memory requirements for model training.

This memory is used for intermediate results, gradients and momentums.

Use aialgo_schedule_training_memory() to set the memory to the model.

Parameters

*model	The model
*optimizer	The optimizer that is used for training

Returns

Required memory size in bytes

aialgo_train_model()

uint8_t aialgo_train_model	(	aimodel_t *	model,
		aitensor_t *	input_tensor,
		aitensor_t *	target_tensor,
		aiopti_t *	optimizer,
		uint32_t	batch_size
	)

Perform one training epoch on all data batches of the dataset using backpropagation.

Make shure to initialize the model (aialgo_compile_model()) and schedule the training memory (for example with aialgo_schedule_training_memory()) and initialize the training memory (aialgo_init_model_for_training()) before calling this function.

Example: Training of an F32 model for multiple epochs

int epochs = 100;
int batch_size = 4;
int print_interval = 10;
 
float loss;
for(i = 0; i < epochs; i++)
{
    // One epoch of training. Iterates through the whole data once
    aialgo_train_model(&model, &input_tensor, &target_tensor, optimizer, batch_size);
 
    // Calculate and print loss every print_interval epochs
    if(i % print_interval == 0)
    {
        aialgo_calc_loss_model_f32(&model, &input_tensor, &target_tensor, &loss);
        printf("Epoch %5d: loss: %f\n", i, loss);
    }
}

Parameters

*model	The model
*input_tensor	The tensor containing the input data
*target_tensor	The tensor containing the target data / labels
*optimizer	The optimizer that is used for training
batch_size	Size of a batch / Number of input vektors

aialgo_update_params_model()

void aialgo_update_params_model	(	aimodel_t *	model,
		aiopti_t *	optimizer
	)

Perform the optimization step on the model parameters.

Parameters

*model	The model
*optimizer	The optimizer that is used for training

aialgo_zero_gradients_model()

void aialgo_zero_gradients_model	(	aimodel_t *	model,
		aiopti_t *	optimizer
	)

Set the gradients to zero.

Parameters

*model	The model
*optimizer	The optimizer that is used for training