ailayer_batch_normalization_default.h File Reference

Default implementation of the Batch Normalization layer . More...

Go to the source code of this file.

Data Structures
struct	ailayer_batch_norm_f32
	Data-type specific Batch Normalization layer struct for F32 . More...

Typedefs
typedef struct ailayer_batch_norm_f32	ailayer_batch_norm_f32_t

Functions
ailayer_t *	ailayer_batch_norm_f32_default (ailayer_batch_norm_f32_t *layer, ailayer_t *input_layer)
	Initializes and connect a Batch Normalization layer with the F32 default implementation. More...
ailayer_t *	ailayer_batch_norm_cfirst_f32_default (ailayer_batch_norm_f32_t *layer, ailayer_t *input_layer)
	Initializes and connect a Batch Normalization layer (channels first) with the F32 default implementation. More...
ailayer_t *	ailayer_batch_norm_chw_f32_default (ailayer_batch_norm_f32_t *layer, ailayer_t *input_layer)
	Initializes and connect a Batch Normalization layer (channels first) with the F32 default implementation. More...
ailayer_t *	ailayer_batch_norm_cl_f32_default (ailayer_batch_norm_f32_t *layer, ailayer_t *input_layer)
	Initializes and connect a Batch Normalization layer (channels first) with the F32 default implementation. More...
ailayer_t *	ailayer_batch_norm_clast_f32_default (ailayer_batch_norm_f32_t *layer, ailayer_t *input_layer)
	Initializes and connect a Batch Normalization layer (channels last) with the F32 default implementation. More...
ailayer_t *	ailayer_batch_norm_hwc_f32_default (ailayer_batch_norm_f32_t *layer, ailayer_t *input_layer)
	Initializes and connect a Batch Normalization layer (channels first) with the F32 default implementation. More...
ailayer_t *	ailayer_batch_norm_lc_f32_default (ailayer_batch_norm_f32_t *layer, ailayer_t *input_layer)
	Initializes and connect a Batch Normalization layer (channels first) with the F32 default implementation. More...
void	ailayer_batch_norm_init_params_f32_default (ailayer_t *self)
	F32 default implementation of the ailayer.init_params function for the Batch Normalization layer More...

Detailed Description

Default implementation of the Batch Normalization layer .

Version

2.2.0

Copyright

Copyright (C) 2020-2023 Fraunhofer Institute for Microelectronic Circuits and Systems. All rights reserved.

AIfES is free software: you can redistribute it and/or modify it under the terms of the GNU Affero General Public License as published by the Free Software Foundation, either version 3 of the License, or (at your option) any later version.

This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU Affero General Public License for more details.

You should have received a copy of the GNU Affero General Public License along with this program. If not, see https://www.gnu.org/licenses/.

Hardware independent implementations of the Batch Normalization layer in F32 data-type.

For more information about the Batch Normalization layer refer to ailayer_batch_normalization.h.

Function Documentation

ailayer_batch_norm_cfirst_f32_default()

ailayer_t* ailayer_batch_norm_cfirst_f32_default	(	ailayer_batch_norm_f32_t *	layer,
		ailayer_t *	input_layer
	)

Initializes and connect a Batch Normalization layer (channels first) with the F32 default implementation.

Code examples are given in the description of ailayer_batch_norm_f32_default().

Parameters

*layer	The layer structure to initialize.
*input_layer	The prior layer.

Returns

The (successfully) initialized layer structure.

ailayer_batch_norm_chw_f32_default()

ailayer_t* ailayer_batch_norm_chw_f32_default	(	ailayer_batch_norm_f32_t *	layer,
		ailayer_t *	input_layer
	)

Initializes and connect a Batch Normalization layer (channels first) with the F32 default implementation.

Code examples are given in the description of ailayer_batch_norm_f32_default().

Parameters

*layer	The layer structure to initialize.
*input_layer	The prior layer.

Returns

The (successfully) initialized layer structure.

ailayer_batch_norm_cl_f32_default()

ailayer_t* ailayer_batch_norm_cl_f32_default	(	ailayer_batch_norm_f32_t *	layer,
		ailayer_t *	input_layer
	)

Initializes and connect a Batch Normalization layer (channels first) with the F32 default implementation.

Code examples are given in the description of ailayer_batch_norm_f32_default().

Parameters

*layer	The layer structure to initialize.
*input_layer	The prior layer.

Returns

The (successfully) initialized layer structure.

ailayer_batch_norm_clast_f32_default()

ailayer_t* ailayer_batch_norm_clast_f32_default	(	ailayer_batch_norm_f32_t *	layer,
		ailayer_t *	input_layer
	)

Initializes and connect a Batch Normalization layer (channels last) with the F32 default implementation.

Code examples are given in the description of ailayer_batch_norm_f32_default().

Parameters

*layer	The layer structure to initialize.
*input_layer	The prior layer.

Returns

The (successfully) initialized layer structure.

ailayer_batch_norm_f32_default()

ailayer_t* ailayer_batch_norm_f32_default	(	ailayer_batch_norm_f32_t *	layer,
		ailayer_t *	input_layer
	)

Initializes and connect a Batch Normalization layer with the F32 default implementation.

Example: Create the layer structure with pretrained weights, means and variances:
In C:

// Use constant data only for inference. For training remove the const qualifier!!
const float means_data_bn[3] = {-0.743f, 1.133f, -0.05f};
const float vars_data_bn[3] = {0.9f, 1.0f, 0.87f};
const float betas_data_bn[3] = {0.1f, -0.1f, 0.0f};
const float gammas_data_bn[3] = {4.0f, 2.0f, 1.0f};
ailayer_batch_norm_f32_t bn_layer = {
    .momentum =                 0.9f,
    .eps =                      1e-6f,
    .base.moving_means =        means_data_bn,
    .base.moving_variances =    vars_data_bn,
    .base.betas =               betas_data_bn,
    .base.gammas =              gammas_data_bn
};

In C, C++ and on Arduino:

// Use constant data only for inference. For training remove the const qualifier!!
const float means_data_bn[3] = {-0.743f, 1.133f, -0.05f};
const float vars_data_bn[3] = {0.9f, 1.0f, 0.87f};
const float betas_data_bn[3] = {0.1f, -0.1f, 0.0f};
const float gammas_data_bn[3] = {4.0f, 2.0f, 1.0f};
ailayer_batch_norm_f32_t bn_layer = AILAYER_BATCH_NORM_F32_M(
                                                                /* momentum =*/         0.9f,
                                                                /* eps =*/              1e-6f,
                                                                /* means =*/            means_data_bn,
                                                                /* variances =*/        vars_data_bn,
                                                                /* betas (offset) =*/   betas_data_bn,
                                                                /* gammas (scale) =*/   gammas_data_bn
                                                             );

Example: Create the layer structure for automatic parameter distribution (e.g. for training):
In C:

ailayer_batch_norm_f32_t bn_layer = {
    .momentum =                 0.9f,
    .eps =                      1e-6f
};

In C, C++ and on Arduino:

ailayer_batch_norm_f32_t bn_layer = AILAYER_BATCH_NORM_F32_A(
                                                                /* momentum =*/         0.9f,
                                                                /* eps =*/              1e-6f
                                                             );

Example: Initialize and connect the layer for data with channels first (all options are equivalent):

bn_layer.channel_axis = 1;
x = ailayer_batch_norm_f32_default(&bn_layer, x);

or

x = ailayer_batch_norm_cfirst_f32_default(&bn_layer, x);

or

x = ailayer_batch_norm_chw_f32_default(&bn_layer, x);

or

x = ailayer_batch_norm_cl_f32_default(&bn_layer, x);

Example: Initialize and connect the layer for data with channels last (all options are equivalent):

bn_layer.channel_axis = -1;
x = ailayer_batch_norm_f32_default(&bn_layer, x);

or

x = ailayer_batch_norm_clast_f32_default(&bn_layer, x);

or

x = ailayer_batch_norm_hwc_f32_default(&bn_layer, x);

or

x = ailayer_batch_norm_lc_f32_default(&bn_layer, x);

Parameters

*layer	The layer structure to initialize.
*input_layer	The prior layer.

Returns

The (successfully) initialized layer structure.

ailayer_batch_norm_hwc_f32_default()

ailayer_t* ailayer_batch_norm_hwc_f32_default	(	ailayer_batch_norm_f32_t *	layer,
		ailayer_t *	input_layer
	)

Initializes and connect a Batch Normalization layer (channels first) with the F32 default implementation.

Code examples are given in the description of ailayer_batch_norm_f32_default().

Parameters

*layer	The layer structure to initialize.
*input_layer	The prior layer.

Returns

The (successfully) initialized layer structure.

ailayer_batch_norm_init_params_f32_default()

void ailayer_batch_norm_init_params_f32_default

(

ailayer_t *

self

)

F32 default implementation of the ailayer.init_params function for the Batch Normalization layer

Implementation of ailayer.init_params.

The function will initialize the tensors for moving_means and betas with zeros and for moving_variances and gammas with ones.

Parameters

*self

The layer structure

ailayer_batch_norm_lc_f32_default()

ailayer_t* ailayer_batch_norm_lc_f32_default	(	ailayer_batch_norm_f32_t *	layer,
		ailayer_t *	input_layer
	)

Initializes and connect a Batch Normalization layer (channels first) with the F32 default implementation.

Code examples are given in the description of ailayer_batch_norm_f32_default().

Parameters

*layer	The layer structure to initialize.
*input_layer	The prior layer.

Returns

The (successfully) initialized layer structure.