ailoss_crossentropy_default.h File Reference

Default implementation of the Cross-Entropy loss . More...

Go to the source code of this file.

Typedefs
typedef struct ailoss_crossentropy	ailoss_crossentropy_f32_t

Functions
ailoss_t *	ailoss_crossentropy_f32_default (ailoss_crossentropy_f32_t *loss, ailayer_t *input_layer)
	Initializes and connect a Cross-Entropy loss with the F32 default implementation using a mean reduction. More...
ailoss_t *	ailoss_crossentropy_sum_f32_default (ailoss_crossentropy_f32_t *loss, ailayer_t *input_layer)
	Initializes and connect a Cross-Entropy loss with the F32 default implementation using a sum reduction. More...
ailoss_t *	ailoss_crossentropy_mean_f32_default (ailoss_crossentropy_f32_t *loss, ailayer_t *input_layer)
	Initializes and connect a Cross-Entropy loss with the F32 default implementation using a mean reduction. More...
ailoss_t *	ailoss_crossentropy_sparse8_f32_default (ailoss_crossentropy_f32_t *loss, ailayer_t *input_layer)
	Initializes and connect a Cross-Entropy loss with the F32 default implementation for sparse labels using a mean reduction. More...
ailoss_t *	ailoss_crossentropy_sum_sparse8_f32_default (ailoss_crossentropy_f32_t *loss, ailayer_t *input_layer)
	Initializes and connect a Cross-Entropy loss with the F32 default implementation for sparse labels using a sum reduction. More...
ailoss_t *	ailoss_crossentropy_mean_sparse8_f32_default (ailoss_crossentropy_f32_t *loss, ailayer_t *input_layer)
	Initializes and connect a Cross-Entropy loss with the F32 default implementation for sparse labels using a mean reduction. More...

Detailed Description

Default implementation of the Cross-Entropy loss .

Version

2.2.0

Copyright

Copyright (C) 2020-2023 Fraunhofer Institute for Microelectronic Circuits and Systems. All rights reserved.

AIfES is free software: you can redistribute it and/or modify it under the terms of the GNU Affero General Public License as published by the Free Software Foundation, either version 3 of the License, or (at your option) any later version.

This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU Affero General Public License for more details.

You should have received a copy of the GNU Affero General Public License along with this program. If not, see https://www.gnu.org/licenses/.

Hardware independent implementations of the Cross-Entropy loss in F32 data-type. For more information about the Cross-Entropy loss refer to ailoss_mse.h.

Function Documentation

ailoss_crossentropy_f32_default()

ailoss_t* ailoss_crossentropy_f32_default	(	ailoss_crossentropy_f32_t *	loss,
		ailayer_t *	input_layer
	)

Initializes and connect a Cross-Entropy loss with the F32 default implementation using a mean reduction.

The labels must me either binary (when the output layer is a Sigmoid layer), for example

\[ \left( \begin{array}{ccc} 1 & 0 & 0 & 1 \\ 1 & 1 & 1 & 0 \\ 0 & 0 & 1 & 0 \end{array}\right) \]

or row wise one-hot encoded (when the output layer is a Softmax layer), for example

\[ \left( \begin{array}{ccc} 0 & 0 & 0 & 1 \\ 1 & 0 & 0 & 0 \\ 0 & 0 & 1 & 0 \end{array}\right) \]

If you want to provide labels as integers, please use ailoss_crossentropy_sparse8_f32_default() loss.

Example: Create the loss structure:

ailoss_crossentropy_f32_t crossentropy_loss;

Example: Initialize and connect the loss to the layer structure:

aimodel_t model;
...
model.output_layer = ailayer_sigmoid_f32_default(&sigmoid_layer, x);
 
model.loss = ailoss_crossentropy_f32_default(&crossentropy_loss, model.output_layer);

Parameters

*loss	The loss structure to initialize.
*input_layer	The output layer of the model (Must be either a Sigmoid or a Softmax layer!).

Returns

The (successfully) initialized loss structure.

ailoss_crossentropy_mean_f32_default()

ailoss_t* ailoss_crossentropy_mean_f32_default	(	ailoss_crossentropy_f32_t *	loss,
		ailayer_t *	input_layer
	)

Initializes and connect a Cross-Entropy loss with the F32 default implementation using a mean reduction.

The labels must me either binary (when the output layer is a Sigmoid layer), for example

\[ \left( \begin{array}{ccc} 1 & 0 & 0 & 1 \\ 1 & 1 & 1 & 0 \\ 0 & 0 & 1 & 0 \end{array}\right) \]

or row wise one-hot encoded (when the output layer is a Softmax layer), for example

\[ \left( \begin{array}{ccc} 0 & 0 & 0 & 1 \\ 1 & 0 & 0 & 0 \\ 0 & 0 & 1 & 0 \end{array}\right) \]

If you want to provide labels as integers, please use ailoss_crossentropy_sparse8_f32_default() loss.

Example: Create the loss structure:

ailoss_crossentropy_f32_t crossentropy_loss;

Example: Initialize and connect the loss to the layer structure:

aimodel_t model;
...
model.output_layer = ailayer_sigmoid_f32_default(&sigmoid_layer, x);
 
model.loss = ailoss_crossentropy_f32_default(&crossentropy_loss, model.output_layer);

Parameters

*loss	The loss structure to initialize.
*input_layer	The output layer of the model (Must be either a Sigmoid or a Softmax layer!).

Returns

The (successfully) initialized loss structure.

ailoss_crossentropy_mean_sparse8_f32_default()

ailoss_t* ailoss_crossentropy_mean_sparse8_f32_default	(	ailoss_crossentropy_f32_t *	loss,
		ailayer_t *	input_layer
	)

Initializes and connect a Cross-Entropy loss with the F32 default implementation for sparse labels using a mean reduction.

This loss is meant for single label classification purposes. It expects the target data / labels to be an 8-bit integer tensor (U8 ) with the true-class index.

For example the matrix

\[ \left( \begin{array}{ccc} 0 & 0 & 0 & 1 \\ 1 & 0 & 0 & 0 \\ 0 & 0 & 1 & 0 \end{array}\right) \]

in sparse representation is

\[ \left( \begin{array}{ccc} 3 \\ 0 \\ 2 \end{array}\right) \]

and can be created with

uint16_t t_shape[2] = {3, 1};
uint8_t t_data[2*1] = {3,
                       0,
                       2};
aitensor_t t = AITENSOR_2D_U8(t_shape, t_data);

Example: Create the loss structure:

ailoss_crossentropy_f32_t crossentropy_loss;

Example: Initialize and connect the loss to the layer structure:

aimodel_t model;
...
model.output_layer = ailayer_softmax_f32_default(&softmax_layer, x);
 
model.loss = ailoss_crossentropy_sparse8_f32_default(&crossentropy_loss, model.output_layer);

Parameters

*loss	The loss structure to initialize.
*input_layer	The output layer of the model (Must be a Softmax layer!).

Returns

The (successfully) initialized loss structure.

ailoss_crossentropy_sparse8_f32_default()

ailoss_t* ailoss_crossentropy_sparse8_f32_default	(	ailoss_crossentropy_f32_t *	loss,
		ailayer_t *	input_layer
	)

Initializes and connect a Cross-Entropy loss with the F32 default implementation for sparse labels using a mean reduction.

This loss is meant for single label classification purposes. It expects the target data / labels to be an 8-bit integer tensor (U8 ) with the true-class index.

For example the matrix

\[ \left( \begin{array}{ccc} 0 & 0 & 0 & 1 \\ 1 & 0 & 0 & 0 \\ 0 & 0 & 1 & 0 \end{array}\right) \]

in sparse representation is

\[ \left( \begin{array}{ccc} 3 \\ 0 \\ 2 \end{array}\right) \]

and can be created with

uint16_t t_shape[2] = {3, 1};
uint8_t t_data[2*1] = {3,
                       0,
                       2};
aitensor_t t = AITENSOR_2D_U8(t_shape, t_data);

Example: Create the loss structure:

ailoss_crossentropy_f32_t crossentropy_loss;

Example: Initialize and connect the loss to the layer structure:

aimodel_t model;
...
model.output_layer = ailayer_softmax_f32_default(&softmax_layer, x);
 
model.loss = ailoss_crossentropy_sparse8_f32_default(&crossentropy_loss, model.output_layer);

Parameters

*loss	The loss structure to initialize.
*input_layer	The output layer of the model (Must be a Softmax layer!).

Returns

The (successfully) initialized loss structure.

ailoss_crossentropy_sum_f32_default()

ailoss_t* ailoss_crossentropy_sum_f32_default	(	ailoss_crossentropy_f32_t *	loss,
		ailayer_t *	input_layer
	)

Initializes and connect a Cross-Entropy loss with the F32 default implementation using a sum reduction.

The labels must me either binary (when the output layer is a Sigmoid layer), for example

\[ \left( \begin{array}{ccc} 1 & 0 & 0 & 1 \\ 1 & 1 & 1 & 0 \\ 0 & 0 & 1 & 0 \end{array}\right) \]

or row wise one-hot encoded (when the output layer is a Softmax layer), for example

\[ \left( \begin{array}{ccc} 0 & 0 & 0 & 1 \\ 1 & 0 & 0 & 0 \\ 0 & 0 & 1 & 0 \end{array}\right) \]

If you want to provide labels as integers, please use ailoss_crossentropy_sparse8_f32_default() loss.

Example: Create the loss structure:

ailoss_crossentropy_f32_t crossentropy_loss;

Example: Initialize and connect the loss to the layer structure:

aimodel_t model;
...
model.output_layer = ailayer_sigmoid_f32_default(&sigmoid_layer, x);
 
model.loss = ailoss_crossentropy_f32_default(&crossentropy_loss, model.output_layer);

Parameters

*loss	The loss structure to initialize.
*input_layer	The output layer of the model (Must be either a Sigmoid or a Softmax layer!).

Returns

The (successfully) initialized loss structure.

ailoss_crossentropy_sum_sparse8_f32_default()

ailoss_t* ailoss_crossentropy_sum_sparse8_f32_default	(	ailoss_crossentropy_f32_t *	loss,
		ailayer_t *	input_layer
	)

Initializes and connect a Cross-Entropy loss with the F32 default implementation for sparse labels using a sum reduction.

This loss is meant for single label classification purposes. It expects the target data / labels to be an 8-bit integer tensor (U8 ) with the true-class index.

For example the matrix

\[ \left( \begin{array}{ccc} 0 & 0 & 0 & 1 \\ 1 & 0 & 0 & 0 \\ 0 & 0 & 1 & 0 \end{array}\right) \]

in sparse representation is

\[ \left( \begin{array}{ccc} 3 \\ 0 \\ 2 \end{array}\right) \]

and can be created with

uint16_t t_shape[2] = {3, 1};
uint8_t t_data[2*1] = {3,
                       0,
                       2};
aitensor_t t = AITENSOR_2D_U8(t_shape, t_data);

Example: Create the loss structure:

ailoss_crossentropy_f32_t crossentropy_loss;

Example: Initialize and connect the loss to the layer structure:

aimodel_t model;
...
model.output_layer = ailayer_softmax_f32_default(&softmax_layer, x);
 
model.loss = ailoss_crossentropy_sparse8_f32_default(&crossentropy_loss, model.output_layer);

Parameters

*loss	The loss structure to initialize.
*input_layer	The output layer of the model (Must be a Softmax layer!).

Returns

The (successfully) initialized loss structure.