Main page

Welcome to the official AIfES 2 documentation! This guideline should give you an overview of the functions and the application of AIfES. The recommendations are based on best practices from various sources and our own experiences in the use of neural networks.

Vocabulary and Abbreviations

• AIfES: Artificial Intelligence for Embedded Systems
• ANN: Artificial Neural Network
• FNN: Feedforward Neural Network (used in this documentation for simple multi-layer perceptrons)
• RNN: Recurrent Neural Network
• CNN: Convolutional Neural Network
• Inference: The calculation of an ANN (foreward pass / prediction)
• Backpropagation: A training algorithm for ANNs that is based on gradient descent

Overview (Documentation)

• Tutorial inference F32: Guides you through the necessary steps to perform an inference with AIfES, based on an example. (Float 32 model)
• Tutorial training F32: Guides you through the training process with AIfES, based on an example. (Float 32 model)
• Tutorial inference Q7: Guides you through the necessary steps to perform an inference with AIfES, based on an example. (Int 8 model)

Overview (Features)

AIfES 2 is a modular toolbox designed to enable developers to execute and train an ANN on resource constrained edge devices as efficient as possible with as little programming effort as possible. The structure is closely based on python libraries such as keras and pytorch to make it easier to get started with the library.

The AIfES basic module contains currently the following features:

Data types

• F32 (32 bit floating point values)
• Q31 (32 bit quantized fixed point values)
• Q7 (8 bit quantized fixed point values)

General

• aialgo_compile_model()
• Q7 specific
  • aialgo_quantize_model_f32_to_q7()
• Debug prints
  • aialgo_print_model_structure()
  • aialgo_print_loss_specs()
  • aialgo_print_optimizer_specs()

For Inference

Layer:

Layer	f32	q31	q7
Dense	ailayer_dense_f32_default() ailayer_dense_f32_cmsis() ailayer_dense_f32_avr_pgm()	ailayer_dense_q31_default()	ailayer_dense_q7_default() ailayer_dense_wt_q7_default() ailayer_dense_wt_q7_cmsis() ailayer_dense_q7_avr_pgm() ailayer_dense_wt_q7_avr_pgm()
Input	ailayer_input_f32_default()	ailayer_input_q31_default()	ailayer_input_q7_default()
ReLU	ailayer_relu_f32_default()	ailayer_relu_q31_default()	ailayer_relu_q7_default() ailayer_relu_q7_avr_pgm()
Sigmoid	ailayer_sigmoid_f32_default()	ailayer_sigmoid_q31_default()	ailayer_sigmoid_q7_default() ailayer_sigmoid_q7_avr_pgm()
Softmax	ailayer_softmax_f32_default()	ailayer_softmax_q31_default()	ailayer_softmax_q7_default() ailayer_softmax_q7_avr_pgm()
Leaky ReLU	ailayer_leaky_relu_f32_default()	ailayer_leaky_relu_q31_default()	ailayer_leaky_relu_q7_default() ailayer_leaky_relu_q7_avr_pgm()
ELU	ailayer_elu_f32_default()	ailayer_elu_q31_default()	ailayer_elu_q7_default() ailayer_elu_q7_avr_pgm()
Tanh	ailayer_tanh_f32_default()	ailayer_tanh_q31_default()	ailayer_tanh_q7_default() ailayer_tanh_q7_avr_pgm()
Softsign	ailayer_softsign_f32_default()	ailayer_softsign_q31_default()	ailayer_softsign_q7_default() ailayer_softsign_q7_avr_pgm()
Conv2D	ailayer_conv2d_f32_default()
Batch Normalization	ailayer_batch_norm_f32_default()
MaxPool2D	ailayer_maxpool2d_f32_default()
Reshape	ailayer_reshape_f32_default()
Flatten	ailayer_flatten_f32_default()

Algorithmic:

• Inference memory (required for intermediate results of an inference)
  • aialgo_sizeof_inference_memory()
  • aialgo_schedule_inference_memory()
• aialgo_inference_model()
• Advanced inference controls
  • aialgo_forward_model()

For Training

Layer:

Layer	f32	q31	q7
Dense	ailayer_dense_f32_default() ailayer_dense_f32_cmsis()	ailayer_dense_q31_default()
Input	ailayer_input_f32_default()	ailayer_input_q31_default()	ailayer_input_q7_default()
ReLU	ailayer_relu_f32_default()	ailayer_relu_q31_default()	ailayer_relu_q7_default()
Sigmoid	ailayer_sigmoid_f32_default()	ailayer_sigmoid_q31_default()
Softmax	ailayer_softmax_f32_default()	ailayer_softmax_q31_default()
Leaky ReLU	ailayer_leaky_relu_f32_default()	ailayer_leaky_relu_q31_default()
ELU	ailayer_elu_f32_default()	ailayer_elu_q31_default()
Tanh	ailayer_tanh_f32_default()	ailayer_tanh_q31_default()
Softsign	ailayer_softsign_f32_default()	ailayer_softsign_q31_default()
Conv2D	ailayer_conv2d_f32_default()
Batch Normalization	ailayer_batch_norm_f32_default()
MaxPool2D	ailayer_maxpool2d_f32_default()
Reshape	ailayer_reshape_f32_default()
Flatten	ailayer_flatten_f32_default()

Loss:

Loss	f32	q31	q7
Mean Squared Error (MSE)	ailoss_mse_f32_default()	ailoss_mse_q31_default()
Crossentropy	ailoss_crossentropy_f32_default() ailoss_crossentropy_sparse8_f32_default()

Optimizer:

Optimizer	f32	q31	q7
Stochastic Gradient Descent (SGD)	aiopti_sgd_f32_default()	aiopti_sgd_q31_default()
Adam	aiopti_adam_f32_default()

Algorithmic:

• Parameter memory (required for the trainable parameters like weights, bias etc. if there are no pre-trained parameters)
  • aialgo_sizeof_parameter_memory()
  • aialgo_distribute_parameter_memory()
• Training memory (required for example for gradients, momentums and intermediate results of the training)
  • aialgo_sizeof_training_memory()
  • aialgo_schedule_training_memory()
• aialgo_initialize_parameters_model()
• aialgo_init_model_for_training()
• aialgo_train_model()
• F32 specific
  • aialgo_calc_loss_model_f32()
• Q31 specific
  • aialgo_set_model_result_precision_q31()
  • aialgo_set_model_delta_precision_q31()
  • aialgo_set_model_gradient_precision_q31()
  • aialgo_calc_loss_model_q31()
• Advanced training controls
  • aialgo_zero_gradients_model()
  • aialgo_forward_model()
  • aialgo_backward_model()
  • aialgo_update_params_model()

AIfES Express

High level functions to build a simple multi layer perceptron / fully-connected neural network with a few lines of code.

• F32
  • AIFES_E_flat_weights_number_fnn_f32()
  • AIFES_E_inference_fnn_f32()
  • AIFES_E_training_fnn_f32()
• Q7
  • AIFES_E_flat_weights_number_fnn_q7()
  • AIFES_E_quantisation_fnn_f32_to_q7()
  • AIFES_E_inference_fnn_q7()

Python tools

To help you to get your neural network models from Python to AIfES and to perform the weights quantization to integer types, we provide some AIfES Python tools.

You can install the tools via pip from our GitHub repository with:

pip install https://github.com/Fraunhofer-IMS/AIfES_for_Arduino/raw/main/etc/python/aifes_tools.zip

You can have a look at the automatic quantization example to see how the Python tools can be used to quantize a model to Q7 integer type.

Structure and design concepts of AIfES

AIfES was designed as a flexible and extendable toolbox for running and training or artificial neural networks on microcontrollers. All layers, losses and optimizers are modular and can be optimized for different data types and hardware platforms.

Example structure of a small FNN with one hidden layer in AIfES 2:

Modular structure of AIfES 2 backend: