Posts by Tags

CMake

05 Nov 2014 » Unit testing with GoogleTest and CMake
03 Nov 2014 » A typical Linux project using CMake

GoogleTest

05 Nov 2014 » Unit testing with GoogleTest and CMake

gcc

08 Jan 2015 » Better understanding Linux secondary dependencies solving with examples

ld

08 Jan 2015 » Better understanding Linux secondary dependencies solving with examples

ELF

17 Aug 2016 » Decentralized modules declarations in C using ELF sections

sections

17 Aug 2016 » Decentralized modules declarations in C using ELF sections

build

23 Sep 2016 » Build and boot a minimal Linux system with qemu
01 Sep 2016 » Benchmarking build systems for a large C project

make

01 Sep 2016 » Benchmarking build systems for a large C project

ninja

01 Sep 2016 » Benchmarking build systems for a large C project

kbuild

01 Sep 2016 » Benchmarking build systems for a large C project

cmake

01 Sep 2016 » Benchmarking build systems for a large C project

linux

23 Sep 2016 » Build and boot a minimal Linux system with qemu

croostool-ng

23 Sep 2016 » Build and boot a minimal Linux system with qemu

qemu

23 Sep 2016 » Build and boot a minimal Linux system with qemu

busybox

23 Sep 2016 » Build and boot a minimal Linux system with qemu

tensorflow

26 Jul 2018 » Identify Repeating Patterns using Spiking Neural Networks in Tensorflow
25 Jul 2018 » Leaky Integrate and Fire neuron with Tensorflow
24 Jul 2018 » Simulating spiking neurons with Tensorflow
26 Jun 2017 » Explore Tensorflow features with the CIFAR10 dataset

CIFAR10

26 Jun 2017 » Explore Tensorflow features with the CIFAR10 dataset

CNN

26 Jun 2017 » Explore Tensorflow features with the CIFAR10 dataset

machine learning

26 Jul 2018 » Identify Repeating Patterns using Spiking Neural Networks in Tensorflow
25 Jul 2018 » Leaky Integrate and Fire neuron with Tensorflow
24 Jul 2018 » Simulating spiking neurons with Tensorflow

spiking neural nets

26 Jul 2018 » Identify Repeating Patterns using Spiking Neural Networks in Tensorflow
25 Jul 2018 » Leaky Integrate and Fire neuron with Tensorflow
24 Jul 2018 » Simulating spiking neurons with Tensorflow

snn

26 Jul 2018 » Identify Repeating Patterns using Spiking Neural Networks in Tensorflow
25 Jul 2018 » Leaky Integrate and Fire neuron with Tensorflow
24 Jul 2018 » Simulating spiking neurons with Tensorflow

lif

25 Jul 2018 » Leaky Integrate and Fire neuron with Tensorflow

stdp

26 Jul 2018 » Identify Repeating Patterns using Spiking Neural Networks in Tensorflow

quantization

30 May 2023 » Aligning quantization scales before incompatible operations
29 May 2023 » Resolve quantization scales after an operation
26 May 2023 » Fixed-point representation for quantization
25 May 2023 » A brief introduction to Machine Learning models quantization

Hi

I am David Corvoysier, versatile developer and open Source enthusiast.

Recent Posts

Aligning quantization scales before incompatible operations

30 May 2023 by David Corvoysier

As explained in my introduction to Machine Learning quantization, important restrictions apply to operations performed on quantized inputs.

First, additions between the integer mantissa of quantized inputs can only be performed if they are in the same scale.

This comes from the representation of the quantized numbers:

$a = (n - zeropoint_a) * scale_a$

$b = (m - zeropoint) * scale_b$

$a$ and $b$ integer mantissa can only be added if $scale_a == scale_b$, allowing us to write directly:

$a + b = (n - zeropoint_a + m - zeropoint_b) * scale_a$

Intuitively, this is analog to say that you cannot add two quantities expressed in different units (like bytes and kilobytes) without converting one number representation to the other.

Resolve quantization scales after an operation

29 May 2023 by David Corvoysier

As explained in my introduction to Machine Learning quantization, the inputs, weights and outputs of a quantized operation are quantized each with a different scale.

In the same post, I explain how these scales can be folded into a single output scale, allowing the operation to be performed on the integer mantissa of the quantized inputs and weights:

$scale_{folded} = \frac{scale_{out}}{scale_{in} . scale_{w}}$

In another post I explain how heterogenous input scales could be converted to a fixed-point representation and aligned before the operation, resulting in yet another implicit scale expressed as a power-of-two that needs to be applied to the output scale.

In this post I explain how these output scales can be applied using integer arithmetics only.

Fixed-point representation for quantization

26 May 2023 by David Corvoysier

As explained in my introduction to Machine Learning quantization, the quantization of a ML model produces a graph of operations applied on quantized tensors.

Quantized tensors are actually integer tensors that share the same float scale and integer zero-point.

The implementation of the quantized operations is device-specific.

One of the main design decision is how the inputs, weights and output float scales are propagated and applied in the quantized graph.

In two other posts I will explain how is is possible to use integer arithmetic operators for that purpose if the scales are represented as fixed-point numbers.

This posts is a brief introduction to the fixed-point representation and to the fixed-point arithmetic operators.

A brief introduction to Machine Learning models quantization

25 May 2023 by David Corvoysier

Even before the development of Large Language Models (LLM), the increasing memory and computing requirements of Deep Neural Networks (DNN) has been a concern.

Functionally, DNN are graphs of arithmetic operations: the inputs are fed at the stem and the chain of operations produces the outputs at the head.

From an implementation perspective, the operations are performed on floating point numbers, which are a digital representation of decimal numbers composed of a mantissa and an exponent:

\[x = mantissa . 2^{exponent}\]