# Zero Knowledge Machine Learning

## Worldcoin Use-Case

• IrisCode self-custody and upgrade ability.
• Making the Orb trustless, provide proof that fraud filters are applied.

## Deep learning networks

               layer |    output shape |     #parameters |            #ops
-------------------- | --------------- | --------------- | ---------------
conv 32x5x5x3 |   (116, 76, 32) |            2400 |        21158400
max-pool |    (58, 38, 32) |               0 |          282112
relu |    (58, 38, 32) |               0 |           70528
conv 32x5x5x32 |    (54, 34, 32) |           25600 |        47001600
max-pool |    (27, 17, 32) |               0 |           58752
relu |    (27, 17, 32) |               0 |           14688
flatten |        (14688,) |               0 |               0
full 1000x14688 |         (1000,) |        14689000 |        14688000
relu |         (1000,) |               0 |            1000
full 5x1000 |            (5,) |            5005 |            5000
normalize |            (5,) |               0 |               6

• Linear operations
• Matrix multiplications: $A ⋅ \vec x$
• Convolutions
• Biases: $\vec x + \vec b$
• Pooling
• Max pooling: $\max_i x_i$
• Sum/Avg pooling: $\sum_i x_i$
• Activation functions:
• ReLU: $\max(0,x)$
• PReLU: $\max(α⋅x,x)$ with $0 ≤ α < 1$
• Logistic: $\p{1 + \mathrm{e}^{-x}}^{-1}$
• Polynomial: $x^2 + x$,
• Argmax: $\argmax_i x_i$
• Softmax: $\mathrm{e}^{x_i}$ and rescaling $\norm{\vec x}_1 = 1$
• Normalize: Rescale such that 2-norm $\norm{\vec x}_2 = 1$.

## Relevant ML evaluation work

Generally: lot's of similarities with embedded implementations: use fixed-point to avoid floating point math, model is held constant, can do precompute.

Differences: Fixed-point numbers in ZK can be large (very very large in some proof systems). Simple comparisons are expensive. Can do inverses trivially. (i.e. zk development we are all familiar with).

## Fix point

$$\mathtt{a} = \floor{\frac{a}{2^{32}}}$$

$$\mathtt{a} ⋅ \mathtt{b} = \floor{\frac{a}{2^{32}}}$$

## Strategy ideas

• Results in low level optimized implementation of matrix multiplications and convolutions.
• Recursion around layers. Vectors passed between layers are smallish, so can be committed to.
• Target 16-bit fixpoint so we can use PLookup for Relu.
• dot product without renormalization.
• Flip max-pool and relu 😁
• Clever algorithms to compute convolutions (from simple tricks to FFT based)
• Efficient non-zk convolution algorithms: https://arxiv.org/abs/1903.01521 http://people.ece.umn.edu/users/parhi/SLIDES/chap8.pdf

Remco Bloemen
Math & Engineering
https://2π.com