SiLU AVX SIMD Vectorization
-
Book Excerpt from "Generative AI in C++" (full text online)
-
by David Spuler, Ph.D.
SiLU AVX SIMD Vectorization
The SiLU function has a sequence of operations that could be vectorized with AVX: negation, exponentiation, scalar addition, reciprocal,
and multiplication.
With so many distinct AVX operations required in sequence, even if parallelized across 4 or 8 float values, it will probably be slow.
Hence, our best option is probably a vectorized lookup-table instead.