Exponential Function Approximation

  • Last Updated 19 June, 2026
  • by David Spuler, Ph.D.

Exponential Function Approximation: Book Excerpts and Blog Articles

Free online book excerpts with full text chapters online and free PDF downloads, and the Aussie AI blog, including related articles:

Research on Exponential Function Approximation

Research papers include:

  1. Charles Frye, Nathan Wang, Timothy Feng, September 26, 2025, We reverse-engineered Flash Attention 4, https://modal.com/blog/reverse-engineer-flash-attention-4 (Flash Attention 4 has Blackwell CUDA C++ improvements, approximate Softmax via exponential approximation, and faster scaling factor updates.)
  2. Nicol N. Schraudolph, 1999, A Fast, Compact Approximation of the Exponential Function https://nic.schraudolph.org/pubs/Schraudolph99.pdf
  3. Titopoulos, V., Alexandridis, K. & Dimitrakopoulos, G. Vectorized FlashAttention with low-cost exponential computation in RISC-V vector processors. J Supercomput 82, 189 (2026). https://doi.org/10.1007/s11227-026-08322-x https://link.springer.com/article/10.1007/s11227-026-08322-x
  4. Tom Hubrecht, Orégane Desrentes, Florent de Dinechin, 2024, Activations in Low Precision with High Accuracy, ⟨hal-04776745) https://inria.hal.science/hal-04776745v1/document
  5. Raaida Noor Mahbub 0009-0005-8343-7340, Hani Saleh, and Ghada Alsuhli, Hardware Accelerators for Softmax in Large Language Models: A Survey, 26 February 2026, https://doi.org/10.36227/techrxiv.177208047.71459791/v1 https://www.techrxiv.org/doi/full/10.36227/techrxiv.177208047.71459791/v1 https://www.techrxiv.org/doi/pdf/10.36227/techrxiv.177208047.71459791/v1?onload=true (Survey of Softmax optimization through hardware, approximations, LUTs, numeric representations, Flash Attention, and more.)
  6. A. Cristiano I. Malossi, Yves Ineichen, Costas Bekas, and Alessandro Curioni, 2015, https://research.ibm.com/publications/fast-exponential-computation-on-simd-architectures https://wapco.e-ce.uth.gr/2015/papers/SESSION3/WAPCO_3_5.pdf https://www.researchgate.net/profile/A-Cristiano-I-Malossi/publication/272178514 _Fast_Exponential_Computation_on_SIMD_Architectures/links/54de344f0cf22a26721fbdc9/Fast-Exponential-Computation-on-SIMD-Architectures.pdf
  7. Simon PF, 2018, fastexp, https://github.com/simonpf/fastexp
  8. Run Wang, Gamze Islamoglu, Andrea Belano, Viviane Potocnik, Francesco Conti, Angelo Garofalo, Luca Benini, 15 Apr 2025, VEXP: A Low-Cost RISC-V ISA Extension for Accelerated Softmax Computation in Transformers, https://arxiv.org/abs/2504.11227
  9. Gavin C. Cawley, September 01 2000, On a Fast, Compact Approximation of the Exponential Function, Neural Computation, Volume 12, Issue 9 September 2000, https://direct.mit.edu/neco/article-abstract/12/9/2009/6446/On-a-Fast-Compact-Approximation-of-the-Exponential
  10. Akio Yamamoto, Yasunori Kitamura, Yoshihiro Yamane, 2004, Computational efficiencies of approximated exponential functions for transport calculations of the characteristics method, Annals of Nuclear Energy, Volume 31, Issue 9, Pages 1027-1037, ISSN 0306-4549, https://doi.org/10.1016/j.anucene.2004.01.003 https://www.sciencedirect.com/science/article/abs/pii/S0306454904000234
  11. L. Moroz, V. Samotyy, Z. Kokosiński and P. Gepner, "Simple Multiple Precision Algorithms for Exponential Functions [Tips & Tricks]," in IEEE Signal Processing Magazine, vol. 39, no. 4, pp. 130-137, July 2022, doi: 10.1109/MSP.2022.3157460, https://ieeexplore.ieee.org/abstract/document/9810030
  12. J. Partzsch et al., "A fixed point exponential function accelerator for a neuromorphic many-core system," 2017 IEEE International Symposium on Circuits and Systems (ISCAS), Baltimore, MD, USA, 2017, pp. 1-4, doi: 10.1109/ISCAS.2017.8050528, https://ieeexplore.ieee.org/abstract/document/8050528
  13. S. K. Narayanaswami, G. Srinivasan and B. Ravindran, "QuAKE: Speeding up Model Inference Using Quick and Approximate Kernels for Exponential Non-Linearities," 2025 IEEE/ACM International Symposium on Low Power Electronics and Design (ISLPED), Reykjavík, Iceland, 2025, pp. 1-7, doi: 10.1109/ISLPED65674.2025.11261772, https://ieeexplore.ieee.org/abstract/document/11261772
  14. J. Wei, A. Kuwana, H. Kobayashi and K. Kubo, "Divide and Conquer: Floating-Point Exponential Calculation Based on Taylor-Series Expansion," 2021 IEEE 14th International Conference on ASIC (ASICON), Kunming, China, 2021, pp. 1-4, doi: 10.1109/ASICON52560.2021.9620253, https://ieeexplore.ieee.org/abstract/document/9620253
  15. Q. CHEN, "A Robust Exponential Integrator Method for Generic Nonlinear Circuit Simulation," 2020 57th ACM/IEEE Design Automation Conference (DAC), San Francisco, CA, USA, 2020, pp. 1-6, doi: 10.1109/DAC18072.2020.9218556, https://ieeexplore.ieee.org/document/9218556
  16. P. Nilsson, A. U. R. Shaik, R. Gangarajaiah and E. Hertz, "Hardware implementation of the exponential function using Taylor series," 2014 NORCHIP, Tampere, Finland, 2014, pp. 1-4, doi: 10.1109/NORCHIP.2014.7004740, https://ieeexplore.ieee.org/document/7004740
  17. Federico Perini, Rolf D. Reitz, Fast approximations of exponential and logarithm functions combined with efficient storage/retrieval for combustion kinetics calculations, Combustion and Flame, Volume 194, 2018, Pages 37-51, ISSN 0010-2180, https://doi.org/10.1016/j.combustflame.2018.04.013 https://www.sciencedirect.com/science/article/abs/pii/S0010218018301652
  18. Ted Zadouri (Princeton University, Together AI), Markus Hoehnerbach (Meta), Jay Shah (Colfax Research), Timmy Liu (NVIDIA), Vijay Thakkar (Meta, Georgia Tech), Tri Dao (Princeton University, Together AI), 5th March 2026, FlashAttention-4: Algorithm and Kernel Pipelining Co-Design for Asymmetric Hardware Scaling, https://www.together.ai/blog/flashattention-4
  19. Ted Zadouri, Markus Hoehnerbach, Jay Shah, Timmy Liu, Vijay Thakkar, Tri Dao, 5 Mar 2026, FlashAttention-4: Algorithm and Kernel Pipelining Co-Design for Asymmetric Hardware Scaling, https://arxiv.org/abs/2603.05451
  20. David Spuler, May 31st, 2026, Chapter 35. , in book LLM Inference Optimization: State-of-the-Art Research, Table of Contents: https://www.aussieai.com/book/llm-inference-optimization https://www.amazon.com/dp/B0H3FKR39T

More AI Research Topics

Read more about: