Division Optimization

Last Updated 14 June, 2026

by David Spuler, Ph.D.

Division is an expensive operation, and has been largely avoided in neural networks (with preference given to multiplication, addition and bitwise operators). However, there is some research in regard to division arithmetic. Related research areas include:

Division algorithms: Faster ways to implement division, now mainly for hardware designers.
Approximate division algorithms: see below.
Power-of-two quantization: Bitshifting can optimize division, as it can for multiplication. Right bitshift is an obvious optimization for integer division involving power-of-2 divisors. This has relevance in relation to logarithmic quantization.
Integer division: For some thoughts on the use of general integer division of weights in quantization, see division quantization.
Advanced number system division: See dyadic numbers and dyadic quantization for an obscure number system involving power-of-two division.

Division in AI: Book Excerpts and Blog Articles

Free online book excerpts with full text chapters online and free PDF downloads, and the Aussie AI blog, including related articles:

David Spuler, March 2024, Chapter 53. Arithmetic Optimization Research, in book "Generative AI in C++", https://www.aussieai.com/book/ch53-arithmetic-optimization
David Spuler, March 2024, Generative AI in C++: Coding Transformers and LLMs, https://www.aussieai.com/book/toc PDF: https://www.aussieai.com/pdf/BOOK-Generative-AI-CPP-Spuler-2024.pdf
David Spuler, May 31st, 2026, Chapter 16. Rare Quantization Techniques, in book LLM Inference Optimization: State-of-the-Art Research, Table of Contents: https://www.aussieai.com/book/llm-inference-optimization https://www.amazon.com/dp/B0H3FKR39T

Division Algorithms and Approximate Division

Some research on fast division algorithms:

LibDivide, https://libdivide.com/ and https://github.com/ridiculousfish/libdivide
Benchmarking division and libdivide on Apple M1 and Intel AVX512, May 12th, 2021, https://ridiculousfish.com/blog/posts/benchmarking-libdivide-m1-avx512.html
S. Hashemi, R. Bahar, and S. Reda, A low-power dynamic divider for approximate applications, Proceedings of the 53rd Annual Design Automation Conference, ACM (2016), p. 105, https://ieeexplore.ieee.org/document/7544348
Suganthi Venkatachalam; Elizabeth Adams; Seok-Bum Ko, May 2019, Design of approximate restoring dividers, 2019 IEEE International Symposium on Circuits and Systems (ISCAS), https://ieeexplore.ieee.org/document/8702363
Chitlu Subhasri, Bhaskara Rao Jammu, L. Guna Sekhar Sai Harsha, Nalini Bodasingi, Visweswara Rao Samoju, Hardware‐efficient approximate logarithmic division with improved accuracy, Journal of Circuit Theory and Applications, 2021, Wiley Online Library, https://onlinelibrary.wiley.com/doi/abs/10.1002/cta.2900, https://doi.org/10.1002/cta.2900
Mohsen Imani; Ricardo Garcia; Andrew Huang; Tajana Rosing, May 2019, CADE: Configurable approximate divider for energy efficiency, 2019 Design, Automation & Test in Europe Conference & Exhibition (DATE) https://ieeexplore.ieee.org/document/8715112
Hiromasa Nakayama, June 2006, Algorithm computing the local b function by an approximate division algorithm in D, https://www.sciencedirect.com/science/article/pii/S0747717108001375, https://arxiv.org/abs/math/0606437
Jackson Melchert; Setareh Behroozi; Jingjie Li; Younghyun Kim, 2019, SAADI-EC: A quality-configurable approximate divider for energy efficiency, IEEE Transactions on Very Large Scale Integration (VLSI) Systems (Volume 27, Issue 11, November 2019, pp.2680-2692), https://ieeexplore.ieee.org/document/8766885
Reza Zendegani; Mehdi Kamal; Arash Fayyazi; Ali Afzali-Kusha; Saeed Safari; Massoud Pedram, 2016, SEERAD: A high speed yet energy-efficient rounding-based approximate divider, 2016 Design, Automation & Test in Europe Conference & Exhibition (DATE), 14-18 March 2016, https://ieeexplore.ieee.org/document/7459545
X Li, B Liu, RH Yang, V Courville, C Xing, VP Nia, 2023, DenseShift: Towards Accurate and Efficient Low-Bit Power-of-Two Quantization, Proceedings of the IEEE/CVF, https://openaccess.thecvf.com/content/ICCV2023/papers/Li_DenseShift_Towards_Accurate_and_Efficient_Low-Bit_Power-of-Two_Quantization_ICCV_2023_paper.pdf (Shows how division by a power-of-two, which is a bitshift in integers, can be done using integer addition on the sign and exponent bits of a floating point number.)
David Spuler, March 2024, Chapter 53. Arithmetic Optimization Research, Generative AI in C++: Coding Transformers and LLMs, https://www.amazon.com/dp/B0CXJKCWX9
David Monniaux, Alice Pain, 18 Jul 2022, Formally verified 32- and 64-bit integer division using double-precision floating-point arithmetic, https://arxiv.org/abs/2207.08420
Michael Lunglmayr, 9 Sep 2022 (v2), Efficient Non-sequential Division for FPGAs, https://arxiv.org/abs/2105.05747
X. Fang, Y. Wang, L. Chen and F. An, "A Reconfigurable Floating-Point Division and Square Root Architecture for High-Precision Softmax," Jan 2025, IEEE Transactions on Circuits and Systems I: Regular Papers, doi: 10.1109/TCSI.2024.3524307. https://ieeexplore.ieee.org/abstract/document/10830536/
Zahra Ebrahimi, Muhammad Zaid, Mark Wijtvliet, Akash Kumar, 28 Jun 2022, RAPID: AppRoximAte Pipelined Soft Multipliers and Dividers for High-Throughput and Energy-Efficiency, https://arxiv.org/abs/2206.13970
Jaros{\l}aw Byrka, Franciszek Malinka, Tomasz Ponitka, 20 Jul 2025, Probing EFX via PMMS: (Non-)Existence Results in Discrete Fair Division, https://arxiv.org/abs/2507.14957
Alexandre Durrmeyer, Jean-Christophe Palauqui and Philippe Andrey, 30 Jul 2025, Deep learning of geometrical cell division rules, https://arxiv.org/abs/2507.22587
Davin Choo, Winston Fu, Derek Khu, Tzeh Yuan Neoh, Tze-Yang Poon, Nicholas Teh, 5 Aug 2025, Approximate Proportionality in Online Fair Division, https://arxiv.org/abs/2508.03253
Vincent Lauinger, Lennart Schmitz, Patrick Matalla, Andrej Rode, Sebastian Randel, and Laurent Schmalen, 17 Sep 2025, Novel Phase-Noise-Tolerant Variational-Autoencoder-Based Equalization Suitable for Space-Division-Multiplexed Transmission, https://arxiv.org/abs/2509.14072
Kaiqi Zhang, Shuai Yuan, Honghan Zhao, 24 Sep 2025, TALEC: Teach Your LLM to Evaluate in Specific Domain with In-house Criteria by Criteria Division and Zero-shot Plus Few-shot, https://arxiv.org/abs/2407.10999
Raaida Noor Mahbub 0009-0005-8343-7340, Hani Saleh, and Ghada Alsuhli, Hardware Accelerators for Softmax in Large Language Models: A Survey, 26 February 2026, https://doi.org/10.36227/techrxiv.177208047.71459791/v1 https://www.techrxiv.org/doi/full/10.36227/techrxiv.177208047.71459791/v1 https://www.techrxiv.org/doi/pdf/10.36227/techrxiv.177208047.71459791/v1?onload=true (Survey of Softmax optimization through hardware, approximations, LUTs, numeric representations, Flash Attention, and more.)
David Spuler, March 2024, Chapter 53. Arithmetic Optimization Research, in book "Generative AI in C++", https://www.aussieai.com/book/ch53-arithmetic-optimization
David Spuler, March 2024, Generative AI in C++: Coding Transformers and LLMs, https://www.aussieai.com/book/toc PDF: https://www.aussieai.com/pdf/BOOK-Generative-AI-CPP-Spuler-2024.pdf
David Spuler, May 31st, 2026, Chapter 16. Rare Quantization Techniques, in book LLM Inference Optimization: State-of-the-Art Research, Table of Contents: https://www.aussieai.com/book/llm-inference-optimization https://www.amazon.com/dp/B0H3FKR39T
J. Wei, A. Kuwana, H. Kobayashi and K. Kubo, "Revisit to Floating-Point Division Algorithm Based on Taylor-Series Expansion," 2020 IEEE Asia Pacific Conference on Circuits and Systems (APCCAS), Ha Long, Vietnam, 2020, pp. 240-243, doi: 10.1109/APCCAS50809.2020.9301675, https://ieeexplore.ieee.org/document/9301675