Floating-Point Adders and Subtractors

"A bunch of tensors getting multiplied and added": this is what 70% of AI workloads boil down to. Nothing fancy. My passion for arithmetic hardware in AI led me to focus on the two main building blocks: adders and multipliers. Adders, however, consume 4 to 30 times less energy than multipliers. Interestingly, since 2019, a growing body of research has explored designing neural networks that rely solely on addition, eliminating multipliers entirely. In these notes, I'll walk you through the fundamentals of adding two floating-point numbers in hardware! pretty trivial, eh? Not really! Let’s do some bit-smashing together and see what is happening under the hood of addition in hardware!

Download the design here

Feel free to reach out if you have questions or suggestions!