Validating floating point implementations Pierre Ciadoux NIST CAVP / PQC MPTS 2026 01/27/2026
Content (12 min presentation) Definition of floating points and fixed points Inaccuracy and Variation examples Real world impact
undefined Floating points (as specified in IEEE754) Most processors use floating points based on standard IEEE754 Useful for very large or very small numbers Efficient using hardware computations (FPU) By Codekaizen - Own work, CC BY-SA 4.0, https://commons.wikimedia.org/w/index.php?curid=3595583

Fixed points

16-bit fixed point example (Q8.8)

Inaccuracy
Inaccuracy a should be 2.6 but is: 2.599999 GCC: warning: floating-point comparison is always false; constant cannot be represented exactly in type'float' [-Wliteral-range]
Variations Implementation differences Example: (a + b) + c ≠ a + (b + c)
Variations Compiler induced Fused-instructions

Real world impact

A variation can impact execution path and/or numerical values

ComputeValue(5.48) ComputeValue(5.51) ComputeValue(5.53)
5 6 -1 (error)
Validating different numerical values Potential validation Small difference Detectable difference Large difference Numerical difference Hidden difference
Validating different execution path Potential validation Small difference Reasonable way to enumerate branches Large difference Different execution path Large number of potential branches
Conclusion Validation can decide to allow for some variation in floating (and fixed) point computations Depends on the scheme used Using floating points can impact values but will only have a real impact on a small percentage of computations It is likely that a validator computes the same values and doesn’t notices variations -> Need for more test vectors than deterministic
Questions? Upcoming article on Validating Falcon with: Maxime Bros Chris Celi Ray Perlner Contact: pierre.ciadoux@nist.gov cavp@nist.gov