Floating-point tolerance tests

Absolute tolerance:

  • Not good for big numbers
abs(x–y) <= tol1

Relative tolerance:

  • Not good for small numbers
abs(x–y) <= tol2 * max(abs(x), abs(y))

Combined tolerance test:

(abs(x-y) <= tol1) ||
  (abs(x-y) <= tol2 * max(abs(x), abs(y))

equal to:

abs(x-y) <= tol1 * Max(1.0f, tol2/tol1 * max(abs(x), abs(y))

which for tol1=tol2 becomes:

abs(x-y) <= tol1 * Max(1.0f, abs(x), abs(y))

 

References:

  • Real time collision detection, Christer Ericson