Truncation and rounding of numbers

From Planimate Knowledge Base
Jump to navigation Jump to search
The printable version is no longer supported and may have rendering errors. Please update your browser bookmarks and please use the default browser print function instead.

The trunc routine operation uses the C floor(x) function, this returns the highest integer that is less than or equal to x. So trunc(1.5) is 1, trunc(0.5) is 0 and trunc(-0.5) is -1.

Very small -ve values will trunc() to the value below it, just as 1.999999999999 would trunc to 1 but display as 2. The internal precision is about 15 digits.

It all comes down to the application you have for the trunc-ed number, normally round is much safer and should be used to convert a number which *should* be integer to an actual integer (with the precision limits of a C "double" that is).

For internal double to int conversion, I actually use a delta of 1.0e-6. Values are display-rouded to 4 decimal places. And values are compared for equality using a dynamic precision based on the highest value, of 10 digits,




idkbase note 68