# [Pharo-project] Unexpected behavior of roundTo:

Andres Valloud avalloud at smalltalk.comcastbiz.net
Sat Apr 9 20:19:32 CEST 2011

```On 4/9/11 0:28 , Marcus Denker wrote:
>
> On Apr 9, 2011, at 9:09 AM, Andres Valloud wrote:
>
>> I think the real solution to these "problems" is to implement
>> decimal floating point as per IEEE-754/2008.  \\
>
> The MPFR lib seems to me (as far as I understand) is inspired by IEEE
> 754
>
> <quote from the docs>

The docs seem to suggest MPRF implements variable precision binary
floating point numbers.  IEEE-754/2008, as opposed to IEEE-754/1985,
also specifies types for base 10 (i.e.: decimal) floating point numbers.
In other words, 2.5 wouldn't be represented as (101)_2, but (25)_10.
Thus, going back to the original problem that caused this discussion,

> 2.8011416510246336 roundTo: 0.01
>
> returns
>
>  2.8000000000000003

the problem here is that there is no binary floating point number that
can represent the number 0.01.  The arguments why this is so are the
usual (and I just put them here for the sake of completeness): binary
floating point numbers represent fractions of the form

+/- m * 2^k

where m, k are integers such that 0 <= m < M, and e <= k <= K.  So let's
try to find which values of m and k match for 0.01:

1/100 = m * 2^k

Well, m is not zero, so it must be at least 1, and so k must be
negative.  Set k' = -k and rearrange:

2^k' = 100m

Now we invoke the help of the fundamental theorem of arithmetic that
says integers can be factored into primes in only one way.  On the left
we only have 2 as a prime factor.  On the right, there is a factor of 5.
Therefore, we conclude the above equality is impossible.  So, m and k
do not exist, and so no binary floating point number is equal to 0.01.

To make the story short, you cannot assume lossless operation just
because you can see a base 10 print string.  If loss of precision is the
issue because we only have 2^k' on the left, let's try base 10 instead.
In other words, floating point numbers in which the base is 10
represent values of the form

+/- m * 10^k

where m, k are integers such that 0 <= m < M, and e <= k <= K.  So let's
try to find which values of m and k match for 0.01:

1/100 = m * 10^k

So now we just match the fraction on the left with m=1 and k=-2.  Done.
Of course, there are values that cannot be represented without loss of
precision, but the loss of precision involves only m and k.  For example,

67423523487542376543765376234576235726345623472 = m * 10^k

when 0 <= m < M and M = 9.  There are not enough digits in the mantissa,
so the result is going to be 7*10^k for some k.  But this issue happens
as well with binary floating point numbers when k is positive.  Try

67423523487542376543765376234576235726345623472 asFloat truncated

Increasing the precision of binary floating point numbers to arbitrary
precision help you if you need a bigger mantissa, but a bigger mantissa
does not solve the fundamental problem of binary floating point numbers
that causes imprecision whenever you use base 10 to write down values.

> MPFR is LGPL, so linking that in the VM or using it via FFI is no

I am not convinced that's what you need:

http://en.wikipedia.org/wiki/Decimal_floating_point

And it looks like Intel wrote a decimal floating point library, too...
now I need to check the license on this one as well:

http://software.intel.com/en-us/blogs/2008/03/06/intel-decimal-floating-point-math-library/

Andres.

```