Decimal Fractions Suffer Binary Round-off

Product Version(s): 3.13 3.20 3.3x 4.00
Operating System:   MS-DOS
Flags: ENDUSER | TAR21448
Last Modified: 29-SEP-1988    ArticleIdent: Q11001

Problem:

The following bankers rounding described in the "Microsoft Pascal
User's Guide" on Page 165 does not work correctly:

trunc(4.5)   result is 4         correct
trunc(207.5) result is 207       should be 208

This problem occurs with PASCAL.LIB and ALTMATH.LIB.

Response:

This is not a bug. Certain rational decimal numbers are not precisely
representable in binary. As a result, the representation of a fraction
may actually be slightly less than the fraction. Therefore, when the
TRUNC function is performed, the result may be unexpected. When real
numbers are decoded, the number is rounded to ensure correct results.

The following example also illustrates this problem. When you use real
numbers, performing a loop that adds 0.1 each time and then uses
TRUNC, the results will be off eventually. In this example it becomes
inaccurate after five increments of the loop. It becomes inaccurate
because it truncates to 4.00 when it should have returned
5.00.

The following sample code demonstrates the problem:

    PROGRAM DEGTST(INPUT,OUTPUT);
    VAR PRTF:TEXT;
        I,J,DEG,TMP1:REAL;
    BEGIN
      ASSIGN(PRTF,'PRN');
      REWRITE(PRTF);
        TMP1:=4.5;
      REPEAT
        I:=TRUNC(TMP1);
        DEG:=TMP1 * 100.0;
        J:=TRUNC(DEG);
        WRITELN(PRTF,'I',I:3:2,' TMP1 ',TMP1,'DEG',DEG,' J',J:3:2);
        TMP1:= TMP1 + 0.1;
      UNTIL TMP1>6.0
    END.

In your particular case, the following program illustrates the
problem:

    PROGRAM TEST(OUTPUT);
    VAR I1 , I2: INTEGER;
    BEGIN
            I1:=TRUNC(4.5);
            I2:=TRUNC(207.5);
            WRITELN(I1);
            WRITELN(I2)
    END.

If you check, the number 207.5 is indeed not precisely representable
using binary arithmetic. It is in fact somewhat less than 207.5 in
binary arithmetic. Thus, when you truncate it, the number returned will
be 207.