Several float to int potential rounding errors/inconsistencies

Linus Torvalds torvalds at linux-foundation.org
Sat Mar 11 08:32:06 PST 2017


On Sat, Mar 11, 2017 at 5:41 AM, Lubomir I. Ivanov <neolit123 at gmail.com> wrote:
>
> of course, there is no need of lrint() in a lot of locations and
> rint() can be used instead, like here:
>
> - sample->depth.mm = depth * FEET * 1000;
> + sample->depth.mm = lrint(depth * FEET * 1000);
>
> "mm" is of type int.

No.

It's an understandable confusion: thinking that rint() returns 'int'
and lrint() returns 'long'. But no, that's not how it works.

'rint()' returns a double that has been rounded to an integer. Note
the difference between: "integer" and "int". The end result not only
isn't an 'int', it might not even fit in an int. In fact, it might not
fit in a long.

So 'rint()' does work, but it results in the float-conversion warnings
because of the implicit conversion from 'double' to 'int' in the final
assignment.

So 'lrint()' is the right thing to do, because FEET is 0.3048, so
'depth * FEET * 1000' is a floating point value, and it does actually
want rounding in theory.

Then the long->int conversion might drop bits, but we don't care.

Not that anybody really cares about the nearest mm when doing feet
conversion, but still...

> here, t1 is of type int as well:
> + int t1 = lrint(max_d / slope);

Again, same issue. "rint()" doesn't return an "int". So "lrint()" is
actually better.

                       Linus


More information about the subsurface mailing list