Several float to int potential rounding errors/inconsistencies
Lubomir I. Ivanov
neolit123 at gmail.com
Sat Mar 11 09:34:10 PST 2017
On 11 March 2017 at 18:32, Linus Torvalds <torvalds at linux-foundation.org> wrote:
> On Sat, Mar 11, 2017 at 5:41 AM, Lubomir I. Ivanov <neolit123 at gmail.com> wrote:
>> of course, there is no need of lrint() in a lot of locations and
>> rint() can be used instead, like here:
>> - sample->depth.mm = depth * FEET * 1000;
>> + sample->depth.mm = lrint(depth * FEET * 1000);
>> "mm" is of type int.
> It's an understandable confusion: thinking that rint() returns 'int'
> and lrint() returns 'long'. But no, that's not how it works.
> 'rint()' returns a double that has been rounded to an integer. Note
> the difference between: "integer" and "int". The end result not only
> isn't an 'int', it might not even fit in an int. In fact, it might not
> fit in a long.
> So 'rint()' does work, but it results in the float-conversion warnings
> because of the implicit conversion from 'double' to 'int' in the final
so rint() returns a double and lrint() returns a long.
makes sense for the stdlib folks, i guess:
so with lrint() we avoid the float -> integer cast.
but in the case of:
int mm = lrint(some_float);
if one enables -Wconversation on a 64bit toolchain, where sizeof(long) == 8:
warning: conversion to ‘int’ from ‘long int’ may alter its value [-Wconversion]
which is essentially truncation, but probably doesn't matter in our
case as it's unlikely some of those properties will receive values
larger than (2^31) - 1...
for instance, 2.147483E+6 meters is *very* deep.
More information about the subsurface