[FRPythoneers] Numerical Accuracy

Sean Reifschneider jafo at tummy.com
Sat May 18 14:16:19 MDT 2002


On Sat, May 18, 2002 at 01:44:45PM -0600, Evelyn Mitchell wrote:
>>>> 29.8
>29.800000000000001
>
>Looks like a bug to me.

Fast floating point approximations sometimes are very bad at dealing with
certain values.  They go under the assumption that it's ok to be close
enough (and to me, 15 digits of precision is close enough for my use).

It's not so much a bug as a trade-off definied by (probably) IEEE fast
floating point mechanisms.  The bigger problem is when you find that 2.0 +
2.0 != 4.0 -- because of rounding errors and the like...  Which is part
of the reason I will usually check for "< 1" instead of "== 0"...

Sean
-- 
 "Who's the guy in the suit?"  "FBI."  "He looks a little to much like a
 lounge lizard for FBI."  "He's working under-cover."  -- Sean & Evelyn 2000
Sean Reifschneider, Inimitably Superfluous <jafo at tummy.com>
tummy.com - Linux Consulting since 1995. Qmail, KRUD, Firewalls, Python



More information about the FRPythoneers mailing list