[RESOLVED] 3 * 0.6 = 1.7777777777779

**Mr Initial Man** · 05-26-2010 02:28 AM

What's going on when I do a simple JavaScript calculation in FireFox? The title is literally the result I got. Is it because I'm converting strings to numbers?

**tirna** · 05-26-2010 04:08 AM

Maybe post your code and FF version because the following code gives me an answer of

1.7999999999999998 to 3 * 0.6 in both IE8 and FF3.6

The code displays the same answer for textbox inputs that are both parseFloated()'ed and not parseFloated()'ed.

The calculated answer is near enough to the correct answer for me

You can use .toFixed() to round the answer to fewer decimal places.

PHP Code:


<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN"

 "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">

<html xmlns="http://www.w3.org/1999/xhtml">

<head>

<title></title>

<script type="text/javascript">

<!--

 

function calc() {

     var num1      = document.getElementById("txtNum1").value;

     var num2      = document.getElementById("txtNum2").value;

     var num1Parse = parseFloat(document.getElementById("txtNum1").value);

     var num2Parse = parseFloat(document.getElementById("txtNum2").value);

     

     alert("UnparseFloat Product = "+num1*num2+"\n\nparseFloat Product = "+num1Parse*num2Parse);

     

}

//-->

</script>



</head>

<body>

 

<div>

Enter a number: <input type="text" id="txtNum1" />

</div>



<div>

Enter a number: <input type="text" id="txtNum2" />

</div>



<div>

<a href="javascript:calc();">Calculate Product</a>

</div>

 

</body>

</html>

**Kor** · 05-26-2010 04:21 AM

Originally Posted by Mr Initial Man

Is it because I'm converting strings to numbers?

No. It is because the CPU uses binary to calculate, or some finite floated decimal numbers become periodical when translated into binary. Thus, the CPU has to perform an approximation, a rounding of that binary. Sometimes, the sum or those small roundings superadd into a final error while translating decimal-binary back and forth.

In fact, it is only an error of display, not calculation, thus it is enough to use toFixed() method in order to bypass that, but only after you finished all the math operations and you want to display the final result. Or, if you want, you may first multiply the floated number with 10 risen to the proper power(^n) before the math operation - in order to avoid using floated values - and divide, later, the result with the same 10^n factor.

**Mr Initial Man** · 05-26-2010 04:55 AM

Originally Posted by tirna

Maybe post your code and FF version because the following code gives me an answer of

1.7999999999999998 to 3 * 0.6 in both IE8 and FF3.6

Sorry, the title was a typo.

Thank you both for your suggestions, I shall used toFixed().

Thread: [RESOLVED] 3 * 0.6 = 1.7777777777779

Thread Tools

Display

[RESOLVED] 3 * 0.6 = 1.7777777777779

Thread Information

Users Browsing this Thread

Bookmarks

Bookmarks

Posting Permissions