www.webdeveloper.com
Results 1 to 4 of 4

Thread: [RESOLVED] 3 * 0.6 = 1.7777777777779

  1. #1
    Join Date
    Sep 2004
    Location
    At the corner of WALK and DONT WALK
    Posts
    1,745

    resolved [RESOLVED] 3 * 0.6 = 1.7777777777779

    What's going on when I do a simple JavaScript calculation in FireFox? The title is literally the result I got. Is it because I'm converting strings to numbers?

  2. #2
    Join Date
    Mar 2010
    Posts
    2,803
    Maybe post your code and FF version because the following code gives me an answer of

    1.7999999999999998 to 3 * 0.6 in both IE8 and FF3.6

    The code displays the same answer for textbox inputs that are both parseFloated()'ed and not parseFloated()'ed.

    The calculated answer is near enough to the correct answer for me

    You can use .toFixed() to round the answer to fewer decimal places.

    PHP Code:
    <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN"
     "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd"
    >
    <
    html xmlns="http://www.w3.org/1999/xhtml">
    <
    head>
    <
    title></title>
    <
    script type="text/javascript">
    <!--
     
    function 
    calc() {
         var 
    num1      document.getElementById("txtNum1").value;
         var 
    num2      document.getElementById("txtNum2").value;
         var 
    num1Parse parseFloat(document.getElementById("txtNum1").value);
         var 
    num2Parse parseFloat(document.getElementById("txtNum2").value);
         
         
    alert("UnparseFloat Product = "+num1*num2+"\n\nparseFloat Product = "+num1Parse*num2Parse);
         
    }
    //-->
    </script>

    </head>
    <body>
     
    <div>
    Enter a number: <input type="text" id="txtNum1" />
    </div>

    <div>
    Enter a number: <input type="text" id="txtNum2" />
    </div>

    <div>
    <a href="javascript:calc();">Calculate Product</a>
    </div>
     
    </body>
    </html> 
    Last edited by tirna; 05-26-2010 at 05:11 AM.

  3. #3
    Join Date
    Dec 2003
    Location
    Bucharest, ROMANIA
    Posts
    15,428
    Quote Originally Posted by Mr Initial Man
    Is it because I'm converting strings to numbers?
    No. It is because the CPU uses binary to calculate, or some finite floated decimal numbers become periodical when translated into binary. Thus, the CPU has to perform an approximation, a rounding of that binary. Sometimes, the sum or those small roundings superadd into a final error while translating decimal-binary back and forth.

    In fact, it is only an error of display, not calculation, thus it is enough to use toFixed() method in order to bypass that, but only after you finished all the math operations and you want to display the final result. Or, if you want, you may first multiply the floated number with 10 risen to the proper power(^n) before the math operation - in order to avoid using floated values - and divide, later, the result with the same 10^n factor.
    Last edited by Kor; 05-26-2010 at 05:40 AM.

  4. #4
    Join Date
    Sep 2004
    Location
    At the corner of WALK and DONT WALK
    Posts
    1,745
    Quote Originally Posted by tirna View Post
    Maybe post your code and FF version because the following code gives me an answer of

    1.7999999999999998 to 3 * 0.6 in both IE8 and FF3.6
    Sorry, the title was a typo.

    Thank you both for your suggestions, I shall used toFixed().

Thread Information

Users Browsing this Thread

There are currently 1 users browsing this thread. (0 members and 1 guests)

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  
HTML5 Development Center



Recent Articles