Click to See Complete Forum and Search --> : Stack overflow with division


digitaldi
07-01-2003, 03:09 PM
I get a stack overflow in the division operation of this script because the resulting number can be a repeating decimal, such as 91.666666666666666666666666666666666 ...

The ECMAscript spec indicates this is planned behavior - how do I do the division and round so the stack overflow does not occur in javascript?

<code>
<script>
function DoOnDataChanged()
{
var spread = document.all("FpSpread1");
var c = spread.ActiveCol;
var r = spread.ActiveRow;
if ((c==2)||(c==5)||(c==8)||(c==11)||(c==14)||(c==17))
{
var adjVal = spread.GetValue(r,c);
var actVal = spread.GetValue(r, (c-1));

// the result of the next line causes a stack overflow:
var adjMar = (1 - (actVal / adjVal))*100;
spread.SetValue(r, (c+1), adjMar);
return true;
}
if ((c==3)||(c==6)||(c==9)||(c==12)||(c==15)||(c==18))
{
var adjMar = spread.GetValue(r,c);
var actVal = spread.GetValue(r, (c-2));
var adjVal = actVal / (1 - (adjMar/100));
spread.SetValue(r, (c-1), adjVal);
return true;
}
}
</script>
</code>

David Harrison
07-01-2003, 04:44 PM
There's two ways to do this, the hard way:

// if your answer is stored in a variable called num just to make it easier and the number of decimal places you want is in a variable called dp.

num = (Math.round(num*Math.pow(10,dp)))/Math.pow(10,dp);

and there's the easy way using toFixed, but I'm not sure how this is supposed to be used (since I've never used it) so someone will come along after this post and tell you about it.