Number – Talk
(→Precision loss: new section) |
Lou Montana (talk | contribs) m (Text replacement - "<code>" to "<code style="display: block">") |
||
| (2 intermediate revisions by 2 users not shown) | |||
| Line 1: | Line 1: | ||
I delete a reference to | I delete a reference to ''integer'', because there is no such type in OFP scripting. | ||
--[[User:Suma|Suma]] 11:36, 30 June 2006 (CEST) | --[[User:Suma|Suma]] 11:36, 30 June 2006 (CEST) | ||
| Line 26: | Line 26: | ||
== Precision loss == | == Precision loss == | ||
There are problems with adding big numbers, for example (16 777 216 + 1) - 16 777 216 = 0 but it is 1... so you loss precision... | |||
The result describe from which number it is not possible to add without loss precision. | |||
<code>_value = 0; | <code style="display: block">_value = 0; | ||
_array = [100000, 10000, 1000, 100, 10, 1]; | _array = [100000, 10000, 1000, 100, 10, 1]; | ||
for "_i" from 1 to 1000000000 step 1000000 do | for "_i" from 1 to 1000000000 step 1000000 do | ||
| Line 39: | Line 40: | ||
{ | { | ||
_value = (_k + _x) - _k; | _value = (_k + _x) - _k; | ||
if (_value != _x) exitWith {diag_log format["precision loss from %1 with %2", [_k] call BIS_fnc_numberText, _x]}; | if (_value != _x) exitWith {diag_log format["precision loss from %1 with adding %2", [_k] call BIS_fnc_numberText, _x]}; | ||
}; | }; | ||
_array = _array - [_x]; | _array = _array - [_x]; | ||
| Line 48: | Line 49: | ||
Result: | Result: | ||
# precision loss from 16 777 216 with 1 | # precision loss from 16 777 216 with adding 1 | ||
# precision loss from 33 554 440 with 10 | # precision loss from 33 554 440 with adding 10 | ||
# precision loss from 67 108 800 with 100 | # precision loss from 67 108 800 with adding 100 | ||
# precision loss from 134 218 000 with 1000 | # precision loss from 134 218 000 with adding 1000 | ||
# precision loss from 268 440 000 with 10000 | # precision loss from 268 440 000 with adding 10000 | ||
# precision loss from 536 800 000 with 100000 | # precision loss from 536 800 000 with adding 100000 | ||
--[[User:Will|Will]] ([[User talk:Will|talk]]) 13:56, 28 May 2016 (CEST) | --[[User:Will|Will]] ([[User talk:Will|talk]]) 13:56, 28 May 2016 (CEST) | ||
Latest revision as of 11:53, 11 January 2023
I delete a reference to integer, because there is no such type in OFP scripting.
--Suma 11:36, 30 June 2006 (CEST)
Is anyone sure on the precision to which numbers can be stored as variables?
OFP example :
var1 = 123456789 var1 = var1 - 123000000 var1 will return 456792
As a non-programmer, I find this behaviour quite strange. --Ceeeb 08:03, 15 February 2007 (CET)
A test script:
player globalchat format["Precision Test: %1, %2, %3, %4, %5, %6, %7, %8, %9", 1234567 - 1234566, 1234567, 999998+1, 999998+2, 999998+3, (9999999+6)-(10999999), (10000000)-(10999999), 9999999/2 - 4999998, 9999999/2 - 4999998.4];
Which gives:
PRECISION TEST: 1, 1.23457E+006, 999999, 1E+006, 1E+006, -999994, -999999, 1.5, 1
This seems to suggest that you can only store a maximum value of 999999 in a variable. (6 digits)
But, you can use up to 8 digits in integer equations, provided the output is 6 digits or less.
Please do not take the above two statements as fact, since I have done only a few hours of limited testing.
--hellop 09:30, 11 July 2008 (PST)
Precision loss
There are problems with adding big numbers, for example (16 777 216 + 1) - 16 777 216 = 0 but it is 1... so you loss precision... The result describe from which number it is not possible to add without loss precision.
_value = 0;
_array = [100000, 10000, 1000, 100, 10, 1];
for "_i" from 1 to 1000000000 step 1000000 do
{
{
_value = (_i + _x) - _i;
if (_value != _x) exitWith
{
for "_k" from (_i - 1000000) to _i step _x do
{
_value = (_k + _x) - _k;
if (_value != _x) exitWith {diag_log format["precision loss from %1 with adding %2", [_k] call BIS_fnc_numberText, _x]};
};
_array = _array - [_x];
};
}
forEach _array;
};
Result:
- precision loss from 16 777 216 with adding 1
- precision loss from 33 554 440 with adding 10
- precision loss from 67 108 800 with adding 100
- precision loss from 134 218 000 with adding 1000
- precision loss from 268 440 000 with adding 10000
- precision loss from 536 800 000 with adding 100000