Math errors in Lua Player for Windows and crashes PSP

Discuss using and improving Lua and the Lua Player specific to the PSP.

Moderators: Shine, Insert_witty_name

Post Reply
WinBear
Posts: 1
Joined: Fri Jan 06, 2006 1:48 am
Location: Denton, Tx

Math errors in Lua Player for Windows and crashes PSP

Post by WinBear »

I've been working on a Lua Player game. I'm requiring compatibility with the Microsoft random number generator/seed code, which I have. This particular code depends on the behavior of "long integers" wrapping when exceeding the capacity of 32 bit numbers. Lua Player uses floating point and I read some of the arguments about that. I'm fairly confident I can work around that feature and make the numbers behave the same way. However, Lua Player for Windows introduces a rounding error which I cannot work around.

Here's what I'm seeing:

Using the following formula
x = 2745024 * 214013 + 2531011

Lua Player for Windows gives a result of x = 587473354752
Windows Calculator says the result should be x = 587473352323
The difference of 2429 looks a little odd until you see the differences in hexadecimal.

In Lua Player for Windows, x = 0x88C8240000
In Calculator for Windows, x = 0x88C823F683

It's rounding up by 2 to the 16th power.

I went to try this on the PSP and Lua Player for the PSP completely crashes trying to do the math calculation, shutting down the PSP after about 10 seconds.
Tommy in Denton
romero126
Posts: 200
Joined: Sat Dec 24, 2005 2:42 pm

Post by romero126 »

0/0 also crashes it
Post Reply