Giters

dcodeIO / long.js

A Long class for representing a 64-bit two's-complement integer value.

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Division for unsigned values produces incorrect result

vladsch opened this issue · comments

commented

Test sample

var Long = dcodeIO.Long;
        var a = new Long(0,8, true);
        var b = Long.fromNumber(2656901066, true);
        var x = a.div(b); // x = 12 is corrupt, its object value->unsigned = false, should be true
        //x = 12; // Explicitly setting Long.fromNumber(12, true) yields correct result.
        var y = Long.fromNumber(3850086465, true);
        var z = Long.fromNumber(2476925576, true);
        var res = x.mul(y).compare(z.shl(32)) > 0;

        console.log("x = " + x + "; (" + (x.mul(y)) + ") > (" + (z.shl(32)) + ") = " + res);

Output:
x = 12; (46201037580) > (10638314343545962496) = true

Expected:
x = 12; (46201037580) > (10638314343545962496) = false

commented

Well, I see where this fails, but this doesn't seem to be an easy thing to fix. The derived algorithm (from the closure library) seems to be designed for signed values only. Making the inputs properly unsigned only results in infinite loops for me so far.

commented

What about to replace 'res = ZERO' (https://github.com/dcodeIO/long.js/blob/master/src/long.js#L961) to 'res = this.unsigned ? UZERO : ZERO'
It seems it resolves the issue

commented

Well, yes, partly, but there is also this test case: https://github.com/dcodeIO/long.js/blob/master/tests/suite.js#L159

Using toNumber() there is bogus of course, but let's say we do this instead:

        "max_unsigned_div_neg_signed": function(test) {
            var a = Long.MAX_UNSIGNED_VALUE;
            var b = Long.MAX_UNSIGNED_VALUE.sub(1);
            var longVal = a.div(b); // here
            console.log(longVal.toString());
            test.done();
        },

With your proposed fix applied, this (still) results in an infinite loop.

commented

@dcodeIO, thank you for fixing this issue!