[BigInt] Using tests from “Violet - Python VM written in Swift”

Question

[BigInt] Using tests from “Violet - Python VM written in Swift”

LiarPrincess opened this issue a year ago · comments

This is a parent issue for merging BigInt tests from Violet - Python VM written in Swift. In total is is about 400 tests (about 70% of Violet test cases) split into a multiple smaller pull requests.

Please note that this may seem like a lot of tests, but I still can't guarantee that everything works even after passing all of them. A few minutes ago I was debugging crash in division, just to discover that this will also fail:

// -9223372036854775808 = Int64.min, obviously '-Int64.min' overflows
let int: Int64 = -9223372036854775808
let big = BigInt(int)
XCTAssertEqual(-big, big * -1)

Also, I have NOT looked into your implementation. I just know that you use 2 complement, but nothing more. Once we fix all of the crashes/failures I may look into your code and add some more targeted tests.

Reference implementation

BigInt module inside Violet - Python VM written in Swift passes all of the tests (look at the swift-numerics branch).

Internal Violet implementation (see documentation for more details):

Under the hood it is a union (via tagged pointer) of Int32 (called Smi, after V8) and a heap allocation (magnitude + sign representation) with ARC for garbage collection.

Since Violet contains 2 different representations of BigInt (Smi for small numbers and BigIntHeap for the rest) in a lot of tests you will see separate test_int_xxx (for small integers) and test_big_xxx (for big integers). All of the test cases finish in ~10s (2014 rMBP, lowest spec, serial execution), so they are fast enough to keep int tests even if swift-numerics implementation does not have a Smi equivalent. This also tests if all of the BigInt operations have the same semantic as Int (div/shift rounding, reminder sign etc.).

Platforms

All of the tests results come from 2014 rMBP -> mac 11.7 (Big Sur), Xcode 13.2.1, Intel.

Not tested:

M1 - I don't have one.
32 bit (or more precisely: configuration where Int.bitWidth != 64, which is not the same as 32 bit, but whatever) - the whole Violet was designed for 64 bit (BigInt was actually the main reason for this), so I may have unintentionally backed some assumptions here and there. Code review needed.

I can confirm that Violet implementation passes tests on:

mac 11.7 (Big Sur) + Xcode 13.2.1 (Swift 5.5) + Intel
Ubuntu 21.04 + Swift 5.4.2 + Intel G4560
Docker
- swift:latest (5.6.0)
- swift:5.3.2

So, the test cases should be resistant to mac/linux/docker differences.

Missing stuff

CMake - in other modules you have CMakeLists.txt, but not in BigIntTests. I don't want to be the person to introduce it, so lets just say that I will follow the BigInt convention of not having one.
~~floating point tests - you already have some tests in BigIntTests.swift.~~ ADDED
missing Sendable

Other engines

In some pull requests/code I will be referencing other BigInt implementations:

WolframAlpha.com - almost all of the operations have the same semantic as Swift, except for >>. For example: -1932735284 >> 5:

Engine	Result	Comment
Wolfram Alpha	-60397977
Swift 5.3.2*	-60397978	This is in `Int64` range, so you can just `-1932735284 >> 5` to test it.
Node v17.5.0	-60397978
Python 3.7.4	-60397978

Both are correct, it depends on the rounding mode, but you already know this: swift-numerics/IntegerUtilities/ShiftWithRounding.swift.

Python - there is a difference in div/mod for negative numbers:

(-5)//4 (-5)%4

Python 3.7.4 -2 3

Swift 5.3.2 -1 -1

Both correct: lhs/rhs = q rem r -> lhs = q * rhs + r.

`Helpers` directory

Shared code for all of the test cases is inside Helpers directory.

Since all of the pull requests contain the same Helpers code, it may be easier for us to first review and merge tests while ignoring Helpers. Then, after all is merged, do any changes in Helpers. Well… unless there is something really broken in Helpers.

License

Unless stated otherwise I am the author of this code, so feel free to slap any license you need.

Recommended merging order

In total this is around 10k hand written + 22k generated lines of code. I split them up, so that each PR contains 2-4 files (excluding Helpers, see above).

(✅ - pass, ❌ - fail, 💀 - crash, 🐰 - to discuss)

Node
- [BigInt tests] ❌ Nodejs tests - they can be either 1st or last. Fixing them will solve most of the problems in other tests. But also, fixing problems other tests will fix Node tests. There is a certain overlap.
Inits and protocols
Operations
Other
No merge
- [BigInt tests][No merge] ❌🐰 Apple
- [BigInt tests][No merge] 🐰 How NOT to write performance tests

Final

I know that with so many pull requests it may be a bit hard to see the big picture, but eventually we should arrive at the state from the image below (please ignore the Apple and Performance directories, BigIntTests.swift are the tests currently present). I think that this is quite clean (1 file for every operation), and it will be quite easy to add/modify tests.

LiarPrincess · Answer 1 · Fri Feb 03 2023 21:42:31 GMT+0800 (China Standard Time)

[3.2.2023] EDIT to main issue:

LiarPrincess · Answer 2 · Thu Feb 23 2023 00:33:00 GMT+0800 (China Standard Time)

Things to do in Helpers (but I'm not going to update 15 PRs, so we will do it later):

add reserveCapacity in generate[Big]Ints - this does not matter in normal tests (count ~1000), but it matters in performance tests where we need to generate 100_000 values
rename BigIntPrototype.compare -> BigIntPrototype.compareMagnitude

rewrite BigIntPrototype.create - shifts 10x faster than multiplication:

internal static func create<T: FixedWidthInteger>(
  isPositive: Bool,
  magnitude: [T]
) -> BigInt {
  assert(!T.isSigned)
  var result = BigInt()

  for (index, word) in magnitude.enumerated() {
    var bits = BigInt(word)
    bits <<= index * T.bitWidth
    result |= bits
  }

  if !isPositive {
    result.negate()
  }

  return result
}

If we use sign+magnitude with UInt as Word then there is another possible optimisation that avoids calculations altogether.

LiarPrincess · Answer 3 · Mon Mar 20 2023 17:43:55 GMT+0800 (China Standard Time)

I wrote Violet XsProMax which is a Violet implementation with following changes:

no small inlined integer (Smi) - magnitude is always stored on the heap
no restrictions on the size - isNegative is stored in-line (and not on the heap like in Violet); count and capacity are on the heap because I don't want to stray too much from Violet.

Which gives us:

struct BigInt {
  struct Header {
    var count: UInt32
    var capacity: UInt32
  }

  var flags: UInt8
  var buffer: ManagedBufferPointer<Header, UInt>
}

I updated the performance PR with the results.

mgriebling · Answer 4 · Wed Apr 19 2023 23:09:54 GMT+0800 (China Standard Time)

The issue below (from @LiarPrincess) appears fixed in my latest updates (see pull request #261)

// -9223372036854775808 = Int64.min, obviously '-Int64.min' overflows
let int: Int64 = -9223372036854775808
let big = BigInt(int)
XCTAssertEqual(-big, big * -1)

mgriebling · Answer 5 · Thu Apr 20 2023 06:01:38 GMT+0800 (China Standard Time)

All remaining issues (at least the published ones) appear to have been fixed with the pull request #262.
There were some problems with division/modulo signs, initialization, corner-case fixes.

	`(-5)//4`	`(-5)%4`
Python 3.7.4	-2	3
Swift 5.3.2	-1	-1

[BigInt] Using tests from “Violet - Python VM written in Swift”

Reference implementation

Platforms

Missing stuff

Other engines

Helpers directory

License

Recommended merging order

Final

`Helpers` directory