Error compiling magnus v0.5.1 on arm-linux

[Christopher Yammine]

Hello 👋
First off, thank you for this project it's great.

I've noticed a compilation error on arm-linux after updating from version 0.4 to 0.5

The error:

error[E0308]: mismatched types
   --> /home/runner/work/roaring-rb/roaring-rb/tmp/cargo-vendor/magnus/src/r_bignum.rs:295:13
    |
291 |     fn sign(self) -> u64 {
    |                      --- expected `u64` because of return type
...
295 |             r_basic.as_ref().flags & (ruby_fl_type::RUBY_FL_USER1 as VALUE)
    |             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ expected `u64`, found `u32`
    |
help: you can convert a `u32` to a `u64`
    |
295 |             (r_basic.as_ref().flags & (ruby_fl_type::RUBY_FL_USER1 as VALUE)).into()
    |             +                                                               ++++++++

Here is a link to the GH Action run where I've observed this: link
All other platforms successfully compile magnus so I think it's isolated to the arm-linux platform.

If someone knows off the top of their head how to fix this I'd love to meaningfully contribute by submitting a patch.

Thanks for your attention

[Mat Sadler]

Hey, thanks for the bug report.

So the issue here is arm-linux is a 32 bit platform (aarch64-linux is 64 bit arm), and when I wrote this code I wasn't thinking about 32 bit platforms (I'm happy to get Magnus compiling for 32 bit platforms, but no guarantee it'll actually do what it's meant to do).

I'll explain what's going on from the ground up so that you have the context if you'd like to put together a patch. Sorry if any of this is stuff you know already.

In Ruby's internals / the Ruby C API all Ruby objects are represented by VALUE, this is effectively a pointer to the object (all of Magnus's types are just wrappers around a VALUE).

Ruby defines VALUE as an integer the size of a pointer, which is basically Rust's usize. But something gets lost in translation so it ends up in Rust as VALUE being defined as an alias to u64 on 64 bit platforms, and an alias to u32 on 32 bit platforms.

So VALUE is a pointer to a Ruby object, and r_basic is the thing it points to. r_basic usually holds some very basic information about the object and then further pointers to specifics. One of the basic bits of data is flags this is a bitfield that encodes some information about the object. In the case of RBignum one of the things in the flags is if the bignum is positive or negative. For some weird historical reason Ruby defines the type of flags as VALUE, even though it's not a Ruby object (relying on knowing that under the hood VALUE is just an integer).

To check the bit in flags that represents whether the bignum is positive or negative we & (bitwise and) the flags with the constant ruby_fl_type::RUBY_FL_USER1. The type of this isn't VALUE, but Rust's & operator wants the types on both sides to be the same, so we cast it to VALUE. As both sides of the & are a VALUE, the end result is VALUE. As the result is most definitely not a Ruby object I didn't want to use VALUE as the return type, so used u64.

This works because on all the platforms I test on VALUE is an alias of u64, but it breaks on arm-linux because there VALUE is an alias of u32, which then won't match the return type.

One other detail, sign/is_positive/is_negative exist in Ruby's C API as macros, which can't be used from Rust, so have to be reimplemented for Rust. The sign function isn't public, it's only here to implement is_positive. While sign does exist in Ruby's C API I didn't have any intention of exposing it publicly as it's just kind of weird, like it's called sign and the return value is 0 for negative or any other number for positive, how does that make sense? So sign doesn't need to exist, can have a different return type, whatever, along as is_positive stays working the same.

I can think of a few different ways to fix this, I don't have any strong reason to prefer one over another, and you might be able to think of another. Hopefully that's enough to go off without just telling you exactly what to do (which seems like it'd be boring for you).

[Christopher Yammine]

Thank you for the very thoughtful reply! :)
I didn't know any of this, so it was very educational for me.

I actually would appreciate some pointers on where I might begin to implement support for 32bit platforms, though no promises on my ability to actually get it done lol

I'll just drop support for that platform for my gem in the meantime

[Mat Sadler]

Sorry, this is a bit long winded, I'm trying to be quick and weirdly that means writing more because I'm not spending the time to edit it down.

The body of the sign function works just fine on 64 bit or 32 bit, but the result doesn't match up with the return type declared in the function signature. This is because the function body is evaluating to VALUE, which only sometimes matches u64.

One way to fix this is to set the return value in the function signature to VALUE, that way it'll always match. The one place it's used is is_positive where it's compared with 0. Rust infers the type of integer literals, so as it is being compared against the result of sign the 0 is automatically the same type as whatever sign returns.
I don't really like this approach as while VALUE is technically just an alias of u64/u32 it is semantically a Ruby object, which the return of sign is not.

Another way would be the suggestion given in the error message, to tack on an .into(). This would work. On 32 bit platforms it becomes a conversion from u32 to u64, which is fine as u32 fits in u64, and on 64 bit platforms it's a conversion from u64 to u64, which obviously works. The problem is it so obviously works that it's pointless, and cargo clippy warns us with useless conversion to the same type .

A third possibility would be to use usize as the return value. usize is a distinct type, not an alias like VALUE, but has similar properties to VALUE in that it's the same size as u64 on 64 bit platforms and the same size as u32 on 32 bit platforms. This can't be done with .into() because the Rust compiler knows that sometimes usize is only 32 bits, but it doesn't know that in that case VALUE will always alias to u32, so it errors when VALUE is u64 because u64 isn't guaranteed to fit into usize. This can be done with an as cast though, because as is kind of lazy and doesn't care about that (some people are opposed to using as because of this. I'm fine with as in this code). Note that with the & the default precedence of a & b as usize would be a & (b as usize), so you have to wrap the whole expression in parens, like (a & b) as usize.

And the final option I can think of is, sign is only used in one place. I don't plan on using it anywhere else, or making it public, so the body of sign could just be inlined into is_positive. As we're never declaring a conflicting return type, and the type of the 0 it's compared against is inferred, it'll work regardless of what VALUE is an alias to.

[Mat Sadler]

Hey, sorry if you've not had time to contribute. I went ahead and fixed this in 7f42447 and put out a 0.5.2 release. I went with the 'inline sign to is_positive approach.