CRC Computation Question

[Chris Holcombe]

I'm having trouble getting the crc32 computation to match what I'm seeing in Ceph.
I have an example here where at the top is a C++ unit test and on the bottom is the Rust code I'm using to try and reproduce it. It's close but doesn't match.

https://play.rust-lang.org/?gist=6bca7e3630d48f9225e5&version=stable

[Chris Holcombe]

Ceph notes in their source that they use this algorithm:
http://create.stephan-brumme.com/crc32/#slicing-by-8-overview

[Rui Hu]

@cholcombe973 0 is used as polynomial rather than initial in rust code while document of ceph crc32 states first argument is initial

/**
 * calculate crc32c
 *
 * Note: if the data pointer is NULL, we calculate a crc value as if
 * it were zero-filled.
 *
 * @param crc initial value
 * @param data pointer to data buffer
 * @param length length of buffer
 */
static inline uint32_t ceph_crc32c(uint32_t crc, unsigned char const *data, unsigned length)
{
    return ceph_crc32c_func(crc, data, length);
}

#endif

you should use correct polynomial and initial to match ceph result unless ceph use IEEE, Castagnoli or Koopman standard which is supported by this library by default

[Chris Holcombe]

Ok so I changed my code to this now:

extern crate crc;
use crc::{crc32, Hasher32};

fn main(){
    let foo = String::from("foo bar baz");
    let foo_bytes = foo.into_bytes();
    println!("foo bytes: {:?}", &foo_bytes);
    let mut digest = crc32::Digest::new_with_initial(crc32::CASTAGNOLI, 0u32);
    digest.write(&foo_bytes);
    println!("foo bar baz: {}", digest.sum32());
}

That still produces:

foo bar baz: 1599983188

which doesn't match up with Ceph's:
ASSERT_EQ(4119623852u, ceph_crc32c(0, (unsigned char *)a, strlen(a)));

I tried this in Python also and got the same answer.

[Rui Hu]

pls reopen if it's still an issue