Unit tests failing in windows
HexedAgain opened this issue · comments
The unit tests (and general usage) seem to fail on my windows device. I get the following stack trace when running the open_file test in integrations.rs
thread 'open_file' panicked at pdf\tests\integration.rs:23:13:
Try at pdf\src\file.rs:157:31:, caused by
Try at pdf\src\backend.rs:62:40:, caused by
Try at pdf\src\parser\parse_xref.rs:63:23:, caused by
Error parsing from string, caused by
invalid digit found in string
stack backtrace:
0: std::panicking::begin_panic_handler
at /rustc/2dceda4f32b97f60b122f2b32491e0267ef5cc0c/library\std\src\panicking.rs:645
1: core::panicking::panic_fmt
at /rustc/2dceda4f32b97f60b122f2b32491e0267ef5cc0c/library\core\src\panicking.rs:72
2: core::panicking::panic_display<enum2$<pdf::error::PdfError> >
at /rustc/2dceda4f32b97f60b122f2b32491e0267ef5cc0c\library\core\src\panicking.rs:196
3: integration::open_file
at .\tests\integration.rs:23
4: integration::open_file::closure$0
at .\tests\integration.rs:22
5: core::ops::function::FnOnce::call_once<integration::open_file::closure_env$0,tuple$<> >
at /rustc/2dceda4f32b97f60b122f2b32491e0267ef5cc0c\library\core\src\ops\function.rs:250
6: core::ops::function::FnOnce::call_once
at /rustc/2dceda4f32b97f60b122f2b32491e0267ef5cc0c/library\core\src\ops\function.rs:250
note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.
I was not able to reproduce this on mac os.
I did try to rummage around sticking logs here and there (even single-stepping through gdb), but my rust knowledge is still very much beginner (atm) and it seems to be dropping down in to the t! macro and failing for some reason. (if I had to guess, I'd go for it being something to do with character encoding in windows)
I pushed a fix (947466c) for this. I used naive string concatenation for paths.. that obviously doesn't work on windows.
Thanks for the response; unfortunately, pulling down this latest change the same issue still occurs. It seems to be the case that the file is actually opened (and this is true without your change here), but something is wrong when parsing it.
Investigating this further (and expanding macros). I think the problem is that a windows newline is "\r\n" whilst on linux/mac its "\n". :(
I was suspecting newline things, but I can't figure out why..
Are the files themself corrupt?
sha1sum 4acc57b5b11de74f99f9c0cd3c1fb17ab9bbe151 example.pdf
Hmm ... I'm getting sha1 as follows:
PS D:\Documents\programming\rust\third_party\pdf> Get-FileHash .\files\example.pdf -Algorithm SHA1
Algorithm Hash Path
--------- ---- ----
SHA1 6BE9550A823EBDF6A6A9FF402BBA5E9C3DFCCBD8 D:\Documents\programming\rust\third_party\pdf\files\example.pdf
Don't know if it helps, but the actual raw bytes on windows when first opening the file are:
[37, 80, 68, 70, 45, 49, 46, 55, 13, 10, 37, 181, 237, 174, 251, 13, 10, 51, 32, 48, 32, 111, 98, 106, 13, 10, 60, 60, 32, 47, 76, 101, 110, 103, 116, 104, 32, 52, 32, 48, 32, 82, 32, 62, 62, 13, 10, 115, 116, 114, 101, 97, 109, 13, 10, 47, 68, 101, 118, 105, 99, 101, 82, 71, 66, 32, 99, 115, 32, 47, 68, 101, 118, 105, 99, 101, 82, 71, 66, 32, 67, 83, 13, 10, 48, 32, 48, 32, 48, 46, 57, 55, 50, 53, 52, 57, 32, 83, 67, 13, 10, 50, 49, 46, 54, 56, 32, 49, 57, 52, 32, 49, 51, 54, 46, 54, 52, 32, 50, 54, 32, 114, 101, 13, 10, 49, 48, 32, 49, 48, 32, 109, 32, 50, 48, 32, 50, 48, 32, 108, 32, 83, 13, 10, 66, 84, 13, 10, 47, 70, 48, 32, 50, 52, 32, 84, 102, 13, 10, 50, 53, 46, 54, 56, 32, 50, 48, 48, 32, 84, 100, 13, 10, 40, 72, 101, 108, 108, 111, 32, 87, 111, 114, 108, 100, 33, 41, 32, 84, 106, 13, 10, 69, 84, 13, 10, 101, 110, 100, 115, 116, 114, 101, 97, 109, 13, 10, 101, 110, 100, 111, 98, 106, 13, 10, 52, 32, 48, 32, 111, 98, 106, 13, 10, 49, 51, 50, 13, 10, 101, 110, 100, 111, 98, 106, 13, 10, 53, 32, 48, 32, 111, 98, 106, 13, 10, 60, 60, 32, 47, 84, 121, 112, 101, 32, 47, 70, 111, 110, 116, 32, 47, 83, 117, 98, 116, 121, 112, 101, 32, 47, 84, 121, 112, 101, 49, 32, 47, 66, 97, 115, 101, 70, 111, 110, 116, 32, 47, 84, 105, 109, 101, 115, 45, 82, 111, 109, 97, 110, 32, 47, 69, 110, 99, 111, 100, 105, 110, 103, 32, 47, 87, 105, 110, 65, 110, 115, 105, 69, 110, 99, 111, 100, 105, 110, 103, 32, 62, 62, 13, 10, 101, 110, 100, 111, 98, 106, 13, 10, 54, 32, 48, 32, 111, 98, 106, 13, 10, 60, 60, 32, 47, 84, 121, 112, 101, 32, 47, 80, 97, 103, 101, 13, 10, 32, 32, 32, 47, 80, 97, 114, 101, 110, 116, 32, 50, 32, 48, 32, 82, 13, 10, 32, 32, 32, 47, 82, 101, 115, 111, 117, 114, 99, 101, 115, 32, 60, 60, 32, 47, 70, 111, 110, 116, 32, 60, 60, 32, 47, 70, 48, 32, 53, 32, 48, 32, 82, 32, 62, 62, 32, 62, 62, 13, 10, 32, 32, 32, 47, 77, 101, 100, 105, 97, 66, 111, 120, 32, 91, 32, 48, 32, 48, 32, 49, 56, 48, 32, 50, 52, 48, 32, 93, 13, 10, 32, 32, 32, 47, 67, 111, 110, 116, 101, 110, 116, 115, 32, 51, 32, 48, 32, 82, 13, 10, 62, 62, 13, 10, 101, 110, 100, 111, 98, 106, 13, 10, 50, 32, 48, 32, 111, 98, 106, 13, 10, 60, 60, 32, 47, 84, 121, 112, 101, 32, 47, 80, 97, 103, 101, 115, 13, 10, 32, 32, 32, 47, 67, 111, 117, 110, 116, 32, 49, 13, 10, 32, 32, 32, 47, 75, 105, 100, 115, 32, 91, 32, 54, 32, 48, 32, 82, 32, 93, 13, 10, 62, 62, 13, 10, 101, 110, 100, 111, 98, 106, 13, 10, 49, 32, 48, 32, 111, 98, 106, 13, 10, 60, 60, 32, 47, 84, 121, 112, 101, 32, 47, 67, 97, 116, 97, 108, 111, 103, 13, 10, 32, 32, 32, 47, 80, 97, 103, 101, 115, 32, 50, 32, 48, 32, 82, 13, 10, 62, 62, 13, 10, 101, 110, 100, 111, 98, 106, 13, 10, 120, 114, 101, 102, 13, 10, 48, 32, 55, 13, 10, 48, 48, 48, 48, 48, 48, 48, 48, 48, 48, 32, 54, 53, 53, 51, 53, 32, 102, 32, 13, 10, 48, 48, 48, 48, 48, 48, 48, 53, 50, 50, 32, 48, 48, 48, 48, 48, 32, 110, 32, 13, 10, 48, 48, 48, 48, 48, 48, 48, 52, 53, 55, 32, 48, 48, 48, 48, 48, 32, 110, 32, 13, 10, 48, 48, 48, 48, 48, 48, 48, 48, 49, 53, 32, 48, 48, 48, 48, 48, 32, 110, 32, 13, 10, 48, 48, 48, 48, 48, 48, 48, 49, 57, 57, 32, 48, 48, 48, 48, 48, 32, 110, 32, 13, 10, 48, 48, 48, 48, 48, 48, 48, 50, 49, 56, 32, 48, 48, 48, 48, 48, 32, 110, 32, 13, 10, 48, 48, 48, 48, 48, 48, 48, 51, 49, 55, 32, 48, 48, 48, 48, 48, 32, 110, 32, 13, 10, 116, 114, 97, 105, 108, 101, 114, 13, 10, 60, 60, 32, 47, 83, 105, 122, 101, 32, 55, 13, 10, 32, 32, 32, 47, 82, 111, 111, 116, 32, 49, 32, 48, 32, 82, 13, 10, 62, 62, 13, 10, 115, 116, 97, 114, 116, 120, 114, 101, 102, 13, 10, 53, 55, 52, 13, 10, 37, 37, 69, 79, 70, 13, 10]
can you hop on zulip?
hmm ... never used it before, I have something open in a browser :)
Your data is definitely not the same. Mine is 780 bytes long
Yep, from a mac I have the following:
[37, 80, 68, 70, 45, 49, 46, 55, 10, 37, 181, 237, 174, 251, 10, 51, 32, 48, 32, 111, 98, 106, 10, 60, 60, 32, 47, 76, 101, 110, 103, 116, 104, 32, 52, 32, 48, 32, 82, 32, 62, 62, 10, 115, 116, 114, 101, 97, 109, 10, 47, 68, 101, 118, 105, 99, 101, 82, 71, 66, 32, 99, 115, 32, 47, 68, 101, 118, 105, 99, 101, 82, 71, 66, 32, 67, 83, 10, 48, 32, 48, 32, 48, 46, 57, 55, 50, 53, 52, 57, 32, 83, 67, 10, 50, 49, 46, 54, 56, 32, 49, 57, 52, 32, 49, 51, 54, 46, 54, 52, 32, 50, 54, 32, 114, 101, 10, 49, 48, 32, 49, 48, 32, 109, 32, 50, 48, 32, 50, 48, 32, 108, 32, 83, 10, 66, 84, 10, 47, 70, 48, 32, 50, 52, 32, 84, 102, 10, 50, 53, 46, 54, 56, 32, 50, 48, 48, 32, 84, 100, 10, 40, 72, 101, 108, 108, 111, 32, 87, 111, 114, 108, 100, 33, 41, 32, 84, 106, 10, 69, 84, 10, 101, 110, 100, 115, 116, 114, 101, 97, 109, 10, 101, 110, 100, 111, 98, 106, 10, 52, 32, 48, 32, 111, 98, 106, 10, 49, 51, 50, 10, 101, 110, 100, 111, 98, 106, 10, 53, 32, 48, 32, 111, 98, 106, 10, 60, 60, 32, 47, 84, 121, 112, 101, 32, 47, 70, 111, 110, 116, 32, 47, 83, 117, 98, 116, 121, 112, 101, 32, 47, 84, 121, 112, 101, 49, 32, 47, 66, 97, 115, 101, 70, 111, 110, 116, 32, 47, 84, 105, 109, 101, 115, 45, 82, 111, 109, 97, 110, 32, 47, 69, 110, 99, 111, 100, 105, 110, 103, 32, 47, 87, 105, 110, 65, 110, 115, 105, 69, 110, 99, 111, 100, 105, 110, 103, 32, 62, 62, 10, 101, 110, 100, 111, 98, 106, 10, 54, 32, 48, 32, 111, 98, 106, 10, 60, 60, 32, 47, 84, 121, 112, 101, 32, 47, 80, 97, 103, 101, 10, 32, 32, 32, 47, 80, 97, 114, 101, 110, 116, 32, 50, 32, 48, 32, 82, 10, 32, 32, 32, 47, 82, 101, 115, 111, 117, 114, 99, 101, 115, 32, 60, 60, 32, 47, 70, 111, 110, 116, 32, 60, 60, 32, 47, 70, 48, 32, 53, 32, 48, 32, 82, 32, 62, 62, 32, 62, 62, 10, 32, 32, 32, 47, 77, 101, 100, 105, 97, 66, 111, 120, 32, 91, 32, 48, 32, 48, 32, 49, 56, 48, 32, 50, 52, 48, 32, 93, 10, 32, 32, 32, 47, 67, 111, 110, 116, 101, 110, 116, 115, 32, 51, 32, 48, 32, 82, 10, 62, 62, 10, 101, 110, 100, 111, 98, 106, 10, 50, 32, 48, 32, 111, 98, 106, 10, 60, 60, 32, 47, 84, 121, 112, 101, 32, 47, 80, 97, 103, 101, 115, 10, 32, 32, 32, 47, 67, 111, 117, 110, 116, 32, 49, 10, 32, 32, 32, 47, 75, 105, 100, 115, 32, 91, 32, 54, 32, 48, 32, 82, 32, 93, 10, 62, 62, 10, 101, 110, 100, 111, 98, 106, 10, 49, 32, 48, 32, 111, 98, 106, 10, 60, 60, 32, 47, 84, 121, 112, 101, 32, 47, 67, 97, 116, 97, 108, 111, 103, 10, 32, 32, 32, 47, 80, 97, 103, 101, 115, 32, 50, 32, 48, 32, 82, 10, 62, 62, 10, 101, 110, 100, 111, 98, 106, 10, 120, 114, 101, 102, 10, 48, 32, 55, 10, 48, 48, 48, 48, 48, 48, 48, 48, 48, 48, 32, 54, 53, 53, 51, 53, 32, 102, 32, 10, 48, 48, 48, 48, 48, 48, 48, 53, 50, 50, 32, 48, 48, 48, 48, 48, 32, 110, 32, 10, 48, 48, 48, 48, 48, 48, 48, 52, 53, 55, 32, 48, 48, 48, 48, 48, 32, 110, 32, 10, 48, 48, 48, 48, 48, 48, 48, 48, 49, 53, 32, 48, 48, 48, 48, 48, 32, 110, 32, 10, 48, 48, 48, 48, 48, 48, 48, 49, 57, 57, 32, 48, 48, 48, 48, 48, 32, 110, 32, 10, 48, 48, 48, 48, 48, 48, 48, 50, 49, 56, 32, 48, 48, 48, 48, 48, 32, 110, 32, 10, 48, 48, 48, 48, 48, 48, 48, 51, 49, 55, 32, 48, 48, 48, 48, 48, 32, 110, 32, 10, 116, 114, 97, 105, 108, 101, 114, 10, 60, 60, 32, 47, 83, 105, 122, 101, 32, 55, 10, 32, 32, 32, 47, 82, 111, 111, 116, 32, 49, 32, 48, 32, 82, 10, 62, 62, 10, 115, 116, 97, 114, 116, 120, 114, 101, 102, 10, 53, 55, 52, 10, 37, 37, 69, 79, 70, 10]
no 13 chars following the 10 like the windows one
I added a .gitattributes, pull and see if that fixed it.
hmm ... windows here still putting up a fight, but I will have a play around. Think it's clear what the root cause is on this! (manually removing the carriage returns in vim then retesting "fixes" the issue). Not sure why windows/git is introducing those characters in the pdfs
Ok, adding the following line in ~/.gitattributes
*.sln text eol=lf`
ensures that git won't add in the carriage return, all tests are passing.
Thanks for your help in investigating this