Investigate dotnet-symbol binary mismatch

Question

Investigate dotnet-symbol binary mismatch

sdmaclea opened this issue 5 years ago · comments

After installing Linux x64 3.0.100 release, I tried to download the symbols for the runtime.

$ dotnet --list-runtimes
Microsoft.AspNetCore.All 2.1.13 [/usr/share/dotnet/shared/Microsoft.AspNetCore.All]
Microsoft.AspNetCore.All 2.2.7 [/usr/share/dotnet/shared/Microsoft.AspNetCore.All]
Microsoft.AspNetCore.App 2.1.13 [/usr/share/dotnet/shared/Microsoft.AspNetCore.App]
Microsoft.AspNetCore.App 2.2.7 [/usr/share/dotnet/shared/Microsoft.AspNetCore.App]
Microsoft.AspNetCore.App 3.0.0 [/usr/share/dotnet/shared/Microsoft.AspNetCore.App]
Microsoft.NETCore.App 2.1.13 [/usr/share/dotnet/shared/Microsoft.NETCore.App]
Microsoft.NETCore.App 2.2.7 [/usr/share/dotnet/shared/Microsoft.NETCore.App]
Microsoft.NETCore.App 3.0.0 [/usr/share/dotnet/shared/Microsoft.NETCore.App]

$ dotnet-symbol --recurse-subdirectories /usr/share/dotnet/shared/Microsoft.NETCore.App/3.0.0/libcoreclr.so -o 3.0.0-symbols
Downloading from http://msdl.microsoft.com/download/symbols/
Writing files to 3.0.0-symbols
Writing: 3.0.0-symbols/libcoreclr.so
Writing: 3.0.0-symbols/libcoreclr.so.dbg
Writing: 3.0.0-symbols/libmscordaccore.so
Writing: 3.0.0-symbols/libmscordbi.so
ERROR: HttpSymbolStore: 404 Not Found 'http://msdl.microsoft.com/download/symbols/libsos.so/elf-buildid-coreclr-b4c030672d8bb8cffdae6ebfac0717763f6677f8/libsos.so'
ERROR: HttpSymbolStore: 404 Not Found 'http://msdl.microsoft.com/download/symbols/sos.netcore.dll/elf-buildid-coreclr-b4c030672d8bb8cffdae6ebfac0717763f6677f8/sos.netcore.dll'

$ md5sum 3.0.0-symbols/libcoreclr.so* /usr/share/dotnet/shared/Microsoft.NETCore.App/3.0.0/libcoreclr.so
b47b275c67a0669c16bdc4f877ef999a  3.0.0-symbols/libcoreclr.so
f3476f8f604902fc320ee7e43ea55529  3.0.0-symbols/libcoreclr.so.dbg
b31e674bd71a6a552c7effd2d7ba3838  /usr/share/dotnet/shared/Microsoft.NETCore.App/3.0.0/libcoreclr.so

As shown the versions do not match.

The downloaded 3.0.0-symbols/libcoreclr.so is missing debug info, but does have a symtab

Mike McLaughlin · Answer 1 · Wed Oct 16 2019 08:12:57 GMT+0800 (China Standard Time)

@dagood @JohnTortugo do either of you understand why the libcoreclr.so included in the runtime install is slightly different from the libcoreclr.so we publish (and then download)? Does our release pipeline do something differently between how the symbol packages are created and the actual product ones?

In the end, the downloaded libcoreclr.so (and libcoreclr.so.dbg) work just fine in lldb (at least for me) so I'm not sure if there is anything broken by this. @sdmaclea is that true?

Steve MacLean · Answer 2 · Wed Oct 16 2019 23:04:52 GMT+0800 (China Standard Time)

I'm not sure if there is anything broken by this. @sdmaclea is that true?

It was confusing, but I don't know that anything is impacted.

Davis Goodin · Answer 3 · Wed Oct 16 2019 23:11:41 GMT+0800 (China Standard Time)

Can you attach the files to make sure I look at the same stuff?

I'm confused this is possible, because ELF files should be identified by their build ID, which I expect to be different for every single build because it's based on hashing parts of the file (so I read). If the symbol downloader is able to find symbols for a libcoreclr.so, I'd expect the build id to match, and for that to mean it's definitely the same exact build.

I think the linker option we're using is this: http://sourceware.org/binutils/docs-2.23.1/ld/Options.html#Options --build-id (--build-id=sha for Core-Setup and CoreCLR).

Davis Goodin · Answer 4 · Wed Oct 16 2019 23:14:01 GMT+0800 (China Standard Time)

It would also be good to know how you acquired .NET Core for that machine in case there's something weird going on for a specific channel.

Davis Goodin · Answer 5 · Wed Oct 16 2019 23:42:00 GMT+0800 (China Standard Time)

Ah... forgot about this. The Debian package has it stripped while the tarball and the symbol package have it unstripped. (Edit: Rather, "not all the way stripped". file reports unstripped but some stripping has been done.)

Since stripping happens after the build, this doesn't affect build ID:

#> for x in *; do du $x ; sha256sum $x | cut '-d ' -f1 ; file $x | cut -d, -f5-6; echo; done
7332    libcoreclr_debpkg.so
43a52aed991d928db33d3c2a22ba9d8b10d154cac07a1ee88c8c55864c54cf58
 BuildID[sha1]=b4c030672d8bb8cffdae6ebfac0717763f6677f8, stripped

8960    libcoreclr_sympkg.so
00dc11b4ab45831ef1648e3d183918df77ef28b234e907c921d5dbb0a821e8ca
 BuildID[sha1]=b4c030672d8bb8cffdae6ebfac0717763f6677f8, not stripped

8960    libcoreclr_zip.so
00dc11b4ab45831ef1648e3d183918df77ef28b234e907c921d5dbb0a821e8ca
 BuildID[sha1]=b4c030672d8bb8cffdae6ebfac0717763f6677f8, not stripped

CoreCLR isn't stripping it "all the way", and the Deb package tool is doing the rest at packaging time. Stripping is entirely handled by the Deb/RPM packaging process for traditional Linux builds, which would explain why it does this by default.

https://github.com/dotnet/core-setup/issues/1607 tracks normalizing this.
(https://github.com/dotnet/core-setup/issues/8356 tracks embracing debuginfo/dbgpkg to fit in with the normal Linux ecosystem.)

This means I don't think we'll have any issues with mismatching bits here, since they're the same build for sure and all that's different is how far they're stripped. 😄