Closure size for ruby is 10x bigger than in nixpkgs
andrewhamon opened this issue · comments
I noticed that a ruby built with nixpkgs-ruby has a significantly larger closure size compared to ruby from nixpkgs, roughly 10x. That leads to a lot of wasted bandwidth.
Here is the sorted, recursive closure size from nixpkgs:
nix path-info --size --closure-size --human-readable --recursive "$(nix build --print-out-paths nixpkgs#ruby_3_1)" | awk '{print $NF,$0}' | sort -h | cut -f2- -d' '
/nix/store/p9hgfznfn44daynmczv1l5vcpzzi5chj-libffi-3.4.4 119.6K 119.6K
/nix/store/gsj9m0sl6rr3xriw1844yn4y79yc89w3-libffi-3.4.4-dev 18.6K 138.2K
/nix/store/ksf888bg7q27bdl7h55wigbgvlll6v6b-libyaml-0.2.5 138.9K 138.9K
/nix/store/m29ibzvjrwcxl29774gbms2haw9q3jdw-libyaml-0.2.5-dev 54.9K 193.8K
/nix/store/d438n7jifwn8z6pk1zslfii1pp8b4k5p-libobjc-11.0.0 262.4K 262.4K
/nix/store/y08crgb0j7bniwwpcsfwn0dikpcim948-zlib-1.3 366.4K 366.4K
/nix/store/z3vip09xdxcfp3dgvwr1fccs2irqg6da-zlib-1.3-dev 112.7K 479.0K
/nix/store/1gplr6pcfgpqp37ia161z4ndw9cd50cv-ncurses-6.4-man 597.8K 597.8K
/nix/store/vycw971nx8byymairwkql4nlqxb7jz03-gdbm-1.23 660.6K 660.6K
/nix/store/iglxg6ay8ryli8r009ckgkggfgrpvycs-libiconv-50 2.1M 2.1M
/nix/store/hm8sxwvz7qbf5kzgprdlggj231l5g2s4-bash-5.2-p15 2.6M 2.6M
/nix/store/wai9z91s1yg0zdkq93yds01906nxanyk-ncurses-6.4 3.6M 3.6M
/nix/store/rba3cfa3aybqwmm044315rqjmddz5szx-readline-8.2p1 378.3K 4.0M
/nix/store/gkxrqf9ahq8czv6q6xdifg0jzprmppz6-openssl-3.0.10 4.3M 4.3M
/nix/store/40bkxvyrp997k905v7in8zal3266zszh-ncurses-6.4-dev 417.0K 7.2M
/nix/store/an422j8j98gn56p62677l6ypc49s825r-readline-8.2p1-dev 69.5K 7.6M
/nix/store/vfhqzf1nq4cadzkcz2yfip67zl69x5h1-openssl-3.0.10-bin 812.3K 7.7M
/nix/store/77barq0w09bh4pkr6n2rs1d0xalh29yl-openssl-3.0.10-dev 1.9M 9.6M
/nix/store/13070az7ny152x6lnbvjlpz1ds188fqx-libSystem-11.0.0 10.8M 10.8M
/nix/store/a885iclj5znshdp9k2zbg7h8xqxg5hn5-ruby-3.1.4 29.8M 59.0M
and here is the same for a ruby I built with nixpkgs-ruby:
nix path-info --size --closure-size --human-readable --recursive "$(nix build --print-out-paths nixpkgs#ruby_3_1)" | awk '{print $NF,$0}' | sort -h | cut -f2- -d' '
/nix/store/f2mr648ckhv6wj31cghq9719kkaxz7zj-expand-response-params 51.2K 51.2K
/nix/store/h3drppybb4792hmigwqvqfziyif4ysjg-libffi-3.4.4 119.6K 119.6K
/nix/store/ah00sz2j896qcxj88qrw9dnb2pa1z6bh-libffi-3.4.4-dev 18.6K 138.2K
/nix/store/zd20wzldz49gwij6nlikx890spliq8nk-libobjc-11.0.0 262.4K 262.4K
/nix/store/66g1x7shb0f910nz5kkah8vl7q40ashb-compiler-rt-libc-11.1.0 326.6K 326.6K
/nix/store/ycc0w7ai9kp7q932mx6fhqmx3il07wnk-compiler-rt-libc-11.1.0-dev 30.6K 357.2K
/nix/store/40m6vc2mv932p6ysjicfzm6mbfpqgwhn-pcre-8.45 367.0K 367.0K
/nix/store/zqyhb058al6lb182n1lyclvwp11k7kjj-zlib-1.2.13 414.8K 414.8K
/nix/store/2yvhjy73swqi35i6c95ra5p8p70n081p-zlib-1.2.13-dev 113.2K 528.0K
/nix/store/cz252na1jzk8d8f0r0x8g9lpsxfqj25y-ncurses-6.4-man 597.7K 597.7K
/nix/store/ldjmd9ry137lb5f4s7lklp6scij0j9b5-libcxxabi-11.1.0 632.5K 632.5K
/nix/store/jwjrd2k1l53lj947i38q858dc3vrcdlp-libcxxabi-11.1.0-dev 10.6K 643.1K
/nix/store/nhq1zhs10j2ap2f3cjliqcag1x6jdm6h-apple-framework-CoreFoundation-11.0.0 679.5K 941.9K
/nix/store/yqwxqfdsqhm9115r8v4c76gfbbd1z6p6-libyaml-0.2.5 193.1K 1.1M
/nix/store/54vn6p4rjydalab36qs76hm6q4j0g2q8-gdbm-1.23 660.6K 1.6M
/nix/store/8qnyan4ym6p0x7hbv7r4z1sf2nnid007-libiconv-50 2.2M 2.2M
/nix/store/6rzl0qzi13ydp8caf9zsg1fw64wfxcsr-libcxx-11.1.0 1.9M 2.5M
/nix/store/s5srxq0cjf87yifx36qwidw6gdim7qdc-gmp-with-cxx-6.2.1 570.6K 3.1M
/nix/store/nzzzgjcc8568iyyja0vvv840hnka6l37-bash-5.2-p15 3.2M 3.2M
/nix/store/xrafw3rm7da99009icrdpcpvcsad0p22-ncurses-6.4 3.7M 3.7M
/nix/store/7gmsbmlzxhl2r7lnr34hwpk0an3f2bhb-libxml2-2.10.1 1.3M 3.8M
/nix/store/azs9zxf936f1y6akiyj775db9v2kw7r9-openssl-3.0.9 4.3M 4.3M
/nix/store/8f32i5p9rng23vcrpdfvq3mqmwpi106n-coreutils-9.1 1.3M 4.4M
/nix/store/84igwxvvy033xy1bzri1xx2fhjlppl3j-readline-8.2p1 378.3K 5.0M
/nix/store/n46b1b7h0nr18ynsirg9n00jy3zgs6jq-openssl-3.0.9 4.3M 5.2M
/nix/store/nd90drzi8lc6fnsb7pzy3b671778z4n4-gnugrep-3.7 299.7K 6.0M
/nix/store/g6bbdiax55bd89ygcngxprcnbfjbci0f-libcxx-11.1.0-dev 4.6M 7.1M
/nix/store/rrgq1n557m8ik8v226q1b2gxgvxsczyf-sigtool-0.1.3 839.2K 7.6M
/nix/store/y92rz06kjkb3fyc20am4w8kfb62r7k7f-post-link-sign-hook 336.0 7.6M
/nix/store/zami8ybb6wsk59knav39bv2zcfz3c2aj-ncurses-6.4-dev 432.4K 7.9M
/nix/store/yjkyc0fnh2ya3a1d7xkknb269iy65pjd-openssl-3.0.9-bin 812.3K 9.2M
/nix/store/71x9zirhlmf3x7x5755gpsic2xz4l9hj-readline-8.2p1-dev 69.5K 9.3M
/nix/store/hg3vg4b2r1zgfcilx9l3q0gwi5cpcl3y-libSystem-11.0.0 10.8M 10.8M
/nix/store/v84ramgj4jcvqpbpdj4g734id23n0v4q-openssl-3.0.9-dev 1.9M 11.1M
/nix/store/szdhphwhpx4lgcx0qbfa5lvhprml6sql-gettext-0.21 8.9M 16.8M
/nix/store/id8rkb6l76f5nr56wdixkczcf4mmql06-binutils-2.40-lib 2.9M 20.1M
/nix/store/p5sf021g9a1g085x1y49yw1k0380aly7-binutils-2.40 9.6M 29.7M
/nix/store/p657y8gl0cyizf7xls2zdghr39gn2ir2-libtapi-1100.0.11 24.7M 30.9M
/nix/store/rj9c61ifmsxd0032f6j8gz8djp40w904-cctools-port-973.0.1 10.6M 41.6M
/nix/store/ni7m17vjvi9wxi0y8xq62bcahkibxrwm-signing-utils 1.5K 46.7M
/nix/store/0ic7lxy45bpbwp7bdphxz9m7kif0dxny-llvm-11.1.0-lib 194.4M 204.6M
/nix/store/d0ha59yka4adfpkr3lkqx65npq5lw7al-llvm-11.1.0 38.2M 242.8M
/nix/store/ishlkzp350ffgnhc8n1993gqdi6nm10v-clang-11.1.0-lib 168.8M 373.4M
/nix/store/avvd92dfzssp3haj0ahmrpvq0in29zdm-clang-11.1.0 23.1M 396.6M
/nix/store/wdxv5yy9aaimhs66x3g5dhmc6j736bhf-cctools-binutils-darwin-973.0.1 4.7K 494.7M
/nix/store/d3rvi35by3676nb1zxnf1sy7swy2xlqx-cctools-binutils-darwin-wrapper-973.0.1 35.5K 512.6M
/nix/store/85cdiqs7d2shwbkf6lm978jpv06vd384-clang-wrapper-11.1.0 58.7K 518.2M
/nix/store/40g2xzhlf07ncdjyh3vigk470lyrzar4-ruby-3.1.4 31.2M 559.8M
I think the reason nixpkgs ruby is much smaller is because they use removeReferencesTo to remove stdenv.cc.
.... except they don't do this if JIT support is enabled, so ruby 3.2+ are comparable in size both in nixpkgs as well as in nixpkgs-ruby.
Thanks a lot for the issue and the hint how to resolve this. I've created #94 with some of the cleanup code adopted from nixpkgs. Using nix-tree it showed that 3.2.2 did not refer to stdenv.cc anymore. It's now building the other versions, which will take a while. Your review would be helpful 👍