on linux x86-64 the binary is very large / can be 4.66x smaller with strip/upx --lzma
alexmyczko opened this issue · comments
although strip and compress cut the size, the binary does not work anymore. any idea why? any chance to make it smaller?
the binary is, as i understand it, the deno binary with our code as a .DATA block. we know that patchelf breaks that loading, so i'm not surprised strip does as well.
if you can reduce the size of deno without breaking it, then compile pkgx, that's probably the way forward.
basically it will be large until deno can make smaller binaries or we switch to rust. Sorry.
So I've gotten the deno binary for x86-64
original size 139310360 # 139 MB
after strip and upx
new size 39192472 # 39 MB
Now I'm having a hard time to build deno from source, and even a harder time to build pkgx from source.
Would it be possible to use this binary to build a pkgx binary for x86-64 linux to see/test if it would actually work?
https://sid.ethz.ch/debian/deno/bin
It’s unlikely that stripping deno would result in a smaller pkgx binary.
-rwxr-xr-x 1 root root 139310360 Jan 25 17:06 deno
-rwxr-xr-x 1 root root 101139872 Jan 26 13:18 deno.strip
-rwxr-xr-x 1 root root 39192472 Jan 26 08:32 deno.strip.upx
-rwxr-xr-x 1 root root 29051348 Jan 26 13:18 deno.strip.upx--lzma
and
deno: ELF 64-bit LSB pie executable, x86-64, version 1 (SYSV), dynamically linked, interpreter /lib64/ld-linux-x86-64.so.2, for GNU/Linux 2.6.32, BuildID[xxHash]=4fec45b0112d0342, with debug_info, not stripped
deno.strip: ELF 64-bit LSB pie executable, x86-64, version 1 (SYSV), dynamically linked, interpreter /lib64/ld-linux-x86-64.so.2, for GNU/Linux 2.6.32, BuildID[xxHash]=4fec45b0112d0342, stripped
deno.strip.upx: ELF 64-bit LSB pie executable, x86-64, version 1 (SYSV), statically linked, no section header
using upx with --lzma it gets even below 30 mb!!! (factor 4.66 smaller)
like, I'm just saying that the size of deno is unlikely to effect the size of the binaries it produces. But who knows, maybe.
Anyway, building pkgx is trivial. We have instructions in the readme, but basically it's deno task compile in a checkout.
@mxcl my hero! it works, well... spits some warnings but it works:
https://sid.ethz.ch/debian/pkgx/build/pkgx/ the binary is 42 MB
and running it with
pkgx bc
gives
warning: Use of deprecated "Deno.stderr.rid" API. This API will be removed in Deno 2.
Stack trace:
at file:///var/www/debian/pkgx/build/pkgx/entrypoint.ts:7:38
hint: Use `Deno.stderr` instance methods instead.
warning: Use of deprecated "Deno.isatty()" API. This API will be removed in Deno 2.
Stack trace:
at default (file:///var/www/debian/pkgx/build/pkgx/src/utils/clicolor.ts:10:36)
at file:///var/www/debian/pkgx/build/pkgx/entrypoint.ts:7:17
hint: Use `Deno.stdin.isTerminal()`, `Deno.stdout.isTerminal()` or `Deno.stderr.isTerminal()` instead.
warning: Use of deprecated "Deno.stdin.rid" API. This API will be removed in Deno 2.
Stack trace:
at logger_prefix (file:///var/www/debian/pkgx/build/pkgx/entrypoint.ts:66:53)
at file:///var/www/debian/pkgx/build/pkgx/entrypoint.ts:59:19
hint: Use `Deno.stdin` instance methods instead.
warning: Use of deprecated "Deno.isatty()" API. This API will be removed in Deno 2.
Stack trace:
at logger_prefix (file:///var/www/debian/pkgx/build/pkgx/entrypoint.ts:66:35)
at file:///var/www/debian/pkgx/build/pkgx/entrypoint.ts:59:19
hint: Use `Deno.stdin.isTerminal()`, `Deno.stdout.isTerminal()` or `Deno.stderr.isTerminal()` instead.
warning: Use of deprecated "Deno.stderr.rid" API. This API will be removed in Deno 2.
Stack trace:
at make_logger (file:///var/www/debian/pkgx/build/pkgx/src/app.ts:148:51)
at default (file:///var/www/debian/pkgx/build/pkgx/src/app.ts:33:18)
at file:///var/www/debian/pkgx/build/pkgx/entrypoint.ts:59:9
hint: Use `Deno.stderr` instance methods instead.
warning: Use of deprecated "Deno.isatty()" API. This API will be removed in Deno 2.
Stack trace:
at make_logger (file:///var/www/debian/pkgx/build/pkgx/src/app.ts:148:32)
at default (file:///var/www/debian/pkgx/build/pkgx/src/app.ts:33:18)
at file:///var/www/debian/pkgx/build/pkgx/entrypoint.ts:59:9
hint: Use `Deno.stdin.isTerminal()`, `Deno.stdout.isTerminal()` or `Deno.stderr.isTerminal()` instead.
bc 1.07.1
Copyright 1991-1994, 1997, 1998, 2000, 2004, 2006, 2008, 2012-2017 Free Software Foundation, Inc.
This is free software with ABSOLUTELY NO WARRANTY.
For details type `warranty'.
k well, this is interesting
the more so since UPX is portable and you could also strip/upx --lzma your other binaries!
but instead opening a new issue. are you interested in distributing pkgx in debian, like in the archive? i can package software for it, however deno is not there yet...
Interested in how it effects startup time.
Also we clearly need to pkg upx
And would love to get into Debian, but yeah, since they don't have deno yet it seems tricky
memory/cpu > disk (no matter rotational, ssd, nvme). i don't notice anything on my 2011 machine.
for deno in debian there's https://bugs.debian.org/961337
found some further info here: denoland/deno#9198
So indeed, stripping deno will lead to smaller compiled binaries. I worry that it may result in non functional binaries. Defaulting it is a risk for just-works but maybe that's the wrong trade off and we should aim for performance
aside: stack traces for deprecated calls seems extreme.
i tried the same strip/upx/upx --lzma on macOS. no luck
is there a place to see all pkgx available binaries? and a way to search for it???
did you mean pkgx? if so, you can review the packages here: https://pkgx.dev/pkgs/
most of the binaries are lists in the provides: keys in the various package.yml files. as always, the pkgxdev/pantry repository is canon.
the search box will search for programs (binaries) that pkgs provide
BTW #978 will make pkgx binaries a little smaller.