cmd/compile: rc1.14rc1 build internal/ssa out of memory
n2vi opened this issue · comments
What version of Go are you using (go version)?
go version go1.13.7 openbsd/amd64
Does this issue reproduce with the latest release?
yes
What operating system and processor architecture are you using (go env)?
See attached file.
What did you do?
git pull git checkout go1.14rc1 cd src ./all.bash
What did you expect to see?
compile and test results as usual
What did you see instead?
Building go toolchain3 using go_bootstrap and Go toolchain2 # cmd/compile/internal/ssa fatal error: runtime: out of memory
There was actually plenty of available RAM at the time, according to top.
There was actually plenty of available RAM at the time, according to top.
What does "plenty" mean here? I don't know how quickly top refreshes, but building package ssa eats ram (and potentially dies) very quickly.
I suspect this is probably a duplicate of #27739. I have a series of CLs ready for 1.15 that will shrink package ssa by about 25%.
Yeah, 12gb really ought to be enough. :) Does it reproduce consistently?
cc @mknyszek @aclements just in case
Yes, reproduces 100%. Happy to re-run with any additional debugging you can teach me.
In any event, if a normal Go install requires tons of memory, it is probably worth measuring this at release candidate build time and adding a comment to the install doc, or even better finding a way to determine at runtime that the process is likely to fail for a known reason.
Also, just reproduced it on another OpenBSD machine running stable+syspatch, so this is probably not some recent OpenBSD issue.
I'm happy to give Austin a login if constructing such a machine would be burdensome to y'all. I know I'm running on a niche OS and aspire to the skill to give you a better bug report.
To rule out some things, let's try disabling parallel and concurrent compilation:
GOFLAGS="-p=1 -gcflags=cmd/compile/internal/ssa=-c=1" ./all.bash
If that still fails, let's see how much RAM the shell thinks it is using:
GOFLAGS=-toolexec="/usr/bin/time -l" ./all.bash
OpenBSD by default has a very low RLIMIT_DATA. What's the output of ulimit?
Even if RLIMIT_DATA is raised, OpenBSD limits anonymous mappings to a fairly small factor larger than physical memory (even if they're untouched), and Go 1.14 does create some larger sparse mappings than Go 1.13 did. For this limit, the relevant top column to watch would be VSZ, I believe.
What's the output of ulimit?
Sorry, ulimit -a.
"ulimit -a" on this 16GB physical NUQkit reports stack 8MB, lockedmem 5GB, mem 16GB.
GOFLAGS="-p=1 -gcflags=cmd/compile/internal/ssa=-c=1" ./all.bash
still fails in the same way.
Maybe I'm blind, but
GOFLAGS=-toolexec="/usr/bin/time -l" ./all.bash
seems to do the same with no extra output. Sorry, I don't know your build system enough to do better yet.
Sounds like Austin probably has a better bead on what’s going on here.
I’m curious though whether my planned changes will help sufficiently. To test,
git fetch "https://go.googlesource.com/go" refs/changes/03/213703/6 && git checkout FETCH_HEAD
and then try again.
Hurray and thank you! With your change, builds and tests successfully.
That’s good to hear. That won’t land until the 1.15 release, though.