Chrome Headless doesn't launch on Debian
fortes opened this issue · comments
Running this example code from the README:
const puppeteer = require('puppeteer');
(async() => {
const browser = await puppeteer.launch();
const page = await browser.newPage();
await page.goto('https://example.com');
await page.screenshot({path: 'example.png'});
browser.close();
})();I get the following error output:
(node:30559) UnhandledPromiseRejectionWarning: Unhandled promise rejection (rejection id: 1): Error: Failed to connect to chrome!
(node:30559) [DEP0018] DeprecationWarning: Unhandled promise rejections are deprecated. In the future, promise rejections that are not handled will terminate the Node.js process with a non-zero exit code.
Platform info:
% uname -a
Linux localhost 3.14.0 #1 SMP PREEMPT Thu Jul 13 12:08:15 PDT 2017 x86_64 GNU/Linux
% lsb_release -a
Distributor ID: Debian
Description: Debian GNU/Linux 9.0 (stretch)
Release: 9.0
Codename: stretch
% node --version
v8.1.1
% cat package.json
{
"dependencies": {
"puppeteer": "^0.9.0"
}
}
This is working fine here on Elementary OS. I'm downloading a Debian installer now to try this out in a VM.
That's interesting. Could you please:
- try launching chromium manually (chromium is downloaded at
node_modules/puppeteer/.local-chromium) - if chromium launches for you, run the following (notice the added
dumpioflag to thepuppeteer.launch) and check what's in the stderr:
const puppeteer = require('puppeteer');
(async() => {
const browser = await puppeteer.launch({dumpio: true});
const page = await browser.newPage();
await page.goto('https://example.com');
await page.screenshot({path: 'example.png'});
browser.close();
})();Can't launch the local chrome:
% ~/p /home/fortes/p/node_modules/puppeteer/.local-chromium/linux-494755/chrome-linux/chrome --help
/home/fortes/p/node_modules/puppeteer/.local-chromium/linux-494755/chrome-linux/chrome: error while loading shared libraries: libX11-xcb.so.1: cannot open shared object file: No such file or directory
I should have mentioned that this is a headless machine that I'm ssh'd into. Given that this is for headless Chrome, I assume that scenario is still supported?
I think in the case of Debian systems you still need https://packages.debian.org/sid/libx11-xcb1 to run headless. That way the system has some of the API calls it needs to to do the rendering calculations.
The action to resolve this (which I'm working on now) is getting a list of all the required dependencies to run Chromium. Straight from the Debian the requirements are below. Documenting this for a PR shortly.
Dependencies for debian
gconf-service
libasound2
libatk1.0-0
libc6
libcairo2
libcups2
libdbus-1-3
libexpat1
libfontconfig1
libgcc1
libgconf-2-4
libgdk-pixbuf2.0-0
libglib2.0-0
libgtk-3-0
libnspr4
libpango-1.0-0
libpangocairo-1.0-0
libstdc++6
libx11-6
libx11-xcb1
libxcb1
libxcomposite1
libxcursor1
libxdamage1
libxext6
libxfixes3
libxi6
libxrandr2
libxrender1
libxss1
libxtst6
ca-certificates
fonts-liberation
libappindicator1
libnss3
lsb-release
xdg-utils
wget
Already have that installed, perhaps a different package is needed?
% ~/p sudo apt-get install libx11-xcb1
Reading package lists... Done
Building dependency tree
Reading state information... Done
libx11-xcb1 is already the newest version (2:1.6.4-3).
0 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.
@fortes so do dependencies from #290 (comment) help?
I've installed those and can now run chrome --help. However, if I try to run chrome -v, I get the following:
febian:~/p /home/fortes/p/node_modules/puppeteer/.local-chromium/linux-494755/chrome-linux/chrome -v
[11104:11104:0816/105455.434188:FATAL:zygote_host_impl_linux.cc(123)] No usable sandbox! Update your kernel or see https://chromium.googlesource.com/chromium/src/+/master/docs/linux_suid_sandbox_development.md for more information on developing with the SUID sandbox. If you want to live dangerously and need an immediate workaround, you can try using --no-sandbox.
#0 0x556f97804657 base::debug::StackTrace::StackTrace()
#1 0x556f97818311 logging::LogMessage::~LogMessage()
#2 0x556f96a091f1 content::ZygoteHostImpl::Init()
#3 0x556f966a3da0 content::BrowserMainLoop::EarlyInitialization()
#4 0x556f966aa4c3 content::BrowserMainRunnerImpl::Initialize()
#5 0x556f966a3532 content::BrowserMain()
#6 0x556f9750f7fd content::ContentMainRunnerImpl::Run()
#7 0x556f97517314 service_manager::Main()
#8 0x556f9750e462 content::ContentMain()
#9 0x556f9614eb74 ChromeMain
#10 0x7fa1f27c92b1 __libc_start_main
#11 0x556f9614e9d0 <unknown>
Received signal 6
#0 0x556f97804657 base::debug::StackTrace::StackTrace()
#1 0x556f978041cf base::debug::(anonymous namespace)::StackDumpSignalHandler()
#2 0x7fa1f8b690c0 <unknown>
#3 0x7fa1f27dbfcf gsignal
#4 0x7fa1f27dd3fa abort
#5 0x556f97803202 base::debug::BreakDebugger()
#6 0x556f978187cc logging::LogMessage::~LogMessage()
#7 0x556f96a091f1 content::ZygoteHostImpl::Init()
#8 0x556f966a3da0 content::BrowserMainLoop::EarlyInitialization()
#9 0x556f966aa4c3 content::BrowserMainRunnerImpl::Initialize()
#10 0x556f966a3532 content::BrowserMain()
#11 0x556f9750f7fd content::ContentMainRunnerImpl::Run()
#12 0x556f97517314 service_manager::Main()
#13 0x556f9750e462 content::ContentMain()
#14 0x556f9614eb74 ChromeMain
#15 0x7fa1f27c92b1 __libc_start_main
#16 0x556f9614e9d0 <unknown>
r8: 0000000000000000 r9: 00007fff8e2bda50 r10: 0000000000000008 r11: 0000000000000246
r12: 00007fff8e2be160 r13: 000000000000016d r14: 00007fff8e2be158 r15: 00007fff8e2be150
di: 0000000000000002 si: 00007fff8e2bda50 bp: 00007fff8e2bdd00 bx: 0000000000000006
dx: 0000000000000000 ax: 0000000000000000 cx: 00007fa1f27dbfcf sp: 00007fff8e2bdac8
ip: 00007fa1f27dbfcf efl: 0000000000000246 cgf: 002b000000000033 erf: 0000000000000000
trp: 0000000000000000 msk: 0000000000000000 cr2: 0000000000000000
[end of stack trace]
Calling _exit(1). Core file will not be generated.
Check the sandbox docs linked to in that error and see if they can help you get it working. Either the security sandbox is messed up on Debian right now or something funky is happening to trigger it to need a non-kernel one. I'm looking into this once this VM server gets installed.
This should be solved with the '--no-sandbox' flag:
const puppeteer = require('puppeteer');
(async() => {
const browser = await puppeteer.launch({args: ['--no-sandbox']});
const page = await browser.newPage();
await page.goto('https://example.com');
await page.screenshot({path: 'example.png'});
browser.close();
})();
It's worth considering adding both --no-sandbox --disable-setuid-sandbox to the default flags on linux.
In chrome-launcher/lighthouse we're already including --disable-setuid-sandbox and plan to add --no-sandbox soon for this reason.
I have always strongly urged people to never turn off the sandbox without a good cause, even in tests. It is a major part of the security system from what I understand.
I'm setting up a squeaky clean and fresh Debian VM to run some install steps in. We should be able to have it well documented how to get it operating without compromising system security.
I recall PHPStorm for example having an issue where it was serving on localhost, so a remote code execution exploit was opened up for any site including code that would look for the port in use and take advantage of a flaw in that server. Let's not open people up to security issues by disabling the sandbox here. Where they could be visiting any number of sites including code that looks for exploits to abuse.
Still no luck w/ those two flags:
febian:~/p /home/fortes/p/node_modules/puppeteer/.local-chromium/linux-494755/chrome-linux/chrome --no-sandbox --disable-setuid-sandbox
(chrome:12521): Gtk-WARNING **: cannot open display:
[0816/111850.260959:ERROR:nacl_helper_linux.cc(310)] NaCl helper process running without a sandbox!
Most likely you need to configure your SUID sandbox correctly
Same warning when just with --no-sandbox
Just as a quick update, I'm like 20-40ish minutes out from starting on testing the install procedure. Downloading packages now for a fresh net install of Debian 9. So, I should get back to you shortly with exact steps to reproduce. It just won't be as fast as something that doesn't require a full OS install. 😄
Thanks for going through effort @Garbee! Will you be testing a headless Debian install, or using via ssh?
SSH is not relevant to the problem. Only a Debian server without any X/wayland system pre-installed (like any remote web server) will be enough to fully recreate the problem and steps to reproduce. So, that's what I'll be working with locally in a VM.
This may or may not be helpful...
I run Electron in a Docker instance on Docker Cloud.
I run this command:
xvfb-run -a --server-args="-screen 0 1024x1024x24" ./node_modules/.bin/electron ./index.js
and my Dockerfile contains the following commands to install dependencies:
RUN apt-get update -y -q
RUN apt-get install -y -q xvfb libgtk2.0-0 libxtst6 libxss1 libgconf-2-4 libnss3 libasound2
headless exists to not need xvfb to virtualize the X instance for Chrome. Since it is all done in software internally.
@Garbee awesome. This stuff isn't my strong suit.
I'm going to attempt to replace the Electron usage with puppeteer, so hopefully this will simplify our environment.
@fortes Are you running this as root by chance?
If you're running as root (after having all the required deps installed as listed earlier) you need to run without a sandbox since Chromium requires that (no clue why exactly yet.) If running as a normal user, then it should run just fine on a fresh debian install with the required dependencies.
Not running as root
That's interesting. I setup a fresh Linode box on Debian 9. Installed the packages listed above, then setup nodesource to install node 8. Then the yarn repository. New folder, yarn add puppeteer and then created the index.js and ran it. Everything works perfectly fine without a sandbox error.
Is the box you're running on under you complete control or is it someone else's like a VPS/shared host? If it is a remote host could you share the provider so I can look into if they do anything funny with their kernel configurations.
Ah yea, that's it. Your box host is messing you up. It has a very old Kernel. Debian 9 ships with 4.9.0-3 and you're running 3.14.0. So the security features of the kernel are extremely different. So, you may be in a case where you need to fallback to using the older file-based sandbox to have some level of security.
Although, in all honesty... Upgrade the kernel or get a host that doesn't keep you back. It's very important that the kernel gets updated for the best security and you're being left vulnerable.
@fortes If you could, please try running this script on your server to see if you have user namespacing enabled. https://gist.githubusercontent.com/Garbee/dfd33ba0a62f1cbc023b8d57de18efca/raw/f82ed513014062554a3536980a67390b57f1f858/user-namespace-check-linux.sh
for the dockerisers amongst us - i've launched successfully with this setup:
FROM node:8
RUN apt-get update && \
apt-get install -yq gconf-service libasound2 libatk1.0-0 libc6 libcairo2 libcups2 libdbus-1-3 \
libexpat1 libfontconfig1 libgcc1 libgconf-2-4 libgdk-pixbuf2.0-0 libglib2.0-0 libgtk-3-0 libnspr4 \
libpango-1.0-0 libpangocairo-1.0-0 libstdc++6 libx11-6 libx11-xcb1 libxcb1 libxcomposite1 \
libxcursor1 libxdamage1 libxext6 libxfixes3 libxi6 libxrandr2 libxrender1 libxss1 libxtst6 \
ca-certificates fonts-liberation libappindicator1 libnss3 lsb-release xdg-utils wget
RUN npm i puppeteer
RUN echo "\
const puppeteer = require('puppeteer');\n\
(async () => {\n\
const browser = await puppeteer.launch({args: ['--no-sandbox', '--disable-setuid-sandbox']});\n\
const page = await browser.newPage();\n\
await page.goto('https://example.com');\n\
await page.screenshot({path: 'example.png'});\n\
browser.close();\n\
})();\
" > index.js
CMD ["node", "index.js"]
PR #311 is open to start looking at expanding the install script to make it much more interactive and friendly to help catch installation problems. You can check the code out from that PR and give it a spin. Please report on the PR of any problems you face or things you think could improve the flow.
Running into the problem using Windows Subsystem for Linux (WSL), as the "emulated" kernel doesn't support namespacing, and its not possible to upgrade the kernel. No combination of flags seemed to work, I switched to native Node.js on Windows and everything works fine.
I run into this problem trying to use puppeteer in Heroku
Don't use untrusted, prebuilt binaries. Use the trusted package from your distro:
https://packages.debian.org/stretch/chromium
Install it and then you can run chromium --headless. Puppeteer should also use it instead of bloating your home directory.
@orangecms currently puppeteer only runs with bleeding edge Chromium. Running against stable won't work at the moment.
Hmm that is unfortunate. 🤔
How about providing a Docker image or something like that with the correct libs preinstalled? Would that be feasible?
Addendum: Thanks a lot for the info @JoelEinbinder - I couldn't run it on my machine, either, so I installed =www-client/google-chrome-unstable-62.0.3178.0 (on Gentoo GNU/Linux). I am running a grsec kernel, which is known to break the Chrome sandbox.
So I pass options to launch():
const options = { executablePath: '/usr/bin/google-chrome-unstable' };
const browser = await puppeteer.launch(options);
I would add this to the docs, but I'm unsure where to put it. I just signed up for the CLA. Can anyone guide me there?
Using an external build is already detailed in the api docs and in the main README under Default Runtime Settings. I don't think we need to go adding that anywhere else for the time being.
How about providing a Docker image or something like that with the correct libs preinstalled? Would that be feasible?
IMO a Docker image while nice, should be something internally waited on until after a stable tag. Right now we should focus on the issues with getting it running directly on machines. Improve this experience. And then once we are stable, we can assess how to best provide a docker image for people to use.
currently puppeteer only runs with bleeding edge Chromium. Running against stable won't work at the moment.
@JoelEinbinder Do you plan to switch to stable builds once Chrome 62 lands?
Ah yea, that's it. Your box host is messing you up. It has a very old Kernel. Debian 9 ships with 4.9.0-3 and you're running 3.14.0. So the security features of the kernel are extremely different. So, you may be in a case where you need to fallback to using the older file-based sandbox to have some level of security.
On the latest kernel:
febian:~/p uname -a
Linux febian 4.9.0-3-amd64 #1 SMP Debian 4.9.30-2+deb9u2 (2017-06-26) x86_64 GNU/Linux
febian:~/p node --version
v8.1.1
febian:~/p lsb_release -a
No LSB modules are available.
Distributor ID: Debian
Description: Debian GNU/Linux 9.1 (stretch)
Release: 9.1
Codename: stretch
Still get this warning:
febian:~/p /home/fortes/p/node_modules/puppeteer/.local-chromium/linux-494755/chrome-linux/chrome --no-sandbox --disable-setuid-sandbox
(chrome:23907): Gtk-WARNING **: cannot open display:
[0817/080600.484767:ERROR:nacl_helper_linux.cc(310)] NaCl helper process running without a sandbox!
Most likely you need to configure your SUID sandbox correctly
@Garbee I ran your bash script and got the following:
febian:~/p ./test.sh
You have user namespacing in the kernel. You should be good to go.
FYI, I ran this script: https://raw.githubusercontent.com/Garbee/puppeteer/cd00f7d936c942f8f378a332401024e501b186ce/utils/linux/debian-check.sh -- there are no missing dependencies.
So this is a different system than earlier. The original system in question from the post had different system information.
Do not turn off the sandbox and you also are not starting it as headless. Which means it will fail anyways since you don't have a window system. When testing, continue to run the example script in the original issue. Don't start changing how to test now, it only makes debugging more difficult.
Same problem on Arch Linux / Manjaro Linux.
Arch Linux / Manjaro default kernel doesn't have the USER_NS feature enabled.
see: https://bugs.archlinux.org/task/36969
I think that's the problem, but i have no time to check it. 🙈
Dockerfile also doesn't work on my system. No wonder, if the kernel feature on the host system is missing.
Update
Using chromium 60 with executablePath: "/usr/bin/chromium", headless: false shows the chromium instance but socket connection doesn't show up and it runs endless
Compiling chromium-dev version 62 and will try again ...
Using Wercker CI, on a Docker Node 8.4 box I am getting similar error. I launched with options:
{ dumpio: true, args: ['--no-sandbox', '--disable-setuid-sandbox'] }
export WERCKER_STEP_ROOT="/pipeline/script-d4cfb19f-79aa-4afa-922d-3ce406cf42f6"
export WERCKER_STEP_ID="script-d4cfb19f-79aa-4afa-922d-3ce406cf42f6"
export WERCKER_STEP_OWNER="wercker"
export WERCKER_STEP_NAME="script"
export WERCKER_REPORT_NUMBERS_FILE="/report/script-d4cfb19f-79aa-4afa-922d-3ce406cf42f6/numbers.ini"
export WERCKER_REPORT_MESSAGE_FILE="/report/script-d4cfb19f-79aa-4afa-922d-3ce406cf42f6/message.txt"
export WERCKER_REPORT_ARTIFACTS_DIR="/report/script-d4cfb19f-79aa-4afa-922d-3ce406cf42f6/artifacts"
source "/pipeline/script-d4cfb19f-79aa-4afa-922d-3ce406cf42f6/run.sh" < /dev/null
/pipeline/source/node_modules/puppeteer/.local-chromium/linux-494755/chrome-linux/chrome: error while loading shared libraries: libX11-xcb.so.1: cannot open shared object file: No such file or directory
Unhandled rejection Error: Failed to connect to chrome!
at Function.launch (/pipeline/source/node_modules/puppeteer/lib/Launcher.js:96:13)
at <anonymous>
at process._tickCallback (internal/process/next_tick.js:188:7)
From previous event:
at Object.<anonymous> (/pipeline/source/build/prerender.js:8:9)
at Module._compile (module.js:573:30)
at Object.Module._extensions..js (module.js:584:10)
at Module.load (module.js:507:32)
at tryModuleLoad (module.js:470:12)
at Function.Module._load (module.js:462:3)
at Function.Module.runMain (module.js:609:10)
at startup (bootstrap_node.js:158:16)
at bootstrap_node.js:598:3
@AndrewBarba You're missing a dependency there.
/pipeline/source/node_modules/puppeteer/.local-chromium/linux-494755/chrome-linux/chrome: error while loading shared libraries: libX11-xcb.so.1: cannot open shared object file: No such file or directory
Please reference the full list I posted earlier and make sure they are installed.
Also, do not disable the sandbox. Do not disable the sandbox. We need to debug why things are happening with the sandbox and address those via documentation so you have a secure system to run tests on. Disabling the sandbox will leave your applications vulnerable to exploit the host machines if any malicious code ever makes it inside of them.
So this is a different system than earlier. The original system in question from the post had different system information.
Yes, I switched machines in order to use one that had the latest kernel, as suggested.
Running w/o the sandbox flags gives the following (same output with or without --headless):
febian:~ /home/fortes/p/node_modules/puppeteer/.local-chromium/linux-494755/chrome-linux/chrome --headless
[0817/085628.462576:FATAL:zygote_host_impl_linux.cc(123)] No usable sandbox! Update your kernel or see https://chromium.googlesource.com/chromium/src/+/master/docs/linux_suid_sandbox_development.md for more information on developing with the SUID sandbox. If you want to live dangerously and need an immediate workaround, you can try using --no-sandbox.
#0 0x55719ffd5657 base::debug::StackTrace::StackTrace()
#1 0x55719ffe9311 logging::LogMessage::~LogMessage()
#2 0x55719f1da1f1 content::ZygoteHostImpl::Init()
#3 0x55719ee74da0 content::BrowserMainLoop::EarlyInitialization()
#4 0x55719ee7b4c3 content::BrowserMainRunnerImpl::Initialize()
#5 0x5571a39d87a5 headless::HeadlessContentMainDelegate::RunProcess()
#6 0x55719fcdfec7 content::RunNamedProcessTypeMain()
#7 0x55719fce07fd content::ContentMainRunnerImpl::Run()
#8 0x55719fce8314 service_manager::Main()
#9 0x55719fcdf462 content::ContentMain()
#10 0x5571a2786325 headless::(anonymous namespace)::RunContentMain()
#11 0x5571a278639c headless::HeadlessBrowserMain()
#12 0x55719fce60e2 headless::HeadlessShellMain()
#13 0x55719e91fb6d ChromeMain
#14 0x7f3bce5832b1 __libc_start_main
#15 0x55719e91f9d0 <unknown>
Received signal 6
#0 0x55719ffd5657 base::debug::StackTrace::StackTrace()
#1 0x55719ffd51cf base::debug::(anonymous namespace)::StackDumpSignalHandler()
#2 0x7f3bd49230c0 <unknown>
#3 0x7f3bce595fcf gsignal
#4 0x7f3bce5973fa abort
#5 0x55719ffd4202 base::debug::BreakDebugger()
#6 0x55719ffe97cc logging::LogMessage::~LogMessage()
#7 0x55719f1da1f1 content::ZygoteHostImpl::Init()
#8 0x55719ee74da0 content::BrowserMainLoop::EarlyInitialization()
#9 0x55719ee7b4c3 content::BrowserMainRunnerImpl::Initialize()
#10 0x5571a39d87a5 headless::HeadlessContentMainDelegate::RunProcess()
#11 0x55719fcdfec7 content::RunNamedProcessTypeMain()
#12 0x55719fce07fd content::ContentMainRunnerImpl::Run()
#13 0x55719fce8314 service_manager::Main()
#14 0x55719fcdf462 content::ContentMain()
#15 0x5571a2786325 headless::(anonymous namespace)::RunContentMain()
#16 0x5571a278639c headless::HeadlessBrowserMain()
#17 0x55719fce60e2 headless::HeadlessShellMain()
#18 0x55719e91fb6d ChromeMain
#19 0x7f3bce5832b1 __libc_start_main
#20 0x55719e91f9d0 <unknown>
r8: 0000000000000000 r9: 00007ffc9d9eeb00 r10: 0000000000000008 r11: 0000000000000246
r12: 00007ffc9d9ef210 r13: 0000000000000161 r14: 00007ffc9d9ef208 r15: 00007ffc9d9ef200
di: 0000000000000002 si: 00007ffc9d9eeb00 bp: 00007ffc9d9eedb0 bx: 0000000000000006
dx: 0000000000000000 ax: 0000000000000000 cx: 00007f3bce595fcf sp: 00007ffc9d9eeb78
ip: 00007f3bce595fcf efl: 0000000000000246 cgf: 002b000000000033 erf: 0000000000000000
trp: 0000000000000000 msk: 0000000000000000 cr2: 0000000000000000
[end of stack trace]
Calling _exit(1). Core file will not be generated.
Running the original example code still gives this:
febian:~/p cat test.js
const puppeteer = require('puppeteer');
(async() => {
const browser = await puppeteer.launch({args: []});
const page = await browser.newPage();
await page.goto('https://example.com');
await page.screenshot({path: 'example.png'});
browser.close();
})();
febian:~/p node test.js
(node:27072) UnhandledPromiseRejectionWarning: Unhandled promise rejection (rejection id: 1): Error: Failed to connect to chrome!
(node:27072) [DEP0018] DeprecationWarning: Unhandled promise rejections are deprecated. In the future, promise rejections that are not handled will terminate the Node.js process with a non-zero exit code.
@fortes Is there a certain set of VPS providers you're trying this on? Any of this done with a local debian install using an official ISO?
This is a physical machine, not a VM or cloud host. Was a normal install of Debian 8 that got upgraded to 9 once it was released.
Hmm, perhaps the upgrade is the issue. Debian 8 didn't ship with the SECCOMP support for the kernel sandboxing. It was backported later.
I'm going to download a copy of Jessie for testing this theory...
I've tried to run Puppeteer scripts on Docker based CI service(e.g. Wercker CI or Circle CI 2.x). And
thanks to @Garbee 's adding dependencies shell, I've launch it successfully. See https://github.com/Quramy/puppeteer-example/blob/master/wercker.yml
We need to debug why things are happening with the sandbox and address those via documentation so you have a secure system to run tests on.
That's what I meant; I couldn't find something like a "known issues" section - how about that? 🙂
Seems to be the same problem trying to run this on Heroku. Any recommendations for cloud hosts that this will work on easily?
@campbecf For the dependencies, just make sure the whole list is installed. For the sandbox problems, we need to figure out why in different environments the sandbox stuff isn't detecting support properly or it isn't available. Then the thread is to figure out how to make what is needed available. At the very least, following the Chromium guide to setup the legacy SUID sandbox file. While it isn't the best protection it is at least something on systems without the proper kernel modules for namespacing execution.
@Garbee For reference, installing all of the dependencies fixed this problem for me.
Ubuntu 16.04
Kernel: 4.9.7-x86_64-linode80
Node v8.2.1
Not running as root.
When running as root, I have to disable sandbox to get it to work.
You probably know all of this, but I figured it might be useful to add another data point.
@campbecf @danielsantiago I've hacked together a fork of Heroku's Chrome buildpack that appears to work with Puppeteer. heroku buildpacks:add https://github.com/mikeraimondi/heroku-buildpack-google-chrome. { args: ['--no-sandbox'] } is required in the call to launch()
Yea @JoelEinbinder root requires no sandbox. No idea why. So if anyone is running in an environment where root is the operation user (like a Docker container) that's where it is necessary.
The difficult part here is not knowing the context of operations with any given system. So it's hard to track down why the sandboxes aren't operating when the details of the kernel at boot says they should.
@mikeraimondi I try using your method without succeed. Now I get the following error:
Unhandled promise rejection (rejection id: 1): Error: socket hang up
and sometimes:
Unhandled promise rejection (rejection id: 1): Error: Protocol error (Network.enable): Target closed.
instead of:
Unhandled promise rejection (rejection id: 1): Error: Failed to connect to chrome!
This is my scenario:
- Heroku
- Puppeteer: 0.9.0
- Node: 7.10.1
- Buildpack: https://github.com/mikeraimondi/heroku-buildpack-google-chrome
- Code:
#!/usr/bin/env node
const puppeteer = require('puppeteer');
const input = process.argv[2];
const output = process.argv[3];
(async() => {
const browser = await puppeteer.launch({args: ['--no-sandbox']});
const page = await browser.newPage();
await page.goto(input, {waitUntil: 'networkidle'});
await page.pdf({path: output, format: 'Letter', displayHeaderFooter: false, printBackground: true});
browser.close();
})();
- Heroku Build Info for Chrome buildpack:
remote: -----> Google Chrome app detected
remote: -----> Updating apt caches
remote: Hit http://apt.postgresql.org trusty-pgdg InRelease
remote: Ign http://archive.ubuntu.com trusty InRelease
remote: Hit http://archive.ubuntu.com trusty-security InRelease
remote: Hit http://archive.ubuntu.com trusty-updates InRelease
remote: Hit http://archive.ubuntu.com trusty Release.gpg
remote: Hit http://archive.ubuntu.com trusty Release
remote: Hit http://apt.postgresql.org trusty-pgdg/main amd64 Packages
remote: Hit http://archive.ubuntu.com trusty-security/main amd64 Packages
remote: Ign http://apt.postgresql.org trusty-pgdg/main Translation-en_US
remote: Ign http://apt.postgresql.org trusty-pgdg/main Translation-en
remote: Hit http://archive.ubuntu.com trusty-security/main Translation-en
remote: Hit http://archive.ubuntu.com trusty-updates/main amd64 Packages
remote: Hit http://archive.ubuntu.com trusty-updates/main Translation-en
remote: Hit http://archive.ubuntu.com trusty/main amd64 Packages
remote: Hit http://archive.ubuntu.com trusty/universe amd64 Packages
remote: Hit http://archive.ubuntu.com trusty/main Translation-en
remote: Hit http://archive.ubuntu.com trusty/universe Translation-en
remote: Ign http://archive.ubuntu.com trusty/main Translation-en_US
remote: Ign http://archive.ubuntu.com trusty/universe Translation-en_US
remote: Reading package lists...
remote: -----> Fetching .debs for libxss1
remote: Reading package lists...
remote: Building dependency tree...
remote: The following NEW packages will be installed:
remote: libxss1
remote: 0 upgraded, 1 newly installed, 0 to remove and 145 not upgraded.
remote: Need to get 0 B/8,582 B of archives.
remote: After this operation, 60.4 kB of additional disk space will be used.
remote: Download complete and in download only mode
remote: -----> Installing google-chrome-stable_current_amd64.deb
remote: -----> Installing libxss1_1%3a1.2.2-1_amd64.deb
remote: -----> Writing profile script
remote: -----> Rewrite package-config files
remote: -----> Discovering process types
@danielsantiago we should probably take this off-thread, but based on your logs it looks like you may need to clear your build cache
@mikeraimondi I ran on heroku (using heroku ps:exec):
/app/node_modules/puppeteer/.local-chromium/linux-494755/chrome-linux/chrome --headless --disable-gpu --no-sandbox --remote-debugging-port=0
And get the following error:
[0818/171943.586886:FATAL:nss_util.cc(627)] NSS_VersionCheck("3.26") failed. NSS >= 3.26 is required. Please upgrade to the latest NSS, and if you still get this error, contact your distribution maintainer.
I clear the heroku cache everytime I try a new buildpack. Also I add the buildpack first with
heroku buildpacks:add -i 1 https://github.com/mikeraimondi/heroku-buildpack-google-chrome
I try to create a issue on your repository but I couldn't find the option there. Any other suggetion to keep this off-thread?
Thanks for the help!
@danielsantiago I turned issues on for my repo. Try again? Thanks.
After adding flags (see further) it started to work on Ubuntu 16 LTS (on GCP Compute engine).
const browser = await puppeteer.launch({
args: [
'--no-sandbox',
'--disable-setuid-sandbox',
],
})But still not working on Netlify CI docker image (there: https://github.com/netlify/build-image). Here's their Dockerfile: https://github.com/netlify/build-image/blob/master/Dockerfile
Would be really nice to have it fixed..
Same promise on Centos6, but --no-sandbox does not work for me.
There is no "fixing this" internally. The problem on the host machines needs to be traced down and documented for people how to get the sandboxes working right. In the cases (such as running root on Docker) where you in fact need to disable the sandboxes, the flags will need to be added by end users. The sandbox should not be disabled internally by default since it is a critical security component that protects machines from malicious code on the internet. It should be manually opted-into when a known situation requires it, and only then.
On Heroku, Ubuntu 16.04, Node 8.4.0, Yarn 0.27.5, having installed the required deps for Debian, and '--no-sandbox', '--disable-setuid-sandbox' I am getting:
/app/node_modules/puppeteer/.local-chromium/linux-494755/chrome-linux/chrome: error while loading shared libraries: libcairo-gobject.so.2: cannot open shared object file: No such file or directory
Error: Failed to connect to chrome!
at Function.launch (/app/node_modules/puppeteer/lib/Launcher.js:96:13)
I also tried downgrading to Ubuntu 14.04 and got a different error:
/app/node_modules/puppeteer/.local-chromium/linux-494755/chrome-linux/chrome: error while loading shared libraries: libgnutls.so.30: cannot open shared object file: No such file or directory
Error: Failed to connect to chrome!
at Function.launch (/app/node_modules/puppeteer/lib/Launcher.js:96:13)
Any ideas how to solve either of these?
@jasondonnette I just got it running on Heroku an hour ago or so.
I used the google chrome buildpack even though it downloads Chrome in addition to just installing the dependencies. It's probably a little more bloat than the absolute bare minimum (since puppeteer downloads it's own chromium...) but it works without any extra effort on my part.
EDIT: I am using the two flags also...
await puppeteer.launch({
args: ['--no-sandbox', '--disable-setuid-sandbox']
})
@maxschmeling As mentioned repeatedly in the thread, you need to make sure the right dependencies for debian are installed. That is the source of both of your original errors.
@Garbee that list of dependencies wasn't sufficient. It was missing libcairo-gobject2 at the very least.
Adding libcairo-gobject2 to the list may have solved it, but I found the buildpack to be a more palatable solution that required less configuration on my end.
Works perfectly on ubuntu xenial - no gtk - remote - ssh - non root
Linux ip---- 4.4.0-1022-aws #31-Ubuntu SMP Tue Jun 27 11:27:55 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux
Ubuntu Xenial
update
sudp apt-get update
node v8.4.0
curl -sL https://deb.nodesource.com/setup_8.x | sudo -E bash - sudo apt-get install -y nodejs
Install npm
sudo apt-get install npm
Install puppeteer
npm i puppeteer
Installed all the dependencies
sudo apt-get install gconf-service libasound2 libatk1.0-0 libc6 libcairo2 libcups2 libdbus-1-3 libexpat1 libfontconfig1 libgcc1 libgconf-2-4 libgdk-pixbuf2.0-0 libglib2.0-0 libgtk-3-0 libnspr4 libpango-1.0-0 libpangocairo-1.0-0 libstdc++6 libx11-6 libx11-xcb1 libxcb1 libxcomposite1 libxcursor1 libxdamage1 libxext6 libxfixes3 libxi6 libxrandr2 libxrender1 libxss1 libxtst6 ca-certificates fonts-liberation libappindicator1 libnss3 lsb-release xdg-utils wget
pass this to launch
({args: ['--no-sandbox', '--disable-setuid-sandbox']}
running as non root
Works perfectly.
Same on Debian GNU/Linux 8.
node.js v6.11.2
Works without sandbox...
thats resolve my problem.
OS -Linux 4.9.0-3-amd64 #1 SMP Debian 4.9.30-2+deb9u3 (2017-08-06) x86_64 GNU/Linux
npm - 5.3.0
node - v8.4.0
- set packages
- Run with root
echo 1 > /proc/sys/kernel/unprivileged_userns_clone(work untill reboot, i guess) - run example
node example.js
const puppeteer = require( 'puppeteer' );
(async () =>{
const browser = await puppeteer.launch( ) // , '--no-sandbox', '--disable-setuid-sandbox'
const page = await browser.newPage()
await page.goto( 'https://google.com', {waitUntil :'networkidle'} )
// Type our query into the search bar
await page.type( 'puppeteer' )
await page.click( 'input[type="submit"]' )
// Wait for the results to show up
await page.waitForSelector( 'h3 a' )
// Extract the results from the page
const links = await page.evaluate( () =>{
const anchors = Array.from( document.querySelectorAll( 'h3 a' ) )
return anchors.map( anchor => anchor.textContent )
} )
console.log( links.join( '\n' ) )
browser.close()
})()P.S. for ubuntu kernel:
It is turned on by default, but can be turned off if admins prefer or,
more importantly, if a security vulnerability is found.
The intent is to use this as mitigation so long as Ubuntu is on the
cutting edge of enablement for things like unprivileged filesystem
mounting.
(This patch is tweaked from the one currently still in Debian sid, which
in turn came from the patch we had in saucy)
@sh00tingstar this worked for me. Same environmet here (Debian 9, node 8.4.0 and npm 5.3.0)
Worked for me on Circle CI with following pre-sets(Ubuntu 14.04, v8.4.0 (npm v5.3.0)):
circle.yml
machine:
node:
version: 8
test:
pre:
- sudo apt-get update
part of the test includes:
const browser = await puppeteer.launch({
args: ["--no-sandbox", "--disable-setuid-sandbox"]
});
Tests are executed using jest and running in parallel in Chromium without any issues.
Run with root
echo 1 > /proc/sys/kernel/unprivileged_userns_clone(work untill reboot, i guess)
This fixed it for me, running the Debian Jessie AWS image, --no-sandbox and --disable-setuid-sandbox is not required. I updated the kernel from 3.something to 4.9 via backports earlier, so maybe this is required, too.
To persist this setting, add kernel.unprivileged_userns_clone=1 to /etc/sysctl.conf.
thanks, it work perfectly @GirishPatil
doesn't seem to work on nixos even with all the solutions above. dumpio shows no output :(
I'm disappointed that installing X-related dependencies are the most upvoted answers. Chrome Headless is made to avoid that dependency so it sounds like a terrible workaround.
It's not a work around. Chrome requires these things to operate. The API calls to them still need to be made even to emulate the environment.
It looks like the permissions on the bundled executables were lost. Most things are 0700 (only readable by the owner) and the root setuid on the chrome_sandbox binary is missing.
@stefanor That depends on your host machine configuration. Simply unzipping the files should leave all of them as packaged. On the note of permissions on the sandbox file, that is expected. You need to follow the setup instructions. You can't simply unzip a file as a normal user and let it have root permissions, that is a major security vulnerability.
I don't see anything in the README for puppeteer that tell you about this. npm install puppeteer does not leave you with a working chrome, as you'd expect it to (and as the README implies it will).
I finally get it work on centos 7.3.
- install all dependencies:
yum install pango.x86_64 libXcomposite.x86_64 libXcursor.x86_64 libXdamage.x86_64 libXext.x86_64 libXi.x86_64 libXtst.x86_64 cups-libs.x86_64 libXScrnSaver.x86_64 libXrandr.x86_64 GConf2.x86_64 alsa-lib.x86_64 atk.x86_64 gtk3.x86_64 -y
yum install ipa-gothic-fonts xorg-x11-fonts-100dpi xorg-x11-fonts-75dpi xorg-x11-utils xorg-x11-fonts-cyrillic xorg-x11-fonts-Type1 xorg-x11-fonts-misc -y- modify the example script as follow:
const puppeteer = require('puppeteer');
(async () => {
const browser = await puppeteer.launch({args: ['--no-sandbox', '--disable-setuid-sandbox']});
const page = await browser.newPage();
await page.goto('https://example.com');
await page.screenshot({path: 'example.png'});
await browser.close();
})();To run chromium from command line, you should add --headless to prevent GUI related error, like this:
node_modules/puppeteer/.local-chromium/linux-499413/chrome-linux/chrome -v --headless --no-sandbox --disable-setuid-sandbox
Unable to launch chromium on Ubuntu Linux Server 14.04.5 LTS. I even tried to launch the chromium manually. But it doesn't start or launch. It just throws me the prompt back. node_modules/puppeteer/.local-chromium/linux-499413/chrome-linux/chrome -v --headless --no-sandbox --disable-setuid-sandbox
Times out every time with the following:
25 09 2017 13:56:19.075:INFO [karma]: Karma v1.5.0 server started at http://0.0.0.0:9876/build/
25 09 2017 13:56:19.076:INFO [launcher]: Launching browser ChromeHeadless with unlimited concurrency
25 09 2017 13:56:19.080:INFO [launcher]: Starting browser Chrome
25 09 2017 13:56:19.263:INFO [HeadlessChrome 0.0.0 (Linux 0.0.0)]: Connected on socket zDM7md2A7Je8kqsPAAAA with id 1679961
25 09 2017 13:58:19.552:WARN [HeadlessChrome 0.0.0 (Linux 0.0.0)]: Disconnected (1 times), because no message in 120000 ms.
HeadlessChrome 0.0.0 (Linux 0.0.0) ERROR
Disconnected, because no message in 120000 ms.
Note: The same exact configuration works and runs all the tests perfectly on OSX without any problems. But unable to make it work on Ubuntu.
Here are the related packages in my package.json:
"karma": "1.7.1",
"karma-chrome-launcher": "2.2.0",
"karma-htmlfile-reporter": "0.3.5",
"karma-jasmine": "1.1.0",
"karma-junit-reporter": "1.2.0",
"puppeteer": "0.11.0"
Here's my configuration:
const ChromiumRevision = require('puppeteer/package.json').puppeteer.chromium_revision;
const Downloader = require('puppeteer/utils/ChromiumDownloader');
const revisionInfo = Downloader.revisionInfo(Downloader.currentPlatform(), ChromiumRevision);
process.env.CHROME_BIN = revisionInfo.executablePath;
module.exports = function (config) {
config.set({
basePath: 'build/',
frameworks: ['jasmine'],
plugins: [
require('karma-jasmine'),
require('karma-chrome-launcher'),
],
customLaunchers: {
'ChromeHeadless': {
base: 'Chrome',
flags: [
'--headless',
'--disable-gpu',
// Without a remote debugging port, Google Chrome exits immediately.
'--remote-debugging-port=9222',
'--no-sandbox',
'--disable-setuid-sandbox'
],
debug: true
}
},
autoWatch: false,
browsers: ['ChromeHeadless'],
singleRun: true,
browserNoActivityTimeout: 120000,
urlRoot: '/build/'
})
}
I have installed all the debian/linux dependencies which have been mentioned across all the other related issues. Does anyone think that the Ubuntu Server [headless] OS version is causing any problems? I am not sure. Looking for some suggestions and workarounds. Thank you. ✌️
We really need this documentation for required options to run added to the README in an "Ubuntu/Debian/Linux" section:
await puppeteer.launch({
args: ['--no-sandbox', '--disable-setuid-sandbox']
})I anticipate a lot of users don't know to search GitHub issues for this issue, therefore can't find this thread, and are then not using this package because "it doesn't work".
Edit: I didn't even see https://github.com/GoogleChrome/puppeteer/blob/master/docs/troubleshooting.md#chrome-headless-fails-due-to-sandbox-issues at first despite it being in the warning error message:
Error {
message: `Failed to launch chrome!␊
/var/www/foobar/source/node_modules/puppeteer/.local-chromium/linux-499413/chrome-linux/chrome: error while loading shared libraries: libX11-xcb.so.1: cannot open shared object file: No such file or directory␊
␊
␊
TROUBLESHOOTING: https://github.com/GoogleChrome/puppeteer/blob/master/docs/troubleshooting.md␊
`,
}I think putting this in the Install section would be appropriate to prevent people from wasting so much time here with this.
Edit: These options didn't work, see my comment below #290 (comment)
We really need this documentation for required options
The options aren't required. They are optional if you chose to not setup a proper security sandbox.
It doesn't work out of the box, that's the issue. And there's zero decent documentation that's straight forward in the Install instructions in the README. Also, these options didn't even work and this thread has 100's of comments. I think it'd be appropriate to post a comment here and lock the issue/thread.
@niftylettuce how about a clear link to https://github.com/GoogleChrome/puppeteer#installation and possibly this thread in the readme install section? I'd hate clutter the readme with growing details in https://github.com/GoogleChrome/puppeteer/blob/master/docs/troubleshooting.md#chrome-headless-fails-due-to-sandbox-issues.
The only thing I could get to work is by outright disabling the sandbox. I tried everything in the other links to avoid this but the only feasible way is to disable the sandbox when using Docker.
Ugh.
Aren't these tools supposed to solve problems? :P
Docker runs everything as root, which causes problems with the sandbox. I think our repo has an example docker setup somewhere. You essentially need to make a standard user in the container to operate as. Running as root is the ultimate "security failure" on Linux. That is why even if the sandbox works, it wouldn't be nearly as effective. Since if an attack existed, you'd just have full box privileges to bypass the sandbox.
Ha, I checked my notifications at the right time ;)
I tried pretty much everything else. I was downloading unstable chrome, creating groups and users and then shifting my new users. I tried pretty much everything and the only thing that did wind up working was running everything as root and disabling the sandbox.
Normally this is bad but in this instance, each container runs in its own separate kernel space so if something did begin arbitrarily executing code, it'd be somewhat self-contained.
I'm trying to read up more and more on the sandbox and it might only be valuable for non-containerized instances.
@LeonineKing1199 there's a Docker example in https://github.com/GoogleChrome/puppeteer/blob/master/docs/troubleshooting.md#running-puppeteer-in-docker that shows how to setup a user and keep the sandbox :)
If you remove pupeteer as a en explicit dependency, then it just works O_o
Alright, thanks, guys!
But yeah, you really have to follow that thing verbatim. I'm glad it's working no though.
each container runs in its own separate kernel space so if something did begin arbitrarily executing code, it'd be somewhat self-contained.
Until you learn to break out of that kernel space. Then you have full root against the host!
It's worth considering adding both --no-sandbox --disable-setuid-sandbox to the default flags on linux.
Is this still the recommended solution as a lib author where I can't control the actual Linux setup of my clients?
Is this still the recommended solution as a lib author where I can't control the actual Linux setup of my clients?
No, this would be wrong and a security risk. The proper solution is to enable the sandboxing in your OS:
Run echo 1 > /proc/sys/kernel/unprivileged_userns_clone as root and add kernel.unprivileged_userns_clone=1 to /etc/sysctl.conf. This works on debian, may be different on other distributions.
Thank you. I'll try this.
For CentOS 7.4, kernel 3.10.0-693.5.2 (I'm running on VirtualBox):
yum install pango.x86_64 libXcomposite.x86_64 libXcursor.x86_64 libXdamage.x86_64 libXext.x86_64 libXi.x86_64 libXtst.x86_64 cups-libs.x86_64 libXScrnSaver-1.2.2-6.1.el7.x86_64 libXrandr.x86_64 GConf2 alsa-lib.x86_64 atk.x86_64 gtk3.x86_64 -y
yum install ipa-gothic-fonts xorg-x11-fonts-100dpi xorg-x11-fonts-75dpi xorg-x11-utils xorg-x11-fonts-cyrillic xorg-x11-fonts-Type1 xorg-x11-fonts-misc -y
const browser = await puppeteer.launch({args: ['--no-sandbox', '--disable-setuid-sandbox']});
FWIW I’m working on a web service that handles most of the issues involved in running Chrom in a production like environment. Comes with preset versions that are directly compatible with certain puppeteer versions.
Launches tomorrow 🚀 https://browserless.io
@joelgriffith looks really interesting! Let us know when that goes out.
One word of caution are the Chrome logos on the site. Distorting, changing it's orientation, etc. is technically against the brand trademark usage. The colored line versions are probably ok if you made those, but I'd be cautious about the altering the official logo.
Thanks @ebidel, I've cleaned up the images in question. Let me know if there's anything else that looks like a violation.
The service is live, I'd be more then happy to answer any questions folks might have here about launching in a linux environment or about the service itself. Not exactly an easy task...
@LeonineKing1199 there's a Docker example in https://github.com/GoogleChrome/puppeteer/blob/master/docs/troubleshooting.md#running-puppeteer-in-docker that shows how to setup a user and keep the sandbox :)
Except, your advice still relies on running with --cap-add=SYS_ADMIN, which is arguably worse than --no-sandbox and it does not work in Kubernetes. Without --cap-add=SYS_ADMIN, the container will fail with an error:
Failed to move to new namespace: PID namespaces supported, Network namespace supported, but failed: errno = Operation not permitted
On systems like App Engine, that's not needed in production. Just locally.
If there's a better flag to use, we can recommend that. Feel free to send a PR suggestion.
I'm using ubuntu in virtualbox, this is the result after a basic npm install puppeteer:
bash: node_modules/puppeteer/.local-chromium/linux-515411/chrome-linux/chrome: cannot execute binary file: Exec format error
uname -a returns: Linux R 4.9.0-4-686 #1 SMP Debian 4.9.51-1 (2017-09-28) i686 GNU/Linux
file: node_modules/puppeteer/.local-chromium/linux-515411/chrome-linux/chrome: ELF 64-bit LSB shared object, x86-64, version 1 (SYSV), dynamically linked, interpreter /lib64/ld-linux-x86-64.so.2, for GNU/Linux 2.6.32, not stripped
Considering the comment #290 (comment) in my case I'm blocked at the point 1.
I think you have a mismatched architecture, vecna.
You're likely trying to run a 64 bit executable on a 32 bit machine.
I agree @LeonineKing1199, but I can't control that from the command npm install puppeteer, the condition has to be managed in the install script
Update, reading in install.js seems that Linux has not a 32bit version anymore. Is that correct?