PhantomJS hangs on even simple examples
nathanejohnson opened this issue · comments
I have built PhantomJS on Alpine Linux 3.3.3 . I have applied these two pull requests in order to get it to build:
examples/hello.js fails (as with all others tried.)
I see the same issues running 2.1.1 tag as well as current master (842715b as of this writing)
Here is an example: (note that I wait quite a while before hitting Control C)
alpineamd64:~/phantomjs$ uname -a
Linux alpineamd64 4.1.20 #1-Alpine SMP Mon Mar 21 10:06:39 GMT 2016 x86_64 Linux
alpineamd64:~/phantomjs$
alpineamd64:~/phantomjs$ cd bin
alpineamd64:~/phantomjs/bin$ ldd phantomjs
/lib/ld-musl-x86_64.so.1 (0x55b3f8a2e000)
libz.so.1 => /lib/libz.so.1 (0x7fb1aa93c000)
libxml2.so.2 => /usr/lib/libxml2.so.2 (0x7fb1aa611000)
libsqlite3.so.0 => /usr/lib/libsqlite3.so.0 (0x7fb1aa366000)
libfontconfig.so.1 => /usr/lib/libfontconfig.so.1 (0x7fb1aa12f000)
libfreetype.so.6 => /usr/lib/libfreetype.so.6 (0x7fb1a9e81000)
libssl.so.1.0.0 => /lib/libssl.so.1.0.0 (0x7fb1a9c18000)
libcrypto.so.1.0.0 => /lib/libcrypto.so.1.0.0 (0x7fb1a97fb000)
libicui18n.so.56 => /usr/lib/libicui18n.so.56 (0x7fb1a93d0000)
libicuuc.so.56 => /usr/lib/libicuuc.so.56 (0x7fb1a9064000)
libicudata.so.56 => /usr/lib/libicudata.so.56 (0x7fb1a7681000)
libstdc++.so.6 => /usr/lib/libstdc++.so.6 (0x7fb1a7331000)
libgcc_s.so.1 => /usr/lib/libgcc_s.so.1 (0x7fb1a711e000)
libc.musl-x86_64.so.1 => /lib/ld-musl-x86_64.so.1 (0x55b3f8a2e000)
libexpat.so.1 => /usr/lib/libexpat.so.1 (0x7fb1a6efe000)
libpng16.so.16 => /usr/lib/libpng16.so.16 (0x7fb1a6cd1000)
alpineamd64:~/phantomjs/bin$
alpineamd64:~/phantomjs/bin$ cd ../examples/
alpineamd64:~/phantomjs/examples$../bin/phantomjs --debug=true hello.js
2016-04-09T14:21:23 [DEBUG] CookieJar - Created but will not store cookies (use option '--cookies-file=<filename>' to enable persistent cookie storage)
2016-04-09T14:21:23 [DEBUG] Set "http" proxy to: "" : 1080
2016-04-09T14:21:23 [DEBUG] Phantom - execute: Configuration
2016-04-09T14:21:23 [DEBUG] 0 objectName : ""
2016-04-09T14:21:23 [DEBUG] 1 cookiesFile : ""
2016-04-09T14:21:23 [DEBUG] 2 diskCacheEnabled : "false"
2016-04-09T14:21:23 [DEBUG] 3 maxDiskCacheSize : "-1"
2016-04-09T14:21:23 [DEBUG] 4 diskCachePath : ""
2016-04-09T14:21:23 [DEBUG] 5 ignoreSslErrors : "false"
2016-04-09T14:21:23 [DEBUG] 6 localUrlAccessEnabled : "true"
2016-04-09T14:21:23 [DEBUG] 7 localToRemoteUrlAccessEnabled : "false"
2016-04-09T14:21:23 [DEBUG] 8 outputEncoding : "UTF-8"
2016-04-09T14:21:23 [DEBUG] 9 proxyType : "http"
2016-04-09T14:21:23 [DEBUG] 10 proxy : ":1080"
2016-04-09T14:21:23 [DEBUG] 11 proxyAuth : ":"
2016-04-09T14:21:23 [DEBUG] 12 scriptEncoding : "UTF-8"
2016-04-09T14:21:23 [DEBUG] 13 webSecurityEnabled : "true"
2016-04-09T14:21:23 [DEBUG] 14 offlineStoragePath : ""
2016-04-09T14:21:23 [DEBUG] 15 localStoragePath : ""
2016-04-09T14:21:23 [DEBUG] 16 localStorageDefaultQuota : "-1"
2016-04-09T14:21:23 [DEBUG] 17 offlineStorageDefaultQuota : "-1"
2016-04-09T14:21:23 [DEBUG] 18 printDebugMessages : "true"
2016-04-09T14:21:23 [DEBUG] 19 javascriptCanOpenWindows : "true"
2016-04-09T14:21:23 [DEBUG] 20 javascriptCanCloseWindows : "true"
2016-04-09T14:21:23 [DEBUG] 21 sslProtocol : "default"
2016-04-09T14:21:23 [DEBUG] 22 sslCiphers : "ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-SHA:ECDHE-ECDSA-AES128-SHA:ECDHE-RSA-AES128-SHA:ECDHE-RSA-AES256-SHA:ECDHE-ECDSA-RC4-SHA:ECDHE-RSA-RC4-SHA:DHE-RSA-AES128-SHA:DHE-DSS-AES128-SHA:DHE-RSA-AES256-SHA:AES128-GCM-SHA256:AES128-SHA:AES256-SHA:DES-CBC3-SHA:RC4-SHA:RC4-MD5"
2016-04-09T14:21:23 [DEBUG] 23 sslCertificatesPath : ""
2016-04-09T14:21:23 [DEBUG] 24 sslClientCertificateFile : ""
2016-04-09T14:21:23 [DEBUG] 25 sslClientKeyFile : ""
2016-04-09T14:21:23 [DEBUG] 26 sslClientKeyPassphrase : ""
2016-04-09T14:21:23 [DEBUG] 27 webdriver : ":"
2016-04-09T14:21:23 [DEBUG] 28 webdriverLogFile : ""
2016-04-09T14:21:23 [DEBUG] 29 webdriverLogLevel : "INFO"
2016-04-09T14:21:23 [DEBUG] 30 webdriverSeleniumGridHub : ""
2016-04-09T14:21:23 [DEBUG] Phantom - execute: Script & Arguments
2016-04-09T14:21:23 [DEBUG] script: "hello.js"
2016-04-09T14:21:23 [DEBUG] Phantom - execute: Starting normal mode
2016-04-09T14:21:23 [DEBUG] WebPage - setupFrame ""
^C
alpineamd64:~/phantomjs/examples$ ulimit -a
core file size (blocks, -c) 0
data seg size (kbytes, -d) unlimited
scheduling priority (-e) 0
file size (blocks, -f) unlimited
pending signals (-i) 7945
max locked memory (kbytes, -l) 64
max memory size (kbytes, -m) unlimited
open files (-n) 1024
pipe size (512 bytes, -p) 8
POSIX message queues (bytes, -q) 819200
real-time priority (-r) 0
stack size (kbytes, -s) 8192
cpu time (seconds, -t) unlimited
max user processes (-u) 7945
virtual memory (kbytes, -v) unlimited
file locks (-x) unlimited
alpineamd64:~/phantomjs/examples$
Here is strace output:
phantomjs.strace.txt
Any ideas?
Reproduced with the following Dockerfile:
FROM alpine:3.3
MAINTAINER Ryan Schlesinger <ryan@outstand.com>
RUN apk add --no-cache \
build-base \
git \
perl \
openssl-dev \
flex \
bison \
gperf \
ruby \
fontconfig-dev \
freetype-dev \
sqlite-dev \
icu-dev \
libpng-dev \
libjpeg \
python \
libx11-dev \
libxext-dev \
linux-headers
ENV PHANTOMJS_VERSION 2.1.1
RUN git clone git://github.com/ariya/phantomjs.git && \
cd phantomjs && \
git checkout ${PHANTOMJS_VERSION} && \
git submodule init && \
git submodule update
WORKDIR /phantomjs
RUN cd src/qt/qtbase && wget -O - https://github.com/Vitallium/qtbase/pull/4.patch | patch -p1
RUN cd src/qt/qtwebkit && wget -O - https://github.com/Vitallium/qtwebkit/pull/9.patch | patch -p1
RUN python build.py --confirm --release --qt-config="-no-pkg-config" --git-clean-qtbase --git-clean-qtwebkitRepro'd on 6090f54 as well. (Comment out the first patch)
Hitting the same problem. In my case the REPL doesn't even work.
I'm trying to get a musl-libc build of phantomjs working, specifically for use in Alpine Linux without the glibc-compat stuff.
I'm building from source (git tag 2.1.1) with the following patches applied:
The build process doesn't seem to run through any trouble, except for a few warnings on unused variables AFAICT (the build log is ginormous however, can't say I went through it all). However, when launching the binary, I get the REPL, but it doesn't seem to execute anything:
$ phantomjs --debug=true
2016-06-14T09:54:46 [DEBUG] CookieJar - Created but will not store cookies (use option '--cookies-file=<filename>' to enable persistent cookie storage)
2016-06-14T09:54:46 [DEBUG] Set "http" proxy to: "" : 1080
2016-06-14T09:54:46 [DEBUG] Phantom - execute: Configuration
2016-06-14T09:54:46 [DEBUG] 0 objectName : ""
2016-06-14T09:54:46 [DEBUG] 1 cookiesFile : ""
2016-06-14T09:54:46 [DEBUG] 2 diskCacheEnabled : "false"
2016-06-14T09:54:46 [DEBUG] 3 maxDiskCacheSize : "-1"
2016-06-14T09:54:46 [DEBUG] 4 diskCachePath : ""
2016-06-14T09:54:46 [DEBUG] 5 ignoreSslErrors : "false"
2016-06-14T09:54:46 [DEBUG] 6 localUrlAccessEnabled : "true"
2016-06-14T09:54:46 [DEBUG] 7 localToRemoteUrlAccessEnabled : "false"
2016-06-14T09:54:46 [DEBUG] 8 outputEncoding : "UTF-8"
2016-06-14T09:54:46 [DEBUG] 9 proxyType : "http"
2016-06-14T09:54:46 [DEBUG] 10 proxy : ":1080"
2016-06-14T09:54:46 [DEBUG] 11 proxyAuth : ":"
2016-06-14T09:54:46 [DEBUG] 12 scriptEncoding : "UTF-8"
2016-06-14T09:54:46 [DEBUG] 13 webSecurityEnabled : "true"
2016-06-14T09:54:46 [DEBUG] 14 offlineStoragePath : ""
2016-06-14T09:54:46 [DEBUG] 15 localStoragePath : ""
2016-06-14T09:54:46 [DEBUG] 16 localStorageDefaultQuota : "-1"
2016-06-14T09:54:46 [DEBUG] 17 offlineStorageDefaultQuota : "-1"
2016-06-14T09:54:46 [DEBUG] 18 printDebugMessages : "true"
2016-06-14T09:54:46 [DEBUG] 19 javascriptCanOpenWindows : "true"
2016-06-14T09:54:46 [DEBUG] 20 javascriptCanCloseWindows : "true"
2016-06-14T09:54:46 [DEBUG] 21 sslProtocol : "default"
2016-06-14T09:54:46 [DEBUG] 22 sslCiphers : "ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-SHA:ECDHE-ECDSA-AES128-SHA:ECDHE-RSA-AES128-SHA:ECDHE-RSA-AES256-SHA:ECDHE-ECDSA-RC4-SHA:ECDHE-RSA-RC4-SHA:DHE-RSA-AES128-SHA:DHE-DSS-AES128-SHA:DHE-RSA-AES256-SHA:AES128-GCM-SHA256:AES128-SHA:AES256-SHA:DES-CBC3-SHA:RC4-SHA:RC4-MD5"
2016-06-14T09:54:46 [DEBUG] 23 sslCertificatesPath : ""
2016-06-14T09:54:46 [DEBUG] 24 sslClientCertificateFile : ""
2016-06-14T09:54:46 [DEBUG] 25 sslClientKeyFile : ""
2016-06-14T09:54:46 [DEBUG] 26 sslClientKeyPassphrase : ""
2016-06-14T09:54:46 [DEBUG] 27 webdriver : ":"
2016-06-14T09:54:46 [DEBUG] 28 webdriverLogFile : ""
2016-06-14T09:54:46 [DEBUG] 29 webdriverLogLevel : "INFO"
2016-06-14T09:54:46 [DEBUG] 30 webdriverSeleniumGridHub : ""
2016-06-14T09:54:46 [DEBUG] Phantom - execute: Script & Arguments
2016-06-14T09:54:46 [DEBUG] script: ""
2016-06-14T09:54:46 [DEBUG] Phantom - execute: Starting REPL mode
2016-06-14T09:54:46 [DEBUG] WebPage - setupFrame ""
phantomjs> var a = 10
phantomjs> console.log(a);
phantomjs> console.log('Echo?')
phantomjs> This is not even *valid syntax*, yet it doesn't complain /o\
phantomjs> ^C
2016-06-14T09:55:52 [DEBUG] WebPage - setupFrame ""
2016-06-14T09:55:52 [DEBUG] WebPage - updateLoadingProgress: 10
2016-06-14T09:55:52 [DEBUG] WebPage - setupFrame ""
2016-06-14T09:55:52 [DEBUG] WebPage - updateLoadingProgress: 100
If anything is missing, I'd be happy to give more details.
Additional info:
$ phantomjs -v
2.1.1
$ ldd /usr/bin/phantomjs
/lib/ld-musl-x86_64.so.1 (0x559e6abbb000)
libz.so.1 => /lib/libz.so.1 (0x7f1a3da7a000)
libsqlite3.so.0 => /usr/lib/libsqlite3.so.0 (0x7f1a3d7cf000)
libfontconfig.so.1 => /usr/lib/libfontconfig.so.1 (0x7f1a3d598000)
libfreetype.so.6 => /usr/lib/libfreetype.so.6 (0x7f1a3d2ea000)
libssl.so.1.0.0 => /lib/libssl.so.1.0.0 (0x7f1a3d081000)
libcrypto.so.1.0.0 => /lib/libcrypto.so.1.0.0 (0x7f1a3cc64000)
libicui18n.so.56 => /usr/lib/libicui18n.so.56 (0x7f1a3c839000)
libicuuc.so.56 => /usr/lib/libicuuc.so.56 (0x7f1a3c4cd000)
libicudata.so.56 => /usr/lib/libicudata.so.56 (0x7f1a3aaea000)
libstdc++.so.6 => /usr/lib/libstdc++.so.6 (0x7f1a3a79a000)
libgcc_s.so.1 => /usr/lib/libgcc_s.so.1 (0x7f1a3a587000)
libc.musl-x86_64.so.1 => /lib/ld-musl-x86_64.so.1 (0x559e6abbb000)
libexpat.so.1 => /usr/lib/libexpat.so.1 (0x7f1a3a367000)
libpng16.so.16 => /usr/lib/libpng16.so.16 (0x7f1a3a13a000)
$ paxctl -v /usr/bin/phantomjs
PaX control v0.9
Copyright 2004,2005,2006,2007,2009,2010,2011,2012,2014 PaX Team <pageexec@freemail.hu>
- PaX flags: -----m-x-e-- [/usr/bin/phantomjs]
MPROTECT is disabled
RANDEXEC is disabled
EMUTRAMP is disabled
Might be the same as an issue I'm having. I'm using PhantomJS to run some jasmine specs. It locks up when encountering an undeclared variable and memory usage spikes to > 4GB.
The problem has been reproduced on 2 machines using this repo: https://github.com/ptolemybarnes/jasmine-rails-issue
Run bundle then bundle exec spec:javascripts. Notice in ActivityMonitor that PhantomJS is consuming a huge amount of memory.
Curiously if you move the unknown variable within the function passed to jasmine#describe it behaves as expected.
I'm encountering the same issue. Been fighting with this for a day now. Going to stop now that I know this is a bug others are encountering.
May I suggest everyone who is hit by this give thumbs-up to the opening post? I think Alpine Linux is a natural OS to run (headless) automated tests on, and so it is worthwhile to quantify the level of support there is for fixing this issue.
We instead ditched phantomjs because of this.
We should probably rename this issue and specify that this is a musl/Alpine linux specific issue.
I posted what I think might be the issue @ ncopa/docker-phantomjs-alpine#2 (comment) for @ncopa. Hopefully can get some progress going on this issue as this is my last real blocker from leveraging only alpine containers.
+1 for getting this issue fixed. I just spend a few days getting a work around to work on my bamboo + docker w/ alpine.
My stop gap work around fix was as follows:
when setting up the webdriver also set these two.
implicitly_wait should be 0 by default but we set it again
page_load_timeout will allow us to raise TimeoutExceptions helping us later
cls.browser.set_page_load_timeout(60)
cls.browser.implicitly_wait(0)when exiting the webdriver:
import signal
cls.browser.service.process.send_signal(signal.SIGTERM)
cls.browser.kill('SIGTERM')
cls.browser.quit()and then I also use a retry decorator on all of my selenium/phantom JS tests that use the webdriver.get() method. Example:
@retry()
def tearDown(self):
self.browser.get(self.get_reversed_url('logout'))Retry decorator as setup for my bamboo instance:
import time
from functools import wraps
def retry(exceptions=Exception, attempts=3, delay=3, backoff=2, logger=None):
def deco_retry(f):
@wraps(f)
def f_retry(*args, **kwargs):
mtries, mdelay = attempts, delay
while mtries > 1:
try:
return f(*args, **kwargs)
except exceptions, e:
msg = "%s: %s, Retrying in %d seconds..." % (str(e.__class__.__name__), str(e), mdelay)
if logger:
logger.warning(msg)
else:
print msg
time.sleep(mdelay)
mtries -= 1
mdelay *= backoff
return f(*args, **kwargs)
return f_retry # true decorator
return deco_retryYou can set this to pick up any exception or set it explicitly to the TimeoutException from selenium. see the docs http://www.saltycrane.com/blog/2009/11/trying-out-retry-decorator-python/
Also a little more complex way of doing this with trying to avoid the @Retry decorator is to override the unittest.TestCase.run method with a test wrapper; this did not work for us as we were doing .get() calls in teardown and those were not getting retried but if you don't do that this will retry your tests 3 times. http://stackoverflow.com/a/32248305
def run(self, result=None):
self.origTestMethodName = self._testMethodName
self._testMethodName = "_testRetryWrapper"
super(ATIPSeleniumTest, self).run(result)
self._testMethodName = self.origTestMethodName
def _testRetryWrapper(self):
testMethod = getattr(self, self.origTestMethodName)
retryAttemptsLeft = 3
while True:
try:
testMethod()
break
except:
if retryAttemptsLeft == 0:
raise
else:
print "Retrying in 3 seconds"
time.sleep(3)
retryAttemptsLeft = retryAttemptsLeft - 1I hope this helps some people!
I don't know how it took me so long to find this thread - I'm having the same issue. From the refs, it doesn't look like there's been much movement on this yet but has anyone had any luck implementing other workarounds?
My choices at present seem to be limited to:
- find a new base image
I would prefer more choices.
@mattschofield even from other docker base image i can't make phantomjs work on my Alpine linux 3.6, i got a qt error
[WARNING] QThread::start: Thread creation error: Operation not permitted
Did you succed ?
@mattschofield That's the only option I've come up with, compiling it in an alpine based distro seems to have the same problem and I can't figure much out aside from it seems to have issues with a pselect call.
@vitoo @thatsnotright in the end I went with a different base image.
My feeling from all the conversations around this was that the Alpine base image just wasn't ready for phantomjs. I would require far greater knowledge of linux fundamentals to explain why this is the case.
In case it's any use for anyone, we've had pretty good success switching to a ruby:2.3.3-slim base image for our application (which uses screen-scraping powered by phantomjs) and it hasn't been too bad. It's not as small an image as alpine would give us but it's totally workable.
Just if someone finds this helpfull - in docker with alpine 3.7 I just did this to make phantomjs work:
apk add --no-cache wget nodejs
wget -qO- "https://github.com/dustinblackman/phantomized/releases/download/2.1.1a/dockerized-phantomjs.tar.gz" | tar xz -C /
npm config set user 0
npm install -g phantomjs-prebuilt
@aiomaster your solution is the best.
Due to our very limited maintenance capacity (see #14541 for more details), we need to prioritize our development focus on other tasks. Therefore, this issue will be automatically closed. In the future, if we see the need to attend to this issue again, then it will be reopened. Thank you for your contribution!