unable to plot in jupyter notebook using R kernel
fyears opened this issue · comments
I am using default configuration of jupyter/datascience-notebook.
By accessing http://docker.local:8888/, I could start a notebook with R kernel, but I could not generate plots inside the notebook.
The simple official example:
require(stats) # for lowess, rpois, rnorm
plot(cars)
lines(lowess(cars))
The #54 might relevant, but I am using the most up-to-todate image (20160519) .
Thanks for the report. Definitely looks like #54 again. libxrender moved from datascience-notebook to minimal-notebook (https://github.com/jupyter/docker-stacks/blob/master/minimal-notebook/Dockerfile#L33) so it should still be available. Must be something else wrong now.
I'm running into the exact same issue.
Not sure if this helps, but if I run start-notebook --debug is see the following output whenever I try plotting anything. It also creates PDF files on disk.
[1] "Got unhandled msg_type:" "comm_info_request"
[1] "Got unhandled msg_type:" "comm_open"
[1] "Got unhandled msg_type:" "comm_open"
dev.new(): using pdf(file="Rplots1.pdf")
I also spotted this in the output, right before the Kernel crashed:
Warning messages:
1: In doTryCatch(return(expr), name, parentenv, handler) :
unable to load shared object '/opt/conda/lib64/R/modules//R_X11.so':
libjpeg.so.8: cannot open shared object file: No such file or directory
2: In png(tf, width, height, "in", pointsize, bg, res, antialias = antialias) :
unable to load shared object '/opt/conda/lib64/R/library/grDevices/libs//cairo.so':
libjpeg.so.8: cannot open shared object file: No such file or directory
3: In png(tf, width, height, "in", pointsize, bg, res, antialias = antialias) :
failed to load cairo DLL
4: In file(con, "rb") :
cannot open file '/tmp/RtmpjYzEfu/file183dd7bd4f.png': No such file or directory
5: In svg(tf, width, height, pointsize, FALSE, family, bg, antialias) :
failed to load cairo DLL
6: In file(con, "rb") :
cannot open file '/tmp/RtmpjYzEfu/file181f35e31f.svg': No such file or directory
[I 04:52:09.095 NotebookApp] Kernel shutdown: 9fefd796-4a2b-4980-9141-a663fb75e4d6
[D 04:52:09.097 NotebookApp] 204 DELETE /api/sessions/02a18c5b-b2d3-4b7a-a4a6-491b0615858e (192.168.99.1) 106.02ms
Did some digging, it might be related to this issue? conda/conda#2020
I played one round of bisect over recent builds listed here https://github.com/jupyter/docker-stacks/wiki/Docker-build-history
- May. 16, 2016, jupyter/datascience-notebook:dc6ae8bd8209 (latest), plots don't work
- Apr. 13, 2016, jupyter/datascience-notebook:2d878db5cbff (4 builds ago), plots work
- May. 05, 2016, jupyter/datascience-notebook:2bcf753a6695 (3 builds ago), plots don't work
The only explicit diff between the last build that plots and the first build that doesn't is the update to use Notebook 4.2 and ipywidgets 5.1:
Other implicit version changes may have snuck in during the docker image build, but we only allow most packages to vary in patch release number, not major or minor version. Plus, our builds are all done on the same box starting with a pull of the last build to gain the benefit of docker layer caching. This means anything before the notebook 4.2 version change in the Dockerfile will not have changed (e.g., version of libxrender pulled from debian).
@takluyver @jakirkham have you seen any problems with IRkernel in notebook 4.2 with or without conda?
@michael-erasmus hit the nail on the head.
Comparing the last working build to the first non-working one, I see a third-order dependency version changed and broke things. In 2d878db, the libjpeg v8 is present:
jovyan@8eac9da0d24d:/opt/conda/lib$ ls -l | grep jpeg
-rw-r--r-- 2 jovyan users 1769836 Sep 11 2012 libjpeg.a
-rwxr-xr-x 1 jovyan users 918 Apr 14 01:33 libjpeg.la
lrwxrwxrwx 1 jovyan users 16 Apr 14 01:33 libjpeg.so -> libjpeg.so.8.4.0
lrwxrwxrwx 1 jovyan users 16 Apr 14 01:33 libjpeg.so.8 -> libjpeg.so.8.4.0
-rwxr-xr-x 2 jovyan users 973500 Sep 11 2012 libjpeg.so.8.4.0
In 2bcf753 and newer, v8 is gone and v9 is there:
jovyan@910e6e4609dd:~/work$ ls -l /opt/conda/lib | grep libjpeg
-rw-r--r-- 2 jovyan users 2078420 Apr 15 18:42 libjpeg.a
-rwxr-xr-x 1 jovyan users 917 May 5 16:52 libjpeg.la
lrwxrwxrwx 1 jovyan users 16 May 5 16:52 libjpeg.so -> libjpeg.so.9.2.0
lrwxrwxrwx 1 jovyan users 16 May 5 16:52 libjpeg.so.9 -> libjpeg.so.9.2.0
-rwxr-xr-x 2 jovyan users 1073308 Apr 15 18:42 libjpeg.so.9.2.0
@jakirkham Is there a conda-forge feedstock for irkernel that needs to be updated? Or a R_X11 feedstock?
@jakirkham Is there a conda-forge feedstock for irkernel that needs to be updated? Or a R_X11 feedstock?
Not yet. Someone started to get the ball rolling on R, but they are generally very busy and I believe there were a few tricks so it came to a stop.
Though we do provide JPEG 9. So, it is possible that some of these R packages are not properly pinned against JPEG 8. Hence we JPEG is upgraded and that breaks them. I would recommend adding a pinned file to the /opt/conda/conda-meta directory, which simply includes the line jpeg 8*. After running an update, this should bring JPEG back down to the right version and things will start working again. Though it is possible though that we may need a more exact pinning.
I would recommend adding a pinned file to the /opt/conda/conda-meta directory
Do you think we should add it to the images as a temporary fix and do a new build? Or do you think users should add it as a workaround until it's fixed upstream?
Let's ask Continuum about fixing these packages to be properly pinned. If it looks like a long turnaround, we should probably pin it ourselves. Do we know exactly which packages were using JPEG?
Do we know exactly which packages were using JPEG?
I do not. Is there a canonical place to look to figure it out?
Just wanted to confirm that @jakirkham's suggestion did indeed work for me. I managed to get plotting to work by using a Dockerfile like this:
FROM jupyter/datascience-notebook
RUN echo "jpeg 8*" >> /opt/conda/conda-meta/pinned
RUN conda update --all -y
Glad to hear that @michael-erasmus. Could you please share with us what happened during conda update --all -y? It will help us as we talk with Continuum about what packages we need to have rebuilt.
@msarahan, do you know who/where we should notify someone about this pinning issue with R packages? Basically, the stack in this docker container jupyter/datascience-notebook ends up being broken w.r.t. to R because of JPEG 9. Fortunately, pinning to a JPEG 8 seems tenable so it may just be an issue of R packages that are missing pins on Linux.
CC @mingwandroid, who handles our R ecosystem.
This appears to be fixed in the latest build (tag: 9f67ac2ec481).
@fyears, @michael-erasmus, please re-open if you still have problems once you update.
New problem with plotting. This time, pinning to jpeg8 post-hoc does not fix the issue. Looking into what we might do.
Sorry I missed this one. Pinging @mingwandroid who does the r stuff.
The latest (3aa2f70cb868) r-notebook image is OK, but datascience-notebook and all-spark-notebook both have the problem. Something in the scipy-notebook image is triggering the upgrade to libjpeg9 and so R in the two child images have the problem.
I can do the pinning in the scipy-notebook for the time being as a workaround until the R package(s) declare the libjpeg8 dependency properly. Or until we bump versions on all of the R packages so they all work with libjpeg9. I'll try the latter first.
No luck upgrading all the R libs. One of them still wants libjpeg8. So pinning it is.
I can't believe I missed this. Seems the problem is cairo is linked to jpeg. While it doesn't look like cairo itself requires jpeg, it may have discovered it via a dependency that got linked to jpeg. Maybe R is a total red herring in all of this.
unable to load shared object '/opt/conda/lib64/R/library/grDevices/libs//cairo.so':
libjpeg.so.8: cannot open shared object file: No such file or directory
As a workaround for the time being, I pinned to jpeg8. Build 97a5071c5775 has it. If / when cairo (good catch @jakirkham) updates to rely on jpeg9 or properly declares its dependency on 8, we can take out the pinned files.
I did a bit more digging and it seems r-base is where this issue comes into play. Honestly, I'm not sure how this hasn't hit us with other issues. Now I'm not sure what version of r-base we were using at the time. However, it has seen a bit of restructuring since then. Here are the dependencies. We need to start pinning down a lot more of these.
@msarahan, can you please let me know which issue tracker to raise the problem cited above?
What channels are being used here? I took a look at:
https://github.com/jupyter/docker-stacks/blob/master/scipy-notebook/Dockerfile
.. but didn't see any channels specified.
I have no issue with updating R to use jpeg9 and would rather do that (and then pin to that version) than make incompatible islands of binaries.
We are using r, conda-forge, and defaults.
@parente , this is preventing normal conda installation of gdal and others that require jpeg 9*. I've spent the last few days trying to figure out what was wrong, only to find I now need to add --no-pin to the install options. I'm new to conda and the docker-stacks, so I imagine others are having similar problems.
Unfortunately there is a discontinuity between defaults/r and conda-forge. One of the big pieces is jpeg. We made the sensible decision here to not break our core supplied features by pinning jpeg to match defaults/r. However, I don't think the fault lies with docker-stacks, @DCAL12. The fault lies with this discontinuity. The sooner we can resolve it the better.
I understand - though perhaps adding a known issues section to README would be helpful. I wasn't aware of what was causing my problem until stumbling upon this discussion.
Anyway, thanks for providing such a powerful data science platform. I learned quite a bit about Jupyter and Docker just troubleshooting this.
@DCAL12 Note added to README. When we take the pinned file out, we'll have to remember to remove the note too.
I now seeing plot without title in the current version of the r-notebook docker image (haven't tried others). It was working earlier last week and when I pulled the latest images a couple days ago the plots stopped working. Did something change upstream?
I see the same behavior in r-notebook. I haven't tried the datascience and all-spark images to see if they have the same behavior.
None of the recent builds from the last week or so explicitly touched the R dependencies (see https://github.com/jupyter/docker-stacks/wiki/Docker-build-history), but some 2nd-order dependency could have changed. Probably need to play another round of docker image bisect and see what libs are different between the last working image and the first broken one.
I have the same issue as the comment above. The image now no longer works and I can no longer get my notebooks to display images. I ended up reverting to a local install. In future it might be work using the SHA256 digest to run an explicit docker image version.
If anyone rolls back to an older SHA where it works, please do post that SHA here so we have an end point for doing the bisect.
I'm running into this as well I think. It seems to stop working somewhere between 54838ed and 7001472
root@jupyter-eldap docker-stacks]# docker run --rm -it docker.io/jupyter/datascience-notebook:54838ed4acb1 R --quiet -e 'capabilities()'
> capabilities()
jpeg png tiff tcltk X11 aqua
TRUE TRUE TRUE TRUE FALSE FALSE
http/ftp sockets libxml fifo cledit iconv
TRUE TRUE TRUE TRUE FALSE TRUE
NLS profmem cairo ICU long.double libcurl
TRUE FALSE TRUE TRUE TRUE TRUE
>
>
[root@jupyter-eldap docker-stacks]# docker run --rm -it docker.io/jupyter/datascience-notebook:7001472997fa R --quiet -e 'capabilities()'
> capabilities()
jpeg png tiff tcltk X11 aqua
FALSE FALSE FALSE TRUE FALSE FALSE
http/ftp sockets libxml fifo cledit iconv
TRUE TRUE TRUE TRUE FALSE TRUE
NLS profmem cairo ICU long.double libcurl
TRUE TRUE FALSE TRUE TRUE TRUE
Not sure if this helps, but 54838ed and 7001472 pick up different revisions of r-base
[root@jupyter-eldap ~]# docker run --rm -it docker.io/jupyter/datascience-notebook:54838ed4acb1 conda list | grep r-base
r-base 3.3.1 1 r
r-base64enc 0.1_3 r3.3.1_0 r
[root@jupyter-eldap ~]# docker run --rm -it docker.io/jupyter/datascience-notebook:7001472997fa conda list | grep r-base
r-base 3.3.1 3 r
r-base64enc 0.1_3 r3.3.1_0 r
Please do conda info and conda list
On 54838ed...
root@jupyter-eldap ~]# docker run --rm -it docker.io/jupyter/datascience-notebook:54838ed4acb1 conda info
Current conda install:
platform : linux-64
conda version : 4.1.11
conda-env version : 2.5.2
conda-build version : not installed
python version : 3.5.2.final.0
requests version : 2.10.0
root environment : /opt/conda (writable)
default environment : /opt/conda
envs directories : /opt/conda/envs
package cache : /opt/conda/pkgs
channel URLs : https://conda.anaconda.org/r/linux-64/
https://conda.anaconda.org/r/noarch/
https://repo.continuum.io/pkgs/free/linux-64/
https://repo.continuum.io/pkgs/free/noarch/
https://repo.continuum.io/pkgs/pro/linux-64/
https://repo.continuum.io/pkgs/pro/noarch/
config file : /home/jovyan/.condarc
offline mode : False
is foreign system : False
[root@jupyter-eldap ~]# docker run --rm -it docker.io/jupyter/datascience-notebook:54838ed4acb1 conda list
# packages in environment at /opt/conda:
#
bokeh 0.11.1 py35_0
bzip2 1.0.6 3
cairo 1.12.18 6
cloudpickle 0.1.1 py35_0
conda 4.1.11 py35_0
conda-env 2.5.2 py35_0
backports.shutil_get_terminal_size 1.0.0 py35_0 conda-forge
blas 1.1 openblas conda-forge
cycler 0.10.0 py35_0 conda-forge
decorator 4.0.10 py35_0 conda-forge
dill 0.2.5 py35_0 conda-forge
entrypoints 0.2.2 py35_0 conda-forge
ipykernel 4.3.1 py35_1 conda-forge
ipython 5.0.0 py35_0 conda-forge
ipywidgets 5.1.5 py35_0 conda-forge
jinja2 2.8 py35_1 conda-forge
jsonschema 2.5.1 py35_0 conda-forge
jupyter_client 4.3.0 py35_0 conda-forge
jupyter_core 4.1.0 py35_0 conda-forge
libiconv 1.14 3 conda-forge
libpng 1.6.23 0 conda-forge
libsodium 1.0.10 0 conda-forge
markupsafe 0.23 py35_0 conda-forge
mistune 0.7.3 py35_0 conda-forge
mpmath 0.19 py35_0 conda-forge
nbconvert 4.2.0 py35_0 conda-forge
nbformat 4.0.1 py35_0 conda-forge
notebook 4.2.2 py35_0 conda-forge
numpy 1.10.4 py35_blas_openblas_202 [blas_openblas] conda-forge
openblas 0.2.18 4 conda-forge
pandas 0.18.1 np110py35_1 conda-forge
pexpect 4.2.0 py35_1 conda-forge
pickleshare 0.7.3 py35_0 conda-forge
pillow 3.2.0 py35_0 conda-forge
prompt_toolkit 1.0.4 py35_0 conda-forge
ptyprocess 0.5.1 py35_0 conda-forge
pygments 2.1.3 py35_0 conda-forge
pyparsing 2.1.1 py35_0 conda-forge
python-dateutil 2.5.3 py35_0 conda-forge
pytz 2016.6.1 py35_0 conda-forge
pyzmq 15.3.0 py35_1 conda-forge
scikit-learn 0.17.1 np110py35_blas_openblas_201 [blas_openblas] conda-forge
scipy 0.17.1 np110py35_blas_openblas_202 [blas_openblas] conda-forge
simplegeneric 0.8.1 py35_0 conda-forge
sip 4.18 py35_0 conda-forge
six 1.10.0 py35_0 conda-forge
statsmodels 0.6.1 np110py35_0 conda-forge
sympy 1.0 py35_0 conda-forge
terminado 0.6 py35_0 conda-forge
tornado 4.4.1 py35_1 conda-forge
traitlets 4.2.2 py35_0 conda-forge
wcwidth 0.1.7 py35_0 conda-forge
widgetsnbextension 1.2.6 py35_3 conda-forge
zeromq 4.1.5 0 conda-forge
curl 7.49.0 1
cython 0.23.5 py35_0
fastcache 1.0.2 py35_1
fontconfig 2.11.1 6
freetype 2.5.5 1
h5py 2.5.0 np110py35_4
hdf5 1.8.15.1 3
icu 54.1 0
ipython_genutils 0.1.0 py35_0
jbig 2.1 0
jpeg 8d 1
jupyterhub 0.5.0 <pip>
libffi 3.2.1 0
libgfortran 3.0.0 1
libtiff 4.0.6 2
libxml2 2.9.2 0
llvmlite 0.8.0 py35_0
matplotlib 1.5.1 np110py35_0
mkl 11.3.3 0
networkx 1.11 py35_0
numba 0.23.1 np110py35_0
numexpr 2.5.2 np110py35_1
openssl 1.0.2h 1
pamela 0.2.1 <pip>
patsy 0.4.1 py35_0
pcre 8.39 0
pip 8.1.2 py35_0
pixman 0.32.6 0
pycosat 0.6.1 py35_1
pycrypto 2.6.1 py35_4
python 3.5.2 0
pyyaml 3.11 py35_4
glib 2.43.0 2 r
gsl 1.16 1 r
harfbuzz 0.9.35 6 r
libgcc 4.8.5 1 r
ncurses 5.9 8 r
pango 1.36.8 3 r
r 3.3.1 r3.3.1_0 r
r-assertthat 0.1 r3.3.1_2 r
r-base 3.3.1 1 r
r-base64enc 0.1_3 r3.3.1_0 r
r-bh 1.60.0_2 r3.3.1_0 r
r-bitops 1.0_6 r3.3.1_2 r
r-boot 1.3_18 r3.3.1_0 r
r-car 2.1_2 r3.3.1_0 r
r-caret 6.0_70 r3.3.1_0 r
r-catools 1.17.1 r3.3.1_2 r
r-class 7.3_14 r3.3.1_0 r
r-cluster 2.0.4 r3.3.1_0 r
r-codetools 0.2_14 r3.3.1_0 r
r-colorspace 1.2_6 r3.3.1_0 r
r-curl 0.9.7 r3.3.1_0 r
r-dbi 0.4_1 r3.3.1_0 r
r-devtools 1.11.1 r3.3.1_0 r
r-dichromat 2.0_0 1a r
r-digest 0.6.9 r3.3.1_0 r
r-dplyr 0.4.3 r3.3.1_0 r
r-evaluate 0.9 r3.3.1_0 r
r-foreach 1.4.3 r3.3.1_0 r
r-forecast 7.1 r3.3.1_0 r
r-foreign 0.8_66 r3.3.1_0 r
r-formatr 1.4 r3.3.1_0 r
r-fracdiff 1.4_2 r3.3.1_2 r
r-ggplot2 2.1.0 r3.3.1_0 r
r-git2r 0.15.0 r3.3.1_0 r
r-gtable 0.2.0 r3.3.1_0 r
r-highr 0.6 r3.3.1_0 r
r-htmltools 0.3.5 r3.3.1_0 r
r-httpuv 1.3.3 r3.3.1_0 r
r-httr 1.2.0 r3.3.1_0 r
r-irdisplay 0.3 r3.3.1_2 r
r-irkernel 0.6.1 r3.3.1_0 r
r-iterators 1.0.8 r3.3.1_0 r
r-jsonlite 0.9.22 r3.3.1_0 r
r-kernsmooth 2.23_15 r3.3.1_0 r
r-knitr 1.13 r3.3.1_0 r
r-labeling 0.3 r3.3.1_2 r
r-lattice 0.20_33 r3.3.1_0 r
r-lazyeval 0.2.0 r3.3.1_0 r
r-lme4 1.1_12 r3.3.1_0 r
r-magrittr 1.5 r3.3.1_2 r
r-markdown 0.7.7 r3.3.1_2 r
r-mass 7.3_45 r3.3.1_0 r
r-matrix 1.2_6 r3.3.1_0 r
r-matrixmodels 0.4_1 r3.3.1_0 r
r-memoise 1.0.0 r3.3.1_0 r
r-mgcv 1.8_12 r3.3.1_0 r
r-mime 0.4 r3.3.1_0 r
r-minqa 1.2.4 r3.3.1_2 r
r-munsell 0.4.3 r3.3.1_0 r
r-nlme 3.1_128 r3.3.1_0 r
r-nloptr 1.0.4 r3.3.1_2 r
r-nnet 7.3_12 r3.3.1_0 r
r-nycflights13 0.2.0 r3.3.1_0 r
r-openssl 0.9.4 r3.3.1_0 r
r-pbdzmq 0.2_3 r3.3.1_0 r
r-pbkrtest 0.4_6 r3.3.1_0 r
r-plyr 1.8.4 r3.3.1_0 r
r-quadprog 1.5_5 r3.3.1_2 r
r-quantreg 5.26 r3.3.1_0 r
r-r6 2.1.2 r3.3.1_0 r
r-randomforest 4.6_12 r3.3.1_0 r
r-rcolorbrewer 1.1_2 1a r
r-rcpp 0.12.5 r3.3.1_0 r
r-rcpparmadillo 0.7.100.3.1 r3.3.1_0 r
r-rcppeigen 0.3.2.8.1 r3.3.1_0 r
r-rcurl 1.95_4.8 r3.3.1_0 r
r-recommended 3.3.1 r3.3.1_0 r
r-repr 0.7 r3.3.1_0 r
r-reshape2 1.4.1 r3.3.1_2 r
r-rmarkdown 0.9.6 r3.3.1_0 r
r-rpart 4.1_10 r3.3.1_0 r
r-rsqlite 1.0.0 r3.3.1_2 r
r-rstudioapi 0.5 r3.3.1_0 r
r-scales 0.4.0 r3.3.1_0 r
r-shiny 0.13.2 r3.3.1_0 r
r-sparsem 1.7 r3.3.1_0 r
r-spatial 7.3_11 r3.3.1_0 r
r-stringi 1.1.1 r3.3.1_0 r
r-stringr 1.0.0 r3.3.1_0 r
r-survival 2.39_4 r3.3.1_0 r
r-tibble 1.0 r3.3.1_0 r
r-tidyr 0.5.1 r3.3.1_0 r
r-timedate 3012.100 r3.3.1_2 r
r-tseries 0.10_35 r3.3.1_0 r
r-uuid 0.1_2 r3.3.1_0 r
r-whisker 0.3_2 r3.3.1_2 r
r-withr 1.0.2 r3.3.1_0 r
r-xtable 1.8_2 r3.3.1_0 r
r-yaml 2.1.13 r3.3.1_2 r
r-zoo 1.7_13 r3.3.1_0 r
rpy2 2.8.1 py35r3.3.1_1 r
readline 6.2 2
requests 2.10.0 py35_0
ruamel_yaml 0.11.14 py35_0
scikit-image 0.11.3 np110py35_0
seaborn 0.7.1 py35_0
setuptools 23.0.0 py35_0
sqlalchemy 1.0.13 py35_0
SQLAlchemy 1.0.14 <pip>
sqlite 3.13.0 0
tk 8.5.18 0
wheel 0.29.0 py35_0
xz 5.2.2 0
yaml 0.1.6 0
zlib 1.2.8 3
and on 7001472
[root@jupyter-eldap ~]# docker run --rm -it docker.io/jupyter/datascience-notebook:7001472997fa conda info
Current conda install:
platform : linux-64
conda version : 4.1.11
conda-env version : 2.5.2
conda-build version : not installed
python version : 3.5.2.final.0
requests version : 2.10.0
root environment : /opt/conda (writable)
default environment : /opt/conda
envs directories : /opt/conda/envs
package cache : /opt/conda/pkgs
channel URLs : https://conda.anaconda.org/r/linux-64/
https://conda.anaconda.org/r/noarch/
https://repo.continuum.io/pkgs/free/linux-64/
https://repo.continuum.io/pkgs/free/noarch/
https://repo.continuum.io/pkgs/pro/linux-64/
https://repo.continuum.io/pkgs/pro/noarch/
config file : /home/jovyan/.condarc
offline mode : False
is foreign system : False
root@jupyter-eldap ~]# docker run --rm -it docker.io/jupyter/datascience-notebook:7001472997fa conda list
# packages in environment at /opt/conda:
#
bokeh 0.11.1 py35_0
bzip2 1.0.6 3
cairo 1.12.18 6
cloudpickle 0.1.1 py35_0
conda 4.1.11 py35_0
conda-env 2.5.2 py35_0
backports.shutil_get_terminal_size 1.0.0 py35_0 conda-forge
blas 1.1 openblas conda-forge
cycler 0.10.0 py35_0 conda-forge
decorator 4.0.10 py35_0 conda-forge
dill 0.2.5 py35_0 conda-forge
entrypoints 0.2.2 py35_0 conda-forge
h5py 2.6.0 np110py35_6 conda-forge
hdf5 1.8.17 2 conda-forge
ipykernel 4.3.1 py35_1 conda-forge
ipython 5.0.0 py35_0 conda-forge
ipywidgets 5.1.5 py35_0 conda-forge
jinja2 2.8 py35_1 conda-forge
jsonschema 2.5.1 py35_0 conda-forge
jupyter_client 4.3.0 py35_0 conda-forge
jupyter_core 4.1.0 py35_0 conda-forge
libiconv 1.14 3 conda-forge
libpng 1.6.23 0 conda-forge
libsodium 1.0.10 0 conda-forge
markupsafe 0.23 py35_0 conda-forge
mistune 0.7.3 py35_0 conda-forge
mpmath 0.19 py35_0 conda-forge
nbconvert 4.2.0 py35_0 conda-forge
nbformat 4.0.1 py35_0 conda-forge
notebook 4.2.2 py35_0 conda-forge
numpy 1.10.4 py35_blas_openblas_202 [blas_openblas] conda-forge
openblas 0.2.18 4 conda-forge
pandas 0.18.1 np110py35_1 conda-forge
pexpect 4.2.0 py35_1 conda-forge
pickleshare 0.7.3 py35_0 conda-forge
pillow 3.2.0 py35_0 conda-forge
prompt_toolkit 1.0.4 py35_0 conda-forge
ptyprocess 0.5.1 py35_0 conda-forge
pygments 2.1.3 py35_0 conda-forge
pyparsing 2.1.7 py35_0 conda-forge
python-dateutil 2.5.3 py35_0 conda-forge
pytz 2016.6.1 py35_0 conda-forge
pyzmq 15.3.0 py35_1 conda-forge
scikit-learn 0.17.1 np110py35_blas_openblas_201 [blas_openblas] conda-forge
scipy 0.17.1 np110py35_blas_openblas_202 [blas_openblas] conda-forge
simplegeneric 0.8.1 py35_0 conda-forge
sip 4.18 py35_0 conda-forge
six 1.10.0 py35_0 conda-forge
statsmodels 0.6.1 np110py35_0 conda-forge
sympy 1.0 py35_0 conda-forge
terminado 0.6 py35_0 conda-forge
tornado 4.4.1 py35_1 conda-forge
traitlets 4.2.2 py35_0 conda-forge
wcwidth 0.1.7 py35_0 conda-forge
widgetsnbextension 1.2.6 py35_3 conda-forge
zeromq 4.1.5 0 conda-forge
curl 7.49.0 1
cython 0.23.5 py35_0
fastcache 1.0.2 py35_1
fontconfig 2.11.1 6
freetype 2.5.5 1
icu 54.1 0
ipython_genutils 0.1.0 py35_0
jbig 2.1 0
jpeg 8d 1
jupyterhub 0.5.0 <pip>
libffi 3.2.1 0
libgfortran 3.0.0 1
libtiff 4.0.6 2
libxml2 2.9.2 0
llvmlite 0.8.0 py35_0
matplotlib 1.5.1 np110py35_0
mkl 11.3.3 0
networkx 1.11 py35_0
numba 0.23.1 np110py35_0
numexpr 2.5.2 np110py35_1
openssl 1.0.2h 1
pamela 0.2.1 <pip>
patsy 0.4.1 py35_0
pcre 8.39 0
pip 8.1.2 py35_0
pixman 0.32.6 0
pycosat 0.6.1 py35_1
pycrypto 2.6.1 py35_4
python 3.5.2 0
pyyaml 3.11 py35_4
glib 2.43.0 2 r
gsl 1.16 1 r
harfbuzz 0.9.35 6 r
libgcc 4.8.5 1 r
ncurses 5.9 8 r
pango 1.36.8 3 r
r 3.3.1 r3.3.1_0 r
r-assertthat 0.1 r3.3.1_2 r
r-base 3.3.1 3 r
r-base64enc 0.1_3 r3.3.1_0 r
r-bh 1.60.0_2 r3.3.1_0 r
r-bitops 1.0_6 r3.3.1_2 r
r-boot 1.3_18 r3.3.1_0 r
r-car 2.1_2 r3.3.1_0 r
r-caret 6.0_70 r3.3.1_0 r
r-catools 1.17.1 1a r
r-class 7.3_14 r3.3.1_0 r
r-cluster 2.0.4 r3.3.1_0 r
r-codetools 0.2_14 r3.3.1_0 r
r-colorspace 1.2_6 r3.3.1_0 r
r-curl 0.9.7 r3.3.1_0 r
r-dbi 0.4_1 r3.3.1_0 r
r-devtools 1.11.1 r3.3.1_0 r
r-dichromat 2.0_0 1a r
r-digest 0.6.9 r3.3.1_0 r
r-dplyr 0.4.3 r3.3.1_0 r
r-evaluate 0.9 r3.3.1_0 r
r-foreach 1.4.3 r3.3.1_0 r
r-forecast 7.1 r3.3.1_0 r
r-foreign 0.8_66 r3.3.1_0 r
r-formatr 1.4 r3.3.1_0 r
r-fracdiff 1.4_2 r3.3.1_2 r
r-ggplot2 2.1.0 r3.3.1_0 r
r-git2r 0.15.0 r3.3.1_0 r
r-gtable 0.2.0 r3.3.1_0 r
r-highr 0.6 r3.3.1_0 r
r-htmltools 0.3.5 r3.3.1_0 r
r-httpuv 1.3.3 r3.3.1_0 r
r-httr 1.2.0 r3.3.1_0 r
r-irdisplay 0.3 r3.3.1_2 r
r-irkernel 0.6.1 r3.3.1_0 r
r-iterators 1.0.8 r3.3.1_0 r
r-jsonlite 0.9.22 r3.3.1_0 r
r-kernsmooth 2.23_15 r3.3.1_0 r
r-knitr 1.13 r3.3.1_0 r
r-labeling 0.3 r3.3.1_2 r
r-lattice 0.20_33 r3.3.1_0 r
r-lazyeval 0.2.0 r3.3.1_0 r
r-lme4 1.1_12 r3.3.1_0 r
r-magrittr 1.5 r3.3.1_2 r
r-markdown 0.7.7 r3.3.1_2 r
r-mass 7.3_45 r3.3.1_0 r
r-matrix 1.2_6 r3.3.1_0 r
r-matrixmodels 0.4_1 r3.3.1_0 r
r-memoise 1.0.0 r3.3.1_0 r
r-mgcv 1.8_12 r3.3.1_0 r
r-mime 0.4 r3.3.1_0 r
r-minqa 1.2.4 r3.3.1_2 r
r-munsell 0.4.3 r3.3.1_0 r
r-nlme 3.1_128 r3.3.1_0 r
r-nloptr 1.0.4 r3.3.1_2 r
r-nnet 7.3_12 r3.3.1_0 r
r-nycflights13 0.2.0 r3.3.1_0 r
r-openssl 0.9.4 r3.3.1_0 r
r-pbdzmq 0.2_3 r3.3.1_0 r
r-pbkrtest 0.4_6 r3.3.1_0 r
r-plyr 1.8.4 r3.3.1_0 r
r-quadprog 1.5_5 r3.3.1_2 r
r-quantreg 5.26 r3.3.1_0 r
r-r6 2.1.2 r3.3.1_0 r
r-randomforest 4.6_12 r3.3.1_0 r
r-rcolorbrewer 1.1_2 r3.3.1_2 r
r-rcpp 0.12.5 r3.3.1_0 r
r-rcpparmadillo 0.7.100.3.1 r3.3.1_0 r
r-rcppeigen 0.3.2.8.1 r3.3.1_0 r
r-rcurl 1.95_4.8 r3.3.1_0 r
r-recommended 3.3.1 r3.3.1_0 r
r-repr 0.7 r3.3.1_0 r
r-reshape2 1.4.1 r3.3.1_2 r
r-rmarkdown 0.9.6 r3.3.1_0 r
r-rpart 4.1_10 r3.3.1_0 r
r-rsqlite 1.0.0 r3.3.1_2 r
r-rstudioapi 0.5 r3.3.1_0 r
r-scales 0.4.0 r3.3.1_0 r
r-shiny 0.13.2 r3.3.1_0 r
r-sparsem 1.7 r3.3.1_0 r
r-spatial 7.3_11 r3.3.1_0 r
r-stringi 1.1.1 r3.3.1_0 r
r-stringr 1.0.0 r3.3.1_0 r
r-survival 2.39_4 r3.3.1_0 r
r-tibble 1.0 r3.3.1_0 r
r-tidyr 0.5.1 r3.3.1_0 r
r-timedate 3012.100 r3.3.1_2 r
r-tseries 0.10_35 r3.3.1_0 r
r-uuid 0.1_2 r3.3.1_0 r
r-whisker 0.3_2 r3.3.1_2 r
r-withr 1.0.2 r3.3.1_0 r
r-xtable 1.8_2 r3.3.1_0 r
r-yaml 2.1.13 r3.3.1_2 r
r-zoo 1.7_13 r3.3.1_0 r
rpy2 2.8.1 py35r3.3.1_1 r
readline 6.2 2
requests 2.10.0 py35_0
ruamel_yaml 0.11.14 py35_0
scikit-image 0.11.3 np110py35_0
seaborn 0.7.1 py35_0
setuptools 23.0.0 py35_0
sqlalchemy 1.0.13 py35_0
SQLAlchemy 1.0.14 <pip>
sqlite 3.13.0 0
tk 8.5.18 0
wheel 0.29.0 py35_0
xz 5.2.2 0
yaml 0.1.6 0
zlib 1.2.8 3
Besides the r-base version, there are several other packages that are changed between the two commits.
Since I do not think the other upgraded packages are related to the graphics capabilities, I think r-rcolorbrewer is the most suspicious of breaking the plotting capability.
$ docker run --rm -it docker.io/jupyter/datascience-notebook:7001472997fa conda list > 7001472997fa.txt
$ docker run --rm -it docker.io/jupyter/datascience-notebook:54838ed4acb1 conda list > 54838ed4acb1.txt
$ diff 7001472997fa.txt 54838ed4acb1.txt
15,16d14
< h5py 2.6.0 np110py35_6 conda-forge
< hdf5 1.8.17 2 conda-forge
42c40
< pyparsing 2.1.7 py35_0 conda-forge
---
> pyparsing 2.1.1 py35_0 conda-forge
63a62,63
> h5py 2.5.0 np110py35_4
> hdf5 1.8.15.1 3
97c97
< r-base 3.3.1 3 r
---
> r-base 3.3.1 1 r
104c104
< r-catools 1.17.1 1a r
---
> r-catools 1.17.1 r3.3.1_2 r
160c160
< r-rcolorbrewer 1.1_2 r3.3.1_2 r
---
> r-rcolorbrewer 1.1_2 1a r
@takluyver any thoughts on what in the delta above might be causing capabilities() to now show jpeg, png, tiff, and cairo support all to be FALSE?
@jakirkham I tried removing the pin for libjpeg and installing jpeg9 from conda-forge, but to no avail. I get the same plot without title.
The change that broke plotting happened in r-base between 3.3.1-1 and 3.3.1-3.
As a workaround, conda install -c r r-base='3.3.1 1' gets things working again.
@mingwandroid do you have any thoughts on what might have changed between r-base 3.3.1 build number 1 (conda-archive/conda-recipes@601fd5d) and build number 3 (conda-archive/conda-recipes@dfe2d55) that could have flipped jpeg, png, and tiff capabilities from TRUE to FALSE as shown in #210 (comment)? I'm looking but not coming up with much.
On a clean install on latest ArchLinux my capabilities include all of those. Can you try the same and report back?
@mingwandroid, sure. Here's my bash history starting from a fresh docker debian container:
apt-get update
apt-get install wget
wget --quiet https://repo.continuum.io/miniconda/Miniconda3-4.1.11-Linux-x86_64.sh
apt-get install bzip2
bash Miniconda3-4.1.11-Linux-x86_64.sh -b
/root/miniconda3/bin/conda install jupyter
/root/miniconda3/bin/conda install -c r r-base
. /root/miniconda3/bin/activate
R --quiet -e 'capabilities()'
Here's the output from the last command:
(root) root@13ae5375c38c:/# R --quiet -e 'capabilities()'
> capabilities()
jpeg png tiff tcltk X11 aqua
FALSE FALSE FALSE TRUE FALSE FALSE
http/ftp sockets libxml fifo cledit iconv
TRUE TRUE TRUE TRUE FALSE TRUE
NLS profmem cairo ICU long.double libcurl
TRUE TRUE FALSE TRUE TRUE TRUE
I pushed a new image with the older r-base build for the time being. It's tagged 54552d36cf0a and is currently latest.
If someone else can try it, that would be great.
This issue is becoming quite large, so to keep the fire going 😄 or at least make note of it.
It looks like currently this issue appears when a user tries to plot. Although, come the next IRkernel release/version this issue will appear on the first execution of a code cell such as 1+1 and kill the kernel instantly.
It will happen because of the changes this past May IRkernel/IRkernel#340 when initing null devices in the shadow environment commits are here and here. This initialization originates when the executor gets initialized which happens when the kernel is initialized. This is not yet in the 0.6.1 release that is in docker stacks to clarify.
This was discovered when running docker_dev, which is running master in this same docker stacks environment.
cc'ing @flying-sheep as you are familiar with this or at least making sure you are aware.
For reference the error when running 1+1 on master was:
Error in .External2(C_X11, paste("png::", filename, sep = ""), g$width, :
unable to start device PNG
Calls: <Anonymous> ... evaluate -> dev.new -> do.call -> <Anonymous> -> ok_device
In addition: Warning message:
In ok_device(filename, ...) : unable to open connection to X11 display ''
Execution halted
In conclusion this is not a big issue yet, but will be later.
wait: when exactly will this occur?
AFAIK only when there is an R system so extremely broken that a very basic call to a non-interactive device like pdf or png fails, right?
or am i wrong?
@parente datascience-notebook:54552d36cf0a works for me. I can plot again!
@flying-sheep With irkernel 0.7 and conda r-base 3.3.1-3, I can crash the kernel by simply entering x <- 1 in a new notebook. The error on the console is the same as what @poplav stated above.
Switching back to r-base 3.3.1-1 resolves the issue. It seems that conda 3.3.1-3 build is the culprit again.
thanks for the help, all of you!
see ContinuumIO/anaconda-issues#1063 for more
@flying-sheep Thanks for filing this upstream.
I'm going to close this (massive) issue now to avoid giving the impression that plotting is still an issue here. When the anaconda issue closes upstream, we can unpin the R version here.
@parente, would it be possible for you to test a new preview build of r-base?
conda update -c r r-essentials
conda update -c rdonnellyr r-base
Thanks!
PR #324 unpins jpeg and updates to r-base 3.3.2 which does not suffer from the original problems.
Hi, I have exactly the same problem.
Hello,
I have problems creating charts in the jupyter notebook, sparmagic and R.
This is my set up:
R version 3.5.2 (2018-12-20)
Platform: x86_64-pc-linux-gnu (64-bit)
Running under: Db2 Warehouse
Matrix products: default
BLAS: /mnt/blumeta0/R-Install/lib64/R/lib/libRblas.so
LAPACK: /mnt/blumeta0/R-Install/lib64/R/lib/libRlapack.so
locale:
[1] LC_CTYPE=en_US.UTF-8 LC_NUMERIC=C
[3] LC_TIME=en_US.UTF-8 LC_COLLATE=en_US.UTF-8
[5] LC_MONETARY=en_US.UTF-8 LC_MESSAGES=en_US.UTF-8
[7] LC_PAPER=en_US.UTF-8 LC_NAME=C
[9] LC_ADDRESS=C LC_TELEPHONE=C
[11] LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C
attached base packages:
[1] stats graphics grDevices utils datasets methods base
loaded via a namespace (and not attached):
[1] compiler_3.5.2
Cairo packages:
cairo.x86_64 1.14.8-2.el7 @base
cairo-devel.x86_64 1.14.8-2.el7 @base
I use jupyter notebook with livy server and sparkmagic
After I installed cairo, I reinstalled R.
I used pip install everything
I invoke the notebook like this:
in the docker container:
jupyter notebook --ip=0.0.0.0 --allow-root
the steps:
options(bitmapType='cairo') --> output: SparkSession available as 'spark'.
library(ggplot2) --> output: null device 1
p <- ggplot(data.frame(x=1:5, y= 1:5, aes(x,y)) geom_point() ---> output: null device 1
p --> output: Error in (function = if (onefile) "Rplots.pdf" else "Rplots%3d.pdf"
cannot open file Rplots.pdf