Running a Jupyter notebook from a remote server
ljvmiranda921 opened this issue · comments
Written on 01/31/2018 08:29:43
URL: https://ljvmiranda921.github.io/notebook/2018/01/31/running-a-jupyter-notebook/
Comment written by A. on 06/26/2018 21:01:01
Hi. Thanks for the article. How do you make the function jpt() to work in the SSH terminal? Do you put a copy of configuration file in the server as well?
Comment written by Lj Miranda on 06/29/2018 05:01:07
Hi Tetsuo! Yes there's a copy of this config file in the remote server. :)
Comment written by Simran Singhal on 04/09/2019 04:40:08
Hi. Thanks for the article.
But, whenever I am running jupyter notebook its asking for the password. How can I resolve this issue?
Comment written by Lj Miranda on 04/09/2019 05:24:44
Hello, when Jupyter notebook is run in your terminal, it will show an API Key or something. You can use that as your password.
Comment written by Simran Singhal on 04/09/2019 06:39:52
I have tried it. But, it isn't working for me.
Comment written by Kaushik Pavani on 04/17/2019 23:06:12
In your remote server, execute:
$ jupyter notebook list
Currently running servers:
http://localhost:8889/?token=966e9c28cfec8e7d7d648b0bba971db2874ab95dc12d50a7Use the token id (966e9c28cfec8e7d7d648b0bba971db2874ab95dc12d50a7) to access the notebook.
Comment written by Sam Joshva on 06/20/2019 07:48:02
thanks so much for this tutorial, quick question how to fire your local jupyter notebook, after you setup the remote server, cause everytime I try to load my local jupyter notebook I get an acces denied error in browser
Comment written by Manu Goyal on 10/08/2019 18:17:34
Thank you so much for this tutorial. But when I am running jupyter notebook, it is picking up python modules from the local computer rather than remote server.
Comment written by Lj Miranda on 11/17/2019 10:26:45
Hi Sam! Ensure that when you load the notebook, you are passing the token in the URL! You can find it when you type jupyter notebook list. You can use the token ID as a way to access your notebook
Comment written by Lj Miranda on 11/17/2019 10:27:34
Hmm that's weird and it shouldn't happen. Are you sure you're connected to your remote server?
Comment written by Jude Nelson on 12/10/2019 21:27:29
Can i ask if you can connect to a remote Ubuntu server running jupyter notebooks from a local windows machine?
Comment written by John Billington on 01/11/2020 22:11:04
Quick and dirty just like I need! Thank you so much!
Comment written by Lj Miranda on 01/12/2020 04:08:05
Thanks I appreciate that!
Comment written by Lj Miranda on 01/12/2020 04:08:37
Hmm, not pretty sure since I haven't used Windows for a long time. I feel that you can find a solution when using the PuTTy software in Windows
Comment written by Kebei Jiang on 02/03/2020 22:21:48
Yes you can. The way I did it was to fire up a Ubuntu subsystem in Windows 10 and then everything follows the instruction naturally. BTW, very helpful post! Thanks! @ljmiranda:disqus
Comment written by Nani Samireddy on 02/23/2020 13:23:41
i am using a widows machine and i just want to practice python in jupyter notebook i have installed it and i am getting an error in the following picture please help me with this https://uploads.disquscdn.c...
Comment written by Kiara on 04/20/2020 11:58:31
Thank you for this post! I have a follow up issue though: I need to leave my notebook running for a couple of hours, so I am using screen to be able to leave it running and reattach later. The problem is that whenever I detach and close my local browser, the current cell stops running. If I leave the browser open, I can see the notebook saving the progress, just not through the output in the notebook but in the "saved seconds ago". How can I disconnect the ports but leave the notebook running and then reconnect later?
Comment written by Lj Miranda on 04/20/2020 14:26:41
Hmm, to be honest I am not really sure, I also experience the same thing. Can I confirm that even if you disconnected, the process is still running server-side? (It should be)
It's a bit of a hassle but what I usually do is periodically save whatever output I want to log to a file. If you're training a model, maybe it may be good to save outputs on a CSV so that you can check that instead. Bit of a stretch, but I think that's a limitation of this solution
Comment written by Lj Miranda on 04/20/2020 14:28:00
Have you solved this? It seems like an installation problem, have you setup your configs properly?
Comment written by Xabier on 04/24/2020 15:59:39
Thanks for the clear explanation! I've got a problem, though. When trying to import the python module sisl, this is not found and the usual ModuleNotFoundError: No module named 'sisl' message appears. This is strange, as I have this module properly installed in the remote host.
Comment written by Kiara on 04/25/2020 16:01:40
Thank you for replying! Yes, so basically I have a terminal open (from where I opened jupyter remotely) and a promt window (where I forwarded the port). When I detach from screen and disconnect in the terminal window, I can see the process still running server-side. The prompt (the second window) does not let me input anything, so I just close it.
I asked a colleague, apparently saving outputs is how everyone else does it. It just would have been nifty if it had worked with jupyter too :)
Comment written by Lj Miranda on 04/26/2020 00:31:19
My hunch is that this is a Python environment problem. How's Jupyter notebook set up in your remote host? Did you install it via conda or pip? I think sisl should be installed that way. You can either try:
pip3 install sisl
or
conda install sisl
Comment written by Xabier on 04/27/2020 16:30:46
You are right, it was an environment problem. Sisl was installed using conda in the remote host. Trying conda install the following error appears: The python kernel does not appear to be a conda environment. Please use %pip install instead As far as I understand this means jupyter was installed using pip. I thought both were part of the conda environment. Eventually, everything goes fine using pip to install sisl. Thanks again for your help!
Comment written by Azar Tamboli on 05/05/2020 10:31:17
Hi...
I am using kubernates..and I ran jupyter notebook command inside kubernate pod and also ran the remote command from client machine. and then went to browser and fire up localhost:YYYY but aksing me token and when I entered token it says Invalid credential...what could be the issue?
Comment written by Evelyn Rose on 06/18/2020 04:23:25
I too faced this. What I did then was copied the entire line jupyter had spit out along with the access token, changed the port to YYYY and launched that in my local browser. That worked for me.
Comment written by abhishekjain on 06/19/2020 13:36:22
localuser@localhost: ssh -N -f -L localhost:YYYY:localhost:XXXX remoteuser@remotehost - where do i run this command? I am using windows 10 on my local..
Comment written by 『Λהםא』✓ᵛᵉʳᶦᶠᶦᵉᵈ on 07/16/2020 04:47:46
You certainly do not need to have jupyter or python or anything installed on your client machine for this to work. Instaling them on your client computer will make things more complex.
Btw thanks for the great guide. I was doing SSH into Win 10 WSL and it took a day to make it working.
Comment written by mikstone on 07/25/2020 23:16:15
Thanks this was really useful!
Comment written by Paras Qadir on 08/14/2020 22:30:01
I am facing this issue: The kernel appears to have died. It will restart automatically.
Comment written by Lj Miranda on 08/19/2020 00:08:24
Thanks for taking point Evelyn Rose! Hope her answer helped you out Azar!
Comment written by Lj Miranda on 08/19/2020 00:08:58
Yup, can confirm you don't need jupyter in your client for this!
Thanks, glad it helped!
Comment written by Lj Miranda on 08/19/2020 00:10:06
Maybe the issue is on how Jupyter is being run on your server. Check if you installed it correctly!
Comment written by Dave Deriso on 08/19/2020 00:23:13
Love it!
Comment written by Lj Miranda on 08/19/2020 01:07:01
Thanks a lot, Dave!
Comment written by Ed Sponsler on 11/05/2020 08:12:11
Absolutely wonderful! I knew there was a way I did this years ago but couldn't piece it together. Thanks for putting this together. Works great!
Comment written by Lj Miranda on 11/06/2020 12:02:29
I appreciate it, Ed! Thanks a lot!
Comment written by Lj Miranda on 11/06/2020 12:02:39
Thank you mikstone!
Comment written by Lj Miranda on 11/06/2020 12:03:28
I think you can run this in your terminal. Not sure if this will work out of the blue in Windows. Maybe you can try in WSL?
Either way, if you're in Windows. I recommend just using puTTy to make ssh calls
Comment written by Dongji on 11/14/2020 04:08:19
hi, this is a great blog. I use your way and i can see a positive review from terminal as below: The Jupyter Notebook is running at:
[I 21:38:56.259 NotebookApp] http://localhost:8890/?token=9c7d5bae296b1dfde8f878d841b2f53a81827cc1160298
[I 21:38:56.259 NotebookApp] or http://127.0.0.1:8890/?token=9c7d5bae296b1dfde8f878d841b2f53a81827cc1160298
[I 21:38:56.259 NotebookApp] Use Control-C to stop this server and shut down all kernels (twice to skip confirmation).
[C 21:38:56.261 NotebookApp]
To access the notebook, open this file in a browser:
file:///home/xxxx/.local/share/jupyter/runtime/nbserver-329227-open.html
Or copy and paste one of these URLs:
http://localhost:8890/?token=9c7d5bae296b1dfde8f878d841b2f53a81827cc1160298
or http://127.0.0.1:8890/?token=9c7d5bae296b1dfde8f878d841b2f53a81827cc1160298
https://uploads.disquscdn.c...
however, once i want to access jupyter notebook from browser, localhost refused to connect. could you please let me know whats the issue? thank you again!
Comment written by Lj Miranda on 11/15/2020 22:07:48
Hi Dongji, thanks a lot!
When opening Jupyter Notebook in your local browser, you should use the port you forwarded to (not from. which i think in this case is 8890). What is the value of YYYY when you ran this command?
ssh -N -f -L localhost:YYYY:localhost:8890 remoteuser@remotehost
Comment written by Dongji on 11/25/2020 17:32:22
Hi, Lj
Very appreciate your reply, i have get me remote jupyter set up, i after the positive reviews from terminal, i use this "ss-N -f -L localhost:8887:localhost:8887 serverName" and this works for me.
Thank you again!
Comment written by Lj Miranda on 11/29/2020 14:15:42
Glad it helped!
Comment written by Andrew on 12/04/2020 12:02:52
Late response, but this is great! I was wondering whether it was also possible to do this through an intermediate server? (To access my work machine, I have to ssh in via a different server). Maybe using something like proxyjump?
Comment written by Karim on 12/06/2020 17:07:39
Hi,
This is an amazing post. Thank you.
I am now running Jupyter on a jetson nano gpu from my MacBook laptop using ssh. My only problem is that it can't see some packages contents when running on the mac. It can see it only if I use jupyter just on the jetson nano from the jetson nano, but when I do the remote thing, it is not working.
Any advice?
Karim
Comment written by Teodor on 12/17/2020 22:10:24
Hi
When running:
localuser@localhost: ssh -N -f -L localhost:YYYY:localhost:XXXX remoteuser@remotehost
i get
ubuntu@129.16.123.168: Permission denied (publickey)
Help please!
Comment written by Ramesh Babu on 01/26/2021 01:44:24
Thanks @ljmiranda:disqus . your post saved my studies, becuase Mac M1 failed to load tensorflow and I stumbled your post.
Your post is very valuable at this time because lot of people are using Mac M1 chip. I wrote a blog post in medium and gave all credit to your post ..
Thanks
Comment written by Olivier Dex on 01/26/2021 14:01:15
@ljmiranda:disqus what a great post ! It's clear, concise and has all the essential steps explained. A big thanks :)
Comment written by Lj Miranda on 01/28/2021 23:25:12
Hey Olivier, thanks a lot!
Comment written by Lj Miranda on 01/28/2021 23:25:45
Thank you Ramesh!
Hello and thank you for this post. Seeing that it is one of the top results in Google when you search "remote access jupyter notebooks", I would suggest to improve the article and make it even better by adding a section on ssh config port forwarding, like:
host notebooks
HostName host
user myuser
LocalForward YYYY:localhost:XXXX
Then you can simply do in the console ssh -Nf notebooks and be prompted for the password. You can also add your IdentityFile for automatic and other goodies in ~/.ssh/config!
Ofc this requires you to fix YYYY and XXXX, but for most cases I think people just forwards 8888 to 8889 or 8888 and doesn't need that much flexibility.
Hey @a-berg thanks for the advice, will definitely update this within the week!
HI,
Thank you for this cool tutorial.
However, I have a question, I am using jupyter-lab.
How can I show the list of files that are located on the left side (file browser) for my files that are in server ?
So far, it only shows the files that are on my local machine ?
Thanks
IG
I am trying to follow this instruction as suggested by @a-berg
host notebooks
HostName myserver.computer.ac.nz
user abc
LocalForward 8888:localhost:8888
when I run ssh -Nf notebooks, I have got the following error
line 4: Missing target argument
Could you give me some help ?
Thanks
Hi @ihgumilar in case you still haven't solved it, your LocalForward should be LocalForward 8888 localhost:8888 (the first : shouldn't be there).
Hi!
After running ssh -Nf on my local machine how do I start jupyter notebook of remote server on my local browser? localhost: \tree didn't work!
Whenever I run 5/6 notebooks simultaneously on the remote server, it seems to crash and I have to restart the server. The GPU is powerful enough to handle intensive tasks but not sure why multiple notebooks is a problem. Any ideas if this is because they are all running on the same port?
@Bhavana-Bhat Have you solved the issue?
Note that if you're doing this on a new Mac as your local computer, you may end up with some trouble using .bashrc. In such a case, add the same jptt helper function into .zshrc and and add brackets around {2} to account for the fact that zsh syntax is different than bash syntax.
https://stackoverflow.com/questions/70084309/is-the-syntax-for-local-port-forwarding-wrong-on-zsh
Reply to Comment written by Kiara on 04/20/2020 11:58:31
NOHUP (No hang up)
Ancient (Unix) solution to modern problem of disconnecting from a running process.
Back in the day analog phone lines with modems would get disconnected, so a UNIX command nohup (no hang up) was invented. Decades later there is a Linux version used with Jupyter!
"Start jupyter notebook on remote server
In the remote server terminal, run:
nohup jupyter notebook &
(the & sends the process to run in the background, so that the terminal window can be used)"
https://gist.github.com/33eyes/e1da2d78979dc059433849c466ff5996
or in our case
nohup jupyter notebook --no-browser --port=XXXX &
HTH
Jim
Reply to Comment written by Kiara on 04/20/2020 11:58:31
NOHUP (No hang up)
Ancient (Unix) solution to modern problem of disconnecting from a running process.
Back in the day analog phone lines with modems would get disconnected, so a UNIX command nohup (no hang up) was invented. Decades later there is a GNU Linux version used with Jupyter!
"Start jupyter notebook on remote server
In the remote server terminal, run:
nohup jupyter notebook &
(the & sends the process to run in the background, so that the terminal window can be used)" https://gist.github.com/33eyes/e1da2d78979dc059433849c466ff5996
or in our case
nohup jupyter notebook --no-browser --port=XXXX &
HTH Jim