scalingexcellence / scrapybook

Scrapy Book Code

Home Page:http://scrapybook.com/

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Could you please give a brief guide on how to setup the environment without vigrent.

lefeudelavie opened this issue · comments

Hi lookfwd,

I have seen in the readme , it's said after you add the "docker-compose.yml", so there is no need to install the Vagrant. But could you give me a brief guide on how to setup the env without Vagrant?

Sincerely,
Lefeudelavie

Theoretically - you install docker from here. Then you checkout the repo, you go to the root directory and do docker-compose up. This should easily bring up all the containers. Then one should be able to ssh to them with ssh -i insecure_key root@hostname_of_the_container. @normangilmore - what was the easiest way to ssh to containers?

To get into containers, there are two options:

The mildly tedious way is docker ps to get the hex container id, then docker exec -it LONGHEXID bash, but the easy way is docker-compose exec web bash or docker-compose exec spark bash, substituting a container name listed as a services key in docker-compose.yml. The advantage of the first technique is it works from any directory, the second technique requires you to be in or under the directory containing docker-compose.yml.

Note that docker containers are not running sshd as a general rule, so you can't ssh into them. Containers are process wrappers, not virtual machines with a full set of services.

Hope this helps!

If and only if you are running Docker Toolbox (which is different than Docker for Mac or Docker for PC!), then you need to first ssh into the VirtualBox VM that docker is running on your behalf.

ssh docker@$(docker-machine ip)
Your ports will be exposed on the internal docker-machine ip, which is usually something like 192.168.99.100.

If you want your ports exposed on localhost, you can do something like this:
ssh -L 80:$(docker-machine ip):80 -L 8080:$(docker-machine ip):8080 docker@$(docker-machine ip), adjusting source and dest ports as needed. You will get a password request, this is for the docker-machine user account on the VM, the password is tcuser. Just leave that window open, and your ports will be tunneled so you can access with http://localhost:8080 or whatever port you mapped to, instead of http://192.168.99.100:8080/

@lookfwd @normangilmore
Thanks for your prompt response.

I have tried to run the "docker-compose up" in the root, but unfortunately, I get an error as follows:
D:\code\scrpaybook\scrapybook>docker-compose up
Traceback (most recent call last):
File "docker-compose", line 3, in
File "compose\cli\main.py", line 68, in main
File "compose\cli\main.py", line 118, in perform_command
File "compose\cli\main.py", line 928, in up
File "compose\project.py", line 414, in up
File "compose\project.py", line 640, in warn_for_swarm_mode
File "site-packages\docker\api\daemon.py", line 90, in info
File "site-packages\docker\utils\decorators.py", line 46, in inner
File "site-packages\docker\api\client.py", line 189, in _get
File "site-packages\requests\sessions.py", line 488, in get
File "site-packages\requests\sessions.py", line 475, in request
File "site-packages\requests\sessions.py", line 596, in send
File "site-packages\requests\adapters.py", line 423, in send
File "site-packages\requests\packages\urllib3\connectionpool.py", line 595, in urlopen
File "site-packages\requests\packages\urllib3\connectionpool.py", line 363, in _make_request
File "httplib.py", line 1042, in request
File "httplib.py", line 1082, in _send_request
File "httplib.py", line 1038, in endheaders
File "httplib.py", line 882, in _send_output
File "httplib.py", line 844, in send
File "site-packages\docker\transport\npipeconn.py", line 31, in connect
File "site-packages\docker\transport\npipesocket.py", line 22, in wrapped
File "site-packages\docker\transport\npipesocket.py", line 50, in connect
pywintypes.error: (2, 'WaitNamedPipe', 'The system cannot find the file specified.')
Failed to execute script docker-compose

My install steps are:
1, download the dockertoolbox and install the docker and virtualbox.
2, git clone the scrapybook code to a dir.
3, run "docker-compose up" in above-mentioned dir which have downloaded the code. And then the error .

Thanks!

I just installed DockerToolbox and Virtualbox on Windows 7, and after I ran VirtualBox.exe -msiparams NETWORKTYPE=NDIS5 to downgrade the network driver for Windows 7, I was able to docker-compose up in this repo.

Are you in the Docker Quickstart terminal?

What do you get for python --version?

Also, are you able to run docker run hello-world?

@normangilmore
Thanks for your response. I am able to run "docker run hello-world" on my PC (windows 7 64 bits).
I have installed two python, one is 2.7, another is 3.5.

python --version output as following:
D:\code\scrapybook>python --version
Python 2.7.13

@normangilmore
And I also have tried to disable path for Python 2.7.13 and change it to Python 3.5, hence I can get Python 3.5 when run 'python --version'. But get the same error when run 'docker-compose up'

docker quickstart icon
docker quickstart terminal

If you are in the Docker Quickstart Terminal, the proper env variables should be set, as shown in image. But you can set them with the command shown as well.

Enter this command to set them, then try it.
eval $("C:\Program Files\Docker Toolbox\docker-machine.exe" env)

I found the suggestion here, shipping-docker/php-app#19
docs here:
https://docs.docker.com/machine/reference/env/

docker-compose may also want the newer Python I think.

@normangilmore
I just checked the docker as you mentioned:
image
But this didn't solve my problem. I think the problem is caused by some other reason. I am trying to reinstall related sw to see if it work.

commented

Hi, I don't want to establish a virtual envirment. And ,what I care about is, how to output the results of the code programms? I only want to see the results in IDLE or Ipython shell or in a file. But I really don't know how to do, what code should I append in the programm.

The box 'lookfwd/scrapybook' could not be found or
could not be accessed in the remote catalog. If this is a private
box on HashiCorp's Vagrant Cloud, please verify you're logged in via
vagrant login. Also, please double-check the name. The expanded
URL and error message are shown below:

URL: ["https://vagrantcloud.com/lookfwd/scrapybook"]
Error: Failed to connect to vagrantcloud.com port 443: Timed out