How to Build Docker Containers

Question

How to Build Docker Containers

ark- opened this issue 4 years ago · comments

Hi. I had to work through multiple issues and the source code to get it working in my instance so I would like to pay it forward.

Build

git clone https://github.com/RD17/ambar.git
cd ~/ambar/FrontEnd
If compiling this on a different machine than what you're deploying on then before performing compile export PORT={YOUR_WEB_FRONTEND_PORT) and export FRONTEND_HOST={YOUR_HOST_IP}. I used 80 and 192.168.1.1.
node npm install
npm run compile
docker build . -t frontend
cd ~/ambar/WebApi
docker build . -t webapi
cd ~/ambar/ServiceApi
docker build . -t serviceapi
cd ~/ambar/Redis
docker build . -t redis
cd ~/ambar/Rabbit
docker build . -t rabbit
cd ~/ambar/Pipeline
docker build . -t pipeline0
cd ~/ambar/LocalCrawler
docker build . -t localcrawler
cd ~/ambar/MongoDB
docker build . -t mongodb
cd ~/ambar/ElasticSearch
docker build . -t elasticsearch
Set elasticsearch docker host settings, these are referenced here

sysctl -w vm.max_map_count=262144
sysctl -w net.ipv4.ip_local_port_range="15000 61000"
sysctl -w net.ipv4.tcp_fin_timeout=30
sysctl -w net.core.somaxconn=1024
sysctl -w net.core.netdev_max_backlog=2000
sysctl -w net.ipv4.tcp_max_syn_backlog=2048
sysctl -w vm.overcommit_memory=1

Docker Compose

My final docker compose:

version: "2.1"
networks:
  internal_network:
services:      
  db:
    restart: always
    networks:
      - internal_network
    build: ./MongoDB
    environment:
      - cacheSizeGB=2
    volumes:
      - ~/ambar/mapped_vols/db:/data/db
    expose:
      - "27017"   
  es:
    restart: always
    networks:
      - internal_network
    build: ./ElasticSearch
    expose:
      - "9200" 
    environment:
      - cluster.name=ambar-es
      - ES_JAVA_OPTS=-Xms2g -Xmx2g
      - max_map_count=262144
    ulimits:
      memlock:
        soft: -1
        hard: -1  
      nofile:
        soft: 65536
        hard: 65536
    cap_add:
      - IPC_LOCK
    volumes:
      - ~/ambar/mapped_vols/es:/usr/share/elasticsearch/data
  rabbit:
    restart: always
    networks:
      - internal_network
    build: ./Rabbit
    hostname: rabbit
    expose:
      - "15672"
      - "5672"   
    volumes:
      - ~/ambar/mapped_vols/rabbit:/var/lib/rabbitmq
  redis:
    restart: always
    sysctls:
      - net.core.somaxconn=1024
    networks:
      - internal_network
    build: ./Redis
    expose:
      - "6379" 
  serviceapi:
    depends_on:
      redis:
        condition: service_healthy
      rabbit:
        condition: service_healthy
      es:
        condition: service_healthy
      db:
        condition: service_healthy
    restart: always
    networks:
      - internal_network
    build: ./ServiceApi
    expose:
      - "8081"  
    ports:
      - "8081:8081"
    environment:            
      - mongoDbUrl=mongodb://db:27017/ambar_data
      - elasticSearchUrl=http://es:9200
      - redisHost=redis
      - redisPort=6379
      - rabbitHost=amqp://rabbit
      - langAnalyzer=ambar_en
  webapi:
    depends_on:
      serviceapi:
        condition: service_healthy
    restart: always
    networks:
      - internal_network
    build: ./WebApi
    expose:
      - "8080"
    ports:
      - "8080:8080"
    environment:            
      - uiLang=en
      - mongoDbUrl=mongodb://db:27017/ambar_data
      - elasticSearchUrl=http://es:9200
      - redisHost=redis
      - redisPort=6379
      - serviceApiUrl=http://serviceapi:8081
      - rabbitHost=amqp://rabbit  
  frontend:
    depends_on:
      webapi:
        condition: service_healthy
    build: ./FrontEnd
    restart: always
    networks:
      - internal_network
    ports:
      - "82:80"
    expose:
      - "82"
    environment:
      - api=http://192.168.1.1:8080  
  pipeline0:
    depends_on: 
      serviceapi: 
        condition: service_healthy                            
    build: ./Pipeline
    restart: always
    networks:
      - internal_network  
    environment:                
      - id=0
      - api_url=http://serviceapi:8081
      - rabbit_host=amqp://rabbit
  my-docs:
    depends_on: 
      serviceapi: 
        condition: service_healthy 
    build: ./LocalCrawler
    restart: always
    networks:
      - internal_network
    expose:
      - "8082"
    environment:      
      - name=my-docs
      - apiUrl=http://serviceapi:8081
    volumes:
      - ~/Documents:/usr/data

The differences between my docker-compose.yml and those listed https://ambar.cloud/docs/installation-docker/ are:

I performed the modifications suggested on this page.
I switched image attributes to build as I have built these locally
I added - max_map_count=262144 to elasticsearch environment
I changed the external frontend port to 82 as per step 3 above
I added - apiUrl=http://serviceapi:8081 to the crawler as it was polling the wrong port without this

Finally, to get it going:

cd ~/ambarr && docker-compose up -d

Sigi · Answer 1 · Sun Jul 05 2020 00:25:43 GMT+0800 (China Standard Time)

I hacked together a multi stage dockerfile for the Frontend. With it, you don't need to install anything on your host to build it:

I think it works, but i hacked it together and wanted to share it before i forget about it.

Btw. it doesn't build with nodejs 12 and there are plenty of warnings and deprecations for the used libs. So be careful if you put out the Frontend without protection.

FROM ubuntu:20.04 as build

RUN apt update && apt install -y curl sudo

RUN curl -sL https://deb.nodesource.com/setup_10.x | sudo -E bash -

RUN apt update && apt install --no-install-recommends --no-install-suggests -y nodejs python2.7 build-essential

WORKDIR /dist

COPY . .
 
RUN npm install && npm run compile



FROM nginx:latest

RUN apt-get update && apt-get install --no-install-recommends --no-install-suggests -y curl

# Set a timezone
ENV TZ=UTC
RUN ln -snf /usr/share/zoneinfo/$TZ /etc/localtime && echo $TZ > /etc/timezone

COPY default /etc/nginx/conf.d/default.conf
COPY --from=build /dist/dist /usr/share/nginx/html

CMD echo $api > /usr/share/nginx/html/apiUrl.txt && nginx -g "daemon off;"

HEALTHCHECK --interval=5s --timeout=30s --retries=50 \
  CMD curl -f localhost:80 || exit 1

kitsune242 · Answer 2 · Fri Jul 17 2020 03:40:04 GMT+0800 (China Standard Time)

THX
i build all but when docker start i have infinite loading screen and white board on localhost:82 in logs i dont see any errors. Anyone have similar problem ?

ark- · Answer 3 · Fri Jul 17 2020 19:44:29 GMT+0800 (China Standard Time)

I had this same issue. Take a look at step 3

If compiling this on a different machine than what you're deploying on then before performing compile export PORT={YOUR_WEB_FRONTEND_PORT) and export FRONTEND_HOST={YOUR_HOST_IP}. I used 80 and 192.168.1.1.

If you compiling in a docker on the same machine (as I was) you'll also need to do this. Try setting these variables and see if this helps.

kitsune242 · Answer 4 · Fri Jul 17 2020 21:13:03 GMT+0800 (China Standard Time)

Unfortunately, it's still the same.
its look it connect to 172.17.0.2:3000 and cant connect to backend. I use this Export command i rebuild docker but still have this problem.
I may use them incorrectly or something

Marcelo Lucas da Silva · Answer 5 · Wed Jul 29 2020 04:42:20 GMT+0800 (China Standard Time)

Instead of:

npm install && npm run compile

Try:

npm install && npm run deploy:prd

It worked for me.
It generated the files in ./FrontEnd/dist without fixed IP and Port.

Deyraka · Answer 6 · Thu Jul 30 2020 18:18:47 GMT+0800 (China Standard Time)

THX
i build all but when docker start i have infinite loading screen and white board on localhost:82 in logs i dont see any errors. Anyone have similar problem ?

I've following the instruction above. Unfortunately, I face the same issue.
there are no problem while docker-compose up. But, when I open {ipAddress}:82 it's only show white blank page with "Loading..." title
What did I miss?

ark- · Answer 7 · Thu Jul 30 2020 21:36:55 GMT+0800 (China Standard Time)

Unfortunately I cannot test as I have decided to stop using Ambar.

This is because 9 containers for document search is too many. Also ElasticSearch image seemed to always be using the CPU. I think Ambar was overkill for my purposes (home document scan search).

iddqd-dev · Answer 8 · Fri Jul 31 2020 19:05:01 GMT+0800 (China Standard Time)

Unfortunately I cannot test as I have decided to stop using Ambar.

This is because 9 containers for document search is too many. Also ElasticSearch image seemed to always be using the CPU. I think Ambar was overkill for my purposes (home document scan search).

Of course, windows indexing does a great job of searching through less than a million documents.

Deyraka · Answer 9 · Mon Aug 03 2020 21:14:16 GMT+0800 (China Standard Time)

THX
i build all but when docker start i have infinite loading screen and white board on localhost:82 in logs i dont see any errors. Anyone have similar problem ?

I've following the instruction above. Unfortunately, I face the same issue.
there are no problem while docker-compose up. But, when I open {ipAddress}:82 it's only show white blank page with "Loading..." title
What did I miss?

Hi, I think I found solution for my problem. Based on my trial in the last two days, I think the problem is the network config in docker-compose.yml.
When I tried the scenario to expose all service port to the host, frontend page appeared as it should. But, not when all service tried to connect each other via docker network.
It's good, finally the frontend page could be accessed. But, I'm sure that not a good choice to expose all the service port to the host which means our server would be more vulnerable from attacks.
Communication via docker network still being the best practice.
And then, I tried to modify docker-compose.yml by specifying network driver : bridge.

Voila, everything works good now.
Hopefully, this will help everyone who face the same issue

ropheefd · Answer 10 · Mon Aug 10 2020 22:04:24 GMT+0800 (China Standard Time)

THX
i build all but when docker start i have infinite loading screen and white board on localhost:82 in logs i dont see any errors. Anyone have similar problem ?

I've following the instruction above. Unfortunately, I face the same issue.
there are no problem while docker-compose up. But, when I open {ipAddress}:82 it's only show white blank page with "Loading..." title
What did I miss?

Hi, I think I found solution for my problem. Based on my trial in the last two days, I think the problem is the network config in docker-compose.yml.
When I tried the scenario to expose all service port to the host, frontend page appeared as it should. But, not when all service tried to connect each other via docker network.
It's good, finally the frontend page could be accessed. But, I'm sure that not a good choice to expose all the service port to the host which means our server would be more vulnerable from attacks.
Communication via docker network still being the best practice.
And then, I tried to modify docker-compose.yml by specifying network driver : bridge.

Voila, everything works good now.
Hopefully, this will help everyone who face the same issue

I tried modifying docker-compose.yml with network driver: bridge, but I'm getting an error. Can you show your final docker-compose as I'm having the same issue as you are?

stale · Answer 11 · Thu Aug 27 2020 09:29:30 GMT+0800 (China Standard Time)

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.