Giters

marinebon / marinebon_app

docker for marinebon.app

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

marinebon_app

docker for https://marinebon.app

update 2023-03-22

migrated to Ben's ecoquants server on Contabo (154.53.57.44) (see ecoquants/server) off DigitalOcean (164.90.217.11)

Contents:

Server software

TODO: install server software

  • CKAN data catalog
  • ERDDAP data service
  • Custom APIS using Plumber
    1. Extract ERDDAP gridded data from polygon, as informed by mregions ID or uploaded shapefile, return mean, stdev
    2. Aggregate ERDDAP tabular data over space/time/taxa, eg from MARINe or CalCOFI

Server setup & docker run

Create server

Created droplet at https://digitalocean.com with ben@ecoquants.com (Google login):

  • Choose an image : Distributions : Marketplace :
    • Docker by DigitalOcean Version 19.03.12, OS Ubuntu 20.04
  • Choose a plan : Basic :
    • $40 /mo $0.060 /hour
    • 8 GB / 4 CPUs
    • 160 GB SSD disk
    • 5 TB transfer
  • Choose a datacenter region :
    • New York 1
  • Authentication :
    • SSH key
    • ssh-keygen:
      • Your identification has been saved in /Users/bbest/.ssh/id_rsa.
      • Your public key has been saved in /Users/bbest/.ssh/id_rsa.pub.
    • SSH key content: cat ~/.ssh/id_rsa.pub Name: macbook-pro-bbest
  • How many Droplets?
    • 1 Droplet
  • Choose a hostname :
    • marinebon.app

Note IP address generated with new droplet, in this case: 164.90.217.11.

Setup domain

  • Bought domain marinebon.app for $12/yr with account bdbest@gmail.com.

  • DNS matched to server IP 164.90.217.11 to domain marinebon.app via Google Domains, plus the following subdomains added under Custom resource records with:

  • Type: A, Data:164.90.217.11 and Name:

    • @
    • api
    • ckan
    • erddap
    • geo
    • rstudio
    • shiny

  • Name: www, Type: CNAME, Data:marinebon.app

Shell into server

ssh root@marinebon.app

Before using secure shell (SSH), you'll need to the public key of your machine and username added to the server. See How To Set Up SSH Keys | DigitalOcean.

The first time you login from a machine, you'll need to type yes:

The authenticity of host '192.241.142.34 (192.241.142.34)' can't be established.
ECDSA key fingerprint is SHA256:XisfpVW6doSV9WDZH5sHAcDpRK3LR3AQF2rNdfUeDOA.
Are you sure you want to continue connecting (yes/no/[fingerprint])? yes
Warning: Permanently added '192.241.142.34' (ECDSA) to the list of known hosts.

Check docker versions

Confirm that docker and docker-compose are installed:

docker --version
# Docker version 19.03.12, build 48a66213fe

docker-compose --version
# docker-compose version 1.22.0, build f46880fe

systemctl status docker

References:

Test docker

Reference:

docker run --name test-web -p 80:80 -d nginx

# confirm working
docker ps
curl http://localhost

returns:

<!DOCTYPE html>
<html>
<head>
<title>Welcome to nginx!</title>
<style>
    body {
        width: 35em;
        margin: 0 auto;
        font-family: Tahoma, Verdana, Arial, sans-serif;
    }
</style>
</head>
<body>
<h1>Welcome to nginx!</h1>
<p>If you see this page, the nginx web server is successfully installed and
working. Further configuration is required.</p>

<p>For online documentation and support please refer to
<a href="http://nginx.org/">nginx.org</a>.<br/>
Commercial support is available at
<a href="http://nginx.com/">nginx.com</a>.</p>

<p><em>Thank you for using nginx.</em></p>
</body>
</html>

You can also visit http://164.90.217.11 to see this in your browser.

Turn off:

docker stop test-web

Run docker-compose

References:

First, you will create the environment .env file to specify password and host:

# get latest docker-compose files
git clone https://github.com/marinebon/marinebon_app.git
cd marinebon_app

# set environment variables
echo 'PASSWORD=S3cr!tpw' > .env
echo 'HOST=marinebon.app' >> .env
cat .env

# launch
docker-compose up -d

# OR update
git pull; docker-compose up -d

# OR build if Dockerfile updated in subfolder
git pull; docker-compose up --build -d

# OR restart
docker-compose restart

# OR stop
docker-compose stop

Docker maintenance

Push docker image

This image could be pushed as a custom image, eg bdbest/rstudio-shiny:s4w, I docker-compose push to bdbest/rstudio-shiny:s4w | Docker Hub.

# login to docker hub
docker login --username=bdbest

# push updated image
docker-compose push

Develop on local host

Note setting of HOST to local vs marinebon.app:

# get latest docker-compose files
git clone https://github.com/marinebon/marinebon_app.git
cd ~/marinebon_app

# set environment variables
echo "PASSWORD=S3cr!tpw" > .env
echo "HOST=local" >> .env
cat .env

# launch
docker-compose up -d

# see all containers
docker ps -a

Then visit http://localhost or http://rstudio.localhost.

TODO: try migrating volumes in /var/lib/docker onto local machine.

Operate on all docker containers

# stop all running containers
docker stop $(docker ps -q)

# remove all containers
docker rm $(docker ps -aq)

# remove all image
docker rmi $(docker images -q)

# determine how volumes used
docker ps -a --filter volume=[volumename]

# remove all volumes
docker volume rm $(docker volume ls -q)

# remove all stopped containers
docker container prune

Inspect docker logs

To tail the logs from the Docker containers in realtime, run:

docker-compose logs -f

docker inspect rstudio-shiny

After docker setup

Download Github repos & setup symbolic links

In https://rstudio.marinebon.app Terminal as admin:

sudo chown -R admin /share

mkdir -p /share/data
mkdir -p /share/github

# git clone private repos
cd /share/github
git clone https://github.com/marinebon/www_marinebon.git


# link special folders to home folder
user=admin
ln -s /srv/shiny-server     /home/$user/shiny_apps
ln -s /var/log/shiny-server /home/$user/shiny_logs
ln -s /share                /home/$user/share
ln -s /share/github         /home/$user/github

Upload marinebon.app_pass.txt into /share/config/.

Add user

user=robertdcurrier
#user=tylarmurray
#user=ben
pass=secr3t

# delete user
#deluser $user
#docker exec rstudio userdel $user
#docker exec rstudio rm -rf /home/$user

# add user to host
sudo adduser $user --gecos 'First Last,RoomNumber,WorkPhone,HomePhone' --disabled-password
sh -c "echo $user:$pass | sudo chpasswd"
sudo usermod -aG sudo $user

# add user inside rstudio-shiny docker container from host
docker exec rstudio adduser $user --gecos 'First Last,RoomNumber,WorkPhone,HomePhone' --disabled-password
docker exec rstudio sh -c "echo $user:$pass | sudo chpasswd"

# setup (once) staff to be shared by admin, and default permissions 775
docker exec rstudio gpasswd -a admin staff
docker exec rstudio sh -c "echo 'umask 002' >> /etc/profile"

# setup (every user) primary group to staff
docker exec rstudio gpasswd -a $user staff
docker exec rstudio usermod -g staff $user

# check groups for user in container
docker exec rstudio groups $user

# setup symbolic links in home dir
docker exec rstudio bash -c \
  "ln -s /share                      /home/$user/share; \
   ln -s /share/data                 /home/$user/data; \
   ln -s /share/github               /home/$user/github; \
   ln -s /share/github/www_marinebon /home/$user/www; \
   ln -s /srv/shiny-server           /home/$user/shiny; \
   ln -s /var/log/shiny-server       /home/$user/shiny-logs"

Load spatial data into database

Since using docker postgis/postgis 12.3, try on rstudio Terminal:

ogrinfo --version
# GDAL 3.0.4, released 2020/01/28

This was for a different project, but useful for future reference:

passwd=`cat /share/config/marinebon.app_pass.txt`
echo $passwd
ogrinfo -ro PG:"host=postgis dbname=gis user=admin password=$passwd"

ogr2ogr -f "PostgreSQL" PG:"host=postgis dbname=gis user=admin password=$passwd" "/share/mpatlas_mpa.gpkg" -nln "mpa_mpa" # -append

About

docker for marinebon.app

Languages

Language:Shell 87.4%Language:Dockerfile 12.6%