Please consider donating above to support the QNAP Unofficial Discord.

Benefits

Community Supporter role in Discord: Priority assistance with any issues.
Access to my private Gitlab repository of existing Docker Swarm recipes
Request & Vote on new recipes to publish
Read access to my personal eBook library.
Access to and advice on new properties for QNAP Unofficial: (i.e. Subreddit, Blog, Wiki, FAQs, CI/CD, Community Docker images, etc.)

QNAP Docker Swarm Setup

A guide for configuring the docker swarm stack on QNAP devices with Container Station

Preparation

Ports 80, 443, and 8080 must be unused by your NAS. By default, QTS assigns ports 8080 and 443 as the default HTTP and HTTPS ports for the QNAP Web Admin Console, and assigns 80 as the default HTTP port for the native "Web Server" application. Each of these must be modified to proceed with this guide. Modify these ports as follows to ensure there will be no port conflicts with docker stacks:
- Change default System ports: In QNAP GUI, General Settings change the default HTTP port to 8880, and the default HTTPS port to 8443.
- Change default Web Application ports: In QNAP GUI, General Settings, Applications, Web Server, change the default HTTP port to 9880, and the default HTTPS port to 9443.
- Unless currently in use, consider disabling both the Web Server and MySQL applications in the QNAP GUI Settings.
Ports 80 and 443 must be forwarded from your router to your NAS. This is possible using UPNP in the QNAP GUI, but this is not recommended. Instead, disable UPNP at the router and manually forward ports 80 and 443 to your NAS.

In sum:

QTS Default System port settings:

HTTP: 8880
HTTPS: 8443

QTS Web Server application ports to be modified:

HTTP: 9880
HTTPS: 9443

Steps:

Backup what you have running now (if you don't have anything running yet, skip to Step 3 or 5)
Shutdown and remove all Containers:
- Open terminal and run docker system prune
- Run docker network prune for good measure
- Run docker swarm leave --force (just to be sure you don't have a swarm left hanging around)
Remove Container Station:
- In App Center, click the dropdown for Container Station and choose Remove
- In Control Panel, Shared Folders, check the box next to the Container shared folder and click "Remove" (example)
- In the pop-up box, check "Also delete the data" and click "Yes" (example)
Reboot NAS (example)
Install Container Station and launch once installed, creating the Container folder suggested when opening
Create a new user called dockeruser
Create the following folder shares and give dockeruser Read/Write permissions:
- /share/appdata - Here we will add a folder . This is where your application files live... libraries, artifacts, internal application configuration, etc. Think of this directory much like a combination of C:/Windows/Program Files and C:\Users\<UserName>/AppData in Windows.
- /share/appdata/config - Here we will also add a folder . Inside this structure, we will keep our actual stack_name.yml files and any other necessary config files used to configure the docker stacks and images we want to run. This folder makes an excellent GitHub repository for this reason.
- /share/runtime - This is a shared folder on a volume that does not get backed up. It is where living DB files and transcode files reside, so it would appreciate running on the fastest storage group you have or in cache mode or in Qtier (if you use it). Think of this like the C:\Temp\ in Windows.
Run id dockeruser in terminal and note the uid and gid
Run docker network ls. The networks should match the following (except the generated NETWORK ID):

NETWORK ID          NAME                DRIVER              SCOPE
XXXXXXXXXXXX        bridge              bridge              local
XXXXXXXXXXXX        host                host                local
XXXXXXXXXXXX        none                null                local

Run docker swarm init --advertise-addr <YOUR NAS IP HERE> - Use YOUR nas IP
Checkpoint: Run docker network ls. Does the list of networks contain one named docker_gwbridge? The networks should match the following (except the generated NETWORK ID):

NETWORK ID          NAME                   DRIVER              SCOPE
XXXXXXXXXXXX        bridge                 bridge              local
XXXXXXXXXXXX        docker_gwbridge        bridge              local
XXXXXXXXXXXX        host                   host                local
XXXXXXXXXXXX        ingress                overlay             swarm
XXXXXXXXXXXX        none                   null                local

Important: If your configuration is lacking a docker_gwbridge or differs from this list, please contact someone on the QNAP Unofficial Discord (ideally in the #docker-stack channel). Do not proceed beyond this point unless your configuration matches the one above, unless you embrace pain and failure and love very complicated problems that could be QNAP's fault.

Run docker network create --driver=overlay --subnet=172.1.1.0/22 --attachable traefik_public
Install the entware-std package from the third-party QNAP Club repository. This is necessary in order to setup the shortcuts/aliases in Steps 18 & 19 by editing a permanent profile.
- The preferred way to do this is to add the QNAP Club Repository to the App Center. Follow the walkthrough instructions here. Note that I use the English translation of the QNAP Club website, but you may change languages (and urls) in the upper right language dropdown.
- If you don't need the walkthrough, add the repository. (For English, go to App Center, Settings, App Repository, Add, https://www.qnapclub.eu/en/repo.xml).
- If you have trouble locating the correct package below, the correct description begins entware-3x and entware-ng merged to become entware. The working link (as of publication) is here: https://www.qnapclub.eu/en/qpkg/556. If you cannot add the QNAP Club store to the App Center, you may manually download the qpkg file from that link and use it to install manually via the App Center, "Install Manually" button. This is not preferred as QNAP cannot check for and notify you of updates to the package.
- Search for entware-std and install that package.

Important: DO NOT CHOOSE either the entware-ng or entware-3x-std packages. These have merged and been superceded by entware-std.

Open Control Panel, Privilege, Shared Folders

Create one shared folder named

Run mkdir -p /share/appdata/traefik
Run mkdir -p /share/appdata/config/traefik
Run mkdir -p /share/runtime/traefik
Install nano or vi, whichever you are more comfortable with (e.g. Run opkg install nano or opkg install vim)
Run nano /opt/etc/profile (or vi /opt/etc/profile if that is your thing)
Add the following lines to the end of the file and save

dsd(){
  if [[ $1 = "-default" ]]; then
      list=( traefik portainer docker-cleanup ouroboros nextcloud privatebin ); 
    else
      list="$@"
  fi
  for i in "${list[@]}"; do
    docker stack deploy "$i" -c /share/appdata/config/"$i"/"$i".yml
  done
  unset list IFS
}
dsr(){
  if [[ $1 = "-all" ]]; then
      IFS=$'\n'; 
      list=( $(docker stack ls --format {{.Name}}) ); 
    else
      list="$@"
  fi
  for i in "${list[@]}"; do
    docker stack rm "$i"
  done
  unset list IFS
}
bounce(){
  if [[ $1 = "-all" ]]; then
      IFS=$'\n'; 
      list=( $(docker stack ls --format {{.Name}}) ); 
    else
      list="$@"
  fi
  for i in "${list[@]}"; do
    docker stack rm "$i"
  done
  for i in "${list[@]}"; do
    while [ "$(docker service ls --filter label=com.docker.stack.namespace=$i -q)" ] || [ "$(docker network ls --filter label=com.docker.stack.namespace=$i -q)" ]; do sleep 1; done
  done
  for i in "${list[@]}"; do
    docker stack deploy "$i" -c /share/appdata/config/"$i"/"$i".yml
  done
  unset list IFS
}
dcu(){
 docker-compose -f /share/appdata/config/"$1"/"$1".yml up -d
}
dcd(){
 docker-compose -f /share/appdata/config/"$1"/"$1".yml down
}
dfc() {
	bash /share/appdata/scripts/folder_setup.sh "$1" "$2" "$3" "$4" "$5" "$6" "$7" "$8" "$9"
}
dup() {
	bash /share/appdata/scripts/restart_stack.sh
}
dsp() {
	bash docker system prune -f
}
dsrms() {
	bash /share/appdata/scripts/remove_stack.sh
}
dss() {
	bash /share/appdata/scripts/setup_stack.sh
}

Remember these shortcut names:

dsd deploys a single stack - e.g. dsd traefik
dsr removes a single stack - e.g. dsr traefik
bounce removes a single stack and recreates it - e.g. bounce traefik
dfc creates the folder structure for a single (or multiple) stack. If you want to setup multiple stack folders use dfc plex ombi PiHole (up to 9 stacks at a time). Simplest example: e.g. dfc plex would create:
- /share/appdata/plex
- /share/appdata/config/plex
- /share/runtime/plex
dup starts existing stacks declared in /share/appdata/scripts/restart_stack.sh
dsp prunes the docker system. Any containers or networks not running will be removed -e.g. dsp
dsrms will remove all stacks, prune the docker system, remove any overlay networks, and leave the swarm - e.g. dsrms (use with care!)
dss will create a new swarm, create a new overlay network, start all stacks declared in /share/appdata/scripts/setup_stack.sh

** See below for scripts that need to be created and added to /share/appdata/scripts folder

NOTE: You will need to restart your ssh or cli session in order to make the profile changes effective.

NOTE: If you use a Windows client to save the profile (or the scripts), they will be saved with CR LF and will error. Please set the file format to UNIX (LF) in order for the profile and scripts to work correctly.

Edit traefik.env and put your cloudflare email and GLOBAL API KEY in lines 7&8 (If you are not using cloudflare you will need to check with the Traefik documentation to add the correct environment settings to your traefik.env file)
Edit traefik.yml and traefik.toml to include your domain name
Add the provided 3 traefik files to /share/appdata/config/traefik (.yml, .toml, .env)
Create and correctly set permissions on the acme.json file:

rm /share/appdata/config/traefik/acme.json
touch /share/appdata/config/traefik/acme.json
chmod 600 /share/appdata/config/traefik/acme.json

Check traefik.<yourdomain.com> resolves to your WAN IP (Run ping traefik.<yourdomain.com> - Press ctrl+c to stop the ping)
Run dsd traefik to start the traefik container
Follow ForwardAuth Setup Steps below
Enjoy Traefik and add more containers.

ForwardAuth Setup Steps

Go to https://auth0.com
Sign in or register an account
Note Tenant Domain provided by Auth0
Login or create an account with https://github.com
Go to Settings -> Developer Settings - OAuth Apps
Create a new app (call it something to recognize it is linked to Auth0)
Note the client Id and Secret
Add homepage URL as https://<yourauth0accounthere>.auth0.com/
Add authorization callback URL as https://<yourauth0accounthere>.auth0.com/login/callback
Go back to Auth0
Go to Connections -> Social
Select Github and enter in your Github app ClientID and secret Credentials - NOTE: ENSURE Attribute "Email Address" is ticked
Create an application on Auth0 (regular web app)
Use the Auth0 clientID and Client Secret in your application.yaml file
Make sure to specify POST method of token endpoint authentication (Drop down box)
Enter in your Callback URL (https://<service>.<domain>/signin & https://<service>.<domain>/oauth/signin)

For an entire domain, the values should look like this example:

Enter your origin URL (https://<your URL here>) and save changes
Go to Users & Roles and Create a user with a real email address. You will use this later so remember it
Click on Rules -> Whitelist
Enter in your email address into the whitelist field (e.g. Line 8 "const whitelist = [ '<your email here>']; //authorized users")
Open ssh and dsr traefik, wait 10 seconds and dsd traefik
Wait 30 seconds and then launch https://traefik.<yourdomainhere>
Enter Auth0 authentication login to reach traefik dashboard

Scripts Setup

Please create these scripts and save them to /share/appdata/scripts if you want to use the cli shortcuts we created in earlier steps. NOTE: setup_stack.sh requires you to add your NAS IP for it to work. All the stack scripts (xxx_stack.sh) require you to edit the stacks list to match your setup. If you do not edit them they will fail to deploy the stacks you don't have .. nothing blows up, no bunnies die, just a big pile of nothingness in your swarm.

NOTE: Please ensure you save these files in UNIX (LF) format. Windows (CR LF) format will break these scripts. If you are a Windows user, please download the files in the scripts folder above.

folder_setup.sh

#!/bin/bash
# Script to create gkoerk's famously awesome folder structure for stacks

# Set folder paths here (you should not need to change these, but if you do, it will save a load of typing)
appdata=/share/appdata
config=/share/appdata/config
runtime=/share/runtime

# Help message for script
helpFunction() {
    echo ""
    echo "Usage: $0 -f <folder name>"
    echo -f "-f name of folder(s) you wish to add. For more than one folder, use -f <folder name 1> <folder name 2> ... <folder name 9>. You can have 9 folder names in a single command"
    exit 1 # Exit script after printing help
}

# Print helpFunction in case parameters are empty
if [ -z "$1" ]; then
    echo "Please enter at least one folder name"
    helpFunction
fi

# Create folder structure

mkdir -p $appdata/{$1,$2,$3,$4,$5,$6,$7,$8,$9}
mkdir -p $config/{$1,$2,$3,$4,$5,$6,$7,$8,$9}
mkdir -p $runtime/{$1,$2,$3,$4,$5,$6,$7,$8,$9}

echo "The following folders were setup:"
echo " - $@"

restart_stack.sh

#!/bin/bash
# define services - Add stacks to this list between the brackets
# Leave Traefik out of the list as it will be started separately
stacks=(portainer docker-cleanup
    shepherd wetty guacamole
    emby privatebin autopirate hackmd
)
# define config folder
config_folder=/share/appdata/config

echo "**** Starting Traefik ****"
# Traefik needs to be the first stack deployed
docker stack deploy traefik -c ${config_folder}/traefik/traefik.yml

# Pruning the stack is optional ... comment out if you don't want to do this step
echo "***** Pruning the stack *****"
docker system prune -f
echo "**** System pruned ****"

echo "**** Starting Other Stacks ****"
# restart all services in docker swarm

# loop through stacks defined above
for stack in "${stacks[@]}"; do
    echo "**** Starting $stack ****"
    docker stack deploy $stack -c ${config_folder}/${stack}/${stack}.yml
    echo "***** Sleeping for a bit*****"
    sleep 10
    echo "**** $stack has been started ****"
done

echo "**** All Stacks have been started ****"

remove_stack.sh

#!/bin/bash
# define services - Add stacks to this list between the brackets
stacks=(portainer docker-cleanup
    shepherd wetty guacamole
    emby privatebin autopirate hackmd
)
# define config folder
config_folder=/share/appdata/config

echo "**** Removing Stacks ****"
# remove all services in docker swarm

# loop through stacks defined above
for stack in "${stacks[@]}"; do
    echo "**** Removing $stack ****"
    docker stack rm $stack
    echo "***** Sleeping for a bit*****"
    sleep 15
    echo "**** $stack has been removed ****"
done
echo "**** Removing Traefik ****"
# traefik needs to be last
docker stack rm traefik
echo "**** All Stacks have been removed ****"
echo "***** Sleeping for a bit *****"
sleep 15
echo "***** Pruning the stack *****"
docker system prune -f
echo "**** System pruned ****"
docker swarm leave -f
echo "***** Swarm Left *****"

setup_stack.sh

#!/bin/bash
# NOTE YOU NEED TO SET YOUR NAS IP BELOW FOR THE SWARM!!
echo "***** Setting up Swarm *****"
# setup Swarm
docker swarm init --advertise-addr <ENTER YOUR NAS IP HERE>

echo "***** Sleeping for a bit*****"
sleep 15

echo "***** Setting up overlay network *****"
# setup the overlay network
docker network create --driver=overlay --subnet=172.1.1.0/22 --attachable traefik_public

# define services - Add stacks to this list between the brackets
# Leave Traefik out of the list as it will be started seperately
stacks=(portainer docker-cleanup
    shepherd nextcloud graylog wetty
    plex privatebin autopirate
)
# define config folder
config_folder=/share/appdata/config

echo "**** Starting Traefik ****"
# Traefik needs to be the first stack deployed
docker stack deploy traefik -c ${config_folder}/traefik/traefik.yml

echo "**** Starting Other Stacks ****"
# restart all services in docker swarm

# loop through stacks defined above
for stack in "${stacks[@]}"; do
    echo "**** Starting $stack ****"
    docker stack deploy $stack -c ${config_folder}/${stack}/${stack}.yml
    echo "***** Sleeping for a bit*****"
    sleep 15
    echo "**** $stack has been started ****"
done

echo "**** All Stacks have been started ****"

fredericrous / QNAP-Docker-Swarm-Setup