Difference between revisions of "Docker"

From Indie IT Wiki
imported>Plittlefield
 
 
(236 intermediate revisions by 2 users not shown)
Line 22: Line 22:
  
 
[https://fleet.linuxserver.io/ LinuxServer]
 
[https://fleet.linuxserver.io/ LinuxServer]
 +
 +
== Useful Wiki ==
 +
 +
* https://wiki.servarr.com/
  
 
== Installation ==
 
== Installation ==
 +
 +
'''WINDOWS'''
 +
 +
# Make sure your computer supports Hardware Virtualisation by checking in the BIOS.
 +
# Install [https://docs.docker.com/desktop/install/windows-install/ Docker Desktop].
 +
# Install the [https://docs.microsoft.com/windows/wsl/wsl2-kernel Windows Subsystem for Linux Kernel Update].
 +
# Reboot.
 +
 +
'''LINUX'''
 +
 +
=== New All In One Official Method ===
 +
 +
sudo -i
 +
curl -fsSL <nowiki>https://get.docker.com</nowiki> | sh
  
 
=== Engine ===
 
=== Engine ===
Line 29: Line 47:
 
This will remove the old version of 'Docker' and install the new version 'Docker CE'...
 
This will remove the old version of 'Docker' and install the new version 'Docker CE'...
  
−
  sudo apt-get remove docker docker-engine docker.io
+
  sudo apt-get remove docker docker-engine docker.io containerd runc
−
  sudo apt-get -y install apt-transport-https ca-certificates curl software-properties-common
+
sudo apt-get update
−
  curl -fsSL <nowiki>https://download.docker.com/linux/ubuntu/gpg</nowiki> | sudo apt-key add -
+
  sudo apt-get -y install apt-transport-https ca-certificates curl gnupg software-properties-common
−
sudo apt-key fingerprint 0EBFCD88
+
sudo mkdir -p /etc/apt/keyrings
−
  sudo add-apt-repository "deb [arch=amd64] <nowiki>https://download.docker.com/linux/ubuntu</nowiki> $(lsb_release -cs) stable"
+
  curl -fsSL <nowiki>https://download.docker.com/linux/ubuntu/gpg</nowiki> | sudo gpg --dearmor -o /etc/apt/keyrings/docker.gpg
−
  sudo apt-get -y install docker-ce
+
  echo "deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.gpg] <nowiki>https://download.docker.com/linux/ubuntu</nowiki> $(lsb_release -cs) stable" | sudo tee /etc/apt/sources.list.d/docker.list > /dev/null
 +
sudo apt-get update
 +
  sudo apt-get -y install docker-ce docker-ce-cli containerd.io docker-compose-plugin
 +
sudo service docker start
 
  sudo docker run hello-world
 
  sudo docker run hello-world
  
 
https://docs.docker.com/install/linux/docker-ce/ubuntu/#install-docker-ce-1
 
https://docs.docker.com/install/linux/docker-ce/ubuntu/#install-docker-ce-1
  
−
=== Compose ===
+
==== Compose From Command Line ====
  
−
curl -I -s "<nowiki>https://github.com/docker/compose/releases/latest</nowiki>" | grep 'location:' | sed 's/^.*[/]//'
+
http://composerize.com - Turns docker run commands into docker-compose files!
  
−
Check the [https://github.com/docker/compose/releases/latest latest version] of 'Docker Compose' and edit the following command...
+
Install composerize and convert your own commands locally.
  
−
  sudo curl -L "<nowiki>https://github.com/docker/compose/releases/download/1.29.2/docker-compose-$(uname -s)-$(uname -m)</nowiki>" -o /usr/local/bin/docker-compose
+
  sudo apt instal npm
−
sudo chmod +x /usr/local/bin/docker-compose
+
sudo npm install composerize -g
−
/usr/local/bin/docker-compose --version
+
composerize docker run -p 80:80 -v /var/run/docker.sock:/tmp/docker.sock:ro --restart always --log-opt max-size=1g nginx
  
−
https://docs.docker.com/compose/install/#install-compose
+
https://github.com/magicmark/composerize
  
−
== Usage ==
+
=== Using Ansible ===
  
−
Statistics
+
<code>docker-install.yml</code>
  
−
  docker stats
+
  - hosts: all
−
docker stats --no-stream
+
 
 +
  become: yes
 +
  tasks:
 +
 
 +
  # Install Docker
 +
  # --
 +
  #
 +
  - name: install prerequisites
 +
    apt:
 +
      name:
 +
        - apt-transport-https
 +
        - ca-certificates
 +
        - curl
 +
        - gnupg-agent
 +
        - software-properties-common
 +
      update_cache: yes
 +
 
 +
  - name: add apt-key
 +
    apt_key:
 +
      url: <nowiki>https://download.docker.com/linux/ubuntu/gpg</nowiki>
 +
 
 +
  - name: add docker repo
 +
    apt_repository:
 +
      repo: deb <nowiki>https://download.docker.com/linux/ubuntu</nowiki> focal stable
 +
 
 +
  - name: install docker
 +
    apt:
 +
      name:
 +
        - docker-ce
 +
        - docker-ce-cli
 +
        - containerd.io
 +
      update_cache: yes
 +
 
 +
  - name: add user permissions
 +
    shell: "usermod -aG docker ubuntu"
 +
 
 +
  # Installs Docker SDK
 +
  # --
 +
  #
 +
  - name: install python package manager
 +
    apt:
 +
      name: python3-pip
 +
 
 +
  - name: install python sdk
 +
    become_user: ubuntu
 +
    pip:
 +
      name:
 +
        - docker
 +
        - docker-compose
  
−
System information
+
Then run Ansible using your playbook on your server host ...
  
−
  docker system info
+
  ansible-playbook docker-install.yml -l 'myserver'
  
−
Run container
+
You can then deploy a container using Ansible as well. This will deploy Portainer ...
  
−
docker run hello-world
+
<code>docker_deploy-portainer.yml</code>
  
−
List containers
+
- hosts: all
 +
 
 +
  become: yes
 +
  become_user: ubuntu
 +
  tasks:
 +
 
 +
  # Create Portainer Volume
 +
  # --
 +
  #
 +
  - name: Create new Volume
 +
    community.docker.docker_volume:
 +
      name: portainer-data
 +
 
 +
  # Deploy Portainer
 +
  # --
 +
  # 
 +
  - name: Deploy Portainer
 +
    community.docker.docker_container:
 +
      name: portainer
 +
      image: "docker.io/portainer/portainer-ce"
 +
      ports:
 +
        - "9000:9000"
 +
        - "9443:9443"
 +
      volumes:
 +
        - /var/run/docker.sock:/var/run/docker.sock
 +
        - portainer-data:/data
 +
      restart_policy: always
  
−
docker container ls
+
... with this command ...
−
docker container ls -a
 
  
−
List container processes
+
  ansible-playbook docker_deploy-portainer.yml -l 'myserver'
  
−
docker ps
+
=== Moving the Docker Volume Storage Location ===
−
docker ps --format "<nowiki>table {{.ID}}\t{{.Names}}\t{{.Image}}\t{{.Ports}}\t{{.Status}}</nowiki>"
 
  
−
List container names
+
Why?
  
−
docker ps --format '<nowiki>{{.Names}}</nowiki>'
+
Because you only have a limited space on your root volume and you want to use the extra 100 GB volume you mounted on your Oracle VPS :)
−
docker ps -a | awk '{print $NF}'
 
  
−
List volumes
+
How?
  
−
docker volume ls
+
Shut down any containers and the docker services.
−
docker volume ls -f dangling=true
 
  
−
List networks
+
systemctl stop docker.socket docker.service containerd.service
  
−
docker network ls
+
Create a directory on your big disk
  
−
Information about container
+
sudo mkdir /data1/docker
  
−
docker container inspect container_name or id
+
Rsync the current docker directory with the big disk directory
  
−
Stop container
+
sudo rsync -av /var/lib/docker/ /data1/docker/
  
−
docker stop container_name
+
Rename the existing docker library directory
  
−
Delete container
+
sudo mv /var/lib/docker /var/lib/docker.orig
  
−
docker rm container_name
+
Create a symlink for the 'docker directory' to your big disk directory
  
−
Delete volumes
+
sudo ln -s /data1/docker /var/lib/docker
  
−
docker volume rm volume_name
+
Reboot the server
  
−
Delete all unused volumes
+
sudo reboot
  
−
docker volume prune
+
Check to make sure all is well after
  
−
Delete all unused networks
+
docker --info
 +
docker ps
  
−
docker network prune
+
Now, go create your BTCPay Server container ;-)
  
−
Prune everything unused
 
  
−
docker system prune
+
Create a symlink to your nice big disk
  
−
Upgrade a stack
+
== Usage ==
  
−
docker-compose pull
+
Statistics
−
docker-compose up -d
 
  
−
BASH Aliases for use with Docker commands
+
docker stats
 +
docker stats --no-stream
  
−
alias dcd='docker-compose down'
+
System information
−
alias dcr='docker-compose restart'
 
−
alias dcu='docker-compose up -d'
 
−
alias dps='docker ps'
 
  
−
== Volumes ==
+
docker system info
 +
 
 +
Run container
 +
 
 +
docker run hello-world
 +
 
 +
List containers
  
−
=== Multiple Containers ===
+
docker container ls
 +
docker container ls -a
  
−
Use volumes which are bind mounted from the host filesystem between multiple containers.
+
List container processes
  
−
First, create the volume bind mounted to the folder...
+
docker ps
 +
docker ps --format "<nowiki>table {{.ID}}\t{{.Names}}\t{{.Image}}\t{{.Ports}}\t{{.Status}}</nowiki>"
  
−
docker volume create --driver local --opt type=none --opt device=/path/to/folder --opt o=bind volume_name
+
List container names
  
−
Then, use it in your docker compose file...
+
docker ps --format '<nowiki>{{.Names}}</nowiki>'
 +
docker ps -a | awk '{print $NF}'
  
−
services:
+
List volumes
−
  ftp.domain.uk-nginx:
 
−
    image: nginx
 
−
    container_name: ftp.domain.uk-nginx
 
−
    expose:
 
−
      - "80"
 
−
    volumes:
 
−
      - ./data/etc/nginx:/etc/nginx
 
−
      - ftp.domain.uk:/usr/share/nginx:ro
 
−
    environment:
 
−
      - VIRTUAL_HOST=ftp.domain.uk
 
−
networks:
 
−
  default:
 
−
    external:
 
−
      name: nginx-proxy-manager
 
−
volumes:
 
−
  ftp.domain.uk:
 
−
    external: true
 
  
−
[https://devopsheaven.com/docker/docker-compose/volumes/2018/01/16/volumes-in-docker-compose.html Using volumes in Docker Compose]
+
docker volume ls
 +
docker volume ls -f dangling=true
  
−
== Networks ==
+
List networks
  
−
Create your network...
+
docker network ls
  
−
docker network create existing-network
+
Information about container
  
−
Use it in your docker-compose.yml file...
+
docker container inspect container_name or id
  
−
services:
+
Stop container
−
  service_name:
 
−
    image: image_name:latest
 
−
    restart: always
 
−
    networks:
 
−
      - existing-network
 
−
networks:
 
−
  existing-network:
 
−
    external: true
 
  
−
https://poopcode.com/join-to-an-existing-network-from-a-docker-container-in-docker-compose/
+
docker stop container_name
 +
 
 +
Delete container
  
−
== Management ==
+
docker rm container_name
  
−
=== Cleaning Space ===
+
Delete volumes
  
−
Over the last month, a whopping 14Gb of space was being used by /var/lib/docker/overlay2/ and needed a way to safely remove unused data.
+
docker volume rm volume_name
  
−
Check your space usage...
+
Delete all unused volumes
  
−
  du -mcsh /var/lib/docker/overlay2
+
  docker volume prune
−
14G    /var/lib/docker/overlay2
 
  
−
Check what Docker thinks is being used...
+
Delete all unused networks
  
−
  docker system df
+
  docker network prune
−
TYPE            TOTAL    ACTIVE    SIZE      RECLAIMABLE
 
−
Images          36        15        8.368GB  4.491GB (53%)
 
−
Containers      17        15        70.74MB  286B (0%)
 
−
Local Volumes  4        2        0B        0B
 
−
Build Cache    0        0        0B        0B
 
  
−
Clean...
+
Prune everything unused
  
 
  docker system prune
 
  docker system prune
−
docker image prune --all
 
  
−
Check again...
+
Upgrade a stack
  
−
  du -mcsh /var/lib/docker/overlay2
+
  docker-compose pull
−
  9.4G    /var/lib/docker/overlay2
+
  docker-compose up -d
−
 
−
docker system df
 
−
TYPE            TOTAL    ACTIVE    SIZE      RECLAIMABLE
 
−
Images          13        13        4.144GB  144MB (3%)
 
−
Containers      15        15        70.74MB  0B (0%)
 
−
Local Volumes  4        2        0B        0B
 
−
Build Cache    0        0        0B        0B
 
  
−
...job done.
+
BASH Aliases for use with Docker commands
  
−
=== Portainer ===
+
alias dcd='docker-compose down'
 +
alias dcr='docker-compose restart'
 +
alias dcu='docker-compose up -d'
 +
alias dps='docker ps'
  
−
https://www.portainer.io
+
=== Run Command In Docker Container ===
  
−
https://github.com/portainer/portainer
+
e.g.
  
−
https://hub.docker.com/r/portainer/portainer-ce
+
List the mail queue for a running email server ...
  
−
=== Monitoring ===
+
docker exec -it mail.domain.co.uk-mailserver mailq
  
−
'''CTop'''
+
=== Setting Memory And CPU Limits In Docker ===
  
−
Press the Q key to stop it...
+
service:
 +
  image: nginx
 +
  mem_limit: 256m
 +
  mem_reservation: 128M
 +
  cpus: 0.5
 +
  ports:
 +
    - "80:80"
  
−
docker run -ti --name ctop --rm -v /var/run/docker.sock:/var/run/docker.sock wrfly/ctop:latest
+
https://www.baeldung.com/ops/docker-memory-limit
  
−
'''Docker Stats'''
+
== Volumes ==
  
−
docker stats
+
=== Multiple Containers ===
  
−
=== Gotchas ===
+
Use volumes which are bind mounted from the host filesystem between multiple containers.
  
−
https://sosedoff.com/2016/10/05/docker-gotchas.html
+
First, create the volume bind mounted to the folder...
  
−
== Applications ==
+
docker volume create --driver local --opt type=none --opt device=/path/to/folder --opt o=bind volume_name
  
−
I have set up my docker containers in a master docker directory with sub-directories for each stack.
+
Then, use it in your docker compose file...
  
−
  docker
+
  services:
−
|-- backups
+
  ftp.domain.uk-nginx:
−
`-- stacks
+
    image: nginx
−
     |-- bitwarden
+
     container_name: ftp.domain.uk-nginx
−
     |  `-- bwdata
+
     expose:
−
     |-- grafana
+
      - "80"
−
    |  `-- data
+
     volumes:
−
    |-- mailserver
+
      - ./data/etc/nginx:/etc/nginx
−
     |  `-- data
+
      - ftp.domain.uk:/usr/share/nginx:ro
−
     |-- nginx-proxy-manager
+
     environment:
−
    |  `-- data
+
      - VIRTUAL_HOST=ftp.domain.uk
−
     `-- portainer
+
networks:
−
        `-- data
+
  default:
 +
     external:
 +
      name: nginx-proxy-manager
 +
volumes:
 +
  ftp.domain.uk:
 +
     external: true
  
−
=== Backups ===
+
[https://devopsheaven.com/docker/docker-compose/volumes/2018/01/16/volumes-in-docker-compose.html Using volumes in Docker Compose]
  
−
https://github.com/alaub81/backup_docker_scripts
+
== Networks ==
  
−
=== Updates ===
+
=== Basic Usage ===
  
−
==== Tracking ====
+
Create your network...
  
−
===== Watchtower =====
+
docker network create networkname
  
−
A process for automating Docker container base image updates.
+
Use it in your docker-compose.yml file...
  
−
With watchtower you can update the running version of your containerized app simply by pushing a new image to the Docker Hub or your own image registry. Watchtower will pull down your new image, gracefully shut down your existing container and restart it with the same options that were used when it was deployed initially.
+
services:
 +
  service_name:
 +
    image: image_name:latest
 +
    restart: always
 +
    networks:
 +
      - networkname
 +
networks:
 +
  networkname:
 +
    external: true
  
−
'''First Time Run Once Check Only'''
+
https://poopcode.com/join-to-an-existing-network-from-a-docker-container-in-docker-compose/
  
−
This will run and output if there are any updates them stop and remove itself...
+
=== Advanced Usage ===
  
−
docker run --name watchtower -v /var/run/docker.sock:/var/run/docker.sock containrrr/watchtower --run-once --debug --monitor-only --rm
+
'''Static IP Address'''
  
−
'''Automated Scheduled Run Daily'''
+
    networks:
 +
      traefik:
 +
        ipv4_address: 172.19.0.9
 +
      backend: null
  
−
This will start the container and schedule a check at 4am every day...
+
[https://www.youtube.com/watch?v=vUyHGF1HMsw Force Docker Containers to use a VPN for their Network]
  
−
<code>~/watchtower/docker-compose.yml</code>
+
==== Block IP Address Using IPTables ====
  
−
version: "3"
+
The key here is to make sure you use the -I or INSERT command for the rule so that it is FIRST in the chain.
−
services:
 
−
  watchtower:
 
−
    image: containrrr/watchtower
 
−
    container_name: watchtower
 
−
    restart: always
 
−
    volumes:
 
−
      - /var/run/docker.sock:/var/run/docker.sock
 
−
    environment:
 
−
      - TZ=${TZ}
 
−
      - WATCHTOWER_DEBUG=true
 
−
      - WATCHTOWER_MONITOR_ONLY=false
 
−
      - WATCHTOWER_CLEANUP=true
 
−
      - WATCHTOWER_LABEL_ENABLE=false
 
−
      - WATCHTOWER_NOTIFICATIONS=email
 
−
      - WATCHTOWER_NOTIFICATION_EMAIL_FROM=${EMAIL_FROM}
 
−
      - WATCHTOWER_NOTIFICATION_EMAIL_TO=${WATCHTOWER_EMAIL_TO}
 
−
      - WATCHTOWER_NOTIFICATION_EMAIL_SERVER=${SMTP_SERVER}
 
−
      - WATCHTOWER_NOTIFICATION_EMAIL_SERVER_PORT=${SMTP_PORT}
 
−
      - WATCHTOWER_NOTIFICATION_EMAIL_SERVER_USER=${SMTP_USER}
 
−
      - WATCHTOWER_NOTIFICATION_EMAIL_SERVER_PASSWORD=${SMTP_PASSWORD}
 
−
      - WATCHTOWER_SCHEDULE=0 0 4 * * *
 
  
−
https://containrrr.dev/watchtower/
+
'''Block IP addresses from LITHUANIA'''
  
−
https://containrrr.dev/watchtower/arguments/#without_updating_containers
+
iptables -I DOCKER-USER -i eth0 -s 141.98.10.0/24 -j DROP
 +
iptables -I DOCKER-USER -i eth0 -s 141.98.11.0/24 -j DROP
 +
iptables -L DOCKER-USER --line-numbers
 +
Chain DOCKER-USER (1 references)
 +
num  target    prot opt source              destination
 +
1    DROP      all  --  141.98.11.0/24      anywhere
 +
2    DROP      all  --  141.98.10.0/24      anywhere
 +
3    RETURN    all  --  anywhere            anywhere
 +
 
 +
==== Display IP Addresses In A Docker Network Sorted Numerically ====
 +
 
 +
docker network inspect traefik |jq -r 'map(.Containers[].IPv4Address) []' |sort -t . -k 3,3n -k 4,4n
  
−
https://github.com/containrrr/watchtower
+
== Docker Compose ==
  
−
https://www.the-digital-life.com/watchtower/
+
=== Restart Policy ===
  
−
https://www.youtube.com/watch?v=5lP_pdjcVMo
+
The "no" option has quotes around it...
  
−
==== Updating ====
+
restart: "no"
 +
restart: always
 +
restart: on-failure
 +
restart: unless-stopped
  
−
You can either ask Watchtower to update the containers automatically for you, or do it manually.
+
== Management ==
  
−
Manually updating when using docker-compose...
+
=== Cleaning Pruning ===
  
−
  cd /path/to/docker/stack/
+
sudo docker system df
−
  docker-compose stop
+
sudo docker system prune
−
  docker-compose pull
+
  cd /etc/cron.daily
−
  docker-compose start
+
sudo nano docker-prune
 +
 +
#!/bin/bash
 +
  docker system prune -y
 +
   
 +
  sudo chmod +x /etc/cron.daily/docker-prune
  
−
=== Bitwarden ===
+
https://alexgallacher.com/prune-unused-docker-images-automatically/
  
−
<code>~/bitwardenrs/docker-compose.yml</code>
+
==== Delete All Stopped Docker Containers ====
  
−
  version: "2"
+
  docker rm $(docker ps --filter "status=exited" -q)
−
services:
 
−
  bitwardenrs:
 
−
    image: bitwardenrs/server:latest
 
−
    container_name: bitwardenrs
 
−
    volumes:
 
−
      - ./data:/data/
 
−
    ports:
 
−
      - 8100:80
 
−
    restart: unless-stopped
 
−
    environment:
 
−
      - TZ=Europe/London
 
−
      #- SIGNUPS_ALLOWED=false
 
−
      #- INVITATIONS_ALLOWED=false
 
−
      #- WEB_VAULT_ENABLED=false
 
−
networks:
 
−
  default:
 
−
    external:
 
−
      name: nginx-proxy-manager
 
  
−
Uncomment the extra security # lines '''after''' you have signed up, imported your old vault and set up your phone app and browsers, etc.
+
=== Updating with Docker Compose ===
  
−
  docker-compose down
+
  for d in ./*/ ; do (cd "$d" && sudo docker-compose pull && sudo docker-compose --compatibility up -d); done
−
docker-compose up -d
 
  
−
Check that the Bitwarden container environment has all the variables...
+
=== Logs Logging ===
  
−
docker exec -it bitwardenrs env | sort
+
If you want to watch the logs in real time, then add the -f or --follow option to your command ...
−
+
 
−
  HOME=/root
+
  docker logs nginx '''--follow'''
−
HOSTNAME=e5f327deb4dd
+
 
−
INVITATIONS_ALLOWED=false
+
or
−
PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
+
 
−
  ROCKET_ENV=staging
+
  docker logs nginx '''-f --tail 20'''
−
ROCKET_PORT=80
+
 
−
ROCKET_WORKERS=10
+
After executing docker-compose up, list your running containers:
−
SIGNUPS_ALLOWED=false
+
 
−
  TERM=xterm
+
  docker ps
−
TZ=Europe/London
 
−
WEB_VAULT_ENABLED=false
 
  
−
... and then refresh your web vault page to see it see "404: Not Found"  :-)
+
Copy the NAME of the given container and read its logs:
  
−
[https://hub.docker.com/r/bitwardenrs/server Bitwarden RS]
+
docker logs NAME_OF_THE_CONTAINER -f
  
−
[http://wiki.indie-it.com/wiki/Bitwarden#Self_Hosting Bitwarden Official]
+
To only read the error logs:
  
−
=== InfluxDB ===
+
docker logs NAME_OF_THE_CONTAINER -f 1>/dev/null
  
−
You can have InfluxDB on its own but there is little point without something to view the stats so you might as well include InfluxDB in the Grafana stack and start both at the same time... see below :-)
+
To only read the access logs:
  
−
=== Grafana ===
+
docker logs NAME_OF_THE_CONTAINER -f 2>/dev/null
  
−
Here is a stack in docker-compose which starts both containers in their own network so they can talk to one another. I have exposed ports for InfluxDB and Grafana to the host so I can use them from the internet.
+
https://linuxhandbook.com/docker-logging/
  
−
Obviously, put your firewall in place and change the passwords below!
+
To search or grep the logs:
  
−
<code>~/grafana/docker-compose.yml</code>
+
docker logs watchtower 2>&1 | grep 'msg="Found new'
  
−
version: "3"
+
=== Cleaning Space ===
−
services:
+
 
−
  grafana:
+
Over the last month, a whopping 14Gb of space was being used by /var/lib/docker/overlay2/ and needed a way to safely remove unused data.
−
    image: grafana/grafana
+
 
−
    container_name: grafana
+
Check your space usage...
−
    restart: always
+
 
−
    networks:
+
du -mcsh /var/lib/docker/overlay2
−
      - grafana-influxdb-network
+
14G    /var/lib/docker/overlay2
−
    ports:
+
 
−
      - 3000:3000
+
Check what Docker thinks is being used...
−
    volumes:
+
 
−
      - ./data/grafana:/var/lib/grafana
+
docker system df
−
    environment:
+
TYPE            TOTAL    ACTIVE    SIZE     RECLAIMABLE
−
      - INFLUXDB_URL=<nowiki>http://influxdb:8086</nowiki>
+
Images          36       15       8.368GB  4.491GB (53%)
−
    depends_on:
+
Containers     17       15       70.74MB  286B (0%)
−
      - influxdb
+
Local Volumes  4        2        0B       0B
−
  influxdb:
+
Build Cache    0        0        0B       0B
−
    image: influxdb:1.8.4
 
−
    container_name: influxdb
 
−
    restart: always
 
−
     networks:
 
−
       - grafana-influxdb-network
 
−
    ports:
 
−
       - 8086:8086
 
−
    volumes:
 
−
      - ./data/influxdb:/var/lib/influxdb
 
−
     environment:
 
−
       - INFLUXDB_DB=grafana
 
−
       - INFLUXDB_USER=grafana
 
−
       - INFLUXDB_USER_PASSWORD=password
 
−
      - INFLUXDB_ADMIN_ENABLED=true
 
−
      - INFLUXDB_ADMIN_USER=admin
 
−
      - INFLUXDB_ADMIN_PASSWORD=password
 
−
       - INFLUXDB_URL=<nowiki>http://influxdb:8086</nowiki>
 
−
networks:
 
−
  grafana-influxdb-network:
 
−
    external: true
 
  
−
After this, change your Telegraf configuration to point to the '''new host''' and change the database it uses to 'grafana'.
+
Clean...
  
−
=== NGiNX Proxy Manager ===
+
docker system prune
 +
docker image prune --all
  
−
Provide users with an easy way to accomplish reverse proxying hosts with SSL termination that is so easy a monkey could do it.
+
Check again...
  
−
# Set up your host
+
du -mcsh /var/lib/docker/overlay2
−
# Add a proxy to point to the host (in Docker this will be the 'name' and the port)
+
9.4G    /var/lib/docker/overlay2
−
# Go to <nowiki>http://yourhost</nowiki>
+
 +
docker system df
 +
TYPE            TOTAL    ACTIVE    SIZE      RECLAIMABLE
 +
Images          13        13        4.144GB  144MB (3%)
 +
Containers      15        15        70.74MB  0B (0%)
 +
Local Volumes  4        2        0B        0B
 +
Build Cache    0        0        0B        0B
 +
 
 +
...job done.
 +
 
 +
=== Dockge ===
 +
 
 +
Better than Portainer.
 +
 
 +
A fancy, easy-to-use and reactive docker 'compose.yaml' stack-oriented manager
  
−
https://nginxproxymanager.com
+
https://github.com/louislam/dockge
  
−
https://github.com/jc21/nginx-proxy-manager
+
=== Portainer ===
  
−
Create the Docker network...
+
https://www.portainer.io
  
−
sudo -i
+
https://github.com/portainer/portainer
−
docker network create nginx-proxy-manager
 
  
−
<code>/root/stacks/nginx-proxy-manager/docker-compose.yml</code>
+
[https://docs.portainer.io/v/ce-2.6/advanced/reset-admin Reset the Admin user Password]
  
−
version: '3'
+
==== Server ====
−
services:
+
 
−
  app:
+
https://hub.docker.com/r/portainer/portainer-ce
−
    image: 'jc21/nginx-proxy-manager:latest'
+
 
−
    container_name: nginx-proxy-manager_app
+
==== Agent ====
−
    restart: always
+
 
−
    networks:
+
Portainer uses the Portainer Agent container to communicate with the Portainer Server instance and provide access to the node's resources. So, this is great for a small server (like a Raspberry Pi) where you don't need the full Portainer Server install.
−
      - nginx-proxy-manager
+
 
−
    ports:
+
'''Deployment'''
−
      - '80:80'
+
 
−
      - '81:81'
+
Run the following command to deploy the Portainer Agent:
−
      - '443:443'
+
 
−
    environment:
+
docker run -d -p 9001:9001 --name portainer_agent --restart=always -v /var/run/docker.sock:/var/run/docker.sock -v /var/lib/docker/volumes:/var/lib/docker/volumes portainer/agent:2.11.0
−
      TZ: "Europe/London"
+
 
−
      DB_MYSQL_HOST: "db"
+
'''Adding your new environment'''
−
      DB_MYSQL_PORT: 3306
+
 
−
      DB_MYSQL_USER: "npm"
+
Once the agent has been installed you are ready to add the environment to your Portainer Server installation.
−
      DB_MYSQL_PASSWORD: "npm"
+
 
−
      DB_MYSQL_NAME: "npm"
+
From the menu select Environments then click Add environment.
−
    volumes:
+
 
−
      - ./data:/data
+
From the Environment type section, select Agent. Since we have already installed the agent you can ignore the sample commands in the Information section.
−
      - ./data/letsencrypt:/etc/letsencrypt
+
 
−
    depends_on:
+
Name: my-raspberry-pi
−
      - db
+
Environment URL: 192.168.0.106:9001
−
  db:
+
 
−
    image: 'jc21/mariadb-aria:latest'
+
When you're ready, click Add environment.
−
    container_name: nginx-proxy-manager_db
+
 
−
    restart: always
+
Then, on the Portainer Home screen you select your new environment, or server running the Agent, and away you go!
−
    networks:
+
 
−
      - nginx-proxy-manager
+
https://docs.portainer.io/v/ce-2.11/admin/environments/add/docker#method-2-connecting-via-the-portainer-agent
−
    environment:
+
 
−
      TZ: "Europe/London"
+
=== Updating ===
−
      MYSQL_ROOT_PASSWORD: 'npm'
 
−
      MYSQL_DATABASE: 'npm'
 
−
      MYSQL_USER: 'npm'
 
−
      MYSQL_PASSWORD: 'npm'
 
−
    volumes:
 
−
      - ./data/mysql:/var/lib/mysql
 
−
networks:
 
−
  nginx-proxy-manager:
 
−
    external: true
 
  
−
'''Reset Password'''
+
==== Portainer ====
  
−
  docker exec -it nginx-proxy-manager_db sh
+
  Containers > Select > Stop > Recreate > Pull Latest Image > Start
−
mysql -u root -p npm
 
−
select * from user;
 
−
delete from user where id=1;
 
−
quit;
 
−
exit
 
  
−
'''Custom SSL Certificate'''
+
==== Watchtowerr ====
  
−
You can add a custom SSL certificate to NPM by saving the 3 parts of the SSL from Let's Encrypt...
+
List updates ...
  
−
# privkey.pem
+
docker run --rm -v /var/run/docker.sock:/var/run/docker.sock containrrr/watchtower --run-once --log-format pretty --monitor-only
−
# cert.pem
+
 
−
# chain.pem
+
Perform updates ...
  
−
...and then uploading them to NPM.
+
docker run --rm -v /var/run/docker.sock:/var/run/docker.sock containrrr/watchtower --run-once --log-format pretty
  
−
'''Updating'''
+
=== Backups ===
  
−
docker-compose pull
+
https://github.com/SavageSoftware/portainer-backup
−
docker-compose up -d
 
  
−
=== NGiNX ===
+
https://forum.openmediavault.org/index.php?thread/43235-backup-portainer-and-completely-uninstall-to-test-disaster-recovery/
  
−
'''Quick Container'''
+
=== Monitoring ===
  
−
Run and delete everything afterwards (press CTRL+C to stop it)...
+
'''CTop'''
  
−
docker run --rm --name test.domain.org-nginx -e VIRTUAL_HOST=test.domain.org nginx
+
Press the Q key to stop it...
  
−
Run and detach and use a host folder to store the web pages and keep the container afterwards...
+
docker run -ti --name ctop --rm -v /var/run/docker.sock:/var/run/docker.sock wrfly/ctop:latest
 +
 
 +
'''Docker Stats'''
  
−
  docker run --name test.domain.org-nginx -e VIRTUAL_HOST=test.domain.org -v /some/content:/usr/share/nginx/html:ro -d nginx
+
  docker stats
  
−
Run and detach and connect to a specific network (like nginx-proxy-manager) and use a host folder to store the web pages and keep the container afterwards...
+
'''DeUnhealth'''
  
−
  docker run --name test.domain.org-nginx --network nginx-proxy-manager -e VIRTUAL_HOST=test.domain.org -v /some/content:/usr/share/nginx/html:ro -d nginx
+
Restart your unhealthy containers safely.
  
−
Check the logs and always show them (like tail -f)...
+
https://github.com/qdm12/deunhealth
  
−
  docker logs test.domain.org-nginx -f
+
https://www.youtube.com/watch?v=Oeo-mrtwRgE
  
−
'''Load Balancer'''
+
'''Dozzle'''
  
−
This is a simple exmaple test to show multiple backend servers answering web page requests.
+
Dozzle is a small lightweight application with a web based interface to monitor Docker logs. It doesn’t store any log files. It is for live monitoring of your container logs only.
  
−
<code>docker-compose.yml</code>
+
https://github.com/amir20/dozzle
  
−
version: '3'
+
=== Gotchas ===
−
services:
+
 
−
  # The load balancer
+
https://sosedoff.com/2016/10/05/docker-gotchas.html
−
  nginx:
+
 
−
    image: nginx:1.16.0-alpine
+
== Applications ==
−
    volumes:
 
−
      - ./nginx.conf:/etc/nginx/nginx.conf:ro
 
−
    ports:
 
−
      - "80:80"
 
−
  # The web server1
 
−
  server1:
 
−
    image: nginx:1.16.0-alpine
 
−
    volumes:
 
−
      - ./server1.html:/usr/share/nginx/html/index.html
 
−
  # The web server2
 
−
  server2:
 
−
    image: nginx:1.16.0-alpine
 
−
    volumes:
 
−
      - ./server2.html:/usr/share/nginx/html/index.html
 
  
−
<code>nginx.conf</code>
+
I have set up my docker containers in a master docker directory with sub-directories for each stack.
  
−
  events {
+
  docker
−
    worker_connections 1024;
+
  |-- backups
−
  }
+
  `-- stacks
−
  http {
+
     |-- bitwarden
−
     upstream app_servers {    # Create an upstream for the web servers
+
    |  `-- bwdata
−
        server server1:80;    # the first server
+
    |-- grafana
−
        server server2:80;    # the second server
+
     |  `-- data
−
     }
+
     |-- mailserver
−
     server {
+
    |  `-- data
−
        listen 80;
+
    |-- nginx-proxy-manager
−
        location / {
+
    |  `-- data
−
            proxy_pass        http://app_servers;  # load balance the traffic
+
     `-- portainer
−
        }
+
        `-- data
−
     }
 
−
}
 
  
−
https://omarghader.github.io/docker-compose-nginx-tutorial/
+
=== Backups ===
  
−
'''Proxy'''
+
https://github.com/alaub81/backup_docker_scripts
  
−
This is very cool and allows you to run multiple web sites on-the-fly.
+
=== Updates ===
  
−
The container connects to the system docker socket and watches for new containers using the VIRTUAL_HOST environment variable.
+
==== Tracking ====
  
−
Start this, then add another container using the VIRTUAL_HOST variable and the proxy container will change its config file and reload nginx to serve the web site... automatically.
+
===== Watchtower =====
  
−
Incredible.
+
A process for automating Docker container base image updates.
  
−
<code>~/nginx-proxy/docker-compose.yml</code>
+
With watchtower you can update the running version of your containerized app simply by pushing a new image to the Docker Hub or your own image registry. Watchtower will pull down your new image, gracefully shut down your existing container and restart it with the same options that were used when it was deployed initially.
  
−
version: "3"
+
'''First Time Run Once Check Only'''
−
services:
 
−
  nginx-proxy:
 
−
    image: jwilder/nginx-proxy
 
−
    container_name: nginx-proxy
 
−
    ports:
 
−
      - "80:80"
 
−
    volumes:
 
−
      - /var/run/docker.sock:/tmp/docker.sock:ro
 
−
networks:
 
−
  default:
 
−
    external:
 
−
      name: nginx-proxy
 
  
−
'''Normal'''
+
This will run and output if there are any updates them stop and remove itself...
  
−
When using the nginx-proxy container above, you can just spin up a virtual web site using the standard 'nginx' docker image and link it to the 'nginx-proxy' network...
+
docker run --name watchtower -v /var/run/docker.sock:/var/run/docker.sock containrrr/watchtower --run-once --debug --monitor-only --rm
  
−
docker run -d --name nginx-website1.uk --expose 80 --net nginx-proxy -e VIRTUAL_HOST=website1.uk nginx
+
'''Automated Scheduled Run Daily'''
  
−
To use the host filesystem to store the web page files...
+
This will start the container and schedule a check at 4am every day...
  
−
docker run -d --name nginx-website1.uk --expose 80 --net nginx-proxy -v /var/www/website1.uk/html:/usr/share/nginx/html:ro -e VIRTUAL_HOST=website1.uk nginx
+
<code>~/watchtower/docker-compose.yml</code>
−
 
 
−
In Docker Compose, it will look like this...
 
−
 
 
−
<code>~/nginx/docker-compose.yml</code>
 
  
 
  version: "3"
 
  version: "3"
 
  services:
 
  services:
−
   nginx-website1.uk:
+
   watchtower:
−
     image: nginx
+
     image: containrrr/watchtower
−
     container_name: nginx-website1.uk
+
     container_name: watchtower
−
     expose:
+
     restart: always
−
      - "80"
 
 
     volumes:
 
     volumes:
−
       - /var/www/website1.uk/html:/usr/share/nginx/html:ro
+
       - /var/run/docker.sock:/var/run/docker.sock
 
     environment:
 
     environment:
−
       - VIRTUAL_HOST=website1.uk
+
       - TZ=${TZ}
−
networks:
+
      - WATCHTOWER_DEBUG=true
−
  default:
+
      - WATCHTOWER_MONITOR_ONLY=false
−
    external:
+
      - WATCHTOWER_CLEANUP=true
−
      name: nginx-proxy
+
      - WATCHTOWER_LABEL_ENABLE=false
−
 
+
      - WATCHTOWER_NOTIFICATIONS=email
−
'''Multiple Virtual Host Web Sites'''
+
      - WATCHTOWER_NOTIFICATION_EMAIL_FROM=${EMAIL_FROM}
 +
      - WATCHTOWER_NOTIFICATION_EMAIL_TO=${WATCHTOWER_EMAIL_TO}
 +
      - WATCHTOWER_NOTIFICATION_EMAIL_SERVER=${SMTP_SERVER}
 +
      - WATCHTOWER_NOTIFICATION_EMAIL_SERVER_PORT=${SMTP_PORT}
 +
      - WATCHTOWER_NOTIFICATION_EMAIL_SERVER_USER=${SMTP_USER}
 +
      - WATCHTOWER_NOTIFICATION_EMAIL_SERVER_PASSWORD=${SMTP_PASSWORD}
 +
      - WATCHTOWER_SCHEDULE=0 0 4 * * *
 +
 
 +
https://containrrr.dev/watchtower/
 +
 
 +
https://containrrr.dev/watchtower/arguments/#without_updating_containers
 +
 
 +
https://github.com/containrrr/watchtower
 +
 
 +
https://www.the-digital-life.com/watchtower/
 +
 
 +
https://www.youtube.com/watch?v=5lP_pdjcVMo
  
−
<code>~/nginx/docker-compose.yml</code>
+
==== Updating ====
  
−
version: "3"
+
You can either ask Watchtower to update the containers automatically for you, or do it manually.
−
services:
 
−
  nginx-website1.uk:
 
−
    image: nginx
 
−
    container_name: nginx-website1.uk
 
−
    expose:
 
−
      - "80"
 
−
    volumes:
 
−
      - /var/www/website1.uk/html:/usr/share/nginx/html:ro
 
−
    environment:
 
−
      - VIRTUAL_HOST=website1.uk
 
−
  nginx-website2.uk:
 
−
    image: nginx
 
−
    container_name: nginx-website2.uk
 
−
    expose:
 
−
      - "80"
 
−
    volumes:
 
−
      - /var/www/website2.uk/html:/usr/share/nginx/html:ro
 
−
    environment:
 
−
      - VIRTUAL_HOST=website2.uk
 
−
networks:
 
−
  default:
 
−
    external:
 
−
      name: nginx-proxy
 
  
−
'''Viewing Logs'''
+
Manually updating when using docker-compose...
  
−
  docker-compose logs nginx-website1.uk
+
cd /path/to/docker/stack/
−
  docker-compose logs nginx-website2.uk
+
  docker-compose down
 +
docker-compose pull
 +
  docker-compose up -d
  
−
'''Proxy Manager'''
+
=== Bitwarden ===
  
−
This is a web front end to manage 'nginx-proxy', where you can choose containers and create SSL certificates etc.
+
<code>~/vaultwarden/docker-compose.yml</code>
  
−
https://cyberhost.uk/npm-setup/
+
services:
 +
 +
  vaultwarden:
 +
    image: "vaultwarden/server:latest"
 +
    container_name: "vaultwarden"
 +
    restart: "always"
 +
    networks:
 +
      traefik:
 +
        ipv4_address: 172.19.0.4
 +
    ports:
 +
      - "8100:80"
 +
    volumes:
 +
      - "./data:/data/"
 +
    environment:
 +
      - "TZ=Europe/London"
 +
      - "SIGNUPS_ALLOWED=true"
 +
      - "INVITATIONS_ALLOWED=true"
 +
      - "WEB_VAULT_ENABLED=true"
 +
    labels:
 +
      - "traefik.enable=true"
 +
      - "traefik.docker.network=traefik"
 +
      - "traefik.http.routers.vaultmydomaincom.rule=Host(`vault.mydomain.com`)"
 +
      - "traefik.http.routers.vaultmydomaincom.entrypoints=websecure"
 +
      - "traefik.http.routers.vaultmydomaincom.tls.certresolver=letsencrypt-gandi"
 +
      - "com.centurylinklabs.watchtower.enable=true"
 +
 +
networks:
 +
  traefik:
 +
    external: true
  
−
'''Various'''
+
'''After you have created an account, signed in and imported your passwords, please change the SIGNUPS_ALLOWED, INVITATIONS_ALLOWED and WEB_VAULT_ENABLED to false.'''
  
−
https://hub.docker.com/_/nginx
+
docker-compose down
 +
docker-compose up -d
  
−
https://blog.ssdnodes.com/blog/host-multiple-websites-docker-nginx/
+
Check that the Bitwarden container environment has all the variables...
  
−
https://github.com/nginx-proxy/nginx-proxy
+
docker exec -it vaultwarden env | sort
 +
 +
HOME=/root
 +
HOSTNAME=e5f327deb4dd
 +
INVITATIONS_ALLOWED=false
 +
PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
 +
ROCKET_ENV=staging
 +
ROCKET_PORT=80
 +
ROCKET_WORKERS=10
 +
SIGNUPS_ALLOWED=false
 +
TERM=xterm
 +
TZ=Europe/London
 +
WEB_VAULT_ENABLED=false
  
−
=== WordPress ===
+
... and then refresh your web vault page to see it see "404: Not Found"  :-)
  
−
https://hub.docker.com/_/wordpress/
+
[https://github.com/dani-garcia/vaultwarden Vaultwarden]
  
−
==== PHP File Uploads Fix ====
+
=== InfluxDB ===
  
−
Create a new PHP configuration file, and name it docker-uploads.ini. Add the following configuration then save the changes.
+
You can have InfluxDB on its own but there is little point without something to view the stats so you might as well include InfluxDB in the Grafana stack and start both at the same time... see below :-)
  
−
# Allow HTTP file uploads
+
=== Grafana ===
−
file_uploads = On
 
−
 
−
# Maximum size of an uploaded file
 
−
upload_max_filesize = 64M
 
−
 
−
# Maximum size of form post data
 
−
post_max_size = 64M
 
  
−
Update the docker-compose.yml to bind the docker-uploads.ini to the wordpress container and then restart the WordPress container.
+
Here is a stack in docker-compose which starts both containers in their own network so they can talk to one another. I have exposed ports for InfluxDB and Grafana to the host so I can use them from the internet.
  
−
volumes:
+
Obviously, put your firewall in place and change the passwords below!
−
  - ./data/config/docker-uploads.ini:/usr/local/etc/php/conf.d/docker-uploads.ini
 
  
−
==== WordPress Clone ====
+
<code>~/grafana/docker-compose.yml</code>
−
 
 
−
Create your A record in DNS using [http://wiki.indie-it.com/wiki/AWS_Route_53#cli53 AWS Route 53 CLI]...
 
−
 
 
−
cli53 rrcreate domain.co.uk 'staging 300 A 123.456.78.910'
 
−
 
 
−
Create your docker folder for the cloned staging test web site...
 
−
 
 
−
mkdir -p ~/docker/stacks/staging.domain.co.uk/data/{db,html}
 
−
 
 
−
Edit your docker compose file, with 2 containers, making sure you use the same network as your Nginx Proxy Manager...
 
−
 
 
−
<code>~/docker/stacks/staging.domain.co.uk/docker-compose.yml</code>
 
  
 
  version: "3"
 
  version: "3"
 
  services:
 
  services:
−
   staging.domain.co.uk-wordpress_db:
+
   grafana:
−
     image: mysql:5.7
+
     image: grafana/grafana
−
     container_name: staging.domain.co.uk-wordpress_db
+
    container_name: grafana
 +
     restart: always
 +
    networks:
 +
      - grafana-influxdb-network
 +
    ports:
 +
      - 3000:3000
 
     volumes:
 
     volumes:
−
       - ./data/db:/var/lib/mysql
+
       - ./data/grafana:/var/lib/grafana
−
    restart: always
 
 
     environment:
 
     environment:
−
       - TZ=Europe/London
+
       - INFLUXDB_URL=<nowiki>http://influxdb:8086</nowiki>
−
      - MYSQL_ROOT_PASSWORD=siel6aiL
 
−
      - MYSQL_DATABASE=dbname
 
−
      - MYSQL_USER=dbuser
 
−
      - MYSQL_PASSWORD=ru5BeoFa
 
−
  staging.domain.co.uk-wordpress:
 
 
     depends_on:
 
     depends_on:
−
       - staging.domain.co.uk-wordpress_db
+
       - influxdb
−
     image: wordpress:latest
+
  influxdb:
−
    container_name: staging.domain.co.uk-wordpress
+
     image: influxdb:1.8.4
−
     volumes:
+
     container_name: influxdb
−
      - ./data/html:/var/www/html
 
 
     restart: always
 
     restart: always
 +
    networks:
 +
      - grafana-influxdb-network
 +
    ports:
 +
      - 8086:8086
 +
    volumes:
 +
      - ./data/influxdb:/var/lib/influxdb
 
     environment:
 
     environment:
−
       - TZ=Europe/London
+
       - INFLUXDB_DB=grafana
−
       - VIRTUAL_HOST=staging.domain.co.uk
+
       - INFLUXDB_USER=grafana
−
       - WORDPRESS_DB_HOST=staging.domain.co.uk-wordpress_db:3306
+
       - INFLUXDB_USER_PASSWORD=password
−
       - WORDPRESS_DB_NAME=dbname
+
      - INFLUXDB_ADMIN_ENABLED=true
−
       - WORDPRESS_DB_USER=dbuser
+
       - INFLUXDB_ADMIN_USER=admin
−
       - WORDPRESS_DB_PASSWORD=ru5BeoFa
+
       - INFLUXDB_ADMIN_PASSWORD=password
 +
       - INFLUXDB_URL=<nowiki>http://influxdb:8086</nowiki>
 
  networks:
 
  networks:
−
   default:
+
   grafana-influxdb-network:
−
     external:
+
     external: true
−
      name: nginx-proxy-manager
 
  
−
Start containers with correct settings and credentials for existing line web site (so that the docker startup script sets up the MySQL permissions)...
+
After this, change your Telegraf configuration to point to the '''new host''' and change the database it uses to 'grafana'.
  
−
docker-compose up -d
+
=== Traefik ===
  
−
Check the logs to make sure all is well...
+
<code>/root/docker/traefik/docker-compose.yml</code>
  
−
  docker logs staging.domain.co.uk-wordpress
+
  services:
−
docker logs staging.domain.co.uk-wordpress_db
+
 +
  traefik:
 +
    image: "traefik:latest"
 +
    container_name: "traefik"
 +
    networks:
 +
      traefik:
 +
        ipv4_address: 172.19.0.2
 +
    ports:
 +
      - "80:80"
 +
      - "443:443"
 +
    volumes:
 +
      - "/etc/timezone:/etc/timezone:ro"
 +
      - "/etc/localtime:/etc/localtime:ro"
 +
      - "./config/traefik/config.yml:/etc/traefik/config.yml:ro"
 +
      - "./config/traefik/traefik.yml:/etc/traefik/traefik.yml:ro"
 +
      - "./acme/:/acme/"
 +
      - "/var/run/docker.sock:/var/run/docker.sock:ro"
 +
      - "./logs/:/var/log/"
 +
    environment:
 +
      - "TZ=Europe/London"
 +
      - "AWS_ACCESS_KEY_ID=xxxxxxxxxxxxxxxxxxxxxx"
 +
      - "AWS_SECRET_ACCESS_KEY=xxxxxxxxxxxxxxxxxxxxxxxxxxxx"
 +
      - "AWS_REGION=eu-west-2"
 +
      - "GANDIV5_API_KEY=xxxxxxxxxxxxxxxxx"
 +
    labels:
 +
      - "traefik.enable=true"
 +
      - "traefik.http.routers.traefik-dashboard.rule=Host(`traefik-dashboard.mydomain.com`) && (PathPrefix(`/api`) || PathPrefix(`/dashboard`))"
 +
      - "traefik.http.routers.traefik-dashboard.service=api@internal"
 +
      - "traefik.http.routers.traefik-dashboard.middlewares=secured"
 +
      - "traefik.http.middlewares.secured.chain.middlewares=auth,rate"
 +
      - "traefik.http.middlewares.auth.basicauth.users=user-name:$xxxxxxxxxxxxxxxxUlFVbwar4jlRBO1a8K"
 +
      - "traefik.http.middlewares.rate.ratelimit.average=100"
 +
      - "traefik.http.middlewares.rate.ratelimit.burst=50"
 +
      - "com.centurylinklabs.watchtower.enable=true"
 +
    restart: "always"
 +
 +
networks:
 +
  traefik:
 +
    external: true
  
−
Copy the WordPress files to the host folder and correct ownership...
 
  
−
cp -av /path/to/backup_unzipped_wordpress/ ~/docker/stacks/staging.domain.co.uk/html/
+
<code>config/traefik/traefik.yml</code>
−
chown -R www-data:www-data ~/docker/stacks/staging.domain.co.uk/html/
 
  
−
Copy the sql file in to the running '''mysql''' container...
+
accessLog: {}
 +
accesslog:
 +
  filePath: "/var/log/access.log"
 +
  fields:
 +
    names:
 +
      StartUTC: drop
 +
log:
 +
  filePath: "/var/log/traefik.log"
 +
  level: INFO
 +
  maxSize: 10
 +
  maxBackups: 3
 +
  maxAge: 3
 +
  compress: true
 +
api:
 +
  dashboard: true
 +
providers:
 +
  docker:
 +
    endpoint: "unix:///var/run/docker.sock"
 +
    exposedByDefault: false
 +
    network: traefik
 +
  file:
 +
    filename: /etc/traefik/config.yml
 +
    watch: true
 +
entryPoints:
 +
  web:
 +
    address: ":80"
 +
    http:
 +
      middlewares:
 +
        - "crowdsec-bouncer@file"
 +
      redirections:
 +
        entryPoint:
 +
          to: websecure
 +
          scheme: https
 +
  websecure:
 +
    address: ":443"
 +
    http:
 +
      middlewares:
 +
        - "crowdsec-bouncer@file"
 +
      tls:
 +
        certResolver: letsencrypt-aws
 +
        domains:
 +
          - main: "mydomain.co.uk"
 +
            sans:
 +
              - "*.mydomain.co.uk"
 +
          - main: "mydomain.uk"
 +
            sans:
 +
              - "*.mydomain.uk"
 +
certificatesResolvers:
 +
  letsencrypt-http:
 +
    acme:
 +
      httpChallenge:
 +
        entrypoint: web
 +
      email: "myname@mydomain.co.uk"
 +
      storage: "/acme/letsencrypt-http.json"
 +
  letsencrypt-aws:
 +
    acme:
 +
      dnsChallenge:
 +
        provider: route53
 +
      email: "myname@mydomain.co.uk"
 +
      storage: "/acme/letsencrypt-aws.json"
 +
  letsencrypt-gandi:
 +
    acme:
 +
      dnsChallenge:
 +
        provider: gandiv5
 +
      email: "myname@mydomain.co.uk"
 +
      storage: "/acme/letsencrypt-gandi.json"
  
−
docker cp db_name.sql mysql_container_name:/tmp/
 
  
−
Log in to the database container...
+
<code>config/traefik/config.yml</code>
  
−
  docker exec -it mysql_container_name bash
+
  http:
 +
  middlewares:   
 +
    crowdsec-bouncer:
 +
      forwardauth:
 +
        address: <nowiki>http://crowdsec-bouncer-traefik:8080/api/v1/forwardAuth</nowiki>
 +
        trustForwardHeader: true
  
−
Check and if necessary, change the timezone...
+
=== Caddy ===
  
−
date
+
[https://caddyserver.com/ Caddy] is a lightweight, high performance web server and proxy server.  It is much easier to configure than NginX or Traefik.
−
mv /etc/localtime /etc/localtime.backup
 
−
  ln -s /usr/share/zoneinfo/Europe/London /etc/localtime
 
−
date
 
  
−
Delete and create the database...
+
Here are a few examples in the Caddyfile ...
  
−
  mysql -u root -p -e "DROP DATABASE db_name; CREATE DATABASE db_name;"
+
  # proxy to a btcpay server running in a different docker network and static ssl certificate files
 +
btcpay.mydomain.co.uk {
 +
        tls /ssl/certs/fullchain.pem /ssl/certs/key.pem
 +
        reverse_proxy generated_btcpayserver_1:49392
 +
        log {
 +
                output stdout
 +
        }
 +
}
 +
# proxy to a wordpress web site with custom ssl dns verification
 +
oracle-1.mydomain.co.uk {
 +
        tls me@mydomain.co.uk {
 +
                dns route53 {
 +
                        access_key_id "AKIAJxxxxxxxxxxxxxxx"
 +
                        secret_access_key "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxx"
 +
                        region "eu-west-2"
 +
                }
 +
        }
 +
        reverse_proxy oracle-1.mydomain.co.uk-nginx:80
 +
        log
 +
}
  
−
Import the database from the sql file, check and exit out of the container...
+
... and this is the docker-compose.yml file for all that above.  You do need to build your own docker image for caddy which includes the dns-route-53 plugin ...
  
−
  mysql -u root -p mysql_db_name < /tmp/db_name.sql
+
  services:
−
mysql -u root -p -e "use db_name; show tables;"
+
  caddy:
−
  rm /tmp/db_name.sql
+
    #image: caddy:alpine
−
exit
+
    image: paully/caddy-dns-route53:latest
 +
    container_name: caddy
 +
    restart: unless-stopped
 +
    ports:
 +
      - 80:80
 +
      - 443:443
 +
      - 443:443/udp
 +
    networks:
 +
      - caddy
 +
      - generated_default
 +
    volumes:
 +
      - /var/run/docker.sock:/var/run/docker.sock
 +
      - ./Caddyfile:/etc/caddy/Caddyfile
 +
      - ./data:/data
 +
      - ./config:/config
 +
      - ./fullchain.pem:/ssl/certs/fullchain.pem:ro
 +
      - ./key.pem:/ssl/certs/key.pem:ro
 +
    environment:
 +
      - "TZ=Europe/London"
 +
      - "AWS_ACCESS_KEY_ID=xxxxxxxxxxxxxxxxxxxxx"
 +
      - "AWS_SECRET_ACCESS_KEY=xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx"
 +
      - "AWS_REGION=eu-west-2"
 +
  networks:
 +
  caddy:
 +
    external: true
 +
  generated_default:
 +
    external: true
 +
 
 +
==== Do I have to restart the container after each Caddyfile change? ====
 +
 
 +
Caddy does not require a full restart when configuration is changed. Caddy comes with a caddy reload⁠ command which can be used to reload its configuration with zero downtime.
 +
 
 +
When running Caddy in Docker, the recommended way to trigger a config reload is by executing the caddy reload command in the running container.
  
−
Edit the wp-config.php on your host server to match new DB_HOST and also add extra variables to be sure...
+
First, you'll need to determine your container ID or name. Then, pass the container ID to docker exec. The working directory is set to /etc/caddy so Caddy can find your Caddyfile without additional arguments.
  
−
  nano /path/to/docker/folder/html/wp-config.php
+
  caddy_container_id=$(docker ps | grep caddy | awk '{print $1;}')
−
define( 'WP_HOME', <nowiki>'http://staging.domain.co.uk'</nowiki> );
+
  docker exec -w /etc/caddy $caddy_container_id caddy reload
−
  define( 'WP_SITEURL', <nowiki>'http://staging.domain.co.uk'</nowiki> );
 
  
−
Install [http://wiki.indie-it.com/wiki/WordPress#Install WordPress CLI] in the running container...
+
==== How do I disable the Caddy admin endpoint? ====
  
−
docker exec -it wordpress_container_name bash
+
Add this to the top of your Caddyfile ...
  
−
Search and replace the original site url...
+
{
 +
    admin off
 +
}
  
−
./wp --allow-root search-replace <nowiki>'http://www.domain.co.uk/' 'http://staging.domain.co.uk/'</nowiki> --dry-run
+
==== WordPress ====
−
./wp --allow-root search-replace <nowiki>'http://www.domain.co.uk/' 'http://staging.domain.co.uk/'</nowiki>
 
  
−
Start your web browser and go to the test staging web site!
+
This uses Caddy as the proxy container in the stack with WordPress FPM and MariaDB.
  
−
==== WordPress CLI ====
+
So, the files are all in 1 folder and on the host as a bind volume for Caddy and WordPress.  The PHP is handled by Caddy and passed to WordPress PHP-FPM on the container port of 9000, thereby removing the need for the NGINX container because Caddy is doing that part.  So, it's very fast!
  
−
In your stack, set up the usual two DB + WordPress containers, then add a third services section for wp-cli...
+
Caddy > PHP > WordPress
  
−
  version: "3"
+
  example.com {
−
  services:
+
        root * /var/www/html
−
  www.domain.uk-wordpress_db:
+
        php_fastcgi wordpress:9000
−
    image: mysql:5.7
+
        file_server
−
    container_name: www.domain.uk-wordpress_db
+
  }
−
    volumes:
+
 
−
      - ./data/db:/var/lib/mysql
+
https://sumguy.com/wordpress-on-php-fpm-caddy-in-docker/
−
    restart: always
+
 
−
    environment:
+
=== NGiNX Proxy Manager ===
−
      - MYSQL_ROOT_PASSWORD=password
+
 
−
      - MYSQL_DATABASE=wordpress
+
Provide users with an easy way to accomplish reverse proxying hosts with SSL termination that is so easy a monkey could do it.
−
      - MYSQL_USER=wordpress
+
 
−
      - MYSQL_PASSWORD=password
+
# Set up your host
−
  www.domain.uk-wordpress:
+
# Add a proxy to point to the host (in Docker this will be the 'name' and the port)
−
    depends_on:
+
# Go to <nowiki>http://yourhost</nowiki>
−
      - www.domain.uk-wordpress_db
+
 
−
    image: wordpress:latest
+
https://nginxproxymanager.com
−
    container_name: www.domain.uk-wordpress
+
 
−
     volumes:
+
https://github.com/jc21/nginx-proxy-manager
−
      - ./data/html:/var/www/html
+
 
−
     expose:
+
Create the Docker network with a chosen subnet (used later for fixing container IP addresses)...
−
      - 80
+
 
 +
sudo -i
 +
docker network create --subnet=172.20.0.0/16 nginx-proxy-manager
 +
 
 +
<code>/root/stacks/nginx-proxy-manager/docker-compose.yml</code>
 +
 
 +
version: '3'
 +
services:
 +
  db:
 +
     image: 'jc21/mariadb-aria:latest'
 +
     container_name: nginx-proxy-manager_db
 
     restart: always
 
     restart: always
 +
    networks:
 +
      nginx-proxy-manager:
 +
        ipv4_address: 172.20.0.2
 
     environment:
 
     environment:
−
       - VIRTUAL_HOST=www.domain.uk
+
       TZ: "Europe/London"
−
       - WORDPRESS_DB_HOST=www.domain.uk-wordpress_db:3306
+
       MYSQL_ROOT_PASSWORD: 'npm'
−
       - WORDPRESS_DB_USER=wordpress
+
       MYSQL_DATABASE: 'npm'
−
       - WORDPRESS_DB_PASSWORD=password
+
       MYSQL_USER: 'npm'
−
  '''www.domain.uk-wordpress-cli:'''
+
      MYSQL_PASSWORD: 'npm'
−
    image: wordpress:cli
 
−
    container_name: www.domain.uk-wordpress-cli
 
 
     volumes:
 
     volumes:
−
       - ./data/html:/var/www/html
+
       - ./data/mysql:/var/lib/mysql
 +
  app:
 +
    image: 'jc21/nginx-proxy-manager:latest'
 +
    container_name: nginx-proxy-manager_app
 +
    restart: always
 +
    networks:
 +
      nginx-proxy-manager:
 +
        ipv4_address: 172.20.0.3
 +
    ports:
 +
      - '80:80'
 +
      - '81:81'
 +
      - '443:443'
 
     environment:
 
     environment:
−
       - WP_CLI_CACHE_DIR=/tmp/
+
       TZ: "Europe/London"
−
       - VIRTUAL_HOST=www.domain.uk
+
       DB_MYSQL_HOST: "db"
−
       - WORDPRESS_DB_HOST=www.domain.uk-wordpress_db:3306
+
       DB_MYSQL_PORT: 3306
−
       - WORDPRESS_DB_USER=wordpress
+
       DB_MYSQL_USER: "npm"
−
       - WORDPRESS_DB_PASSWORD=password
+
      DB_MYSQL_PASSWORD: "npm"
−
     working_dir: /var/www/html
+
       DB_MYSQL_NAME: "npm"
−
     user: "33:33"
+
     volumes:
 +
      - ./data:/data
 +
      - ./data/letsencrypt:/etc/letsencrypt
 +
     depends_on:
 +
      - db
 
  networks:
 
  networks:
−
   default:
+
   nginx-proxy-manager:
−
     external:
+
     external: true
−
      name: nginx-proxy-manager
 
  
−
...then start it all up.
+
'''Reset Password'''
  
−
  docker-compose up -d
+
  docker exec -it nginx-proxy-manager_db sh
 +
mysql -u root -p npm
 +
select * from user;
 +
delete from user where id=1;
 +
quit;
 +
exit
  
−
Then, run your wp-cli commands (e.g. wp user list) on the end of a docker run command...
+
'''Custom SSL Certificate'''
  
−
docker-compose run --rm www.domain.uk-wordpress-cli '''wp --info'''
+
You can add a custom SSL certificate to NPM by saving the 3 parts of the SSL from Let's Encrypt...
−
docker-compose run --rm www.domain.uk-wordpress-cli '''wp cli version'''
 
−
docker-compose run --rm www.domain.uk-wordpress-cli '''wp user list'''
 
−
docker-compose run --rm www.domain.uk-wordpress-cli '''wp help theme'''
 
−
docker-compose run --rm www.domain.uk-wordpress-cli '''wp theme delete --all'''
 
  
−
=== Email Server ===
+
# privkey.pem
 +
# cert.pem
 +
# chain.pem
  
−
https://github.com/docker-mailserver/docker-mailserver
+
...and then uploading them to NPM.
  
−
https://docker-mailserver.github.io/docker-mailserver/edge/config/security/ssl/#lets-encrypt-recommended
+
'''Updating'''
  
−
==== Postgrey ====
+
docker-compose pull
 +
docker-compose up -d
 +
 
 +
==== Fix Error: Bad Gateway - error create table npm.migrations Permission Denied ====
  
−
[https://github.com/docker-mailserver/docker-mailserver/issues/1957 Local List]
+
https://github.com/NginxProxyManager/nginx-proxy-manager/issues/1499#issuecomment-1656997077
  
−
==== SpamAssassin ====
+
docker exec -it nginx-proxy-manager_db sh
 +
cd /var/lib/mysql
 +
chown -R mysql:mysql npm
 +
exit
  
−
[https://docker-mailserver.github.io/docker-mailserver/edge/faq/#how-can-i-manage-my-custom-spamassassin-rules Custom Rules]
+
=== NGiNX ===
  
−
[https://github.com/docker-mailserver/docker-mailserver/issues/365 Bayes Database]
+
'''Quick Container'''
  
−
SpamAssassin is controlled by Amavis (a fork of MailScanner) with the user 'amavis'.
+
Run and delete everything afterwards (press CTRL+C to stop it)...
  
−
'''Show Bayes Database Stats'''
+
docker run --rm --name test.domain.org-nginx -e VIRTUAL_HOST=test.domain.org -p 80:80 nginx:alpine
  
−
docker exec --user amavis mail.mydomain.org.uk-mailserver sa-learn --dump magic --dbpath /var/lib/amavis/.spamassassin
+
Run and detach and use a host folder to store the web pages and keep the container afterwards...
  
−
'''Learn Ham'''
+
docker run --name test.domain.org-nginx -e VIRTUAL_HOST=test.domain.org -v /some/content:/usr/share/nginx/html:ro -d -p 80:80 nginx:alpine
  
−
docker exec --user amavis mail.mydomain.org.uk-mailserver sa-learn --ham --progress /var/mail/mydomain.org.uk/info/cur --dbpath /var/lib/amavis/.spamassassin
+
Run and detach and connect to a specific network (like nginx-proxy-manager) and use a host folder to store the web pages and keep the container afterwards...
  
−
==== Backups ====
+
  docker run --name test.domain.org-nginx --network nginx-proxy-manager -e VIRTUAL_HOST=test.domain.org -v /some/content:/usr/share/nginx/html:ro -d -p 80:80 nginx:alpine
  
−
[https://docker-mailserver.github.io/docker-mailserver/edge/faq/#what-about-backups Backups]
+
Check the logs and always show them (like tail -f)...
  
−
==== Autodiscover ====
+
  docker logs test.domain.org-nginx -f
  
−
Create SRV and A record entries in your DNS for the services...
+
<code>docker-compose.yml</code>
−
 
 
−
$ORIGIN domain.org.uk.
 
−
@ 300 IN TXT "v=spf1 mx ~all; mailconf=<nowiki>https://autoconfig.domain.org.uk/mail/config-v1.1.xml</nowiki>"
 
−
_autodiscover._tcp 300 IN SRV 0 0 443 autodiscover.domain.org.uk.
 
−
_imap._tcp 300 IN SRV 0 0 0 .
 
−
_imaps._tcp 300 IN SRV 0 1 993 mail.domain.org.uk.
 
−
_ldap._tcp 300 IN SRV 0 0 636 mail.domain.org.uk.
 
−
_pop3._tcp 300 IN SRV 0 0 0 .
 
−
_pop3s._tcp 300 IN SRV 0 0 0 .
 
−
_submission._tcp 300 IN SRV 0 1 587 mail.domain.org.uk.
 
−
autoconfig 300 IN A 3.10.67.19
 
−
autodiscover 300 IN A 3.10.67.19
 
−
imap 300 IN CNAME mail
 
−
mail 300 IN A 3.10.67.19
 
−
smtp 300 IN CNAME mail
 
−
www 300 IN A 3.10.67.19
 
−
 
 
−
<code>docker-compose.yml</code>
 
  
 +
version: "3"
 
  services:
 
  services:
−
   mailserver-autodiscover:
+
   nginx:
−
     image: monogramm/autodiscover-email-settings:latest
+
     image: nginx
−
     container_name: mail.domain.org.uk-mailserver-autodiscover
+
     container_name: nginx
 
     environment:
 
     environment:
−
       - COMPANY_NAME=My Company
+
       - TZ=Europe/London
−
       - SUPPORT_URL=<nowiki>https://autodiscover.domain.org.uk</nowiki>
+
    volumes:
−
       - DOMAIN=domain.org.uk
+
       - ./data/html:/usr/share/nginx/html:ro
−
      - IMAP_HOST=mail.domain.org.uk
+
       - /etc/timezone:/etc/timezone:ro
−
      - IMAP_PORT=993
+
    expose:
−
      - IMAP_SOCKET=SSL
+
       - 80
−
      - SMTP_HOST=mail.domain.org.uk
+
     restart: unless-stopped
−
      - SMTP_PORT=587
 
−
       - SMTP_SOCKET=STARTTLS
 
−
     restart: unless-stopped
 
−
networks:
 
−
  default:
 
−
    external:
 
−
      name: nginx-proxy-manager
 
  
−
[https://docker-mailserver.github.io/docker-mailserver/edge/config/best-practices/autodiscover/ Autodiscover]
 
  
−
[https://hub.docker.com/r/monogramm/autodiscover-email-settings/ monogramm/autodiscover-email-settings]
+
'''With PHP'''
  
−
=== Internet Speedtest ===
+
<code>./data/nginx/site.conf</code>
  
−
https://github.com/henrywhitaker3/Speedtest-Tracker
+
server {
 +
    server_name docker-demo.com;
 +
    root /var/www/html;
 +
    index index.php index.html index.htm;
 +
    access_log /var/log/nginx/access.log;
 +
    error_log /var/log/nginx/error.log;
 +
    location / {
 +
        try_files $uri $uri/ /index.php?$query_string;
 +
    }
 +
    # PHP-FPM Configuration Nginx
 +
    location ~ \.php$ {
 +
        try_files $uri = 404;
 +
        fastcgi_split_path_info ^(.+\.php)(/.+)$;
 +
        fastcgi_pass php:9000;
 +
        fastcgi_index index.php;
 +
        include fastcgi_params;
 +
        fastcgi_param REQUEST_URI $request_uri;
 +
        fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
 +
        fastcgi_param PATH_INFO $fastcgi_path_info;
 +
    }
 +
}
  
−
=== Emby Media Server ===
+
<code>docker-compose.yml</code>
  
−
https://emby.media/docker-server.html
+
version: "3"
−
 
+
services:
−
https://hub.docker.com/r/emby/embyserver
+
  nginx:
 +
    image: nginx
 +
    container_name: nginx
 +
    environment:
 +
      - TZ=Europe/London
 +
    volumes:
 +
      - ./data/html:/usr/share/nginx/html:ro
 +
      - ./data/nginx:/etc/nginx/conf.d/
 +
      - /etc/timezone:/etc/timezone:ro
 +
    expose:
 +
      - 80
 +
    restart: unless-stopped
 +
  php:
 +
    image: php:7.2-fpm
 +
    volumes:
 +
      - ./data/html:/usr/share/nginx/html:ro
 +
      - ./data/php:/usr/local/etc/php/php.ini
  
−
=== AWS CLI ===
+
https://adoltech.com/blog/how-to-set-up-nginx-php-fpm-and-mysql-with-docker-compose/
  
−
docker run --rm -it -v ~/.aws:/root/.aws amazon/aws-cli s3 ls
+
'''With PERL'''
  
−
https://docs.aws.amazon.com/cli/latest/userguide/install-cliv2-docker.html
+
This is a way to get the IP address of the visitor (REMOTE_ADDR) displayed...
  
−
=== Let's Encrypt ===
+
# nginx.conf; mostly copied from defaults
 +
load_module "modules/ngx_http_perl_module.so";
 +
 +
user  nginx;
 +
worker_processes  auto;
 +
 +
error_log  /var/log/nginx/error.log notice;
 +
pid        /var/run/nginx.pid;
 +
 +
events {
 +
    worker_connections  1024;
 +
}
 +
 +
http {
 +
    #perl_modules /; # only needed the hello.pm isn't in @INC (e.g. dir specified below)
 +
    perl_modules /usr/lib/perl5/vendor_perl/x86_64-linux-thread-multi/;
 +
    perl_require hello.pm;
 +
 +
    server {
 +
        location / {
 +
            perl hello::handler;
 +
        }
 +
    }
 +
}
  
−
Issue a wildcard certificate...
+
# hello.pm; put in a @INC dir
 +
package hello;
 +
use nginx;
 +
 +
sub handler {
 +
    my $r = shift;
 +
    $r->send_http_header("text/html");
 +
    return OK if $r->header_only;
 +
    $r->print($r->remote_addr);
 +
    return OK;
 +
}
 +
1;
  
−
docker run -it --rm --name certbot -v "/usr/bin:/usr/bin" -v "/root/.aws:/root/.aws" -v "/etc/letsencrypt:/etc/letsencrypt" -v "/var/lib/letsencrypt:/var/lib/letsencrypt" certbot/dns-route53 certonly --dns-route53 --domain "example.com" --domain "*.example.com"
+
https://www.reddit.com/r/docker/comments/oabga4/run_perl_script_in_nginx_container/
  
−
Check your certificates...
+
'''Load Balancer'''
  
−
docker run -it --rm --name certbot -v "/usr/bin:/usr/bin" -v "/root/.aws:/root/.aws" -v "/etc/letsencrypt:/etc/letsencrypt" -v "/var/lib/letsencrypt:/var/lib/letsencrypt" certbot/certbot certificates
+
This is a simple exmaple test to show multiple backend servers answering web page requests.
  
−
https://certbot.eff.org/docs/install.html#running-with-docker
+
<code>docker-compose.yml</code>
  
−
=== VPN ===
+
version: '3'
 +
services:
 +
  # The load balancer
 +
  nginx:
 +
    image: nginx:1.16.0-alpine
 +
    volumes:
 +
      - ./nginx.conf:/etc/nginx/nginx.conf:ro
 +
    ports:
 +
      - "80:80"
 +
  # The web server1
 +
  server1:
 +
    image: nginx:1.16.0-alpine
 +
    volumes:
 +
      - ./server1.html:/usr/share/nginx/html/index.html
 +
  # The web server2
 +
  server2:
 +
    image: nginx:1.16.0-alpine
 +
    volumes:
 +
      - ./server2.html:/usr/share/nginx/html/index.html
  
−
==== OpenVPN ====
+
<code>nginx.conf</code>
  
−
'''Server'''
+
events {
 +
    worker_connections 1024;
 +
}
 +
http {
 +
    upstream app_servers {    # Create an upstream for the web servers
 +
        server server1:80;    # the first server
 +
        server server2:80;    # the second server
 +
    }
 +
    server {
 +
        listen 80;
 +
        location / {
 +
            proxy_pass        http://app_servers;  # load balance the traffic
 +
        }
 +
    }
 +
}
  
−
https://hub.docker.com/r/linuxserver/openvpn-as
+
https://omarghader.github.io/docker-compose-nginx-tutorial/
  
−
'''Client'''
+
'''Proxy'''
  
−
https://hub.docker.com/r/dperson/openvpn-client
+
This is very cool and allows you to run multiple web sites on-the-fly.
  
−
'''Routing Containers Through Container'''
+
The container connects to the system docker socket and watches for new containers using the VIRTUAL_HOST environment variable.
  
−
sudo docker run -it --net=container:vpn -d some/docker-container
+
Start this, then add another container using the VIRTUAL_HOST variable and the proxy container will change its config file and reload nginx to serve the web site... automatically.
  
−
==== WireGuard ====
+
Incredible.
  
−
=== ffmpeg ===
+
<code>~/nginx-proxy/docker-compose.yml</code>
  
−
https://registry.hub.docker.com/r/jrottenberg/ffmpeg
+
version: "3"
 +
services:
 +
  nginx-proxy:
 +
    image: jwilder/nginx-proxy
 +
    container_name: nginx-proxy
 +
    ports:
 +
      - "80:80"
 +
    volumes:
 +
      - /var/run/docker.sock:/tmp/docker.sock:ro
 +
networks:
 +
  default:
 +
    external:
 +
      name: nginx-proxy
  
−
https://github.com/jrottenberg/ffmpeg
+
'''Normal'''
  
−
https://medium.com/coconut-stories/using-ffmpeg-with-docker-94523547f35c
+
When using the nginx-proxy container above, you can just spin up a virtual web site using the standard 'nginx' docker image and link it to the 'nginx-proxy' network...
  
−
https://github.com/linuxserver/docker-ffmpeg
+
docker run -d --name nginx-website1.uk --expose 80 --net nginx-proxy -e VIRTUAL_HOST=website1.uk nginx
  
−
=== MakeMKV ===
+
To use the host filesystem to store the web page files...
  
−
'''This will NOT work on a Raspberry Pi.'''
+
docker run -d --name nginx-website1.uk --expose 80 --net nginx-proxy -v /var/www/website1.uk/html:/usr/share/nginx/html:ro -e VIRTUAL_HOST=website1.uk nginx
  
−
https://github.com/jlesage/docker-makemkv
+
In Docker Compose, it will look like this...
  
−
Use this in combination with [[FFmpeg_DVD|ffmpeg]] or [[HandBrake]] (as shown below) and [[FileBot]] to process your media through to your media server - like [[Emby]] or [[Plex]]..
+
<code>~/nginx/docker-compose.yml</code>
  
−
MakeMKV > HandBrake > FileBot > Emby
+
version: "3"
 +
services:
 +
  nginx-website1.uk:
 +
    image: nginx
 +
    container_name: nginx-website1.uk
 +
    expose:
 +
      - "80"
 +
    volumes:
 +
      - /var/www/website1.uk/html:/usr/share/nginx/html:ro
 +
    environment:
 +
      - VIRTUAL_HOST=website1.uk
 +
networks:
 +
  default:
 +
    external:
 +
      name: nginx-proxy
  
−
To make this work with your DVD drive (/dev/sr0) '''you need to have the second device''' (/dev/sg0) in order for it to work. I don't get it, but it works.
+
'''Multiple Virtual Host Web Sites'''
  
−
<code>/root/docker/stacks/makemkv/docker-compose.yml</code>
+
<code>~/nginx/docker-compose.yml</code>
  
−
  version: '3'
+
  version: "3"
 
  services:
 
  services:
−
   makemkv:
+
   nginx-website1.uk:
−
     image: jlesage/makemkv
+
     image: nginx
−
     container_name: makemkv
+
     container_name: nginx-website1.uk
−
     ports:
+
     expose:
−
       - "0.0.0.0:5801:5800"
+
       - "80"
 +
    volumes:
 +
      - /var/www/website1.uk/html:/usr/share/nginx/html:ro
 +
    environment:
 +
      - VIRTUAL_HOST=website1.uk
 +
  nginx-website2.uk:
 +
    image: nginx
 +
    container_name: nginx-website2.uk
 +
    expose:
 +
      - "80"
 
     volumes:
 
     volumes:
−
       - "/home/user/.MakeMKV_DOCKER:/config:rw"
+
       - /var/www/website2.uk/html:/usr/share/nginx/html:ro
−
      - "/home/user/:/storage:ro"
 
−
      - "/home/user/ToDo/MakeMKV/output:/output:rw"
 
−
    devices:
 
−
      - "/dev/sr0:/dev/sr0"
 
−
      - "/dev/sg0:/dev/sg0"
 
 
     environment:
 
     environment:
−
       - USER_ID=1000
+
       - VIRTUAL_HOST=website2.uk
−
      - GROUP_ID=1000
+
networks:
−
      - TZ=Europe/London
+
  default:
−
      - MAKEMKV_KEY=your_licence_key
+
    external:
−
       - AUTO_DISC_RIPPER=1
+
       name: nginx-proxy
  
−
'''Troubleshooting'''
+
'''Viewing Logs'''
  
−
PROBLEM = "driver failed programming external connectivity on endpoint makemkv: Error starting userland proxy: listen tcp6 [::]:5800: socket: address family not supported by protocol."
+
docker-compose logs nginx-website1.uk
 +
docker-compose logs nginx-website2.uk
  
−
SOLUTION = Put 0.0.0.0:5801 in the published ports line of docker compose to restrict the network to IPv4.
+
'''Proxy Manager'''
  
−
'''Docker Process Output'''
+
This is a web front end to manage 'nginx-proxy', where you can choose containers and create SSL certificates etc.
  
−
CONTAINER ID  IMAGE              COMMAND  CREATED        STATUS        PORTS                              NAMES
+
https://cyberhost.uk/npm-setup/
−
4a8b3106b00b  jlesage/handbrake  "/init"  40 hours ago  Up 40 hours  5900/tcp, 0.0.0.0:5802->5800/tcp  handbrake
 
−
89fe3ba8a31e  jlesage/makemkv    "/init"  40 hours ago  Up 40 hours  5900/tcp, 0.0.0.0:5801->5800/tcp  makemkv
 
  
−
=== HandBrake ===
+
'''Various'''
  
−
'''This will NOT work on a Raspberry Pi.'''
+
https://hub.docker.com/_/nginx
  
−
Use this in combination with [[FFmpeg_DVD|ffmpeg]] or [[MakeMKV]] (as shown below) and [[FileBot]] to process your media through to your media server - like [[Emby]] or [[Plex]]..
+
https://blog.ssdnodes.com/blog/host-multiple-websites-docker-nginx/
  
−
I have changed the port from 5800 to 5802 because Jocelyn's other Docker image for MakeMKV uses the same port (so I move that one as well to 5801 - see above).
+
https://github.com/nginx-proxy/nginx-proxy
  
−
To make this work with your DVD drive (/dev/sr0) '''you need to have the second device''' (/dev/sg0) in order for it to work. I don't get it, but it works.
+
=== Typical LEMP ===
  
−
[https://www.youtube.com/watch?v=yKww_gg1fVI YouTube / DB Tech - How to install HandBrake in Docker]
+
https://adoltech.com/blog/how-to-set-up-nginx-php-fpm-and-mysql-with-docker-compose/
  
−
[https://dbtechreviews.com/2020/03/how-to-install-handbrake-in-openmediavault-and-docker/ Blog / DB Tech - How to install HandBrake in Docker]
+
=== WordPress ===
  
−
[https://github.com/jlesage/docker-handbrake Docker HandBrake by Jocelyn Le Sage]
+
https://hub.docker.com/_/wordpress/
  
−
[https://hub.docker.com/r/jlesage/handbrake Docker Image by Jocelyn Le Sage]
+
==== PHP File Uploads Fix ====
  
−
<code>/root/docker/stacks/handbrake/docker-compose.yml</code>
+
Create a new PHP configuration file, and name it '''php.ini'''. Add the following configuration then save the changes.
  
−
  version: '3'
+
  # Hide PHP version
−
  services:
+
  expose_php = Off
−
  handbrake:
+
−
    image: jlesage/handbrake
+
# Allow HTTP file uploads
−
    container_name: handbrake
+
file_uploads = On
−
    ports:
+
−
      - "0.0.0.0:5802:5800"
+
# Maximum size of an uploaded file
−
    volumes:
+
upload_max_filesize = 64M
−
      - "/home/user:/storage:ro"
+
−
      - "/home/user/ToDo/HandBrake/config:/config:rw"
+
# Maximum size of form post data
−
      - "/home/user/ToDo/HandBrake/watch:/watch:rw"
+
post_max_size = 64M
−
      - "/home/user/ToDo/HandBrake/output:/output:rw"
+
−
    devices:
+
# Maximum Input Variables
−
      - "/dev/sr0:/dev/sr0"
+
max_input_vars = 3000
−
      - "/dev/sg0:/dev/sg0"
 
−
    environment:
 
−
      - USER_ID=1000
 
−
      - GROUP_ID=1000
 
−
      - TZ=Europe/London
 
  
−
'''Docker Process Output'''
+
Update the docker-compose.yml to bind the '''php.ini''' to the wordpress container and then restart the WordPress container.
  
−
  CONTAINER ID  IMAGE              COMMAND  CREATED        STATUS        PORTS                              NAMES
+
  volumes:
−
4a8b3106b00b  jlesage/handbrake  "/init"  40 hours ago  Up 40 hours  5900/tcp, 0.0.0.0:5802->5800/tcp  handbrake
+
  - ./data/config/php.ini:/usr/local/etc/php/conf.d/php.ini
−
89fe3ba8a31e  jlesage/makemkv    "/init"  40 hours ago  Up 40 hours  5900/tcp, 0.0.0.0:5801->5800/tcp  makemkv
 
  
−
=== Automated Downloader ===
+
==== WordPress Clone ====
  
−
This takes the hassle out of going through the various web sites to find stuff and be bombarded with ads and pop-ups.
+
Create your A record in DNS using [http://wiki.indie-it.com/wiki/AWS_Route_53#cli53 AWS Route 53 CLI]...
  
−
# Jackett
+
cli53 rrcreate domain.co.uk 'staging 300 A 123.456.78.910'
−
# Radarr
 
−
# Transmission
 
  
−
Jackett > Radarr > Transmission
+
Create your docker folder for the cloned staging test web site...
  
−
Create a docker network which Jackett and Radarr share to talk to each other...
+
mkdir -p ~/docker/stacks/staging.domain.co.uk/data/{db,html}
  
−
sudo docker network create jackett-radarr
+
Edit your docker compose file, with 2 containers, making sure you use the same network as your Nginx Proxy Manager...
  
−
...then continue setting up the containers below.
+
<code>~/docker/stacks/staging.domain.co.uk/docker-compose.yml</code>
  
−
==== Jackett ====
+
  version: "3"
−
 
 
−
Jackett works as a proxy server: it translates queries from apps (Sonarr, SickRage, CouchPotato, Mylar, etc) into tracker-site-specific http queries, parses the html response, then sends results back to the requesting software. This allows for getting recent uploads (like RSS) and performing searches. Jackett is a single repository of maintained indexer scraping and translation logic - removing the burden from other apps.
 
−
 
 
−
'''So, this is where you build your list of web sites "with content you want" ;-)'''
 
−
 
 
−
https://fleet.linuxserver.io/image?name=linuxserver/jackett
 
−
 
 
−
https://docs.linuxserver.io/images/docker-jackett
 
−
 
 
−
https://hub.docker.com/r/linuxserver/jackett
 
−
 
 
−
https://github.com/Jackett/Jackett
 
−
 
 
−
<code>/root/docker/stacks/docker-compose.yml</code>
 
−
 
 
−
  version: "2.1"
 
 
  services:
 
  services:
−
   jackett:
+
   staging.domain.co.uk-wordpress_db:
−
     image: ghcr.io/linuxserver/jackett
+
     image: mysql:5.7
−
     container_name: jackett
+
    container_name: staging.domain.co.uk-wordpress_db
 +
    volumes:
 +
      - ./data/db:/var/lib/mysql
 +
     restart: always
 
     environment:
 
     environment:
−
      - PUID=1000
 
−
      - PGID=1000
 
 
       - TZ=Europe/London
 
       - TZ=Europe/London
−
       - AUTO_UPDATE=true
+
       - MYSQL_ROOT_PASSWORD=changeme
 +
      - MYSQL_DATABASE=dbname
 +
      - MYSQL_USER=dbuser
 +
      - MYSQL_PASSWORD=changeme
 +
  staging.domain.co.uk-wordpress:
 +
    depends_on:
 +
      - staging.domain.co.uk-wordpress_db
 +
    image: wordpress:latest
 +
    container_name: staging.domain.co.uk-wordpress
 
     volumes:
 
     volumes:
−
       - ./data/config:/config
+
       - ./data/html:/var/www/html
−
       - ./data/downloads:/downloads
+
    restart: always
−
     networks:
+
    environment:
−
       - jackett-radarr
+
      - TZ=Europe/London
−
    ports:
+
      - VIRTUAL_HOST=staging.domain.co.uk
−
       - 0.0.0.0:9117:9117
+
      - WORDPRESS_DB_HOST=staging.domain.co.uk-wordpress_db:3306
−
     restart: unless-stopped
+
      - WORDPRESS_DB_NAME=dbname
 +
      - WORDPRESS_DB_USER=dbuser
 +
      - WORDPRESS_DB_PASSWORD=changeme
 +
  staging.domain.co.uk-wordpress-cli:
 +
    image: wordpress:cli
 +
    container_name: staging.domain.co.uk-wordpress-cli
 +
    volumes:
 +
       - ./data/html:/var/www/html
 +
     environment:
 +
       - TZ=Europe/London
 +
      - WP_CLI_CACHE_DIR=/tmp/
 +
      - VIRTUAL_HOST=staging.domain.co.uk
 +
       - WORDPRESS_DB_HOST=staging.domain.co.uk-wordpress_db:3306
 +
      - WORDPRESS_DB_NAME=dbname
 +
      - WORDPRESS_DB_USER=dbuser
 +
      - WORDPRESS_DB_PASSWORD=changeme
 +
    working_dir: /var/www/html
 +
     user: "33:33"
 
  networks:
 
  networks:
−
   jackett-radarr:
+
   default:
−
     external: true
+
     external:
 +
      name: nginx-proxy-manager
  
−
==== Radarr ====
+
Start containers with correct settings and credentials for existing live web site (so that the docker startup script sets up the MySQL permissions)...
  
−
Radarr is a movie collection manager for Usenet and BitTorrent users. It can monitor multiple RSS feeds for new movies and will interface with clients and indexers to grab, sort, and rename them. It can also be configured to automatically upgrade the quality of existing files in the library when a better quality format becomes available.
+
docker-compose up -d
  
−
'''Radarr is the 'man-in-the-middle' to take lists from Jackett and pass them to Transmission to download.'''
+
Check the logs to make sure all is well...
  
−
Radarr is the web UI to search for "the content you want" ;-)
+
docker logs staging.domain.co.uk-wordpress
 +
docker logs staging.domain.co.uk-wordpress_db
 +
 
 +
Copy the WordPress files to the host folder and correct ownership...
  
−
https://radarr.video/
+
rsync -av /path/to/backup_unzipped_wordpress/ ~/docker/stacks/staging.domain.co.uk/html/
 +
chown -R www-data:www-data ~/docker/stacks/staging.domain.co.uk/html/
  
−
https://docs.linuxserver.io/images/docker-radarr
+
Copy the sql file in to the running '''mysql''' container...
  
−
https://github.com/linuxserver/docker-radarr
+
docker cp /path/to/backup_unzipped_wordpress/db_name.sql mysql_container_name:/tmp/
  
−
https://sasquatters.com/radarr-docker/
+
Log in to the database container...
  
−
https://www.smarthomebeginner.com/install-radarr-using-docker/
+
docker exec -it mysql_container_name bash
  
−
https://trash-guides.info/Radarr/
+
Check and if necessary, change the timezone...
  
−
So, you use Jackett as an Indexer of content, which answers questions from Radarr, which passes a good result to Transmission...
+
date
 +
mv /etc/localtime /etc/localtime.backup
 +
ln -s /usr/share/zoneinfo/Europe/London /etc/localtime
 +
date
  
−
# Settings > Profiles > delete all but 'any' (and edit that to get rid of naff qualities at the bottom)
+
Delete and create the database...
−
# Indexers > Add Indexer > Torznab > complete and TEST then SAVE
 
−
# Download Clients > Add Download Client > Transmission > complete and TEST and SAVE
 
  
−
<code>/root/docker/stacks/docker-compose.yml</code>
+
mysql -u root -p -e "DROP DATABASE db_name; CREATE DATABASE db_name;"
  
−
version: "2.1"
+
Import the database from the sql file, check and exit out of the container...
−
services:
 
−
  radarr:
 
−
    image: ghcr.io/linuxserver/radarr
 
−
    container_name: radarr
 
−
    environment:
 
−
      - PUID=1000
 
−
      - PGID=1000
 
−
      - TZ=Europe/London
 
−
    volumes:
 
−
      - ./data/config:/config
 
−
      - ./data/downloads:/downloads
 
−
      - ./data/torrents:/torrents
 
−
    networks:
 
−
      - jackett-radarr
 
−
    ports:
 
−
      - 0.0.0.0:7878:7878
 
−
    restart: unless-stopped
 
−
networks:
 
−
  jackett-radarr:
 
−
    external: true
 
  
−
==== NZBGet ====
+
mysql -u root -p mysql_db_name < /tmp/db_name.sql
 +
mysql -u root -p -e "use db_name; show tables;"
 +
rm /tmp/db_name.sql
 +
exit
  
−
Nzbget is a usenet downloader. That's all there is to say on that... even the official home page keeps things quiet.
+
Edit the wp-config.php on your host server to match new DB_HOST and also add extra variables to be sure...
  
−
https://nzbget.net
+
nano /path/to/docker/folder/html/wp-config.php
 +
define( 'WP_HOME', <nowiki>'http://staging.domain.co.uk'</nowiki> );
 +
define( 'WP_SITEURL', <nowiki>'http://staging.domain.co.uk'</nowiki> );
  
−
https://hub.docker.com/r/linuxserver/nzbget
+
Install [http://wiki.indie-it.com/wiki/WordPress#Install WordPress CLI] in the running container...
  
−
Webui can be found at <your-ip>:6789 and the default login details (change ASAP) are...
+
docker exec -it wordpress_container_name bash
  
−
username: nzbget
+
Search and replace the original site url...
−
password: tegbzn6789
 
  
−
=== Tandoor Recipe Manager ===
+
./wp --allow-root search-replace <nowiki>'http://www.domain.co.uk/' 'http://staging.domain.co.uk/'</nowiki> --dry-run
 +
./wp --allow-root search-replace <nowiki>'http://www.domain.co.uk/' 'http://staging.domain.co.uk/'</nowiki>
  
−
The recipe manager that allows you to manage your ever growing collection of digital recipes.
+
Start your web browser and go to the test staging web site!
  
−
https://docs.tandoor.dev/install/docker/
+
==== WordPress CLI ====
  
−
https://www.youtube.com/watch?v=7-nb3muJxI0
+
In your stack, set up the usual two DB + WordPress containers, then add a third services section for wp-cli...
−
 
 
−
<code>/root/docker/stacks/tandoor/docker-compose.yml</code>
 
  
 
  version: "3"
 
  version: "3"
 
  services:
 
  services:
−
   db_recipes:
+
   www.domain.uk-wordpress_db:
−
     container_name: tandoor_db
+
    image: mysql:5.7
 +
     container_name: www.domain.uk-wordpress_db
 +
    volumes:
 +
      - ./data/db:/var/lib/mysql
 
     restart: always
 
     restart: always
−
     image: postgres:11-alpine
+
     environment:
−
    volumes:
+
      - MYSQL_ROOT_PASSWORD=password
−
       - ./data/postgresql:/var/lib/postgresql/data
+
      - MYSQL_DATABASE=wordpress
−
     env_file:
+
      - MYSQL_USER=wordpress
−
       - ./.env
+
       - MYSQL_PASSWORD=password
−
  web_recipes:
+
  www.domain.uk-wordpress:
−
    container_name: tandoor_web
+
     depends_on:
−
     image: vabene1111/recipes
+
       - www.domain.uk-wordpress_db
−
    restart: always
+
     image: wordpress:latest
−
     env_file:
+
     container_name: www.domain.uk-wordpress
−
      - ./.env
 
 
     volumes:
 
     volumes:
−
       - ./data/mediafiles:/opt/recipes/mediafiles
+
       - ./data/html:/var/www/html
−
      - ./data/staticfiles:/opt/recipes/staticfiles
+
     expose:
−
      - nginx_config:/opt/recipes/nginx/conf.d
+
       - 80
−
     depends_on:
 
−
       - db_recipes
 
−
  nginx_recipes:
 
−
    container_name: tandoor_nginx
 
−
    image: nginx:mainline-alpine
 
 
     restart: always
 
     restart: always
−
     ports:
+
     environment:
−
       - 80
+
       - VIRTUAL_HOST=www.domain.uk
−
    env_file:
+
      - WORDPRESS_DB_HOST=www.domain.uk-wordpress_db:3306
−
       - ./.env
+
      - WORDPRESS_DB_NAME=wordpress
−
     depends_on:
+
       - WORDPRESS_DB_USER=wordpress
−
      - web_recipes
+
      - WORDPRESS_DB_PASSWORD=password
 +
  '''www.domain.uk-wordpress-cli:'''
 +
     image: wordpress:cli
 +
    container_name: www.domain.uk-wordpress-cli
 
     volumes:
 
     volumes:
−
       - ./data/mediafiles:/media
+
       - ./data/html:/var/www/html
−
       - ./data/staticfiles:/static
+
    environment:
−
       - nginx_config:/etc/nginx/conf.d:ro
+
       - WP_CLI_CACHE_DIR=/tmp/
−
volumes:
+
      - VIRTUAL_HOST=www.domain.uk
−
  nginx_config:
+
      - WORDPRESS_DB_HOST=www.domain.uk-wordpress_db:3306
 +
      - WORDPRESS_DB_NAME=wordpress
 +
      - WORDPRESS_DB_USER=wordpress
 +
       - WORDPRESS_DB_PASSWORD=password
 +
    working_dir: /var/www/html
 +
    user: "33:33"
 
  networks:
 
  networks:
 
   default:
 
   default:
Line 1,276: Line 1,596:
 
       name: nginx-proxy-manager
 
       name: nginx-proxy-manager
  
−
=== Project Send ===
+
...then start it all up.
  
−
Self-hosted file sharing... small, simple, secure.
+
docker-compose up -d
  
−
https://www.projectsend.org
+
Then, run your wp-cli commands (e.g. wp user list) on the end of a docker run command...
 +
 
 +
docker-compose run --rm www.domain.uk-wordpress-cli '''wp --info'''
 +
docker-compose run --rm www.domain.uk-wordpress-cli '''wp cli version'''
 +
docker-compose run --rm www.domain.uk-wordpress-cli '''wp user list'''
 +
docker-compose run --rm www.domain.uk-wordpress-cli '''wp help theme'''
 +
docker-compose run --rm www.domain.uk-wordpress-cli '''wp theme delete --all'''
 +
 
 +
==== SSL Behind A Reverse Proxy ====
 +
 
 +
https://wiki.indie-it.com/wiki/WordPress#SSL_When_Using_A_Reverse_Proxy
  
−
https://docs.linuxserver.io/images/docker-projectsend
+
=== Email Server (mailu) ===
  
−
https://github.com/linuxserver/docker-projectsend
+
Mailu is a simple yet full-featured mail server as a set of Docker images. It is free software (both as in free beer and as in free speech), open to suggestions and external contributions. The project aims at providing people with an easily setup, easily maintained and full-featured mail server while not shipping proprietary software nor unrelated features often found in popular groupware.
  
−
'''What they don't tell you in the docs is that you need a database backend - which is not in the docker compose file.'''
+
https://mailu.io/1.7/
  
−
So, we just add a MariaDB database container to the stack!
+
https://hub.docker.com/u/mailu
  
−
Create your subdomain A record in DNS...
+
https://github.com/Mailu/Mailu
  
−
cli53 rrcreate domain.uk 'send 300 A 123.45.678.90'
+
'''Postfix Admin'''
  
−
Create your Proxy Host in Ngnix Proxy Manager with an SSL...
+
https://hub.docker.com/_/postfixadmin
  
−
<nowiki>https://send.domain.uk</nowiki>
+
=== Email Server (docker-mailserver) ===
  
−
Create directories on the server for the Docker container files...
+
https://github.com/docker-mailserver
  
−
sudo -i
+
https://github.com/docker-mailserver/docker-mailserver
−
mkdir -p /root/docker/stacks/projectsend/data/{config,db,files}
 
−
chown -R 1000:1000 /root/docker/stacks/projectsend/data/files
 
  
−
<code>/root/docker/stacks/projectsend/docker-compose.yml</code>
+
https://github.com/docker-mailserver/docker-mailserver-admin
  
−
  version: "2.1"
+
https://docker-mailserver.github.io/docker-mailserver/edge/config/security/ssl/#lets-encrypt-recommended
−
  services:
+
 
−
   projectsend:
+
==== Postscreen ====
−
     image: ghcr.io/linuxserver/projectsend
+
 
−
     container_name: projectsend
+
Postscreen is an SMTP filter that blocks spambots (or zombie machines) away from the real Postfix smtpd daemon, so Postfix does not feel overloaded and can process legitimate emails more efficiently.
−
     environment:
+
 
−
       - PUID=1000
+
The example below shows a typical spambot attempt at accessing the SMTP service and being stopped...
−
       - PGID=1000
+
 
 +
Jun 24 10:42:26 mail postfix/postscreen[203907]: CONNECT from [212.70.149.56]:19452 to [172.23.0.2]:25
 +
Jun 24 10:42:26 mail postfix/dnsblog[386054]: addr 212.70.149.56 listed by domain b.barracudacentral.org as 127.0.0.2
 +
Jun 24 10:42:26 mail postfix/dnsblog[402550]: addr 212.70.149.56 listed by domain list.dnswl.org as 127.0.10.3
 +
Jun 24 10:42:26 mail postfix/dnsblog[407802]: addr 212.70.149.56 listed by domain bl.mailspike.net as 127.0.0.2
 +
Jun 24 10:42:26 mail postfix/dnsblog[386155]: addr 212.70.149.56 listed by domain psbl.surriel.com as 127.0.0.2
 +
Jun 24 10:42:29 mail postfix/postscreen[203907]: PREGREET 11 after 2.9 from [212.70.149.56]:19452: EHLO User\r\n
 +
Jun 24 10:42:29 mail postfix/postscreen[203907]: DISCONNECT [212.70.149.56]:19452
 +
 
 +
Postscreen is enabled by default but there are a few settings to tweak to get the best out of it.
 +
 
 +
Edit your <code>data/config/postfix-main.cf</code> file and add the following lines, making sure your Docker host IP is in bold...
 +
 
 +
mynetworks = 127.0.0.0/8 [::1]/128 [fe80::]/64 172.19.0.2/32 '''172.19.0.1/32'''
 +
postscreen_greet_action = drop
 +
postscreen_pipelining_enable = yes
 +
postscreen_pipelining_action = drop
 +
postscreen_non_smtp_command_enable = yes
 +
postscreen_non_smtp_command_action = drop
 +
postscreen_bare_newline_enable = yes
 +
postscreen_bare_newline_action = drop
 +
 
 +
[https://www.linuxbabe.com/mail-server/configure-postscreen-in-postfix-to-block-spambots Enable and Configure Postscreen in Postfix to Block Spambots]
 +
 
 +
==== Postgrey ====
 +
 
 +
[https://github.com/docker-mailserver/docker-mailserver/issues/1957 Local List]
 +
 
 +
==== SpamAssassin ====
 +
 
 +
[https://docker-mailserver.github.io/docker-mailserver/edge/faq/#how-can-i-manage-my-custom-spamassassin-rules Custom Rules]
 +
 
 +
[https://github.com/docker-mailserver/docker-mailserver/issues/365 Bayes Database]
 +
 
 +
SpamAssassin is controlled by Amavis (a fork of MailScanner) with the user 'amavis'.
 +
 
 +
'''Show Bayes Database Stats'''
 +
 
 +
docker exec --user amavis mail.mydomain.org.uk-mailserver sa-learn --dump magic --dbpath /var/lib/amavis/.spamassassin
 +
 
 +
'''Learn Ham'''
 +
 
 +
docker exec --user amavis mail.mydomain.org.uk-mailserver sa-learn --ham --progress /var/mail/mydomain.org.uk/info/cur --dbpath /var/lib/amavis/.spamassassin
 +
 
 +
'''Backup and Restore from Existing Mail Server'''
 +
 
 +
On the old server...
 +
 
 +
/bin/su -l -c '/usr/bin/sa-learn --backup > sa-learn_backup.txt' debian-spamd
 +
rsync -avP /var/lib/spamassassin/sa-learn_backup.txt user@mail.mydomain.org.uk:/tmp/
 +
 
 +
On the new server...
 +
 
 +
docker cp /tmp/sa-learn_backup.txt mail.mydomain.org.uk-mailserver:/tmp/
 +
docker exec --user amavis mail.mydomain.org.uk-mailserver sa-learn --sync --dbpath /var/lib/amavis/.spamassassin
 +
docker exec --user amavis mail.mydomain.org.uk-mailserver sa-learn --clear --dbpath /var/lib/amavis/.spamassassin
 +
docker exec --user amavis mail.mydomain.org.uk-mailserver sa-learn --restore /tmp/sa-learn_backup.txt --dbpath /var/lib/amavis/.spamassassin
 +
docker exec --user amavis mail.mydomain.org.uk-mailserver sa-learn --sync --dbpath /var/lib/amavis/.spamassassin
 +
docker exec --user amavis mail.mydomain.org.uk-mailserver sa-learn --dump magic --dbpath /var/lib/amavis/.spamassassin
 +
 
 +
==== Fail2Ban ====
 +
 
 +
List jails...
 +
 
 +
docker exec -it mail.mydomain.org.uk-mailserver fail2ban-client status
 +
 +
Status
 +
|- Number of jail:  3
 +
`- Jail list:  dovecot, postfix, postfix-sasl
 +
 
 +
Manually ban IP address in named jail...
 +
 
 +
docker exec -it mail.mydomain.org.uk-mailserver fail2ban-client set postfix banip 212.70.149.56
 +
 
 +
Check banned IPs...
 +
 
 +
docker exec -it mail.mydomain.org.uk-mailserver fail2ban-client status postfix
 +
 +
Status for the jail: postfix
 +
|- Filter
 +
|  |- Currently failed: 2
 +
|  |- Total failed:    2
 +
|  `- File list:        /var/log/mail.log
 +
`- Actions
 +
    |- Currently banned: 1
 +
    |- Total banned:    1
 +
    `- Banned IP list:  212.70.149.56
 +
 
 +
https://www.the-lazy-dev.com/en/install-fail2ban-with-docker/
 +
 
 +
==== Backups ====
 +
 
 +
[https://docker-mailserver.github.io/docker-mailserver/edge/faq/#what-about-backups Backups]
 +
 
 +
==== Autodiscover ====
 +
 
 +
Create SRV and A record entries in your DNS for the services...
 +
 
 +
$ORIGIN domain.org.uk.
 +
@ 300 IN TXT "v=spf1 mx ~all; mailconf=<nowiki>https://autoconfig.domain.org.uk/mail/config-v1.1.xml</nowiki>"
 +
_autodiscover._tcp 300 IN SRV 0 0 443 autodiscover.domain.org.uk.
 +
_imap._tcp 300 IN SRV 0 0 0 .
 +
_imaps._tcp 300 IN SRV 0 1 993 mail.domain.org.uk.
 +
_ldap._tcp 300 IN SRV 0 0 636 mail.domain.org.uk.
 +
_pop3._tcp 300 IN SRV 0 0 0 .
 +
_pop3s._tcp 300 IN SRV 0 0 0 .
 +
_submission._tcp 300 IN SRV 0 1 587 mail.domain.org.uk.
 +
autoconfig 300 IN A 3.10.67.19
 +
autodiscover 300 IN A 3.10.67.19
 +
imap 300 IN CNAME mail
 +
mail 300 IN A 3.10.67.19
 +
smtp 300 IN CNAME mail
 +
www 300 IN A 3.10.67.19
 +
 
 +
<code>docker-compose.yml</code>
 +
 
 +
services:
 +
  mailserver-autodiscover:
 +
    image: monogramm/autodiscover-email-settings:latest
 +
    container_name: mail.domain.org.uk-mailserver-autodiscover
 +
    environment:
 +
      - COMPANY_NAME=My Company
 +
      - SUPPORT_URL=<nowiki>https://autodiscover.domain.org.uk</nowiki>
 +
      - DOMAIN=domain.org.uk
 +
      - IMAP_HOST=mail.domain.org.uk
 +
      - IMAP_PORT=993
 +
      - IMAP_SOCKET=SSL
 +
      - SMTP_HOST=mail.domain.org.uk
 +
      - SMTP_PORT=587
 +
      - SMTP_SOCKET=STARTTLS
 +
    restart: unless-stopped
 +
networks:
 +
  default:
 +
    external:
 +
      name: nginx-proxy-manager
 +
 
 +
[https://docker-mailserver.github.io/docker-mailserver/edge/config/best-practices/autodiscover/ Autodiscover]
 +
 
 +
[https://hub.docker.com/r/monogramm/autodiscover-email-settings/ monogramm/autodiscover-email-settings]
 +
 
 +
=== Internet Speedtest ===
 +
 
 +
https://github.com/henrywhitaker3/Speedtest-Tracker
 +
 
 +
=== Emby Media Server ===
 +
 
 +
https://emby.media/docker-server.html
 +
 
 +
https://hub.docker.com/r/emby/embyserver
 +
 
 +
=== AWS CLI ===
 +
 
 +
docker run --rm -it -v "/root/.aws:/root/.aws" amazon/aws-cli configure
 +
docker run --rm -it -v "/root/.aws:/root/.aws" amazon/aws-cli s3 ls
 +
docker run --rm -it -v "/root/.aws:/root/.aws" amazon/aws-cli route53 list-hosted-zones
 +
 
 +
https://docs.aws.amazon.com/cli/latest/userguide/install-cliv2-docker.html
 +
 
 +
=== Let's Encrypt ===
 +
 
 +
Force RENEW a standalone certificate with the new preferred chain of "ISRG Root X1"
 +
 
 +
docker run -it --rm --name certbot -v "/usr/bin:/usr/bin" -v "/root/.aws:/root/.aws" -v "/etc/letsencrypt:/etc/letsencrypt" -v "/var/lib/letsencrypt:/var/lib/letsencrypt" certbot/certbot --force-renewal --preferred-chain "ISRG Root X1" certonly --standalone --email me@mydomain.com --agree-tos -d www.mydomain.com
 +
 
 +
Issue a wildcard certificate...
 +
 
 +
docker run -it --rm --name certbot -v "/usr/bin:/usr/bin" -v "/root/.aws:/root/.aws" -v "/etc/letsencrypt:/etc/letsencrypt" -v "/var/lib/letsencrypt:/var/lib/letsencrypt" certbot/dns-route53 certonly --dns-route53 --domain "example.com" --domain "*.example.com"
 +
 
 +
Check your certificates...
 +
 
 +
docker run -it --rm --name certbot -v "/usr/bin:/usr/bin" -v "/root/.aws:/root/.aws" -v "/etc/letsencrypt:/etc/letsencrypt" -v "/var/lib/letsencrypt:/var/lib/letsencrypt" certbot/certbot certificates
 +
 
 +
Renew a certificate...
 +
 
 +
docker run -it --rm --name certbot -v "/usr/bin:/usr/bin" -v "/root/.aws:/root/.aws" -v "/etc/letsencrypt:/etc/letsencrypt" -v "/var/lib/letsencrypt:/var/lib/letsencrypt" certbot/dns-route53 renew
 +
 
 +
If you have multiple profiles in your .aws/config then you will need to pass the AWS_PROFILE variable to the docker container...
 +
 
 +
docker run -it --rm --name certbot -v "/usr/bin:/usr/bin" -v "/root/.aws:/root/.aws" -v "/etc/letsencrypt:/etc/letsencrypt" -v "/var/lib/letsencrypt:/var/lib/letsencrypt" -e '''"AWS_PROFILE=certbot"''' certbot/dns-route53 renew
 +
 
 +
https://certbot.eff.org/docs/install.html#running-with-docker
 +
 
 +
=== VPN ===
 +
 
 +
==== Gluetun ====
 +
 
 +
Gluetun VPN client
 +
 
 +
Lightweight swiss-knife-like VPN client to tunnel to Cyberghost, ExpressVPN, FastestVPN, HideMyAss, IPVanish, IVPN, Mullvad, NordVPN, Perfect Privacy, Privado, Private Internet Access, PrivateVPN, ProtonVPN, PureVPN, Surfshark, TorGuard, VPNUnlimited, VyprVPN, WeVPN and Windscribe VPN servers using Go, OpenVPN or Wireguard, iptables, DNS over TLS, ShadowSocks and an HTTP proxy.
 +
 
 +
[https://github.com/qdm12/gluetun Github]
 +
 
 +
[https://github.com/qdm12/gluetun/wiki/Connect-a-container-to-gluetun Connect a container to Gluetun]
 +
 
 +
==== OpenVPN ====
 +
 
 +
'''Server'''
 +
 
 +
https://hub.docker.com/r/linuxserver/openvpn-as
 +
 
 +
'''Client'''
 +
 
 +
https://hub.docker.com/r/dperson/openvpn-client
 +
 
 +
'''Routing Containers Through Container'''
 +
 
 +
sudo docker run -it --net=container:vpn -d some/docker-container
 +
 
 +
'''OpenVPN-PiHole'''
 +
 
 +
https://github.com/Simonwep/openvpn-pihole
 +
 
 +
==== WireHole ====
 +
 
 +
WireHole is a combination of WireGuard, PiHole, and Unbound in a docker-compose project with the intent of enabling users to quickly and easily create and deploy a personally managed full or split-tunnel WireGuard VPN with ad blocking capabilities (via Pihole), and DNS caching with additional privacy options (via Unbound).
 +
 
 +
https://github.com/IAmStoxe/wirehole
 +
 
 +
To view a QR code, run this ...
 +
 
 +
docker exec -it wireguard /app/show-peer 1
 +
 
 +
==== WireGuard ====
 +
 
 +
'''Use [https://wiki.indie-it.com/wiki/Docker#WireHole WireHole] instead!'''
 +
 
 +
<code>docker-compose.yml</code>
 +
 
 +
  version: "2.1"
 +
  services:
 +
   wireguard:
 +
     image: ghcr.io/linuxserver/wireguard
 +
     container_name: wireguard
 +
    cap_add:
 +
      - NET_ADMIN
 +
      - SYS_MODULE
 +
     environment:
 +
       - PUID=1000
 +
       - PGID=1000
 
       - TZ=Europe/London
 
       - TZ=Europe/London
−
       - MAX_UPLOAD=100
+
      - SERVERURL=wireguard.domain.uk
 +
      - SERVERPORT=51820
 +
      - PEERS=3
 +
      - PEERDNS=auto
 +
      - INTERNAL_SUBNET=10.13.13.0
 +
      - ALLOWEDIPS=0.0.0.0/0
 +
    volumes:
 +
      - ./data/config:/config
 +
      - /lib/modules:/lib/modules
 +
    ports:
 +
      - 51820:51820/udp
 +
    sysctls:
 +
      - net.ipv4.conf.all.src_valid_mark=1
 +
    restart: unless-stopped
 +
 
 +
https://hub.docker.com/r/linuxserver/wireguard
 +
 
 +
'''To show the QR code'''
 +
 
 +
docker exec -it wireguard /app/show-peer 1
 +
docker exec -it wireguard /app/show-peer 2
 +
docker exec -it wireguard /app/show-peer 3
 +
 
 +
'''Error: const struct ipv6_stub'''
 +
 
 +
If you receive an error in the container logs about not being able to compile the kernel module, then follow the instructions to compile the WireGuard kernel module and tools in your host system.
 +
 
 +
https://github.com/linuxserver/docker-wireguard/issues/46#issuecomment-708278250
 +
 
 +
'''[https://www.youtube.com/watch?v=vUyHGF1HMsw Force Docker Containers to use a VPN for connection]'''
 +
 
 +
=== ffmpeg ===
 +
 
 +
docker pull jrottenberg/ffmpeg
 +
docker run jrottenberg/ffmpeg -h
 +
docker run jrottenberg/ffmpeg -i /path/to/input.mkv -stats $ffmpeg_options - > out.mp4
 +
docker run -v $(pwd):$(pwd) -w $(pwd) jrottenberg/ffmpeg -y -i input.mkv -t 00:00:05.00 -vf scale=-1:360 output.mp4
 +
 
 +
https://registry.hub.docker.com/r/jrottenberg/ffmpeg
 +
 
 +
https://github.com/jrottenberg/ffmpeg
 +
 
 +
https://medium.com/coconut-stories/using-ffmpeg-with-docker-94523547f35c
 +
 
 +
https://github.com/linuxserver/docker-ffmpeg
 +
 
 +
=== MediaInfo ===
 +
 
 +
Install ...
 +
 
 +
sudo docker pull jlesage/mediainfo
 +
 
 +
Run ...
 +
 
 +
docker run --rm --name=mediainfo -e USER_ID=$(id -u) -e GROUP_ID=$(id -g) -v $(pwd):$(pwd):ro jlesage/mediainfo su-exec "$(id -u):$(id -g)" /usr/bin/mediainfo --help
 +
 
 +
https://github.com/jlesage/docker-mediainfo
 +
 
 +
=== MKV Toolnix ===
 +
 
 +
Install ...
 +
 
 +
sudo docker pull jlesage/mkvtoolnix
 +
 
 +
Run ...
 +
 
 +
'''mkvextract'''
 +
 
 +
docker run --rm --name=mkvextract -e USER_ID=$(id -u) -e GROUP_ID=$(id -g) -v $(pwd):$(pwd):rw jlesage/mkvtoolnix su-exec "$(id -u):$(id -g)" /usr/bin/mkvextract "$(pwd)/filename.mkv" tracks 3:"$(pwd)/filename.eng.srt"
 +
 
 +
'''mkvpropedit'''
 +
 
 +
docker run --rm --name=mkvextract -e USER_ID=$(id -u) -e GROUP_ID=$(id -g) -v $(pwd):$(pwd):rw jlesage/mkvtoolnix su-exec "$(id -u):$(id -g)" /usr/bin/mkvpropedit "$(pwd)/filename.mkv" --edit track:a1 --set language=eng
 +
 
 +
https://github.com/jlesage/docker-mkvtoolnix
 +
 
 +
=== MakeMKV ===
 +
 
 +
'''This will NOT work on a Raspberry Pi.'''
 +
 
 +
https://github.com/jlesage/docker-makemkv
 +
 
 +
Use this in combination with [[FFmpeg_DVD|ffmpeg]] or [[HandBrake]] (as shown below) and [[FileBot]] to process your media through to your media server - like [[Emby]] or [[Plex]]..
 +
 
 +
MakeMKV > HandBrake > FileBot > Emby
 +
 
 +
To make this work with your DVD drive (/dev/sr0) '''you need to have the second device''' (/dev/sg0) in order for it to work. I don't get it, but it works.
 +
 
 +
<code>/root/docker/stacks/makemkv/docker-compose.yml</code>
 +
 
 +
version: '3'
 +
services:
 +
  makemkv:
 +
    image: jlesage/makemkv
 +
    container_name: makemkv
 +
    ports:
 +
      - "0.0.0.0:5801:5800"
 +
    volumes:
 +
      - "/home/user/.MakeMKV_DOCKER:/config:rw"
 +
      - "/home/user/:/storage:ro"
 +
      - "/home/user/ToDo/MakeMKV/output:/output:rw"
 +
    devices:
 +
      - "/dev/sr0:/dev/sr0"
 +
      - "/dev/sg0:/dev/sg0"
 +
    environment:
 +
      - USER_ID=1000
 +
      - GROUP_ID=1000
 +
      - TZ=Europe/London
 +
      - MAKEMKV_KEY=your_licence_key
 +
      - AUTO_DISC_RIPPER=1
 +
 
 +
'''Troubleshooting'''
 +
 
 +
PROBLEM = "driver failed programming external connectivity on endpoint makemkv: Error starting userland proxy: listen tcp6 [::]:5800: socket: address family not supported by protocol."
 +
 
 +
SOLUTION = Put 0.0.0.0:5801 in the published ports line of docker compose to restrict the network to IPv4.
 +
 
 +
'''Docker Process Output'''
 +
 
 +
CONTAINER ID  IMAGE              COMMAND  CREATED        STATUS        PORTS                              NAMES
 +
4a8b3106b00b  jlesage/handbrake  "/init"  40 hours ago  Up 40 hours  5900/tcp, 0.0.0.0:5802->5800/tcp  handbrake
 +
89fe3ba8a31e  jlesage/makemkv    "/init"  40 hours ago  Up 40 hours  5900/tcp, 0.0.0.0:5801->5800/tcp  makemkv
 +
 
 +
'''Command Line'''
 +
 
 +
docker run --rm -v "/home/user/.MakeMKV_DOCKER:/config:rw" -v "/home/user/:/storage:ro" -v "/home/user/ToDo/MakeMKV/output:/output:rw" --device /dev/sr0 --device /dev/sg0 --device /dev/sg1 jlesage/makemkv /opt/makemkv/bin/makemkvcon mkv disc:0 all /output
 +
 
 +
https://github.com/jlesage/docker-makemkv/issues/141
 +
 
 +
=== HandBrake ===
 +
 
 +
'''This will NOT work on a Raspberry Pi.'''
 +
 
 +
Use this in combination with [[FFmpeg_DVD|ffmpeg]] or [[MakeMKV]] (as shown below) and [[FileBot]] to process your media through to your media server - like [[Emby]] or [[Plex]]..
 +
 
 +
I have changed the port from 5800 to 5802 because Jocelyn's other Docker image for MakeMKV uses the same port (so I move that one as well to 5801 - see above).
 +
 
 +
To make this work with your DVD drive (/dev/sr0) '''you need to have the second device''' (/dev/sg0) in order for it to work. I don't get it, but it works.
 +
 
 +
[https://www.youtube.com/watch?v=yKww_gg1fVI YouTube / DB Tech - How to install HandBrake in Docker]
 +
 
 +
[https://dbtechreviews.com/2020/03/how-to-install-handbrake-in-openmediavault-and-docker/ Blog / DB Tech - How to install HandBrake in Docker]
 +
 
 +
[https://github.com/jlesage/docker-handbrake Docker HandBrake by Jocelyn Le Sage]
 +
 
 +
[https://hub.docker.com/r/jlesage/handbrake Docker Image by Jocelyn Le Sage]
 +
 
 +
<code>/root/docker/stacks/handbrake/docker-compose.yml</code>
 +
 
 +
version: '3'
 +
services:
 +
  handbrake:
 +
    image: jlesage/handbrake
 +
    container_name: handbrake
 +
    ports:
 +
      - "0.0.0.0:5802:5800"
 +
    volumes:
 +
      - "/home/paully:/storage:ro"
 +
      - "/home/paully/ToDo/HandBrake/config:/config:rw"
 +
      - "/home/paully/ToDo/HandBrake/watch:/watch:rw"
 +
      - "/home/paully/ToDo/HandBrake/output:/output:rw"
 +
    devices:
 +
      - "/dev/sr0:/dev/sr0"
 +
      - "/dev/sg0:/dev/sg0"
 +
    environment:
 +
      - USER_ID=1000
 +
      - GROUP_ID=1000
 +
      - TZ=Europe/London
 +
 
 +
'''Docker Process Output'''
 +
 
 +
CONTAINER ID  IMAGE              COMMAND  CREATED        STATUS        PORTS                              NAMES
 +
4a8b3106b00b  jlesage/handbrake  "/init"  40 hours ago  Up 40 hours  5900/tcp, 0.0.0.0:5802->5800/tcp  handbrake
 +
89fe3ba8a31e  jlesage/makemkv    "/init"  40 hours ago  Up 40 hours  5900/tcp, 0.0.0.0:5801->5800/tcp  makemkv
 +
 
 +
'''Command Line'''
 +
 
 +
docker run --rm -v "/home/paully/:/storage:ro" -v "/home/paully/ToDo/HandBrake/config:/config:rw" -v "/home/paully/ToDo/HandBrake/watch:/watch:rw" -v "/home/paully/ToDo/HandBrake/output:/output:rw" --device /dev/sr0 --device /dev/sg0 jlesage/handbrake /usr/bin/HandBrakeCLI --input "/output/file.mkv" --stop-at duration:120 --preset 'Fast 480p30' --non-anamorphic --encoder-preset slow --quality 22 --deinterlace --lapsharp --audio 1 --aencoder copy:ac3 --no-markers --output "/output/file.mp4"
 +
 
 +
docker run --rm -v "/home/paully/:/storage:ro" -v "/home/paully/ToDo/HandBrake/config:/config:rw" -v "/home/paully/ToDo/HandBrake/watch:/watch:rw" -v "/home/paully/ToDo/HandBrake/output:/output:rw" jlesage/handbrake /usr/bin/HandBrakeCLI --input "/storage/input.mkv" --preset 'Super HQ 2160p60 4K HEVC Surround' --encoder x265 --non-anamorphic --audio 1 --aencoder copy:eac3 --no-markers --output "/output/output.mkv"
 +
 
 +
=== FileBot ===
 +
 
 +
==== Setup ====
 +
 
 +
Create your directories for data volumes (https://github.com/jlesage/docker-filebot#data-volumes) ...
 +
 
 +
mkdir -p ~/filebot/{config,output,watch}
 +
 
 +
The license file received via email can be saved on the host, into the configuration directory of the container (i.e. in the directory mapped to /config). Then, start or restart the container to have it automatically installed. NOTE: The license file is expected to have a .psm extension.
 +
 
 +
==== Usage ====
 +
 
 +
WORK IN PROGRESS
 +
 
 +
Rather than running all the time, we run the image ad-hoc with the rm option to delete the old container each time...
 +
 
 +
docker run --rm --name=filebot -v ~/filebot/config:/config:rw -v $HOME:/storage:rw -v ~/filebot/output:/output:rw -v ~/filebot/watch:/watch:rw -e AMC_ACTION=test jlesage/filebot
 +
 
 +
https://github.com/jlesage/docker-filebot
 +
 
 +
https://github.com/filebot/filebot-docker
 +
 
 +
=== Automated Downloaderr ===
 +
 
 +
This takes the hassle out of going through the various web sites to find stuff and be bombarded with ads and pop-ups.
 +
 
 +
# FlareSolverr
 +
# Prowlarr
 +
# Radarr + Sonarr + Bazarr
 +
# Transmission + NZBGet
 +
# Tdarr
 +
 
 +
FlareSolverr > Prowlarr > Radarr + Sonarr + Bazarr > Transmission + NZBGet > Tdarr
 +
 
 +
 
 +
'''HOW TO RESTART THE RRS IN ORDER ON PORTAINER OR OMV'''
 +
 
 +
Stacks > Click on each one > Stop > count to 10 > Start
 +
 
 +
* WireGuard
 +
* FlareSolverr
 +
* Prowlarr
 +
* Radarr
 +
* Sonarr
 +
 
 +
ONE DAY WE WILL GET A SINGLE STACK WITH ALL THE RIGHT CONTAINERS STARTING IN THE RIGHT ORDER
 +
 
 +
 
 +
https://hotio.dev/containers/autoscan/
 +
 
 +
'''ONE APP TO RULE THEM ALL'''
 +
 
 +
https://github.com/JagandeepBrar/LunaSea
 +
 
 +
 
 +
'''GUIDE FOR DIRECTORY STRUCTURE'''
 +
 
 +
https://trash-guides.info/Hardlinks/How-to-setup-for/Docker/
 +
 
 +
This will enable you to automatically rename files but allow you to copy them to your actual Plex or Emby folders.
 +
 
 +
It is possible to do hard linking and let the rrrrs control all the files but I am not a fan of that.
 +
 
 +
This way, the files get renamed and moved to a 'halfway' house where you can check them and then simply move them to your desired location.
 +
 
 +
'''METHOD'''
 +
 
 +
Create the directories ...
 +
 
 +
mkdir -p /path/to/data/{media,torrents,usenet}/{movies,music,tv}
 +
mkdir -p /path/to/docker/appdata/{radarr,sonarr,bazarr,nzbget}
 +
 
 +
Change the ownership and permissions ...
 +
 
 +
chown -R admin:users /path/to/data/ /path/to/docker/
 +
find /path/to/data/ /path/to/docker/ -type d -exec chmod 0755 {} \;
 +
find /path/to/data/ /path/to/docker/ -type d -exec chmod g+s {} \;
 +
 
 +
Finished directory structure ...
 +
 
 +
/srv/dev-disk-by-uuid-7f81e7b6-1a05-4232-893b-f34c046b2bdb/data/
 +
|-- media
 +
|  |-- movies
 +
|  |-- music
 +
|  `-- tv
 +
|-- torrents
 +
|  |-- movies
 +
|  |-- music
 +
|  `-- tv
 +
`-- usenet
 +
    |-- completed
 +
    |  |-- movies
 +
    |  `-- tv
 +
    |-- movies
 +
    |-- music
 +
    |-- nzb
 +
    |  `-- Movie.Name.720p.nzb.queued
 +
    |-- tmp
 +
    `-- tv
 +
 
 +
'''PORTAINER STACK'''
 +
 
 +
This is from Open Media Vault (OMV) so the volume paths are long.
 +
 
 +
This works but needs the whole VPN thing added (which changes ports etc) but for now ...
 +
 
 +
Portainer > Stacks > Add Stack > Datarr
 +
 
 +
version: "3.2"
 +
services:
 +
  prowlarr:
 +
    container_name: prowlarr
 +
    image: hotio/prowlarr:latest
 +
    restart: unless-stopped
 +
    logging:
 +
      driver: json-file
 +
    network_mode: bridge
 +
    ports:
 +
      - 9696:9696
 +
    environment:
 +
      - PUID=998
 +
      - PGID=100
 +
      - TZ=Europe/London
 +
    volumes:
 +
      - /etc/localtime:/etc/localtime:ro
 +
      - /srv/dev-disk-by-uuid-7f81e7b6-1a05-4232-893b-f34c046b2bdb/docker/appdata/prowlarr:/config
 +
      - /srv/dev-disk-by-uuid-7f81e7b6-1a05-4232-893b-f34c046b2bdb/data:/data
 +
  radarr:
 +
    container_name: radarr
 +
    image: hotio/radarr:latest
 +
    restart: unless-stopped
 +
    logging:
 +
      driver: json-file
 +
    network_mode: bridge
 +
    ports:
 +
      - 7878:7878
 +
    environment:
 +
      - PUID=998
 +
      - PGID=100
 +
      - TZ=Europe/London
 +
    volumes:
 +
      - /etc/localtime:/etc/localtime:ro
 +
      - /srv/dev-disk-by-uuid-7f81e7b6-1a05-4232-893b-f34c046b2bdb/docker/appdata/radarr:/config
 +
      - /srv/dev-disk-by-uuid-7f81e7b6-1a05-4232-893b-f34c046b2bdb/data:/data
 +
  sonarr:
 +
    container_name: sonarr
 +
    image: hotio/sonarr:latest
 +
    restart: unless-stopped
 +
    logging:
 +
      driver: json-file
 +
    network_mode: bridge
 +
    ports:
 +
      - 8989:8989
 +
    environment:
 +
      - PUID=998
 +
      - PGID=100
 +
      - TZ=Europe/London
 +
    volumes:
 +
      - /etc/localtime:/etc/localtime:ro
 +
      - /srv/dev-disk-by-uuid-7f81e7b6-1a05-4232-893b-f34c046b2bdb/docker/appdata/sonarr:/config
 +
      - /srv/dev-disk-by-uuid-7f81e7b6-1a05-4232-893b-f34c046b2bdb/data:/data
 +
  bazarr:
 +
    container_name: bazarr
 +
    image: hotio/bazarr:latest
 +
    restart: unless-stopped
 +
    logging:
 +
      driver: json-file
 +
    network_mode: bridge
 +
    ports:
 +
      - 6767:6767
 +
    environment:
 +
      - PUID=998
 +
      - PGID=100
 +
      - TZ=Europe/London
 +
    volumes:
 +
      - /etc/localtime:/etc/localtime:ro
 +
      - /srv/dev-disk-by-uuid-7f81e7b6-1a05-4232-893b-f34c046b2bdb/docker/appdata/bazarr:/config
 +
      - /srv/dev-disk-by-uuid-7f81e7b6-1a05-4232-893b-f34c046b2bdb/data/media:/data/media
 +
  nzbget:
 +
    container_name: nzbget
 +
    image: hotio/nzbget:latest
 +
    restart: unless-stopped
 +
    logging:
 +
      driver: json-file
 +
    network_mode: bridge
 +
    ports:
 +
      - 6789:6789
 +
    environment:
 +
      - PUID=998
 +
      - PGID=100
 +
      - TZ=Europe/London
 +
    volumes:
 +
      - /etc/localtime:/etc/localtime:ro
 +
      - /srv/dev-disk-by-uuid-7f81e7b6-1a05-4232-893b-f34c046b2bdb/docker/appdata/nzbget:/config
 +
      - /srv/dev-disk-by-uuid-7f81e7b6-1a05-4232-893b-f34c046b2bdb/data/usenet:/data/usenet:rw
 +
 
 +
 
 +
OLD NOTES
 +
 
 +
Create a docker network which Jackett and Radarr share to talk to each other...
 +
 
 +
sudo docker network create jackett-radarr
 +
 
 +
...then continue setting up the containers below.
 +
 
 +
==== FlareSolverr ====
 +
 
 +
FlareSolverr is a proxy server to bypass Cloudflare protection.
 +
 
 +
FlareSolverr starts a proxy server and it waits for user requests in an idle state using few resources. When some request arrives, it uses puppeteer with the stealth plugin to create a headless browser (Chrome). It opens the URL with user parameters and waits until the Cloudflare challenge is solved (or timeout). The HTML code and the cookies are sent back to the user, and those cookies can be used to bypass Cloudflare using other HTTP clients.
 +
 
 +
Radarr > Jackett > FlareSolverr > Internet
 +
 
 +
https://github.com/FlareSolverr/FlareSolverr
 +
 
 +
https://hub.docker.com/r/flaresolverr/flaresolverr
 +
 
 +
Some indexers are protected by CloudFlare or similar services and Jackett is not able to solve the challenges. For these cases, FlareSolverr has been integrated into Jackett. This service is in charge of solving the challenges and configuring Jackett with the necessary cookies. Setting up this service is optional, most indexers don't need it.
 +
 
 +
Install FlareSolverr service using a Docker container, then configure '''FlareSolverr API URL''' in Jackett. For example: http://172.17.0.2:8191
 +
 
 +
Command line...
 +
 
 +
docker run -d \
 +
  --name=flaresolverr \
 +
  -p 8191:8191 \
 +
  -e LOG_LEVEL=info \
 +
  --restart unless-stopped \
 +
  ghcr.io/flaresolverr/flaresolverr:latest
 +
 
 +
Docker compose...
 +
 
 +
---
 +
version: "2.1"
 +
services:
 +
  flaresolverr:
 +
    image: ghcr.io/flaresolverr/flaresolverr:latest
 +
    container_name: flaresolverr
 +
    environment:
 +
      - PUID=1000
 +
      - PGID=1000
 +
      - LOG_LEVEL=${LOG_LEVEL:-info}
 +
      - LOG_HTML=${LOG_HTML:-false}
 +
      - CAPTCHA_SOLVER=${CAPTCHA_SOLVER:-none}
 +
      - TZ=Europe/London
 +
    ports:
 +
      - "${PORT:-8191}:8191"
 +
    restart: unless-stopped
 +
 
 +
'''Usage'''
 +
 
 +
To use it, you have to add a Proxy Server under Prowlarr > Settings > Indexers > Indexer Proxies
 +
 
 +
'''Testing'''
 +
 
 +
curl -L -X POST '<nowiki>http://localhost:8191/v1</nowiki>' -H 'Content-Type: application/json' --data-raw '{ "cmd": "request.get", "url":"<nowiki>https://www.paully.co.uk/</nowiki>", "maxTimeout": 60000 }'
 +
 +
{"status":"ok","message":"","startTimestamp":1651659265156,"endTimestamp":1651659269585,"version":"v2.2.4","solution":
 +
 
 +
==== Jackett ====
 +
 
 +
Jackett works as a proxy server: it translates queries from apps (Sonarr, SickRage, CouchPotato, Mylar, etc) into tracker-site-specific http queries, parses the html response, then sends results back to the requesting software. This allows for getting recent uploads (like RSS) and performing searches. Jackett is a single repository of maintained indexer scraping and translation logic - removing the burden from other apps.
 +
 
 +
'''So, this is where you build your list of web sites "with content you want" ;-)'''
 +
 
 +
https://fleet.linuxserver.io/image?name=linuxserver/jackett
 +
 
 +
https://docs.linuxserver.io/images/docker-jackett
 +
 
 +
https://hub.docker.com/r/linuxserver/jackett
 +
 
 +
https://github.com/Jackett/Jackett
 +
 
 +
<code>/root/docker/stacks/docker-compose.yml</code>
 +
 
 +
---
 +
version: "2.1"
 +
services:
 +
  jackett:
 +
    image: ghcr.io/linuxserver/jackett
 +
    container_name: jackett
 +
    environment:
 +
      - PUID=1000
 +
      - PGID=1000
 +
      - TZ=Europe/London
 +
      - AUTO_UPDATE=true
 +
    volumes:
 +
      - ./data/config:/config
 +
      - ./data/downloads:/downloads
 +
    networks:
 +
      - jackett-radarr
 +
    ports:
 +
      - 0.0.0.0:9117:9117
 +
    restart: unless-stopped
 +
networks:
 +
  jackett-radarr:
 +
    external: true
 +
 
 +
==== Prowlarr ====
 +
 
 +
An alternative to Jackett, and now the '''preferred''' application.
 +
 
 +
https://wiki.servarr.com/prowlarr
 +
 
 +
https://wiki.servarr.com/prowlarr/quick-start-guide
 +
 
 +
https://hub.docker.com/r/linuxserver/prowlarr
 +
 
 +
https://github.com/linuxserver/docker-prowlarr
 +
 
 +
---
 +
version: "2.1"
 +
services:
 +
  prowlarr:
 +
    image: lscr.io/linuxserver/prowlarr:develop
 +
    container_name: prowlarr
 +
    environment:
 +
      - PUID=1000
 +
      - PGID=1000
 +
      - TZ=Europe/London
 +
    volumes:
 +
      - /path/to/data:/config
 +
    ports:
 +
      - 9696:9696
 +
    restart: unless-stopped
 +
 
 +
==== Radarr ====
 +
 
 +
Radarr is a movie collection manager for Usenet and BitTorrent users. It can monitor multiple RSS feeds for new movies and will interface with clients and indexers to grab, sort, and rename them. It can also be configured to automatically upgrade the quality of existing files in the library when a better quality format becomes available.
 +
 
 +
'''Radarr is the 'man-in-the-middle' to take lists from Jackett and pass them to Transmission to download.'''
 +
 
 +
Radarr is the web UI to search for "the content you want" ;-)
 +
 
 +
https://radarr.video/
 +
 
 +
https://docs.linuxserver.io/images/docker-radarr
 +
 
 +
https://github.com/linuxserver/docker-radarr
 +
 
 +
https://sasquatters.com/radarr-docker/
 +
 
 +
https://www.smarthomebeginner.com/install-radarr-using-docker/
 +
 
 +
https://trash-guides.info/Radarr/
 +
 
 +
https://discord.com/channels/264387956343570434/264388019585286144
 +
 
 +
'''[https://wiki.servarr.com/radarr/custom-scripts Custom Scripts]'''
 +
 
 +
So, you use Jackett as an Indexer of content, which answers questions from Radarr, which passes a good result to Transmission...
 +
 
 +
# Settings > Profiles > delete all but 'any' (and edit that to get rid of naff qualities at the bottom)
 +
# Indexers > Add Indexer > Torznab > complete and TEST then SAVE
 +
# Download Clients > Add Download Client > Transmission > complete and TEST and SAVE
 +
 
 +
<code>/root/docker/stacks/docker-compose.yml</code>
 +
 
 +
---
 +
version: "2.1"
 +
services:
 +
  radarr:
 +
    image: ghcr.io/linuxserver/radarr
 +
    container_name: radarr
 +
    environment:
 +
      - PUID=1000
 +
      - PGID=1000
 +
      - TZ=Europe/London
 +
    volumes:
 +
      - ./data/config:/config
 +
      - ./data/downloads:/downloads
 +
      - ./data/torrents:/torrents
 +
    networks:
 +
      - jackett-radarr
 +
    ports:
 +
      - 0.0.0.0:7878:7878
 +
    restart: unless-stopped
 +
networks:
 +
  jackett-radarr:
 +
    external: true
 +
 
 +
==== Sonarr ====
 +
 
 +
Sonarr (formerly NZBdrone) is a PVR for usenet and bittorrent users. It can monitor multiple RSS feeds for new episodes of your favorite shows and will grab, sort and rename them. It can also be configured to automatically upgrade the quality of files already downloaded when a better quality format becomes available.
 +
 
 +
https://sonarr.tv/
 +
 
 +
https://docs.linuxserver.io/images/docker-sonarr
 +
 
 +
Docker Compose using a WireGuard VPN container for internet ...
 +
 
 +
---
 +
version: "2.1"
 +
services:
 +
  sonarr:
 +
    image: ghcr.io/linuxserver/sonarr
 +
    container_name: sonarr
 +
    network_mode: container:wireguard
 +
    environment:
 +
      - PUID=1000
 +
      - PGID=1000
 +
      - TZ=Europe/London
 +
    volumes:
 +
      - ./data/config:/config
 +
      - ./data/downloads:/downloads
 +
      - ./data/torrents:/torrents
 +
    restart: "no"
 +
 
 +
'''UPDATE: 16 FEBRUARY 2023 / latest image is based on Ubuntu Jammy and not Alpine, which will cause a problem if your Docker version is below 20.10.10'''
 +
 
 +
https://docs.linuxserver.io/faq#jammy
 +
 
 +
To fix this, you can either upgrade your Docker (https://docs.docker.com/engine/install/ubuntu/#install-using-the-convenience-script) or add the following lines to your docker-compose.yml file ...
 +
 
 +
security_opt:
 +
  - seccomp=unconfined
 +
 
 +
==== Bazarr ====
 +
 
 +
https://www.bazarr.media/
 +
 
 +
Bazarr is a companion application to Sonarr and Radarr that manages and downloads subtitles based on your requirements.
 +
 
 +
Docker compose file ...
 +
 
 +
---
 +
version: "2.1"
 +
services:
 +
  bazarr:
 +
    image: lscr.io/linuxserver/bazarr:latest
 +
    container_name: bazarr
 +
    environment:
 +
      - PUID=1000
 +
      - PGID=1000
 +
      - TZ=Europe/London
 +
    volumes:
 +
      - /path/to/bazarr/config:/config
 +
      - /path/to/movies:/movies #optional
 +
      - /path/to/tv:/tv #optional
 +
    ports:
 +
      - 6767:6767
 +
    restart: unless-stopped
 +
 
 +
==== NZBGet ====
 +
 
 +
NZBGet is a usenet downloader.
 +
 
 +
You will require the following 3 things at a basic level before you are able to use usenet:-
 +
 
 +
* A usenet provider (Reddit provides a detailed [https://www.reddit.com/r/usenet/wiki/providers#wiki_usenet_services_map Provider Map] for usenet)
 +
* An NZB indexer ([https://www.nzbgeek.info/ NZBGeek])
 +
* A usenet client ([https://nzbget.net/ NZBget])
 +
 
 +
https://nzbget.net
 +
 
 +
https://hub.docker.com/r/linuxserver/nzbget
 +
 
 +
https://www.cogipas.com/nzbget-complete-how-to-guide/
 +
 
 +
The Web GUI can be found at <your-ip>:6789 and the default login details (change ASAP) are...
 +
 
 +
username: nzbget
 +
password: tegbzn6789
 +
 
 +
Docker Compose file...
 +
 
 +
---
 +
version: "2.1"
 +
services:
 +
  nzbget:
 +
    image: lscr.io/linuxserver/nzbget:latest
 +
    container_name: nzbget
 +
    environment:
 +
      - PUID=1000
 +
      - PGID=1000
 +
      - TZ=Europe/London
 +
      - NZBGET_USER=nzbget
 +
      - NZBGET_PASS=tegbzn6789
 +
    volumes:
 +
      - /path/to/data:/config
 +
      - /path/to/downloads:/downloads
 +
    ports:
 +
      - 6789:6789
 +
    restart: unless-stopped
 +
 
 +
==== Tdarr ====
 +
 
 +
Tdarr is a popular conditional transcoding application for processing large (or small) media libraries. The application comes in the form of a click-to-run web-app, which you run on your own device and access through a web browser.
 +
 
 +
Tdarr uses two popular transcoding applications under the hood: FFmpeg and HandBrake (which itself is built on top of FFmpeg).
 +
 
 +
Tdarr works in a distributed manner where you can use multiple devices to process your library together. It does this using 'Tdarr Nodes' which connect with a central server and pick up tasks so you can put all your spare devices to use.
 +
 
 +
Each Node can run multiple 'Tdarr Workers' in parallel to maximize the hardware usage % on that Node. For example, a single FFmpeg worker running on a 64 core CPU may only hit ~30% utilization. Running multiple Workers in parallel allows the CPU to hit 100% utilization, allowing you to process your library more quickly.
 +
 
 +
[https://www.tdarr.io Home Page]
 +
 
 +
[https://docs.tdarr.io Documentation]
 +
 
 +
[https://www.reddit.com/r/Tdarr/comments/qsppw6/using_amd_vcn_for_large_h264_to_h265_transcode/ Using AMD GPU for Transcoding]
 +
 
 +
==== Readarr ====
 +
 
 +
https://academy.pointtosource.com/containers/ebooks-calibre-readarr/
 +
 
 +
==== Unpackerr ====
 +
 
 +
Extracts downloads for Radarr, Sonarr, Lidarr, Readarr, and/or a Watch folder - Deletes extracted files after import.
 +
 
 +
https://github.com/Unpackerr/unpackerr
 +
 
 +
Docker Compose - https://github.com/Unpackerr/unpackerr/blob/main/examples/docker-compose.yml
 +
 
 +
==== Servarr (All-In-One) ====
 +
 
 +
This is a docker compose file which starts all the containers in the correct order.  This is achieved by making the next service dependant on the previous service.
 +
 
 +
# tdarr
 +
# unpackerr
 +
# readarr
 +
# bazarr
 +
# sonarr
 +
# radarr
 +
# sabnzbd
 +
# prowlarr
 +
# flaresolverr
 +
# wireguard
 +
 +
services:
 +
 +
  tdarr:
 +
    depends_on:
 +
      - unpackerr
 +
    container_name: tdarr
 +
    image: ghcr.io/haveagitgat/tdarr:latest
 +
    restart: unless-stopped
 +
    network_mode: bridge
 +
    ports:
 +
      - 8265:8265 # webUI port
 +
      - 8266:8266 # server port
 +
    environment:
 +
      - TZ=Europe/London
 +
      - PUID=1000
 +
      - PGID=1000
 +
      - UMASK_SET=002
 +
      - serverIP=0.0.0.0
 +
      - serverPort=8266
 +
      - webUIPort=8265
 +
      - internalNode=true
 +
      - inContainer=true
 +
      - ffmpegVersion=6
 +
      - nodeName=MyInternalNode
 +
    volumes:
 +
      - ./data/tdarr/server:/app/server
 +
      - ./data/tdarr/configs:/app/configs
 +
      - ./data/tdarr/logs:/app/logs
 +
      - ./data/tdarr/transcode_cache:/temp
 +
      - /home/user/data/media/movies:/input
 +
      - /home/user/data/tdarr/movies/output:/output
 +
      - /home/user/Emby:/Emby
 +
 +
  unpackerr:
 +
    depends_on:
 +
      - readarr
 +
    image: golift/unpackerr
 +
    container_name: unpackerr
 +
    network_mode: container:wireguard
 +
    volumes:
 +
      # You need at least this one volume mapped so Unapckerr can find your files to extract.
 +
      # Make sure this matches your Starr apps; the folder mount (/downloads or /data) should be identical.
 +
      - /etc/localtime:/etc/localtime:ro
 +
      - ./data/unpackerr/config:/config
 +
      - /home/user/data:/data
 +
    restart: "no"
 +
    # Get the user:group correct so unpackerr can read and write to your files.
 +
    user: 1000:1000
 +
    # What you see below are defaults for this compose. You only need to modify things specific to your environment.
 +
    environment:
 +
      - TZ=Europe/London
 +
      # General config
 +
      - UN_DEBUG=false
 +
      - UN_LOG_FILE=
 +
      - UN_LOG_FILES=10
 +
      - UN_LOG_FILE_MB=10
 +
      - UN_INTERVAL=2m
 +
      - UN_START_DELAY=1m
 +
      - UN_RETRY_DELAY=5m
 +
      - UN_MAX_RETRIES=3
 +
      - UN_PARALLEL=1
 +
      - UN_FILE_MODE=0644
 +
      - UN_DIR_MODE=0755
 +
      # Radarr Config
 +
      - UN_RADARR_0_URL=<nowiki>http://172.21.0.2:7878</nowiki>
 +
      - UN_RADARR_0_API_KEY=xxxxxxxxxxxxxxxxxxxxxxxxxx
 +
      - UN_RADARR_0_PATHS_0=/downloads
 +
      - UN_RADARR_0_PROTOCOLS=torrent
 +
      - UN_RADARR_0_TIMEOUT=10s
 +
      - UN_RADARR_0_DELETE_ORIG=false
 +
      - UN_RADARR_0_DELETE_DELAY=5m
 +
      # Sonarr Config
 +
      - UN_SONARR_0_URL=<nowiki>http://172.21.0.2:8989</nowiki>
 +
      - UN_SONARR_0_API_KEY=xxxxxxxxxxxxx
 +
      - UN_SONARR_0_PATHS_0=/downloads
 +
      - UN_SONARR_0_PROTOCOLS=torrent
 +
      - UN_SONARR_0_TIMEOUT=10s
 +
      - UN_SONARR_0_DELETE_ORIG=false
 +
      - UN_SONARR_0_DELETE_DELAY=5m
 +
      # Readarr Config
 +
      - UN_READARR_0_URL=<nowiki>http://172.21.0.2:8787</nowiki>
 +
      - UN_READARR_0_API_KEY=xxxxxxxxxxxxxxxxxxx
 +
      - UN_READARR_0_PATHS_0=/downloads
 +
      - UN_READARR_0_PROTOCOLS=torrent
 +
      - UN_READARR_0_TIMEOUT=10s
 +
      - UN_READARR_0_DELETE_ORIG=false
 +
      - UN_READARR_0_DELETE_DELAY=5m
 +
    security_opt:
 +
      - no-new-privileges:true
 +
 +
  readarr:
 +
    depends_on:
 +
      - bazarr
 +
    image: lscr.io/linuxserver/readarr:develop
 +
    container_name: readarr
 +
    network_mode: container:wireguard
 +
    environment:
 +
      - PUID=1000
 +
      - PGID=1000
 +
      - TZ=Europe/London
 +
    volumes:
 +
      - /etc/localtime:/etc/localtime:ro
 +
      - ./data/readarr/config:/config
 +
      - /home/user/data:/data
 +
    # ports:
 +
      # - 8787:8787
 +
    restart: "no"
 +
 +
  bazarr:
 +
    depends_on:
 +
      - sonarr
 +
    image: lscr.io/linuxserver/bazarr:latest
 +
    container_name: bazarr
 +
    network_mode: container:wireguard
 +
    environment:
 +
      - PUID=1000
 +
      - PGID=1000
 +
      - TZ=Europe/London
 +
    volumes:
 +
      - /etc/localtime:/etc/localtime:ro
 +
      - ./data/bazarr/config:/config
 +
      - /home/user/data:/data
 +
    # ports:
 +
      # - 6767:6767
 +
    restart: "no"
 +
 +
  sonarr:
 +
    depends_on:
 +
      - radarr
 +
    image: lscr.io/linuxserver/sonarr:latest
 +
    container_name: sonarr
 +
    network_mode: container:wireguard
 +
    environment:
 +
      - PUID=1000
 +
      - PGID=1000
 +
      - TZ=Europe/London
 +
    volumes:
 +
      - /etc/localtime:/etc/localtime:ro
 +
      - ./data/sonarr/config:/config
 +
      - /home/user/data:/data
 +
    restart: "no"
 +
    security_opt:
 +
      - seccomp=unconfined
 +
 +
  radarr:
 +
    depends_on:
 +
      - sabnzbd
 +
    image: lscr.io/linuxserver/radarr:latest
 +
    container_name: radarr
 +
    network_mode: container:wireguard
 +
    environment:
 +
      - PUID=1000
 +
      - PGID=1000
 +
      - TZ=Europe/London
 +
    volumes:
 +
      - /etc/localtime:/etc/localtime:ro
 +
      - ./data/radarr/config:/config
 +
      - /home/user/data:/data
 +
    restart: "no"
 +
 +
  sabnzbd:
 +
    depends_on:
 +
      - prowlarr
 +
    image: lscr.io/linuxserver/sabnzbd:latest
 +
    container_name: sabnzbd
 +
    network_mode: container:wireguard
 +
    environment:
 +
      - PUID=1000
 +
      - PGID=1000
 +
      - TZ=Europe/London
 +
    volumes:
 +
      - /etc/localtime:/etc/localtime:ro
 +
      - ./data/sabnzbd/config:/config
 +
      - /home/user/data:/data
 +
    restart: "no"
 +
 +
  prowlarr:
 +
    depends_on:
 +
      - flaresolverr
 +
    image: lscr.io/linuxserver/prowlarr:latest
 +
    container_name: prowlarr
 +
    network_mode: container:wireguard
 +
    environment:
 +
      - PUID=1000
 +
      - PGID=1000
 +
      - TZ=Europe/London
 +
    volumes:
 +
      - /etc/localtime:/etc/localtime:ro
 +
      - ./data/prowlarr/config:/config
 +
    restart: "no"
 +
 +
  flaresolverr:
 +
    depends_on:
 +
      - wireguard
 +
    image: ghcr.io/flaresolverr/flaresolverr:latest
 +
    container_name: flaresolverr
 +
    network_mode: container:wireguard
 +
    environment:
 +
      - LOG_LEVEL=${LOG_LEVEL:-info}
 +
      - LOG_HTML=${LOG_HTML:-false}
 +
      - CAPTCHA_SOLVER=${CAPTCHA_SOLVER:-none}
 +
      - TZ=Europe/London
 +
    restart: "no"
 +
 +
  wireguard:
 +
    image: ghcr.io/linuxserver/wireguard
 +
    container_name: wireguard
 +
    cap_add:
 +
      - NET_ADMIN
 +
      - SYS_MODULE
 +
    environment:
 +
      - PUID=1000
 +
      - PGID=1000
 +
      - TZ=Europe/London
 +
      - SERVERURL=wireguard.domain.uk
 +
      - SERVERPORT=51820
 +
      - PEERDNS=auto
 +
      - INTERNAL_SUBNET=10.6.0.0
 +
      - ALLOWEDIPS=0.0.0.0/0
 +
    volumes:
 +
      - ./data/wireguard/config:/config
 +
      - /lib/modules:/lib/modules
 +
    ports:
 +
      - 51820:51820/udp
 +
      - 8191:8191/tcp # FlareSolverr
 +
      - 9117:9117/tcp # Jackett
 +
      - 9696:9696/tcp # Prowlarr
 +
      - 9876:6789/tcp # NZBGet
 +
      - 7878:7878/tcp # Radarr
 +
      - 8989:8989/tcp # Sonarr
 +
      - 6767:6767/tcp # Bazarr
 +
      - 8787:8787/tcp # Readarr
 +
    sysctls:
 +
      - net.ipv4.conf.all.src_valid_mark=1
 +
    restart: "no"
 +
    networks:
 +
      - wireguardvpn
 +
 +
networks:
 +
  wireguardvpn:
 +
    external: true
 +
 
 +
=== Calibre ===
 +
 
 +
eBook management and automation using Calibre, COPS or Calibre-Web, and Readarr.
 +
 
 +
Auto format conversion.
 +
 
 +
https://academy.pointtosource.com/containers/ebooks-calibre-readarr/
 +
 
 +
=== YouTube-DL ===
 +
 
 +
https://registry.hub.docker.com/search?q=youtube&sort=updated_at&order=desc
 +
 
 +
https://registry.hub.docker.com/r/mikenye/youtube-dl#quick-start
 +
 
 +
=== Nagios ===
 +
 
 +
Work in progress.
 +
 
 +
This is an old version of Nagios in the container image, so will look for a newer one.
 +
 
 +
<code>/root/docker/stacks/nagios/docker-compose.yml</code>
 +
 
 +
version: '3'
 +
services:
 +
  nagios:
 +
    image: jasonrivers/nagios
 +
    container_name: nagios
 +
    restart: unless-stopped
 +
    ports:
 +
      - 8181:80
 +
    # volumes:
 +
      # - ./data/etc/:/opt/nagios/etc/
 +
    environment:
 +
      - PUID=999
 +
      - PGID=1000
 +
      - TZ=Europe/London
 +
      - NAGIOS_TIMEZONE=Europe/London
 +
 
 +
Start it with no volume mounts, then copy the etc directory to your host...
 +
 
 +
cd /root/docker/stacks/nagios/
 +
docker cp nagios:/opt/nagios/etc data/
 +
chown -R 999:1000 data/
 +
 
 +
...then uncomment the # lines in the docker-compose file and restart the container.
 +
 
 +
'''Credentials'''
 +
 
 +
The default credentials for the web interface is nagiosadmin / nagios
 +
 
 +
To change the password, generate a new one by logging in to the container and running 'htpasswd'...
 +
 
 +
docker exec -it nagios bash
 +
htpasswd -n nagiosadmin
 +
(copy the output)
 +
 
 +
...then editing the <code>/opt/nagios/etc/htpasswd.users</code> file and refreshing the admin web page.
 +
 
 +
https://github.com/ethnchao/docker-nagios
 +
 
 +
http://www.kraftinfosec.com/running-nagios-in-docker/
 +
 
 +
https://github.com/JasonRivers/Docker-Nagios
 +
 
 +
=== Tandoor Recipe Manager ===
 +
 
 +
The recipe manager that allows you to manage your ever growing collection of digital recipes.
 +
 
 +
https://docs.tandoor.dev/install/docker/
 +
 
 +
https://www.youtube.com/watch?v=7-nb3muJxI0
 +
 
 +
<code>/root/docker/stacks/tandoor/docker-compose.yml</code>
 +
 
 +
version: "3"
 +
services:
 +
  db_recipes:
 +
    container_name: tandoor_db
 +
    restart: always
 +
    image: postgres:11-alpine
 +
    volumes:
 +
      - ./data/postgresql:/var/lib/postgresql/data
 +
    env_file:
 +
      - ./.env
 +
  web_recipes:
 +
    container_name: tandoor_web
 +
    image: vabene1111/recipes
 +
    restart: always
 +
    env_file:
 +
      - ./.env
 +
    volumes:
 +
      - ./data/mediafiles:/opt/recipes/mediafiles
 +
      - ./data/staticfiles:/opt/recipes/staticfiles
 +
      - nginx_config:/opt/recipes/nginx/conf.d
 +
    depends_on:
 +
      - db_recipes
 +
  nginx_recipes:
 +
    container_name: tandoor_nginx
 +
    image: nginx:mainline-alpine
 +
    restart: always
 +
    ports:
 +
      - 80
 +
    env_file:
 +
      - ./.env
 +
    depends_on:
 +
      - web_recipes
 +
    volumes:
 +
      - ./data/mediafiles:/media
 +
      - ./data/staticfiles:/static
 +
      - nginx_config:/etc/nginx/conf.d:ro
 +
volumes:
 +
  nginx_config:
 +
networks:
 +
  default:
 +
    external:
 +
      name: nginx-proxy-manager
 +
 
 +
=== NextCloud ===
 +
 
 +
IMPORTANT
 +
 
 +
 
 +
 
 +
If you are receiving errors about PHP or issues with nginx or certificates:
 +
 
 +
# Switch your image to '''lscr.io/linuxserver/nextcloud:24.0.6-ls204''' and start the container
 +
# Execute '''docker exec -it nextcloud updater.phar''' repeatedly until there are no more updates (as of writing, Nextcloud 25 is the latest version)
 +
# Switch your image to '''lscr.io/linuxserver/nextcloud''' (latest, no tag) and start the container
 +
# Execute '''docker exec -it nextcloud mv /config/nginx/site-confs/default.conf /config/nginx/site-confs/default.conf.bak'''
 +
# Execute '''docker exec -it nextcloud mv /config/nginx/nginx.conf /config/nginx/nginx.conf.bak'''
 +
# Execute <code>docker logs nextcloud</code> and check for any other outdated configs, rename them with a .bak extension (like above)
 +
# Restart the container
 +
# Nextcloud should now be in a working state
 +
 
 +
 
 +
 
 +
 
 +
Nextcloud gives you access to all your files wherever you are.
 +
 
 +
Create your container folders...
 +
 
 +
mkdir -p /root/docker/stacks/nextcloud/data/{config,files}
 +
chown -R 1000:1000 /root/docker/stacks/nextcloud/data/
 +
 
 +
<code>/root/docker/stacks/nextcloud/docker-compose.yml</code>
 +
 
 +
version: "2.1"
 +
services:
 +
  nextcloud:
 +
    image: ghcr.io/linuxserver/nextcloud
 +
    container_name: nextcloud
 +
    environment:
 +
      - PUID=1000
 +
      - PGID=1000
 +
      - TZ=Europe/London
 +
    volumes:
 +
      - ./data/config:/config
 +
      - ./data/files:/data
 +
    ports:
 +
      - 4443:443
 +
    restart: unless-stopped
 +
 
 +
(I have changed the default port it listens on the local machine to 4443 but if you don't need any ports open then change the lines to:-
 +
 
 +
    expose:
 +
      - 443
 +
 
 +
...then use Nginx Proxy Manager to direct traffic to your NextCloud installation)
 +
 
 +
Now visit https://ip.address.of.host:4443
 +
 
 +
'''...and on that setup page, _untick_ the option for "Install recommended apps" which does not install Calendar, Contacts, Mail, Chat, etc.'''
 +
 
 +
Enjoy.
 +
 
 +
https://hub.docker.com/r/linuxserver/nextcloud
 +
 
 +
==== Command Line Admin OCC ====
 +
 
 +
https://docs.nextcloud.com/server/latest/admin_manual/configuration_server/occ_command.html
 +
 
 +
'''List Users'''
 +
 
 +
How do I list the users in NextCloud Docker?
 +
 
 +
sudo docker exec -it nextcloud occ user:list
 +
 
 +
'''Reset Admin Password'''
 +
 
 +
How do you reset the admin user password in NextCloud Docker?
 +
 
 +
sudo docker exec -it nextcloud occ user:resetpassword admin
 +
 
 +
https://docs.nextcloud.com/server/latest/admin_manual/configuration_user/reset_admin_password.html
 +
 
 +
=== Project Send ===
 +
 
 +
Self-hosted file sharing... small, simple, secure.
 +
 
 +
https://www.projectsend.org
 +
 
 +
https://docs.linuxserver.io/images/docker-projectsend
 +
 
 +
https://github.com/linuxserver/docker-projectsend
 +
 
 +
'''What they don't tell you in the docs is that you need a database backend - which is not in the docker compose file.'''
 +
 
 +
So, we just add a MariaDB database container to the stack!
 +
 
 +
Create your subdomain A record in DNS...
 +
 
 +
cli53 rrcreate domain.uk 'send 300 A 123.45.678.90'
 +
 
 +
Create your Proxy Host in Ngnix Proxy Manager with an SSL...
 +
 
 +
<nowiki>https://send.domain.uk</nowiki>
 +
 
 +
Create directories on the server for the Docker container files...
 +
 
 +
sudo -i
 +
mkdir -p /root/docker/stacks/projectsend/data/{config,db,files}
 +
chown -R 1000:1000 /root/docker/stacks/projectsend/data/files
 +
 
 +
<code>/root/docker/stacks/projectsend/docker-compose.yml</code>
 +
 
 +
version: "2.1"
 +
services:
 +
  projectsend:
 +
    image: ghcr.io/linuxserver/projectsend
 +
    container_name: projectsend
 +
    environment:
 +
      - PUID=1000
 +
      - PGID=1000
 +
      - TZ=Europe/London
 +
       - MAX_UPLOAD=100
 +
    volumes:
 +
      - ./data/config:/config
 +
      - ./data/files:/data
 +
      - /etc/timezone:/etc/timezone:ro
 +
    expose:
 +
      - 80
 +
    restart: unless-stopped
 +
  projectsend-db:
 +
    image: mariadb
 +
    container_name: projectsend-db
 +
    environment:
 +
      TZ: Europe/London
 +
      MYSQL_ROOT_PASSWORD: projectsend
 +
      MYSQL_DATABASE: projectsend
 +
      MYSQL_USER: projectsend
 +
      MYSQL_PASSWORD: projectsend
 +
    volumes:
 +
      - ./data/db:/var/lib/mysql
 +
      - /etc/timezone:/etc/timezone:ro
 +
    restart: unless-stopped
 +
networks:
 +
  default:
 +
    external:
 +
      name: nginx-proxy-manager
 +
 
 +
Go to the secure web site URL and complete the installation, using the Docker container name for the 'Database hostname'.
 +
 
 +
projectsend-db
 +
 
 +
Then, log in with your Admin username and password...
 +
 
 +
# Create Group 'Public' which is public.
 +
# Create Group 'Customers' which is not public.
 +
# Create Client 'Customer Name' which is assigned to the 'Customers' group.
 +
# Upload some files and test both the Public and Client links.
 +
 
 +
Enjoy.
 +
 
 +
==== Troubleshooting ====
 +
 
 +
'''Change the Site URL'''
 +
 
 +
If you move host or domain name, you can log in to the DB container and change the 'base_uri'...
 +
 
 +
docker exec -it projectsend-db bash
 +
mysql -u root -p projectsend
 +
 +
MariaDB [projectsend]>
 +
MariaDB [projectsend]> select * from tbl_options where name = 'base_uri';
 +
+----+----------+-------------------------+
 +
| id | name    | value                  |
 +
+----+----------+-------------------------+
 +
|  1 | base_uri | <nowiki>https://send.domain.uk/</nowiki> |
 +
+----+----------+-------------------------+
 +
 
 +
=== MediaWiki ===
 +
 
 +
==== Installation ====
 +
 
 +
Create the docker compose file and use default volume. Go to your browser at http://localhost:8080 and finish setup. Download LocalSettings.php file and copy to it to the container filesystem, then copy the whole folder to the host filsystem...
 +
 
 +
docker cp LocalSettings.php mediawiki:/var/www/html/
 +
docker cp mediawiki:/var/www/html /root/docker/stacks/mediawiki/data/
 +
chown -R www-data:www-data data/html
 +
chmod o-w data/html
 +
docker-compose down
 +
(then edit your docker-compose.yml file so that local folders are used)
 +
docker-compose up -d
 +
 
 +
Now, all the files are on your docker folder, ready to easily backup :-)
 +
 
 +
mediawiki
 +
`-- data
 +
    |-- db
 +
    `-- html
 +
 
 +
<code>~/docker/mediawiki/docker-compose.yml</code>
 +
 
 +
version: '3'
 +
services:
 +
  mediawiki:
 +
    image: mediawiki
 +
    container_name: mediawiki
 +
    restart: always
 +
    ports:
 +
      - 8080:80
 +
    links:
 +
      - database
 +
    volumes:
 +
      #- ./data/html:/var/www/html    <-- '''#2'''
 +
      #- /var/www/html/images      <-- '''#1'''
 +
    environment:
 +
      - PUID=33
 +
      - PGID=33
 +
      - TZ=Europe/London
 +
  database:
 +
    image: mariadb
 +
    container_name: mediawiki_db
 +
    restart: always
 +
    environment:
 +
      MYSQL_DATABASE: my_wiki
 +
      MYSQL_USER: wikiuser
 +
      MYSQL_PASSWORD: example
 +
      MYSQL_RANDOM_ROOT_PASSWORD: 'yes'
 +
    volumes:
 +
      - ./data/db:/var/lib/mysql
 +
 
 +
https://hub.docker.com/_/mediawiki
 +
 
 +
==== Tweaks ====
 +
 
 +
Change default skin to mobile responsive modern one...
 +
 
 +
wfLoadSkin( 'Timeless' );
 +
$wgDefaultSkin = "timeless";
 +
 
 +
Enable the new editing toolbar...
 +
 
 +
wfLoadExtension( 'WikiEditor' );
 +
 
 +
Make the URL shorter...
 +
 
 +
$wgScriptPath = "";
 +
$wgScriptExtension = ".php";
 +
$wgArticlePath = "/wiki/$1";
 +
$wgUsePathInfo = true;
 +
 
 +
File uploads...
 +
 
 +
<code>LocalSettings.php</code>
 +
 
 +
$wgEnableUploads = true;
 +
 
 +
https://www.mediawiki.org/wiki/Manual:Configuring_file_uploads
 +
 
 +
https://kindalame.com/2020/11/25/self-hosting-mediawiki-with-docker/
 +
 
 +
==== Importing ====
 +
 
 +
'''Pages'''
 +
 
 +
OLD SERVER
 +
 
 +
Generate the page dump in XML format...
 +
 
 +
docker exec -it mediawiki bash
 +
php maintenance/dumpBackup.php --current > pages.xml
 +
exit
 +
 
 +
NEW SERVER
 +
 
 +
Import the pages...
 +
 
 +
cp pages.xml ./data/html/
 +
docker exec -it mediawiki bash
 +
php maintenance/importDump.php < pages.xml
 +
php maintenance/update.php
 +
php maintenance/rebuildall.php
 +
exit
 +
 
 +
https://www.hostknox.com/tutorials/mediawiki/pages/export-and-import#import-pages-via-ssh
 +
 
 +
'''Images'''
 +
 
 +
OLD SERVER
 +
 
 +
Generate the image dumps using dumpUploads.php, which creates a txt list of all image filenames in use...
 +
 
 +
mkdir /tmp/workingBackupMediaFiles
 +
php maintenance/dumpUploads.php \
 +
    | sed 's~mwstore://local-backend/local-public~./images~' \
 +
    | xargs cp -t /tmp/workingBackupMediaFiles
 +
zip -r ~/Mediafiles.zip /tmp/workingBackupMediaFiles
 +
rm -r /tmp/workingBackupMediaFiles
 +
 
 +
NEW SERVER
 +
 
 +
Unzip the files to your container filsystem...
 +
 
 +
cd /root/docker/stacks/mediawiki
 +
unzip Mediafiles.zip -d ./data/html/
 +
 
 +
Import the Images...
 +
 
 +
docker exec -it mediawiki bash
 +
php maintenance/importImages.php tmp/workingBackupMediaFiles
 +
php maintenance/update.php
 +
php maintenance/rebuildall.php
 +
exit
 +
 
 +
https://stackoverflow.com/questions/1002258/exporting-and-importing-images-in-mediawiki
 +
 
 +
=== Kuma ===
 +
 
 +
A self-hosted monitoring tool like Uptime Robot and not as complicated as Nagios.
 +
 
 +
https://hub.docker.com/r/louislam/uptime-kuma
 +
 
 +
https://github.com/louislam/uptime-kuma
 +
 
 +
https://youtu.be/dIVf1nhT0mI
 +
 
 +
=== Kasm Containerized Apps and Desktops ===
 +
 
 +
Streaming containerized apps and desktops to end-users. The Workspaces platform provides enterprise-class orchestration, data loss prevention, and web streaming technology to enable the delivery of containerized workloads to your browser.
 +
 
 +
https://kasmweb.com
 +
 
 +
https://hub.docker.com/u/kasmweb
 +
 
 +
https://www.youtube.com/channel/UCgpv4MLH8diVlIiakCBu8eQ
 +
 
 +
=== SFTP ===
 +
 
 +
https://hub.docker.com/r/atmoz/sftp
 +
 
 +
=== Wordle ===
 +
 
 +
https://github.com/cwackerfuss/react-wordle
 +
 
 +
=== Ombi ===
 +
 
 +
Ombi allows you to host your own Emby Request and user management system. If you are sharing your Emby server with other users, allow them to request new content using an easy to manage interface! Manage all your requests for Movies and TV with ease, leave notes for the user and get notification when a user requests something. Allow your users to post issues against their requests so you know there is a problem with the audio etc. Even automatically send them weekly newsletters of new content that has been added to your Emby server :-)
 +
 
 +
https://hub.docker.com/r/linuxserver/ombi
 +
 
 +
=== Emby ===
 +
 
 +
https://fleet.linuxserver.io/image?name=linuxserver/emby
 +
 
 +
https://hub.docker.com/r/linuxserver/emby
 +
 
 +
---
 +
version: "2.1"
 +
services:
 +
  emby:
 +
    image: lscr.io/linuxserver/emby
 +
    container_name: emby
 +
    environment:
 +
      - PUID=998
 +
      - PGID=100
 +
      - TZ=Europe/London
 +
    volumes:
 +
      - /srv/dev-disk-by-uuid-7f81e7b6-1a05-4232-893b-f34c046b2bdb/Emby/Config:/config
 +
      - /srv/dev-disk-by-uuid-7f81e7b6-1a05-4232-893b-f34c046b2bdb/Emby/TV:/data/tvshows
 +
      - /srv/dev-disk-by-uuid-7f81e7b6-1a05-4232-893b-f34c046b2bdb/Emby/Movies:/data/movies
 +
    ports:
 +
      - 8096:8096
 +
    restart: unless-stopped
 +
 
 +
=== Tdarr ===
 +
 
 +
Tdarr is a conditional based transcoding application for automating media library transcoding and remux management which uses cross-platform Tdarr Nodes which work together with Tdarr Server to process your files.
 +
 
 +
https://docs.tdarr.io/docs/installation/docker/run-compose
 +
 
 +
version: "3.4"
 +
 +
services:
 +
 +
# server
 +
  tdarr:
 +
    container_name: tdarr
 +
    image: ghcr.io/haveagitgat/tdarr:latest
 +
    restart: unless-stopped
 +
    network_mode: bridge
 +
    ports:
 +
      - 8265:8265 # webUI port
 +
      - 8266:8266 # server port
 +
      - 8267:8267 # Internal node port
 +
      - 8268:8268 # Example extra node port
 +
    environment:
 +
      - TZ=Europe/London
 +
      - PUID=${PUID}
 +
      - PGID=${PGID}
 +
      - UMASK_SET=002
 +
      - serverIP=0.0.0.0
 +
      - serverPort=8266
 +
      - webUIPort=8265
 +
      - internalNode=true
 +
      - nodeID=MyInternalNode
 +
    volumes:
 +
      - /docker/tdarr/server:/app/server
 +
      - /docker/tdarr/configs:/app/configs
 +
      - /docker/tdarr/logs:/app/logs
 +
      - /media:/media
 +
      - /transcode_cache:/temp
 +
 +
# node example
 +
  tdarr-node:
 +
    container_name: tdarr-node
 +
    image: ghcr.io/haveagitgat/tdarr_node:latest
 +
    restart: unless-stopped
 +
    network_mode: service:tdarr
 +
    environment:
 +
      - TZ=Europe/London
 +
      - PUID=${PUID}
 +
      - PGID=${PGID}
 +
      - UMASK_SET=002
 +
      - nodeID=MainNode
 +
      - serverIP=0.0.0.0
 +
      - serverPort=8266
 +
    volumes:
 +
      - /docker/tdarr/configs:/app/configs
 +
      - /docker/tdarr/logs:/app/logs
 +
      - /media:/media
 +
      - /transcode_cache:/temp
 +
 
 +
=== Unifi Controller ===
 +
 
 +
https://github.com/linuxserver/docker-unifi-controller
 +
 
 +
If you want to fix a particular version of the controller then check the '[https://github.com/linuxserver/docker-unifi-controller/releases?q=%E2%80%9C5.14.23%E2%80%9D&expanded=true releases]' at github and adjust your docker-compose.yml file accordingly.
 +
 
 +
e.g. linuxserver/unifi-controller:'''5.14.23-ls76'''
 +
 
 +
version: "2.1"
 +
services:
 +
  unifi-controller:
 +
    # image: linuxserver/unifi-controller:latest
 +
    image: linuxserver/unifi-controller:'''5.14.23-ls76'''
 +
    container_name: unifi-controller
 +
    environment:
 +
      - PUID=998
 +
      - PGID=100
 +
      # - MEM_LIMIT=1024 #optional
 +
      # - MEM_STARTUP=1024 #optional
 +
    volumes:
 +
      - <path to data>:/config
 +
    ports:
 +
      - 8443:8443
 +
      - 3478:3478/udp
 +
      - 10001:10001/udp
 +
      - 8080:8080
 +
      - 1900:1900/udp #optional
 +
      - 8843:8843 #optional
 +
      - 8880:8880 #optional
 +
      - 6789:6789 #optional
 +
      - 5514:5514/udp #optional
 +
    restart: unless-stopped
 +
 
 +
=== rPort ===
 +
 
 +
Rport helps you to manage your remote servers without the hassle of VPNs, chained SSH connections, jump-hosts, or the use of commercial tools like TeamViewer and its clones.
 +
 
 +
Rport acts as server and client establishing permanent or on-demand secure tunnels to devices inside protected intranets behind a firewall.
 +
 
 +
All operating systems provide secure and well-established mechanisms for remote management, being SSH and Remote Desktop the most widely used. Rport makes them accessible easily and securely.
 +
 
 +
https://rport.io/en/features
 +
 
 +
https://oss.rport.io/
 +
 
 +
https://github.com/cloudradar-monitoring/rport
 +
 
 +
https://hub.docker.com/r/acwhiteglint/rport
 +
 
 +
=== Paperless ===
 +
 
 +
Paperless is an application that indexes your scanned documents and allows you to easily search for documents and store metadata alongside your documents.
 +
 
 +
https://github.com/jonaswinkler/paperless-ng
 +
 
 +
https://paperless-ng.readthedocs.io/en/latest/setup.html#installation
 +
 
 +
=== Ansible ===
 +
 
 +
So, this is not installing Docker using Ansible... this is installing (or running) Ansible using Docker :-)
 +
 
 +
https://iceburn.medium.com/run-ansible-with-docker-9eb27d75285b
 +
 
 +
=== Pi Alert ===
 +
 
 +
WIFI / LAN intruder detector. Scan the devices connected to your WIFI / LAN and alert you the connection of unknown devices. It also warns the disconnection of "always connected" devices.
 +
 
 +
https://github.com/pucherot/Pi.Alert
 +
 
 +
=== Roundcube Webmail ===
 +
 
 +
https://hub.docker.com/search?q=roundcube
 +
 
 +
https://hub.docker.com/r/roundcubeorg/roundcubemail
 +
 
 +
https://github.com/roundcube/roundcubemail/wiki/Configuration
 +
 
 +
docker run -e ROUNDCUBEMAIL_DEFAULT_HOST=ssl://mail.domain.co.uk -e ROUNDCUBEMAIL_DEFAULT_PORT=993 -e ROUNDCUBEMAIL_SMTP_SERVER=tls://mail.domain.co.uk -e ROUNDCUBEMAIL_SMTP_PORT=587 -e ROUNDCUBEMAIL_PLUGINS=archive,zipdownload,managesieve,mobile -p 8000:80 -d roundcube/roundcubemail
 +
 
 +
=== FAST SpeedTest ===
 +
 
 +
docker run -it --rm --name fast-cli mschirrmeister/fast-cli:latest
 +
 
 +
https://hub.docker.com/r/mschirrmeister/fast-cli
 +
 
 +
https://github.com/sindresorhus/fast-cli
 +
 
 +
=== Crypto Mining ===
 +
 
 +
==== XMRig ====
 +
 
 +
https://hub.docker.com/r/minerboy/xmrig
 +
 
 +
Go to the [https://xmrig.com/wizard XMRig Wizard] and generate config like this ...
 +
 
 +
{
 +
    "autosave": true,
 +
    "cpu": true,
 +
    "opencl": false,
 +
    "cuda": false,
 +
    "pools": [
 +
        {
 +
            "coin": "monero",
 +
            "algo": "rx/0",
 +
            "url": "stratum+tcp://randomxmonero.auto.nicehash.com:9200",
 +
            "user": "NHbLd5exQeCGGyWnopVoLHLbzexKN5z8iq7p.NAS",
 +
            "pass": "x",
 +
            "tls": false,
 +
            "keepalive": true,
 +
            "nicehash": true
 +
        }
 +
    ]
 +
}
 +
 
 +
Then save that file as '''config.json''' and run the following commands to start mining ...
 +
 
 +
sudo -i
 +
docker pull minerboy/xmrig
 +
docker run --cap-add=SYS_ADMIN --cap-add=SYS_RAWIO --device=/dev/cpu --device=/dev/mem -v /lib/modules:/lib/modules -v /full/path/to/config.json:/etc/xmrig/config.json minerboy/xmrig:latest --cpu-max-threads-hint 50 --threads 2
 +
 
 +
Here is the same command as a lovely docker compose file ...
 +
 
 +
version: '3.3'
 +
services:
 +
  xmrig:
 +
      image: 'minerboy/xmrig:latest'
 +
      container_name: xmrig
 +
      devices:
 +
        - /dev/cpu
 +
        - /dev/mem
 +
      volumes:
 +
        - '/lib/modules:/lib/modules'
 +
        - '/root/docker/stacks/xmrig/config.json:/etc/xmrig/config.json'
 +
      command:
 +
        - --cpu-max-threads-hint 50
 +
        - --threads 2
 +
      restart: "no"
 +
 
 +
=== BTCPay Server ===
 +
 
 +
https://docs.btcpayserver.org/Docker/
 +
 
 +
sudo -i
 +
mkdir -p /root/docker
 +
git clone <nowiki>https://github.com/btcpayserver/btcpayserver-docker</nowiki>
 +
cd btcpayserver-docker
 +
export BTCPAY_HOST="btcpay.mydomain.com"
 +
export NBITCOIN_NETWORK="mainnet"
 +
export BTCPAYGEN_CRYPTO1="btc"
 +
export BTCPAYGEN_ADDITIONAL_FRAGMENTS="opt-save-storage-xs"
 +
export BTCPAYGEN_REVERSEPROXY="nginx"
 +
export BTCPAYGEN_LIGHTNING="clightning"
 +
export BTCPAY_ENABLE_SSH=false
 +
export REVERSEPROXY_HTTP_PORT=8080
 +
export REVERSEPROXY_HTTPS_PORT=4443
 +
. ./btcpay-setup.sh -i
 +
 
 +
Then, after 5 minutes, you can switch to the '''FastSync''' of the blockchain ...
 +
 
 +
cd $BTCPAY_BASE_DIRECTORY/btcpayserver-docker
 +
./btcpay-down.sh
 +
cd $BTCPAY_BASE_DIRECTORY/btcpayserver-docker/contrib/FastSync
 +
./load-utxo-set.sh
 +
docker volume rm generated_bitcoin_wallet_datadir
 +
cd $BTCPAY_BASE_DIRECTORY/btcpayserver-docker
 +
./btcpay-up.sh
 +
docker logs --tail -100 btcpayserver_bitcoind
 +
 
 +
==== Using Caddy Proxy Separately ====
 +
 
 +
Yes!  I finally worked out how to do this!
 +
 
 +
You can use BTCPay Server with an existing Proxy like NginX or Caddy.
 +
 
 +
The trick is to tell the Docker BTCPay Server to '''disable its' own proxy''' offering and also '''disable the SSL termination'''.
 +
 
 +
These are the magic list of commands to use - setting the environment variables for the setup script ...
 +
 
 +
BTCPAYGEN_REVERSEPROXY="none"
 +
NOREVERSEPROXY_HTTP_PORT="3003"
 +
BTCPAYGEN_EXCLUDE_FRAGMENTS="nginx-https"
 +
 
 +
So, your full command list is as follows ...
 +
 
 +
sudo -i
 +
mkdir -p /root/docker
 +
git clone <nowiki>https://github.com/btcpayserver/btcpayserver-docker</nowiki>
 +
cd btcpayserver-docker
 +
export BTCPAY_HOST="btcpay.mydomain.com"
 +
export NBITCOIN_NETWORK="mainnet"
 +
export BTCPAYGEN_CRYPTO1="btc"
 +
export BTCPAYGEN_ADDITIONAL_FRAGMENTS="opt-save-storage-xs"
 +
export BTCPAYGEN_LIGHTNING="clightning"
 +
export BTCPAY_ENABLE_SSH=false
 +
'''export BTCPAYGEN_REVERSEPROXY="none"'''
 +
'''export NOREVERSEPROXY_HTTP_PORT="3003"'''
 +
'''export BTCPAYGEN_EXCLUDE_FRAGMENTS="nginx-https"'''
 +
. ./btcpay-setup.sh -i
 +
 
 +
This should now give you these dockers with the ports used ...
 +
 
 +
btcpayserver_bitcoind            8332-8333/tcp, 18332-18333/tcp, 18443-18444/tcp, 39388/tcp, 43782/tcp
 +
btcpayserver_clightning_bitcoin  0.0.0.0:9735->9735/tcp, :::9735->9735/tcp, 9835/tcp, 0.0.0.0:32768->3010/tcp, [::]:32768->3010/tcp
 +
generated-bitcoin_rtl-1          3000/tcp
 +
generated_btcpayserver_1          0.0.0.0:3003->'''49392/tcp''', [::]:3003->49392/tcp
 +
generated_nbxplorer_1            32838/tcp
 +
generated_postgres_1              5432/tcp
 +
tor                              9050-9051/tcp
 +
tor-gen
 +
 
 +
You will notice the crucial port 49392/tcp running in the btcpayserver container.
 +
 
 +
THIS IS WHAT YOU PROXY TO :)
 +
 
 +
You do ''not'' proxy to the host's 3003 port (as often mentioned in old instructions on web pages!)
 +
 
 +
This is the Caddyfile which uses static Let's Encrypt files I generated with certbot.  You will also notice I have left my incorrect reverse_proxy line in for reference :) 
 +
 
 +
So, Caddy proxies requests to the '''container name and container port''' ...
 +
 
 +
btcpay.mydomain.com:443 {
 +
        tls /ssl/certs/fullchain.pem /ssl/certs/key.pem
 +
        #reverse_proxy 127.0.0.1:3003
 +
        '''reverse_proxy generated_btcpayserver_1:49392'''
 +
}
 +
 
 +
This is the Caddy docker-compose and you notice it's using the BTCPay Server 'generated_default' network - which is REALLY important!
 +
services:
 +
  caddy:
 +
    image: caddy:alpine
 +
    container_name: caddy
 +
    restart: unless-stopped
 +
    ports:
 +
      - 80:80
 +
      - 443:443
 +
      - 443:443/udp
 +
    networks:
 +
      - generated_default
 +
    volumes:
 +
      - /var/run/docker.sock:/var/run/docker.sock
 +
      - ./Caddyfile:/etc/caddy/Caddyfile
 +
      - ./data:/data
 +
      - ./config:/config
 +
      - ./fullchain.pem:/ssl/certs/fullchain.pem:ro
 +
      - ./key.pem:/ssl/certs/key.pem:ro
 +
    environment:
 +
      - TZ="Europe/London"
 +
networks:
 +
  generated_default:
 +
    external: true
 +
 
 +
... and that's it!  Enjoy.
 +
 
 +
==== Umbrel ====
 +
 
 +
# ssh into your umbrel
 +
ssh umbrel@umbrel.local    (or the IP of your Umbrel server)     
 +
# Password is the same as Umbrel Web UI
 +
 +
# Edit the .env.app_proxy file
 +
nano ~/umbrel/app-data/btcpay-server/.env.app_proxy
 +
 +
# Enter the following into the file
 +
PROXY_TRUST_UPSTREAM=true
 +
 +
# Save using:
 +
Control + X, then: Y, then: <enter>
 +
 +
# Restart BTCPayServer
 +
~/umbrel/scripts/app restart btcpay-server
 +
 +
# You can also right-click on the GUI icon for BTCPay Server and choose 'Restart'
 +
 
 +
https://orange.surf/public-btcpay-umbrel-tailscale/
 +
 
 +
=== AI ===
 +
 
 +
[https://technotim.live/posts/private-practical-local-ai/ Self-Hosted AI That's Actually Useful - Techno Tim]
 +
 
 +
* Open WebUI
 +
* Olama
 +
* SearXNG
 +
* Stable Diffusion + ComfyUI
 +
* Prompt Generator
 +
 
 +
=== Microsoft Windows ===
 +
 
 +
1) Watch ...
 +
 
 +
https://www.youtube.com/watch?v=3h1KtrL3CYQ
 +
 
 +
2) Install ...
 +
 
 +
https://github.com/dockur/windows
 +
 
 +
3) Tweak ...
 +
 
 +
https://christitus.com/windows-utility-improved/
 +
 
 +
https://www.youtube.com/watch?v=5_AaHXrelTE
 +
 
 +
=== Locust Web Site Testing Tool ===
 +
 
 +
[https://locust.io/ Locust] is an open source performance/load testing tool for HTTP and other protocols. Its developer-friendly approach lets you define your tests in regular Python code.
 +
 
 +
Locust tests can be run from command line or using its web-based UI. Throughput, response times and errors can be viewed in real time and/or exported for later analysis.
 +
 
 +
services:
 +
  master:
 +
    image: locustio/locust
 +
    container_name: locust_master
 +
    restart: 'no'
 +
    ports:
 +
      - "8089:8089"
 
     volumes:
 
     volumes:
−
       - ./data/config:/config
+
       - ./:/mnt/locust
−
      - ./data/files:/data
+
    command: -f /mnt/locust/locustfile.py --master -H <nowiki>http://master:8089</nowiki>
−
      - /etc/timezone:/etc/timezone:ro
+
   worker:
−
    expose:
+
     image: locustio/locust
−
      - 80
+
     container_name: locust_worker
−
    restart: unless-stopped
+
     restart: 'no'
−
   projectsend-db:
 
−
     image: mariadb
 
−
     container_name: projectsend-db
 
−
     environment:
 
−
      TZ: Europe/London
 
−
      MYSQL_ROOT_PASSWORD: projectsend
 
−
      MYSQL_DATABASE: projectsend
 
−
      MYSQL_USER: projectsend
 
−
      MYSQL_PASSWORD: projectsend
 
 
     volumes:
 
     volumes:
−
       - ./data/db:/var/lib/mysql
+
       - ./:/mnt/locust
−
      - /etc/timezone:/etc/timezone:ro
+
    command: -f /mnt/locust/locustfile.py --worker --master-host master
−
    restart: unless-stopped
 
−
networks:
 
−
  default:
 
−
    external:
 
−
      name: nginx-proxy-manager
 
−
 
 
−
Go to the secure web site URL and complete the installation, using the Docker container name for the 'Database hostname'.
 
−
 
 
−
projectsend-db
 
−
 
 
−
Then, log in with your Admin username and password...
 
−
 
 
−
# Create Group 'Public' which is public.
 
−
# Create Group 'Customers' which is not public.
 
−
# Create Client 'Customer Name' which is assigned to the 'Customers' group.
 
−
# Upload some files and test both the Public and Client links.
 
  
−
Enjoy.
+
Then connect to your server over SSH with this config to interact with the web gui ...
  
−
==== Troubleshooting ====
+
Host    caddy-test-locust
 +
        User ubuntu
 +
        Port 22
 +
        HostName myserverhostname
 +
        LocalForward 9999 127.0.0.1:8089
 +
        ProxyJump mysshbastionhost
  
−
'''Change the Site URL'''
+
=== Raspberry Pi ===
  
−
If you move host or domain name, you can log in to the DB container and change the 'base_uri'...
+
https://github.com/ptrsr/pi-ci
−
 
 
−
docker exec -it projectsend-db bash
 
−
mysql -u root -p projectsend
 
−
 
−
MariaDB [projectsend]>
 
−
MariaDB [projectsend]> select * from tbl_options where name = 'base_uri';
 
−
+----+----------+-------------------------+
 
−
| id | name    | value                  |
 
−
+----+----------+-------------------------+
 
−
|  1 | base_uri | <nowiki>https://send.domain.uk/</nowiki> |
 
−
+----+----------+-------------------------+
 
−
 
 
−
=== MediaWiki ===
 
−
 
 
−
==== Installation ====
 
−
 
 
−
Create the docker compose file and use default volume. Go to your browser at http://localhost:8080 and finish setup. Download LocalSettings.php file and copy to it to the container filesystem, then copy the whole folder to the host filsystem...
 
−
 
 
−
docker cp LocalSettings.php mediawiki:/var/www/html/
 
−
docker cp mediawiki:/var/www/html /root/docker/stacks/mediawiki/data/
 
−
chown -R www-data:www-data data/html
 
−
chmod o-w data/html
 
−
docker-compose down
 
−
(then edit your docker-compose.yml file so that local folders are used)
 
−
docker-compose up -d
 
  
−
Now, all the files are on your docker folder, ready to easily backup :-)
+
== Swarm ==
  
−
mediawiki
+
'''Docker Swarm''' is a container orchestrator and a clustering and scheduling tool offered by Docker as a application mode, for managing multiples nodes, and it’s deployments like a whole single system. Swarm mode also exists natively for Docker Engine, the layer between the OS and container images. We generally use the Docker CLI to create a swarm, deploy application services to a swarm, and manage swarm behaviour. The activities of the cluster are controlled by a swarm manager, and machines that have joined the cluster are referred to as nodes. Docker Swarm lets you connect containers to multiple hosts similar to Kubernetes.
−
`-- data
 
−
    |-- db
 
−
    `-- html
 
  
−
<code>~/docker/mediawiki/docker-compose.yml</code>
+
The docker swarm function recognises two different types of nodes, each with a different role within the docker swarm ecosystem:
  
−
version: '3'
+
''Manager nodes'': These contain the Swarm Manager, the process in charge of handling the commands in the Swarm mode and reconciling the desired state with the actual cluster state.
−
services:
 
−
  mediawiki:
 
−
    image: mediawiki
 
−
    container_name: mediawiki
 
−
    restart: always
 
−
    ports:
 
−
      - 8080:80
 
−
    links:
 
−
      - database
 
−
    volumes:
 
−
      #- ./data/html:/var/www/html    <-- '''#2'''
 
−
      #- /var/www/html/images      <-- '''#1'''
 
−
    environment:
 
−
      - PUID=33
 
−
      - PGID=33
 
−
      - TZ=Europe/London
 
−
  database:
 
−
    image: mariadb
 
−
    container_name: mediawiki_db
 
−
    restart: always
 
−
    environment:
 
−
      MYSQL_DATABASE: my_wiki
 
−
      MYSQL_USER: wikiuser
 
−
      MYSQL_PASSWORD: example
 
−
      MYSQL_RANDOM_ROOT_PASSWORD: 'yes'
 
−
    volumes:
 
−
      - ./data/db:/var/lib/mysql
 
  
−
https://hub.docker.com/_/mediawiki
+
''Worker nodes'': In a docker swarm with numerous hosts, each worker node functions by receiving and executing the tasks that are allocated to it by manager nodes.
  
−
==== Tweaks ====
+
[https://docs.docker.com/engine/swarm/swarm-tutorial Docker Swarm Tutorial]
  
−
Change default skin to mobile responsive modern one...
+
On AWS, create your VPC and Security Groups ...
  
−
wfLoadSkin( 'Timeless' );
 
−
$wgDefaultSkin = "timeless";
 
  
−
Enable the new editing toolbar...
 
  
−
wfLoadExtension( 'WikiEditor' );
 
  
−
Make the URL shorter...
 
  
−
$wgScriptPath = "";
 
−
$wgScriptExtension = ".php";
 
−
$wgArticlePath = "/wiki/$1";
 
−
$wgUsePathInfo = true;
 
  
−
==== Importing ====
 
  
−
'''Pages'''
+
1) Initialise swarm on the Manager ...
  
−
OLD SERVER
+
docker swarm init --advertise-addr 172.31.7.94
  
−
Generate the page dump in XML format...
+
2) Join swarm on the Workers ...
  
−
  docker exec -it mediawiki bash
+
  docker swarm join --token SWMTKN-1-5zy8ij0t240g4b9xxxxxxxxxxxxxxxxxxxxxxin829mxrsl-amipciivp9t0sbtjugtjrcqlf 172.31.7.94:2377
−
php maintenance/dumpBackup.php --current > pages.xml
 
−
exit
 
  
−
NEW SERVER
+
3) Deploy a Service ...
  
−
Import the pages...
+
docker service create --replicas 1 --name helloworld alpine ping docker.com
  
−
cp pages.xml ./data/html/
+
4) Check a Service ...
−
docker exec -it mediawiki bash
 
−
php maintenance/importDump.php < pages.xml
 
−
php maintenance/update.php
 
−
php maintenance/rebuildall.php
 
−
exit
 
  
−
https://www.hostknox.com/tutorials/mediawiki/pages/export-and-import#import-pages-via-ssh
+
docker service ls
  
−
'''Images'''
+
5) Inspect a Service ...
  
−
OLD SERVER
+
docker service ps helloworld
  
−
Generate the image dumps using dumpUploads.php, which creates a txt list of all image filenames in use...
+
6) Scale up a Service ...
  
−
  mkdir /tmp/workingBackupMediaFiles
+
  docker service scale helloworld=5
−
php maintenance/dumpUploads.php \
 
−
    | sed 's~mwstore://local-backend/local-public~./images~' \
 
−
    | xargs cp -t /tmp/workingBackupMediaFiles
 
−
zip -r ~/Mediafiles.zip /tmp/workingBackupMediaFiles
 
−
rm -r /tmp/workingBackupMediaFiles
 
  
−
NEW SERVER
+
7) Inspect a Service ...
  
−
Unzip the files to your container filsystem...
+
docker service ps helloworld
  
−
cd /root/docker/stacks/mediawiki
+
8) Scale down a Service ...
−
unzip Mediafiles.zip -d ./data/html/
 
  
−
Import the Images...
+
docker service scale helloworld=1
  
−
docker exec -it mediawiki bash
+
9) Stop and remove a Service ...
−
php maintenance/importImages.php tmp/workingBackupMediaFiles
 
−
php maintenance/update.php
 
−
php maintenance/rebuildall.php
 
−
exit
 
  
−
https://stackoverflow.com/questions/1002258/exporting-and-importing-images-in-mediawiki
+
docker service scale helloworld=0
 +
docker service rm helloworld
  
 
== Help ==
 
== Help ==

Latest revision as of 16:22, 12 November 2024

Introduction

Docker vs Virtual Machines

Docker is a set of platform as a service products that use OS-level virtualization to deliver software in packages called containers. Containers are isolated from one another and bundle their own software, libraries and configuration files but they can communicate with each other through well-defined channels.

https://www.docker.com - Official Web Site.

https://hub.docker.com - Official Repository of Container Images.

It was originally developed for programmers to test their software but has now become a fully fledged answer to running servers in mission critical situations.

Each container has a mini operating system plus the software needed to run the program you want, and no more.

All of the 'hard work' for a piece of software has been 'done for you' and the end result is starting a program with one command line.

For example, the WordPress image contains the LAP part of LAMP (Linux + Apache + PHP) all configured and running.

Images

Docker Hub

LinuxServer

Useful Wiki

Installation

WINDOWS

  1. Make sure your computer supports Hardware Virtualisation by checking in the BIOS.
  2. Install Docker Desktop.
  3. Install the Windows Subsystem for Linux Kernel Update.
  4. Reboot.

LINUX

New All In One Official Method

sudo -i
curl -fsSL https://get.docker.com | sh

Engine

This will remove the old version of 'Docker' and install the new version 'Docker CE'... 

sudo apt-get remove docker docker-engine docker.io containerd runc
sudo apt-get update
sudo apt-get -y install apt-transport-https ca-certificates curl gnupg software-properties-common
sudo mkdir -p /etc/apt/keyrings
curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo gpg --dearmor -o /etc/apt/keyrings/docker.gpg
echo "deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.gpg] https://download.docker.com/linux/ubuntu $(lsb_release -cs) stable" | sudo tee /etc/apt/sources.list.d/docker.list > /dev/null
sudo apt-get update
sudo apt-get -y install docker-ce docker-ce-cli containerd.io docker-compose-plugin
sudo service docker start
sudo docker run hello-world

https://docs.docker.com/install/linux/docker-ce/ubuntu/#install-docker-ce-1

Compose From Command Line

http://composerize.com - Turns docker run commands into docker-compose files!

Install composerize and convert your own commands locally. 

sudo apt instal npm
sudo npm install composerize -g
composerize docker run -p 80:80 -v /var/run/docker.sock:/tmp/docker.sock:ro --restart always --log-opt max-size=1g nginx

https://github.com/magicmark/composerize

Using Ansible

docker-install.yml 

- hosts: all
  
  become: yes
  tasks:
  
  # Install Docker
  # --
  # 
  - name: install prerequisites
    apt:
      name:
        - apt-transport-https
        - ca-certificates 
        - curl 
        - gnupg-agent
        - software-properties-common
      update_cache: yes
  
  - name: add apt-key
    apt_key:
      url: https://download.docker.com/linux/ubuntu/gpg
  
  - name: add docker repo
    apt_repository:
      repo: deb https://download.docker.com/linux/ubuntu focal stable
  
  - name: install docker 
    apt:
      name: 
        - docker-ce
        - docker-ce-cli
        - containerd.io
      update_cache: yes
  
  - name: add user permissions
    shell: "usermod -aG docker ubuntu"
  
  # Installs Docker SDK
  # --
  # 
  - name: install python package manager
    apt:
      name: python3-pip
  
  - name: install python sdk
    become_user: ubuntu
    pip:
      name:
        - docker
        - docker-compose

Then run Ansible using your playbook on your server host ... 

ansible-playbook docker-install.yml -l 'myserver'

You can then deploy a container using Ansible as well. This will deploy Portainer ...

docker_deploy-portainer.yml 

- hosts: all
  
  become: yes
  become_user: ubuntu
  tasks:
  
  # Create Portainer Volume
  # --
  # 
  - name: Create new Volume
    community.docker.docker_volume:
      name: portainer-data
  
  # Deploy Portainer
  # --
  #   
  - name: Deploy Portainer
    community.docker.docker_container:
      name: portainer
      image: "docker.io/portainer/portainer-ce"
      ports:
        - "9000:9000"
        - "9443:9443"
      volumes:
        - /var/run/docker.sock:/var/run/docker.sock
        - portainer-data:/data
      restart_policy: always

... with this command ... 

 ansible-playbook docker_deploy-portainer.yml -l 'myserver'

Moving the Docker Volume Storage Location

Why?

Because you only have a limited space on your root volume and you want to use the extra 100 GB volume you mounted on your Oracle VPS :)

How?

Shut down any containers and the docker services. 

systemctl stop docker.socket docker.service containerd.service

Create a directory on your big disk 

sudo mkdir /data1/docker

Rsync the current docker directory with the big disk directory 

sudo rsync -av /var/lib/docker/ /data1/docker/

Rename the existing docker library directory 

sudo mv /var/lib/docker /var/lib/docker.orig

Create a symlink for the 'docker directory' to your big disk directory 

sudo ln -s /data1/docker /var/lib/docker

Reboot the server 

sudo reboot

Check to make sure all is well after 

docker --info
docker ps

Now, go create your BTCPay Server container ;-)


Create a symlink to your nice big disk

Usage

Statistics 

docker stats
docker stats --no-stream

System information 

docker system info

Run container 

docker run hello-world

List containers 

docker container ls
docker container ls -a

List container processes 

docker ps
docker ps --format "table {{.ID}}\t{{.Names}}\t{{.Image}}\t{{.Ports}}\t{{.Status}}"

List container names 

docker ps --format '{{.Names}}'
docker ps -a | awk '{print $NF}'

List volumes 

docker volume ls
docker volume ls -f dangling=true

List networks 

docker network ls

Information about container 

docker container inspect container_name or id

Stop container 

docker stop container_name

Delete container 

docker rm container_name

Delete volumes 

docker volume rm volume_name

Delete all unused volumes 

docker volume prune

Delete all unused networks 

docker network prune

Prune everything unused 

docker system prune

Upgrade a stack 

docker-compose pull
docker-compose up -d

BASH Aliases for use with Docker commands 

alias dcd='docker-compose down'
alias dcr='docker-compose restart'
alias dcu='docker-compose up -d'
alias dps='docker ps'

Run Command In Docker Container

e.g.

List the mail queue for a running email server ... 

docker exec -it mail.domain.co.uk-mailserver mailq

Setting Memory And CPU Limits In Docker

service:
  image: nginx
  mem_limit: 256m
  mem_reservation: 128M
  cpus: 0.5
  ports:
    - "80:80"

https://www.baeldung.com/ops/docker-memory-limit

Volumes

Multiple Containers

Use volumes which are bind mounted from the host filesystem between multiple containers.

First, create the volume bind mounted to the folder... 

docker volume create --driver local --opt type=none --opt device=/path/to/folder --opt o=bind volume_name

Then, use it in your docker compose file... 

services:
  ftp.domain.uk-nginx:
    image: nginx
    container_name: ftp.domain.uk-nginx
    expose:
      - "80"
    volumes:
      - ./data/etc/nginx:/etc/nginx
      - ftp.domain.uk:/usr/share/nginx:ro
    environment:
      - VIRTUAL_HOST=ftp.domain.uk
networks:
  default:
    external:
      name: nginx-proxy-manager
volumes:
  ftp.domain.uk:
    external: true

Using volumes in Docker Compose

Networks

Basic Usage

Create your network... 

docker network create networkname

Use it in your docker-compose.yml file... 

services:
  service_name:
    image: image_name:latest
    restart: always
    networks:
      - networkname
networks:
  networkname:
    external: true

https://poopcode.com/join-to-an-existing-network-from-a-docker-container-in-docker-compose/

Advanced Usage

Static IP Address 

   networks:
     traefik:
       ipv4_address: 172.19.0.9
     backend: null

Force Docker Containers to use a VPN for their Network

Block IP Address Using IPTables

The key here is to make sure you use the -I or INSERT command for the rule so that it is FIRST in the chain.

Block IP addresses from LITHUANIA 

iptables -I DOCKER-USER -i eth0 -s 141.98.10.0/24 -j DROP
iptables -I DOCKER-USER -i eth0 -s 141.98.11.0/24 -j DROP
iptables -L DOCKER-USER --line-numbers
Chain DOCKER-USER (1 references)
num  target     prot opt source               destination
1    DROP       all  --  141.98.11.0/24       anywhere
2    DROP       all  --  141.98.10.0/24       anywhere
3    RETURN     all  --  anywhere             anywhere

Display IP Addresses In A Docker Network Sorted Numerically

docker network inspect traefik |jq -r 'map(.Containers[].IPv4Address) []' |sort -t . -k 3,3n -k 4,4n

Docker Compose

Restart Policy

The "no" option has quotes around it... 

restart: "no"
restart: always
restart: on-failure
restart: unless-stopped

Management

Cleaning Pruning

sudo docker system df
sudo docker system prune
cd /etc/cron.daily
sudo nano docker-prune

#!/bin/bash
docker system prune -y

sudo chmod +x /etc/cron.daily/docker-prune

https://alexgallacher.com/prune-unused-docker-images-automatically/

Delete All Stopped Docker Containers

docker rm $(docker ps --filter "status=exited" -q)

Updating with Docker Compose

for d in ./*/ ; do (cd "$d" && sudo docker-compose pull && sudo docker-compose --compatibility up -d); done

Logs Logging

If you want to watch the logs in real time, then add the -f or --follow option to your command ... 

docker logs nginx --follow

or 

docker logs nginx -f --tail 20

After executing docker-compose up, list your running containers: 

docker ps

Copy the NAME of the given container and read its logs: 

docker logs NAME_OF_THE_CONTAINER -f

To only read the error logs: 

docker logs NAME_OF_THE_CONTAINER -f 1>/dev/null

To only read the access logs: 

docker logs NAME_OF_THE_CONTAINER -f 2>/dev/null

https://linuxhandbook.com/docker-logging/

To search or grep the logs: 

docker logs watchtower 2>&1 | grep 'msg="Found new'

Cleaning Space

Over the last month, a whopping 14Gb of space was being used by /var/lib/docker/overlay2/ and needed a way to safely remove unused data.

Check your space usage... 

du -mcsh /var/lib/docker/overlay2
14G     /var/lib/docker/overlay2

Check what Docker thinks is being used... 

docker system df
TYPE            TOTAL     ACTIVE    SIZE      RECLAIMABLE
Images          36        15        8.368GB   4.491GB (53%)
Containers      17        15        70.74MB   286B (0%)
Local Volumes   4         2         0B        0B
Build Cache     0         0         0B        0B

Clean... 

docker system prune
docker image prune --all

Check again... 

du -mcsh /var/lib/docker/overlay2
9.4G    /var/lib/docker/overlay2

docker system df
TYPE            TOTAL     ACTIVE    SIZE      RECLAIMABLE
Images          13        13        4.144GB   144MB (3%)
Containers      15        15        70.74MB   0B (0%)
Local Volumes   4         2         0B        0B
Build Cache     0         0         0B        0B

...job done.

Dockge

Better than Portainer.

A fancy, easy-to-use and reactive docker 'compose.yaml' stack-oriented manager

https://github.com/louislam/dockge

Portainer

https://www.portainer.io

https://github.com/portainer/portainer

Reset the Admin user Password

Server

https://hub.docker.com/r/portainer/portainer-ce

Agent

Portainer uses the Portainer Agent container to communicate with the Portainer Server instance and provide access to the node's resources. So, this is great for a small server (like a Raspberry Pi) where you don't need the full Portainer Server install.

Deployment

Run the following command to deploy the Portainer Agent: 

docker run -d -p 9001:9001 --name portainer_agent --restart=always -v /var/run/docker.sock:/var/run/docker.sock -v /var/lib/docker/volumes:/var/lib/docker/volumes portainer/agent:2.11.0

Adding your new environment

Once the agent has been installed you are ready to add the environment to your Portainer Server installation.

From the menu select Environments then click Add environment.

From the Environment type section, select Agent. Since we have already installed the agent you can ignore the sample commands in the Information section. 

Name: my-raspberry-pi
Environment URL: 192.168.0.106:9001

When you're ready, click Add environment.

Then, on the Portainer Home screen you select your new environment, or server running the Agent, and away you go!

https://docs.portainer.io/v/ce-2.11/admin/environments/add/docker#method-2-connecting-via-the-portainer-agent

Updating

Portainer

Containers > Select > Stop > Recreate > Pull Latest Image > Start

Watchtowerr

List updates ... 

docker run --rm -v /var/run/docker.sock:/var/run/docker.sock containrrr/watchtower --run-once --log-format pretty --monitor-only

Perform updates ... 

docker run --rm -v /var/run/docker.sock:/var/run/docker.sock containrrr/watchtower --run-once --log-format pretty

Backups

https://github.com/SavageSoftware/portainer-backup

https://forum.openmediavault.org/index.php?thread/43235-backup-portainer-and-completely-uninstall-to-test-disaster-recovery/

Monitoring

CTop

Press the Q key to stop it... 

docker run -ti --name ctop --rm -v /var/run/docker.sock:/var/run/docker.sock wrfly/ctop:latest

Docker Stats 

docker stats

DeUnhealth

Restart your unhealthy containers safely.

https://github.com/qdm12/deunhealth

https://www.youtube.com/watch?v=Oeo-mrtwRgE

Dozzle

Dozzle is a small lightweight application with a web based interface to monitor Docker logs. It doesn’t store any log files. It is for live monitoring of your container logs only.

https://github.com/amir20/dozzle

Gotchas

https://sosedoff.com/2016/10/05/docker-gotchas.html

Applications

I have set up my docker containers in a master docker directory with sub-directories for each stack. 

docker
|-- backups
`-- stacks
    |-- bitwarden
    |   `-- bwdata
    |-- grafana
    |   `-- data
    |-- mailserver
    |   `-- data
    |-- nginx-proxy-manager
    |   `-- data
    `-- portainer
        `-- data

Backups

https://github.com/alaub81/backup_docker_scripts

Updates

Tracking

Watchtower

A process for automating Docker container base image updates.

With watchtower you can update the running version of your containerized app simply by pushing a new image to the Docker Hub or your own image registry. Watchtower will pull down your new image, gracefully shut down your existing container and restart it with the same options that were used when it was deployed initially.

First Time Run Once Check Only

This will run and output if there are any updates them stop and remove itself... 

docker run --name watchtower -v /var/run/docker.sock:/var/run/docker.sock containrrr/watchtower --run-once --debug --monitor-only --rm

Automated Scheduled Run Daily

This will start the container and schedule a check at 4am every day...

~/watchtower/docker-compose.yml 

version: "3"
services:
  watchtower:
    image: containrrr/watchtower
    container_name: watchtower
    restart: always
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock
    environment:
      - TZ=${TZ}
      - WATCHTOWER_DEBUG=true
      - WATCHTOWER_MONITOR_ONLY=false
      - WATCHTOWER_CLEANUP=true
      - WATCHTOWER_LABEL_ENABLE=false
      - WATCHTOWER_NOTIFICATIONS=email
      - WATCHTOWER_NOTIFICATION_EMAIL_FROM=${EMAIL_FROM}
      - WATCHTOWER_NOTIFICATION_EMAIL_TO=${WATCHTOWER_EMAIL_TO}
      - WATCHTOWER_NOTIFICATION_EMAIL_SERVER=${SMTP_SERVER}
      - WATCHTOWER_NOTIFICATION_EMAIL_SERVER_PORT=${SMTP_PORT}
      - WATCHTOWER_NOTIFICATION_EMAIL_SERVER_USER=${SMTP_USER}
      - WATCHTOWER_NOTIFICATION_EMAIL_SERVER_PASSWORD=${SMTP_PASSWORD}
      - WATCHTOWER_SCHEDULE=0 0 4 * * *

https://containrrr.dev/watchtower/

https://containrrr.dev/watchtower/arguments/#without_updating_containers

https://github.com/containrrr/watchtower

https://www.the-digital-life.com/watchtower/

https://www.youtube.com/watch?v=5lP_pdjcVMo

Updating

You can either ask Watchtower to update the containers automatically for you, or do it manually.

Manually updating when using docker-compose... 

cd /path/to/docker/stack/
docker-compose down
docker-compose pull
docker-compose up -d

Bitwarden

~/vaultwarden/docker-compose.yml 

services:

  vaultwarden:
    image: "vaultwarden/server:latest"
    container_name: "vaultwarden"
    restart: "always"
    networks:
      traefik:
        ipv4_address: 172.19.0.4
    ports:
      - "8100:80"
    volumes:
      - "./data:/data/"
    environment:
      - "TZ=Europe/London"
      - "SIGNUPS_ALLOWED=true"
      - "INVITATIONS_ALLOWED=true"
      - "WEB_VAULT_ENABLED=true"
    labels:
      - "traefik.enable=true"
      - "traefik.docker.network=traefik"
      - "traefik.http.routers.vaultmydomaincom.rule=Host(`vault.mydomain.com`)"
      - "traefik.http.routers.vaultmydomaincom.entrypoints=websecure"
      - "traefik.http.routers.vaultmydomaincom.tls.certresolver=letsencrypt-gandi"
      - "com.centurylinklabs.watchtower.enable=true"

networks:
  traefik:
    external: true

After you have created an account, signed in and imported your passwords, please change the SIGNUPS_ALLOWED, INVITATIONS_ALLOWED and WEB_VAULT_ENABLED to false. 

docker-compose down
docker-compose up -d

Check that the Bitwarden container environment has all the variables... 

docker exec -it vaultwarden env | sort

HOME=/root
HOSTNAME=e5f327deb4dd
INVITATIONS_ALLOWED=false
PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
ROCKET_ENV=staging
ROCKET_PORT=80
ROCKET_WORKERS=10
SIGNUPS_ALLOWED=false
TERM=xterm
TZ=Europe/London
WEB_VAULT_ENABLED=false

... and then refresh your web vault page to see it see "404: Not Found" :-)

Vaultwarden

InfluxDB

You can have InfluxDB on its own but there is little point without something to view the stats so you might as well include InfluxDB in the Grafana stack and start both at the same time... see below :-)

Grafana

Here is a stack in docker-compose which starts both containers in their own network so they can talk to one another. I have exposed ports for InfluxDB and Grafana to the host so I can use them from the internet.

Obviously, put your firewall in place and change the passwords below!

~/grafana/docker-compose.yml 

version: "3"
services:
  grafana:
    image: grafana/grafana
    container_name: grafana
    restart: always
    networks:
      - grafana-influxdb-network
    ports:
      - 3000:3000
    volumes:
      - ./data/grafana:/var/lib/grafana
    environment:
      - INFLUXDB_URL=http://influxdb:8086
    depends_on:
      - influxdb
  influxdb:
    image: influxdb:1.8.4
    container_name: influxdb
    restart: always
    networks:
      - grafana-influxdb-network
    ports:
      - 8086:8086
    volumes:
      - ./data/influxdb:/var/lib/influxdb
    environment:
      - INFLUXDB_DB=grafana
      - INFLUXDB_USER=grafana
      - INFLUXDB_USER_PASSWORD=password
      - INFLUXDB_ADMIN_ENABLED=true
      - INFLUXDB_ADMIN_USER=admin 
      - INFLUXDB_ADMIN_PASSWORD=password 
      - INFLUXDB_URL=http://influxdb:8086
networks:
  grafana-influxdb-network:
    external: true

After this, change your Telegraf configuration to point to the new host and change the database it uses to 'grafana'.

Traefik

/root/docker/traefik/docker-compose.yml 

services:

  traefik:
    image: "traefik:latest"
    container_name: "traefik"
    networks:
      traefik:
        ipv4_address: 172.19.0.2
    ports:
      - "80:80"
      - "443:443"
    volumes:
      - "/etc/timezone:/etc/timezone:ro"
      - "/etc/localtime:/etc/localtime:ro"
      - "./config/traefik/config.yml:/etc/traefik/config.yml:ro"
      - "./config/traefik/traefik.yml:/etc/traefik/traefik.yml:ro"
      - "./acme/:/acme/"
      - "/var/run/docker.sock:/var/run/docker.sock:ro"
      - "./logs/:/var/log/"
    environment:
      - "TZ=Europe/London"
      - "AWS_ACCESS_KEY_ID=xxxxxxxxxxxxxxxxxxxxxx"
      - "AWS_SECRET_ACCESS_KEY=xxxxxxxxxxxxxxxxxxxxxxxxxxxx"
      - "AWS_REGION=eu-west-2"
      - "GANDIV5_API_KEY=xxxxxxxxxxxxxxxxx"
    labels:
      - "traefik.enable=true"
      - "traefik.http.routers.traefik-dashboard.rule=Host(`traefik-dashboard.mydomain.com`) && (PathPrefix(`/api`) || PathPrefix(`/dashboard`))"
      - "traefik.http.routers.traefik-dashboard.service=api@internal"
      - "traefik.http.routers.traefik-dashboard.middlewares=secured"
      - "traefik.http.middlewares.secured.chain.middlewares=auth,rate"
      - "traefik.http.middlewares.auth.basicauth.users=user-name:$xxxxxxxxxxxxxxxxUlFVbwar4jlRBO1a8K"
      - "traefik.http.middlewares.rate.ratelimit.average=100"
      - "traefik.http.middlewares.rate.ratelimit.burst=50"
      - "com.centurylinklabs.watchtower.enable=true"
    restart: "always"

networks:
  traefik:
    external: true


config/traefik/traefik.yml 

accessLog: {}
accesslog:
  filePath: "/var/log/access.log"
  fields:
    names:
      StartUTC: drop
log:
  filePath: "/var/log/traefik.log"
  level: INFO
  maxSize: 10
  maxBackups: 3
  maxAge: 3
  compress: true
api:
  dashboard: true
providers:
  docker:
    endpoint: "unix:///var/run/docker.sock"
    exposedByDefault: false
    network: traefik
  file:
    filename: /etc/traefik/config.yml
    watch: true
entryPoints:
  web:
    address: ":80"
    http:
      middlewares:
        - "crowdsec-bouncer@file"
      redirections:
        entryPoint:
          to: websecure
          scheme: https
  websecure:
    address: ":443"
    http:
      middlewares:
        - "crowdsec-bouncer@file"
      tls:
        certResolver: letsencrypt-aws
        domains:
          - main: "mydomain.co.uk"
            sans:
              - "*.mydomain.co.uk"
          - main: "mydomain.uk"
            sans:
              - "*.mydomain.uk"
certificatesResolvers:
  letsencrypt-http:
    acme:
      httpChallenge:
        entrypoint: web
      email: "myname@mydomain.co.uk"
      storage: "/acme/letsencrypt-http.json"
  letsencrypt-aws:
    acme:
      dnsChallenge:
        provider: route53
      email: "myname@mydomain.co.uk"
      storage: "/acme/letsencrypt-aws.json"
  letsencrypt-gandi:
    acme:
      dnsChallenge:
        provider: gandiv5
      email: "myname@mydomain.co.uk"
      storage: "/acme/letsencrypt-gandi.json"


config/traefik/config.yml 

http:
  middlewares:    
    crowdsec-bouncer:
      forwardauth:
        address: http://crowdsec-bouncer-traefik:8080/api/v1/forwardAuth
        trustForwardHeader: true

Caddy

Caddy is a lightweight, high performance web server and proxy server. It is much easier to configure than NginX or Traefik.

Here are a few examples in the Caddyfile ... 

# proxy to a btcpay server running in a different docker network and static ssl certificate files
btcpay.mydomain.co.uk {
        tls /ssl/certs/fullchain.pem /ssl/certs/key.pem
        reverse_proxy generated_btcpayserver_1:49392
        log {
                output stdout
        }
}
# proxy to a wordpress web site with custom ssl dns verification
oracle-1.mydomain.co.uk {
        tls me@mydomain.co.uk {
                dns route53 {
                        access_key_id "AKIAJxxxxxxxxxxxxxxx"
                        secret_access_key "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxx"
                        region "eu-west-2"
                }
        }
        reverse_proxy oracle-1.mydomain.co.uk-nginx:80
        log
}

... and this is the docker-compose.yml file for all that above. You do need to build your own docker image for caddy which includes the dns-route-53 plugin ... 

services:
  caddy:
    #image: caddy:alpine
    image: paully/caddy-dns-route53:latest
    container_name: caddy
    restart: unless-stopped
    ports:
      - 80:80
      - 443:443
      - 443:443/udp
    networks:
      - caddy
      - generated_default
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock
      - ./Caddyfile:/etc/caddy/Caddyfile
      - ./data:/data
      - ./config:/config
      - ./fullchain.pem:/ssl/certs/fullchain.pem:ro
      - ./key.pem:/ssl/certs/key.pem:ro
    environment:
      - "TZ=Europe/London"
      - "AWS_ACCESS_KEY_ID=xxxxxxxxxxxxxxxxxxxxx"
      - "AWS_SECRET_ACCESS_KEY=xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx"
      - "AWS_REGION=eu-west-2"
networks:
  caddy:
    external: true
  generated_default:
    external: true

Do I have to restart the container after each Caddyfile change?

Caddy does not require a full restart when configuration is changed. Caddy comes with a caddy reload⁠ command which can be used to reload its configuration with zero downtime.

When running Caddy in Docker, the recommended way to trigger a config reload is by executing the caddy reload command in the running container.

First, you'll need to determine your container ID or name. Then, pass the container ID to docker exec. The working directory is set to /etc/caddy so Caddy can find your Caddyfile without additional arguments. 

caddy_container_id=$(docker ps | grep caddy | awk '{print $1;}')
docker exec -w /etc/caddy $caddy_container_id caddy reload

How do I disable the Caddy admin endpoint?

Add this to the top of your Caddyfile ... 

{
    admin off
}

WordPress

This uses Caddy as the proxy container in the stack with WordPress FPM and MariaDB.

So, the files are all in 1 folder and on the host as a bind volume for Caddy and WordPress. The PHP is handled by Caddy and passed to WordPress PHP-FPM on the container port of 9000, thereby removing the need for the NGINX container because Caddy is doing that part. So, it's very fast! 

Caddy > PHP > WordPress

example.com {
       root * /var/www/html
       php_fastcgi wordpress:9000
       file_server
}

https://sumguy.com/wordpress-on-php-fpm-caddy-in-docker/

NGiNX Proxy Manager

Provide users with an easy way to accomplish reverse proxying hosts with SSL termination that is so easy a monkey could do it.

  1. Set up your host
  2. Add a proxy to point to the host (in Docker this will be the 'name' and the port)
  3. Go to http://yourhost

https://nginxproxymanager.com

https://github.com/jc21/nginx-proxy-manager

Create the Docker network with a chosen subnet (used later for fixing container IP addresses)... 

sudo -i
docker network create --subnet=172.20.0.0/16 nginx-proxy-manager

/root/stacks/nginx-proxy-manager/docker-compose.yml 

version: '3'
services:
  db:
    image: 'jc21/mariadb-aria:latest'
    container_name: nginx-proxy-manager_db
    restart: always
    networks:
      nginx-proxy-manager:
        ipv4_address: 172.20.0.2
    environment:
      TZ: "Europe/London"
      MYSQL_ROOT_PASSWORD: 'npm'
      MYSQL_DATABASE: 'npm'
      MYSQL_USER: 'npm'
      MYSQL_PASSWORD: 'npm'
    volumes:
      - ./data/mysql:/var/lib/mysql
  app:
    image: 'jc21/nginx-proxy-manager:latest'
    container_name: nginx-proxy-manager_app
    restart: always
    networks:
      nginx-proxy-manager:
        ipv4_address: 172.20.0.3
    ports:
      - '80:80'
      - '81:81'
      - '443:443'
    environment:
      TZ: "Europe/London"
      DB_MYSQL_HOST: "db"
      DB_MYSQL_PORT: 3306
      DB_MYSQL_USER: "npm"
      DB_MYSQL_PASSWORD: "npm"
      DB_MYSQL_NAME: "npm"
    volumes:
      - ./data:/data
      - ./data/letsencrypt:/etc/letsencrypt
    depends_on:
      - db
networks:
  nginx-proxy-manager:
    external: true

Reset Password 

docker exec -it nginx-proxy-manager_db sh
mysql -u root -p npm
select * from user;
delete from user where id=1;
quit;
exit

Custom SSL Certificate

You can add a custom SSL certificate to NPM by saving the 3 parts of the SSL from Let's Encrypt...

  1. privkey.pem
  2. cert.pem
  3. chain.pem

...and then uploading them to NPM.

Updating 

docker-compose pull
docker-compose up -d

Fix Error: Bad Gateway - error create table npm.migrations Permission Denied

https://github.com/NginxProxyManager/nginx-proxy-manager/issues/1499#issuecomment-1656997077 

docker exec -it nginx-proxy-manager_db sh
cd /var/lib/mysql
chown -R mysql:mysql npm
exit

NGiNX

Quick Container

Run and delete everything afterwards (press CTRL+C to stop it)... 

docker run --rm --name test.domain.org-nginx -e VIRTUAL_HOST=test.domain.org -p 80:80 nginx:alpine

Run and detach and use a host folder to store the web pages and keep the container afterwards... 

docker run --name test.domain.org-nginx -e VIRTUAL_HOST=test.domain.org -v /some/content:/usr/share/nginx/html:ro -d -p 80:80 nginx:alpine

Run and detach and connect to a specific network (like nginx-proxy-manager) and use a host folder to store the web pages and keep the container afterwards... 

 docker run --name test.domain.org-nginx --network nginx-proxy-manager -e VIRTUAL_HOST=test.domain.org -v /some/content:/usr/share/nginx/html:ro -d -p 80:80 nginx:alpine

Check the logs and always show them (like tail -f)... 

 docker logs test.domain.org-nginx -f

docker-compose.yml 

version: "3"
services:
  nginx:
    image: nginx
    container_name: nginx
    environment:
      - TZ=Europe/London
    volumes:
      - ./data/html:/usr/share/nginx/html:ro
      - /etc/timezone:/etc/timezone:ro
    expose:
      - 80
    restart: unless-stopped


With PHP

./data/nginx/site.conf 

server {
    server_name docker-demo.com;
    root /var/www/html;
    index index.php index.html index.htm;
    access_log /var/log/nginx/access.log;
    error_log /var/log/nginx/error.log;
    location / {
        try_files $uri $uri/ /index.php?$query_string;
    }
    # PHP-FPM Configuration Nginx
    location ~ \.php$ {
        try_files $uri = 404;
        fastcgi_split_path_info ^(.+\.php)(/.+)$;
        fastcgi_pass php:9000;
        fastcgi_index index.php;
        include fastcgi_params;
        fastcgi_param REQUEST_URI $request_uri;
        fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
        fastcgi_param PATH_INFO $fastcgi_path_info;
    }
}

docker-compose.yml 

version: "3"
services:
  nginx:
    image: nginx
    container_name: nginx
    environment:
      - TZ=Europe/London
    volumes:
      - ./data/html:/usr/share/nginx/html:ro
      - ./data/nginx:/etc/nginx/conf.d/
      - /etc/timezone:/etc/timezone:ro
    expose:
      - 80
    restart: unless-stopped
  php:
    image: php:7.2-fpm
    volumes:
      - ./data/html:/usr/share/nginx/html:ro
      - ./data/php:/usr/local/etc/php/php.ini

https://adoltech.com/blog/how-to-set-up-nginx-php-fpm-and-mysql-with-docker-compose/

With PERL

This is a way to get the IP address of the visitor (REMOTE_ADDR) displayed... 

# nginx.conf; mostly copied from defaults
load_module "modules/ngx_http_perl_module.so";

user  nginx;
worker_processes  auto;

error_log  /var/log/nginx/error.log notice;
pid        /var/run/nginx.pid;

events {
    worker_connections  1024;
}

http {
    #perl_modules /; # only needed the hello.pm isn't in @INC (e.g. dir specified below)
    perl_modules /usr/lib/perl5/vendor_perl/x86_64-linux-thread-multi/;
    perl_require hello.pm;

    server {
        location / {
            perl hello::handler;
        }
    }
}

# hello.pm; put in a @INC dir
package hello;
use nginx;

sub handler {
    my $r = shift;
    $r->send_http_header("text/html");
    return OK if $r->header_only;
    $r->print($r->remote_addr);
    return OK;
}
1;

https://www.reddit.com/r/docker/comments/oabga4/run_perl_script_in_nginx_container/

Load Balancer

This is a simple exmaple test to show multiple backend servers answering web page requests.

docker-compose.yml 

version: '3'
services:
  # The load balancer
  nginx:
    image: nginx:1.16.0-alpine
    volumes:
      - ./nginx.conf:/etc/nginx/nginx.conf:ro
    ports:
      - "80:80"
  # The web server1
  server1:
    image: nginx:1.16.0-alpine
    volumes:
      - ./server1.html:/usr/share/nginx/html/index.html
  # The web server2
  server2:
    image: nginx:1.16.0-alpine
    volumes:
      - ./server2.html:/usr/share/nginx/html/index.html

nginx.conf 

events {
    worker_connections 1024;
}
http {
    upstream app_servers {    # Create an upstream for the web servers
        server server1:80;    # the first server
        server server2:80;    # the second server
    }
    server {
        listen 80;
        location / {
            proxy_pass         http://app_servers;  # load balance the traffic
        }
    }
}

https://omarghader.github.io/docker-compose-nginx-tutorial/

Proxy

This is very cool and allows you to run multiple web sites on-the-fly.

The container connects to the system docker socket and watches for new containers using the VIRTUAL_HOST environment variable.

Start this, then add another container using the VIRTUAL_HOST variable and the proxy container will change its config file and reload nginx to serve the web site... automatically.

Incredible.

~/nginx-proxy/docker-compose.yml 

version: "3"
services:
  nginx-proxy:
    image: jwilder/nginx-proxy
    container_name: nginx-proxy
    ports:
      - "80:80"
    volumes:
      - /var/run/docker.sock:/tmp/docker.sock:ro
networks:
  default:
    external:
      name: nginx-proxy

Normal

When using the nginx-proxy container above, you can just spin up a virtual web site using the standard 'nginx' docker image and link it to the 'nginx-proxy' network... 

docker run -d --name nginx-website1.uk --expose 80 --net nginx-proxy -e VIRTUAL_HOST=website1.uk nginx

To use the host filesystem to store the web page files... 

docker run -d --name nginx-website1.uk --expose 80 --net nginx-proxy -v /var/www/website1.uk/html:/usr/share/nginx/html:ro -e VIRTUAL_HOST=website1.uk nginx

In Docker Compose, it will look like this...

~/nginx/docker-compose.yml 

version: "3"
services:
  nginx-website1.uk:
    image: nginx
    container_name: nginx-website1.uk
    expose:
      - "80"
    volumes:
      - /var/www/website1.uk/html:/usr/share/nginx/html:ro
    environment:
      - VIRTUAL_HOST=website1.uk
networks:
  default:
    external:
      name: nginx-proxy

Multiple Virtual Host Web Sites

~/nginx/docker-compose.yml 

version: "3"
services:
  nginx-website1.uk:
    image: nginx
    container_name: nginx-website1.uk
    expose:
      - "80"
    volumes:
      - /var/www/website1.uk/html:/usr/share/nginx/html:ro
    environment:
      - VIRTUAL_HOST=website1.uk
  nginx-website2.uk:
    image: nginx
    container_name: nginx-website2.uk
    expose:
      - "80"
    volumes:
      - /var/www/website2.uk/html:/usr/share/nginx/html:ro
    environment:
      - VIRTUAL_HOST=website2.uk
networks:
  default:
    external:
      name: nginx-proxy

Viewing Logs 

docker-compose logs nginx-website1.uk
docker-compose logs nginx-website2.uk

Proxy Manager

This is a web front end to manage 'nginx-proxy', where you can choose containers and create SSL certificates etc.

https://cyberhost.uk/npm-setup/

Various

https://hub.docker.com/_/nginx

https://blog.ssdnodes.com/blog/host-multiple-websites-docker-nginx/

https://github.com/nginx-proxy/nginx-proxy

Typical LEMP

https://adoltech.com/blog/how-to-set-up-nginx-php-fpm-and-mysql-with-docker-compose/

WordPress

https://hub.docker.com/_/wordpress/

PHP File Uploads Fix

Create a new PHP configuration file, and name it php.ini. Add the following configuration then save the changes. 

# Hide PHP version
expose_php = Off 

# Allow HTTP file uploads
file_uploads = On

# Maximum size of an uploaded file
upload_max_filesize = 64M

# Maximum size of form post data
post_max_size = 64M

# Maximum Input Variables
max_input_vars = 3000

Update the docker-compose.yml to bind the php.ini to the wordpress container and then restart the WordPress container. 

volumes:
  - ./data/config/php.ini:/usr/local/etc/php/conf.d/php.ini

WordPress Clone

Create your A record in DNS using AWS Route 53 CLI... 

cli53 rrcreate domain.co.uk 'staging 300 A 123.456.78.910'

Create your docker folder for the cloned staging test web site... 

mkdir -p ~/docker/stacks/staging.domain.co.uk/data/{db,html}

Edit your docker compose file, with 2 containers, making sure you use the same network as your Nginx Proxy Manager...

~/docker/stacks/staging.domain.co.uk/docker-compose.yml 

version: "3"
services:
  staging.domain.co.uk-wordpress_db:
    image: mysql:5.7
    container_name: staging.domain.co.uk-wordpress_db
    volumes:
      - ./data/db:/var/lib/mysql
    restart: always
    environment:
      - TZ=Europe/London
      - MYSQL_ROOT_PASSWORD=changeme
      - MYSQL_DATABASE=dbname
      - MYSQL_USER=dbuser
      - MYSQL_PASSWORD=changeme
  staging.domain.co.uk-wordpress:
    depends_on:
      - staging.domain.co.uk-wordpress_db
    image: wordpress:latest
    container_name: staging.domain.co.uk-wordpress
    volumes:
      - ./data/html:/var/www/html
    restart: always
    environment:
      - TZ=Europe/London
      - VIRTUAL_HOST=staging.domain.co.uk
      - WORDPRESS_DB_HOST=staging.domain.co.uk-wordpress_db:3306
      - WORDPRESS_DB_NAME=dbname
      - WORDPRESS_DB_USER=dbuser
      - WORDPRESS_DB_PASSWORD=changeme
  staging.domain.co.uk-wordpress-cli:
    image: wordpress:cli
    container_name: staging.domain.co.uk-wordpress-cli
    volumes:
      - ./data/html:/var/www/html
    environment:
      - TZ=Europe/London
      - WP_CLI_CACHE_DIR=/tmp/
      - VIRTUAL_HOST=staging.domain.co.uk
      - WORDPRESS_DB_HOST=staging.domain.co.uk-wordpress_db:3306
      - WORDPRESS_DB_NAME=dbname
      - WORDPRESS_DB_USER=dbuser
      - WORDPRESS_DB_PASSWORD=changeme
    working_dir: /var/www/html
    user: "33:33"
networks:
  default:
    external:
      name: nginx-proxy-manager

Start containers with correct settings and credentials for existing live web site (so that the docker startup script sets up the MySQL permissions)... 

docker-compose up -d

Check the logs to make sure all is well... 

docker logs staging.domain.co.uk-wordpress
docker logs staging.domain.co.uk-wordpress_db

Copy the WordPress files to the host folder and correct ownership... 

rsync -av /path/to/backup_unzipped_wordpress/ ~/docker/stacks/staging.domain.co.uk/html/
chown -R www-data:www-data ~/docker/stacks/staging.domain.co.uk/html/

Copy the sql file in to the running mysql container... 

docker cp /path/to/backup_unzipped_wordpress/db_name.sql mysql_container_name:/tmp/

Log in to the database container... 

docker exec -it mysql_container_name bash

Check and if necessary, change the timezone... 

date
mv /etc/localtime /etc/localtime.backup
ln -s /usr/share/zoneinfo/Europe/London /etc/localtime
date

Delete and create the database... 

mysql -u root -p -e "DROP DATABASE db_name; CREATE DATABASE db_name;"

Import the database from the sql file, check and exit out of the container... 

mysql -u root -p mysql_db_name < /tmp/db_name.sql
mysql -u root -p -e "use db_name; show tables;"
rm /tmp/db_name.sql
exit

Edit the wp-config.php on your host server to match new DB_HOST and also add extra variables to be sure... 

nano /path/to/docker/folder/html/wp-config.php
define( 'WP_HOME', 'http://staging.domain.co.uk' );
define( 'WP_SITEURL', 'http://staging.domain.co.uk' );

Install WordPress CLI in the running container... 

docker exec -it wordpress_container_name bash

Search and replace the original site url... 

./wp --allow-root search-replace 'http://www.domain.co.uk/' 'http://staging.domain.co.uk/' --dry-run
./wp --allow-root search-replace 'http://www.domain.co.uk/' 'http://staging.domain.co.uk/'

Start your web browser and go to the test staging web site!

WordPress CLI

In your stack, set up the usual two DB + WordPress containers, then add a third services section for wp-cli... 

version: "3"
services:
  www.domain.uk-wordpress_db:
    image: mysql:5.7
    container_name: www.domain.uk-wordpress_db
    volumes:
      - ./data/db:/var/lib/mysql
    restart: always
    environment:
      - MYSQL_ROOT_PASSWORD=password
      - MYSQL_DATABASE=wordpress
      - MYSQL_USER=wordpress
      - MYSQL_PASSWORD=password
  www.domain.uk-wordpress:
    depends_on:
      - www.domain.uk-wordpress_db
    image: wordpress:latest
    container_name: www.domain.uk-wordpress
    volumes:
      - ./data/html:/var/www/html
    expose:
      - 80
    restart: always
    environment:
      - VIRTUAL_HOST=www.domain.uk
      - WORDPRESS_DB_HOST=www.domain.uk-wordpress_db:3306
      - WORDPRESS_DB_NAME=wordpress
      - WORDPRESS_DB_USER=wordpress
      - WORDPRESS_DB_PASSWORD=password
  www.domain.uk-wordpress-cli:
    image: wordpress:cli
    container_name: www.domain.uk-wordpress-cli
    volumes:
      - ./data/html:/var/www/html
    environment:
      - WP_CLI_CACHE_DIR=/tmp/
      - VIRTUAL_HOST=www.domain.uk
      - WORDPRESS_DB_HOST=www.domain.uk-wordpress_db:3306
      - WORDPRESS_DB_NAME=wordpress
      - WORDPRESS_DB_USER=wordpress
      - WORDPRESS_DB_PASSWORD=password
    working_dir: /var/www/html
    user: "33:33"
networks:
  default:
    external:
      name: nginx-proxy-manager

...then start it all up. 

docker-compose up -d

Then, run your wp-cli commands (e.g. wp user list) on the end of a docker run command... 

docker-compose run --rm www.domain.uk-wordpress-cli wp --info
docker-compose run --rm www.domain.uk-wordpress-cli wp cli version
docker-compose run --rm www.domain.uk-wordpress-cli wp user list
docker-compose run --rm www.domain.uk-wordpress-cli wp help theme
docker-compose run --rm www.domain.uk-wordpress-cli wp theme delete --all

SSL Behind A Reverse Proxy

https://wiki.indie-it.com/wiki/WordPress#SSL_When_Using_A_Reverse_Proxy

Email Server (mailu)

Mailu is a simple yet full-featured mail server as a set of Docker images. It is free software (both as in free beer and as in free speech), open to suggestions and external contributions. The project aims at providing people with an easily setup, easily maintained and full-featured mail server while not shipping proprietary software nor unrelated features often found in popular groupware.

https://mailu.io/1.7/

https://hub.docker.com/u/mailu

https://github.com/Mailu/Mailu

Postfix Admin

https://hub.docker.com/_/postfixadmin

Email Server (docker-mailserver)

https://github.com/docker-mailserver

https://github.com/docker-mailserver/docker-mailserver

https://github.com/docker-mailserver/docker-mailserver-admin

https://docker-mailserver.github.io/docker-mailserver/edge/config/security/ssl/#lets-encrypt-recommended

Postscreen

Postscreen is an SMTP filter that blocks spambots (or zombie machines) away from the real Postfix smtpd daemon, so Postfix does not feel overloaded and can process legitimate emails more efficiently.

The example below shows a typical spambot attempt at accessing the SMTP service and being stopped... 

Jun 24 10:42:26 mail postfix/postscreen[203907]: CONNECT from [212.70.149.56]:19452 to [172.23.0.2]:25
Jun 24 10:42:26 mail postfix/dnsblog[386054]: addr 212.70.149.56 listed by domain b.barracudacentral.org as 127.0.0.2
Jun 24 10:42:26 mail postfix/dnsblog[402550]: addr 212.70.149.56 listed by domain list.dnswl.org as 127.0.10.3
Jun 24 10:42:26 mail postfix/dnsblog[407802]: addr 212.70.149.56 listed by domain bl.mailspike.net as 127.0.0.2
Jun 24 10:42:26 mail postfix/dnsblog[386155]: addr 212.70.149.56 listed by domain psbl.surriel.com as 127.0.0.2
Jun 24 10:42:29 mail postfix/postscreen[203907]: PREGREET 11 after 2.9 from [212.70.149.56]:19452: EHLO User\r\n
Jun 24 10:42:29 mail postfix/postscreen[203907]: DISCONNECT [212.70.149.56]:19452

Postscreen is enabled by default but there are a few settings to tweak to get the best out of it.

Edit your data/config/postfix-main.cf file and add the following lines, making sure your Docker host IP is in bold... 

mynetworks = 127.0.0.0/8 [::1]/128 [fe80::]/64 172.19.0.2/32 172.19.0.1/32
postscreen_greet_action = drop
postscreen_pipelining_enable = yes
postscreen_pipelining_action = drop
postscreen_non_smtp_command_enable = yes
postscreen_non_smtp_command_action = drop
postscreen_bare_newline_enable = yes 
postscreen_bare_newline_action = drop

Enable and Configure Postscreen in Postfix to Block Spambots

Postgrey

Local List

SpamAssassin

Custom Rules

Bayes Database

SpamAssassin is controlled by Amavis (a fork of MailScanner) with the user 'amavis'.

Show Bayes Database Stats 

docker exec --user amavis mail.mydomain.org.uk-mailserver sa-learn --dump magic --dbpath /var/lib/amavis/.spamassassin

Learn Ham 

docker exec --user amavis mail.mydomain.org.uk-mailserver sa-learn --ham --progress /var/mail/mydomain.org.uk/info/cur --dbpath /var/lib/amavis/.spamassassin

Backup and Restore from Existing Mail Server

On the old server... 

/bin/su -l -c '/usr/bin/sa-learn --backup > sa-learn_backup.txt' debian-spamd
rsync -avP /var/lib/spamassassin/sa-learn_backup.txt user@mail.mydomain.org.uk:/tmp/

On the new server... 

docker cp /tmp/sa-learn_backup.txt mail.mydomain.org.uk-mailserver:/tmp/
docker exec --user amavis mail.mydomain.org.uk-mailserver sa-learn --sync --dbpath /var/lib/amavis/.spamassassin
docker exec --user amavis mail.mydomain.org.uk-mailserver sa-learn --clear --dbpath /var/lib/amavis/.spamassassin
docker exec --user amavis mail.mydomain.org.uk-mailserver sa-learn --restore /tmp/sa-learn_backup.txt --dbpath /var/lib/amavis/.spamassassin
docker exec --user amavis mail.mydomain.org.uk-mailserver sa-learn --sync --dbpath /var/lib/amavis/.spamassassin
docker exec --user amavis mail.mydomain.org.uk-mailserver sa-learn --dump magic --dbpath /var/lib/amavis/.spamassassin

Fail2Ban

List jails... 

docker exec -it mail.mydomain.org.uk-mailserver fail2ban-client status

Status
|- Number of jail:   3
`- Jail list:   dovecot, postfix, postfix-sasl

Manually ban IP address in named jail... 

docker exec -it mail.mydomain.org.uk-mailserver fail2ban-client set postfix banip 212.70.149.56

Check banned IPs... 

docker exec -it mail.mydomain.org.uk-mailserver fail2ban-client status postfix

Status for the jail: postfix
|- Filter
|  |- Currently failed: 2
|  |- Total failed:     2
|  `- File list:        /var/log/mail.log
`- Actions
   |- Currently banned: 1
   |- Total banned:     1
   `- Banned IP list:   212.70.149.56

https://www.the-lazy-dev.com/en/install-fail2ban-with-docker/

Backups

Backups

Autodiscover

Create SRV and A record entries in your DNS for the services... 

$ORIGIN domain.org.uk.
@	300	IN	TXT	"v=spf1 mx ~all; mailconf=https://autoconfig.domain.org.uk/mail/config-v1.1.xml"
_autodiscover._tcp	300	IN	SRV	0 0 443 autodiscover.domain.org.uk.
_imap._tcp	300	IN	SRV	0 0 0 .
_imaps._tcp	300	IN	SRV	0 1 993 mail.domain.org.uk.
_ldap._tcp	300	IN	SRV	0 0 636 mail.domain.org.uk.
_pop3._tcp	300	IN	SRV	0 0 0 .
_pop3s._tcp	300	IN	SRV	0 0 0 .
_submission._tcp	300	IN	SRV	0 1 587 mail.domain.org.uk.
autoconfig	300	IN	A	3.10.67.19
autodiscover	300	IN	A	3.10.67.19
imap	300	IN	CNAME	mail
mail	300	IN	A	3.10.67.19
smtp	300	IN	CNAME	mail
www	300	IN	A	3.10.67.19

docker-compose.yml 

services:
  mailserver-autodiscover:
    image: monogramm/autodiscover-email-settings:latest
    container_name: mail.domain.org.uk-mailserver-autodiscover
    environment:
      - COMPANY_NAME=My Company
      - SUPPORT_URL=https://autodiscover.domain.org.uk
      - DOMAIN=domain.org.uk
      - IMAP_HOST=mail.domain.org.uk
      - IMAP_PORT=993
      - IMAP_SOCKET=SSL
      - SMTP_HOST=mail.domain.org.uk
      - SMTP_PORT=587
      - SMTP_SOCKET=STARTTLS
    restart: unless-stopped
networks:
  default:
    external:
      name: nginx-proxy-manager

Autodiscover

monogramm/autodiscover-email-settings

Internet Speedtest

https://github.com/henrywhitaker3/Speedtest-Tracker

Emby Media Server

https://emby.media/docker-server.html

https://hub.docker.com/r/emby/embyserver

AWS CLI

docker run --rm -it -v "/root/.aws:/root/.aws" amazon/aws-cli configure
docker run --rm -it -v "/root/.aws:/root/.aws" amazon/aws-cli s3 ls
docker run --rm -it -v "/root/.aws:/root/.aws" amazon/aws-cli route53 list-hosted-zones

https://docs.aws.amazon.com/cli/latest/userguide/install-cliv2-docker.html

Let's Encrypt

Force RENEW a standalone certificate with the new preferred chain of "ISRG Root X1" 

docker run -it --rm --name certbot -v "/usr/bin:/usr/bin" -v "/root/.aws:/root/.aws" -v "/etc/letsencrypt:/etc/letsencrypt" -v "/var/lib/letsencrypt:/var/lib/letsencrypt" certbot/certbot --force-renewal --preferred-chain "ISRG Root X1" certonly --standalone --email me@mydomain.com --agree-tos -d www.mydomain.com

Issue a wildcard certificate... 

docker run -it --rm --name certbot -v "/usr/bin:/usr/bin" -v "/root/.aws:/root/.aws" -v "/etc/letsencrypt:/etc/letsencrypt" -v "/var/lib/letsencrypt:/var/lib/letsencrypt" certbot/dns-route53 certonly --dns-route53 --domain "example.com" --domain "*.example.com"

Check your certificates... 

docker run -it --rm --name certbot -v "/usr/bin:/usr/bin" -v "/root/.aws:/root/.aws" -v "/etc/letsencrypt:/etc/letsencrypt" -v "/var/lib/letsencrypt:/var/lib/letsencrypt" certbot/certbot certificates

Renew a certificate... 

docker run -it --rm --name certbot -v "/usr/bin:/usr/bin" -v "/root/.aws:/root/.aws" -v "/etc/letsencrypt:/etc/letsencrypt" -v "/var/lib/letsencrypt:/var/lib/letsencrypt" certbot/dns-route53 renew

If you have multiple profiles in your .aws/config then you will need to pass the AWS_PROFILE variable to the docker container... 

docker run -it --rm --name certbot -v "/usr/bin:/usr/bin" -v "/root/.aws:/root/.aws" -v "/etc/letsencrypt:/etc/letsencrypt" -v "/var/lib/letsencrypt:/var/lib/letsencrypt" -e "AWS_PROFILE=certbot" certbot/dns-route53 renew

https://certbot.eff.org/docs/install.html#running-with-docker

VPN

Gluetun

Gluetun VPN client

Lightweight swiss-knife-like VPN client to tunnel to Cyberghost, ExpressVPN, FastestVPN, HideMyAss, IPVanish, IVPN, Mullvad, NordVPN, Perfect Privacy, Privado, Private Internet Access, PrivateVPN, ProtonVPN, PureVPN, Surfshark, TorGuard, VPNUnlimited, VyprVPN, WeVPN and Windscribe VPN servers using Go, OpenVPN or Wireguard, iptables, DNS over TLS, ShadowSocks and an HTTP proxy.

Github

Connect a container to Gluetun

OpenVPN

Server

https://hub.docker.com/r/linuxserver/openvpn-as

Client

https://hub.docker.com/r/dperson/openvpn-client

Routing Containers Through Container 

sudo docker run -it --net=container:vpn -d some/docker-container

OpenVPN-PiHole

https://github.com/Simonwep/openvpn-pihole

WireHole

WireHole is a combination of WireGuard, PiHole, and Unbound in a docker-compose project with the intent of enabling users to quickly and easily create and deploy a personally managed full or split-tunnel WireGuard VPN with ad blocking capabilities (via Pihole), and DNS caching with additional privacy options (via Unbound).

https://github.com/IAmStoxe/wirehole

To view a QR code, run this ... 

docker exec -it wireguard /app/show-peer 1

WireGuard

Use WireHole instead!

docker-compose.yml 

version: "2.1"
services:
  wireguard:
    image: ghcr.io/linuxserver/wireguard
    container_name: wireguard
    cap_add:
      - NET_ADMIN
      - SYS_MODULE
    environment:
      - PUID=1000
      - PGID=1000
      - TZ=Europe/London
      - SERVERURL=wireguard.domain.uk
      - SERVERPORT=51820
      - PEERS=3
      - PEERDNS=auto
      - INTERNAL_SUBNET=10.13.13.0
      - ALLOWEDIPS=0.0.0.0/0
    volumes:
      - ./data/config:/config
      - /lib/modules:/lib/modules
    ports:
      - 51820:51820/udp
    sysctls:
      - net.ipv4.conf.all.src_valid_mark=1
    restart: unless-stopped

https://hub.docker.com/r/linuxserver/wireguard

To show the QR code 

docker exec -it wireguard /app/show-peer 1
docker exec -it wireguard /app/show-peer 2
docker exec -it wireguard /app/show-peer 3

Error: const struct ipv6_stub

If you receive an error in the container logs about not being able to compile the kernel module, then follow the instructions to compile the WireGuard kernel module and tools in your host system.

https://github.com/linuxserver/docker-wireguard/issues/46#issuecomment-708278250

Force Docker Containers to use a VPN for connection

ffmpeg

docker pull jrottenberg/ffmpeg
docker run jrottenberg/ffmpeg -h
docker run jrottenberg/ffmpeg -i /path/to/input.mkv -stats $ffmpeg_options - > out.mp4
docker run -v $(pwd):$(pwd) -w $(pwd) jrottenberg/ffmpeg -y -i input.mkv -t 00:00:05.00 -vf scale=-1:360 output.mp4

https://registry.hub.docker.com/r/jrottenberg/ffmpeg

https://github.com/jrottenberg/ffmpeg

https://medium.com/coconut-stories/using-ffmpeg-with-docker-94523547f35c

https://github.com/linuxserver/docker-ffmpeg

MediaInfo

Install ... 

sudo docker pull jlesage/mediainfo

Run ... 

docker run --rm --name=mediainfo -e USER_ID=$(id -u) -e GROUP_ID=$(id -g) -v $(pwd):$(pwd):ro jlesage/mediainfo su-exec "$(id -u):$(id -g)" /usr/bin/mediainfo --help

https://github.com/jlesage/docker-mediainfo

MKV Toolnix

Install ... 

sudo docker pull jlesage/mkvtoolnix

Run ...

mkvextract 

docker run --rm --name=mkvextract -e USER_ID=$(id -u) -e GROUP_ID=$(id -g) -v $(pwd):$(pwd):rw jlesage/mkvtoolnix su-exec "$(id -u):$(id -g)" /usr/bin/mkvextract "$(pwd)/filename.mkv" tracks 3:"$(pwd)/filename.eng.srt"

mkvpropedit 

docker run --rm --name=mkvextract -e USER_ID=$(id -u) -e GROUP_ID=$(id -g) -v $(pwd):$(pwd):rw jlesage/mkvtoolnix su-exec "$(id -u):$(id -g)" /usr/bin/mkvpropedit "$(pwd)/filename.mkv" --edit track:a1 --set language=eng

https://github.com/jlesage/docker-mkvtoolnix

MakeMKV

This will NOT work on a Raspberry Pi.

https://github.com/jlesage/docker-makemkv

Use this in combination with ffmpeg or HandBrake (as shown below) and FileBot to process your media through to your media server - like Emby or Plex..

MakeMKV > HandBrake > FileBot > Emby

To make this work with your DVD drive (/dev/sr0) you need to have the second device (/dev/sg0) in order for it to work. I don't get it, but it works.

/root/docker/stacks/makemkv/docker-compose.yml 

version: '3'
services:
  makemkv:
    image: jlesage/makemkv
    container_name: makemkv
    ports:
      - "0.0.0.0:5801:5800"
    volumes:
      - "/home/user/.MakeMKV_DOCKER:/config:rw"
      - "/home/user/:/storage:ro"
      - "/home/user/ToDo/MakeMKV/output:/output:rw"
    devices:
      - "/dev/sr0:/dev/sr0"
      - "/dev/sg0:/dev/sg0"
    environment:
      - USER_ID=1000
      - GROUP_ID=1000
      - TZ=Europe/London
      - MAKEMKV_KEY=your_licence_key
      - AUTO_DISC_RIPPER=1

Troubleshooting

PROBLEM = "driver failed programming external connectivity on endpoint makemkv: Error starting userland proxy: listen tcp6 [::]:5800: socket: address family not supported by protocol."

SOLUTION = Put 0.0.0.0:5801 in the published ports line of docker compose to restrict the network to IPv4.

Docker Process Output 

CONTAINER ID   IMAGE               COMMAND   CREATED        STATUS        PORTS                              NAMES
4a8b3106b00b   jlesage/handbrake   "/init"   40 hours ago   Up 40 hours   5900/tcp, 0.0.0.0:5802->5800/tcp   handbrake
89fe3ba8a31e   jlesage/makemkv     "/init"   40 hours ago   Up 40 hours   5900/tcp, 0.0.0.0:5801->5800/tcp   makemkv

Command Line 

docker run --rm -v "/home/user/.MakeMKV_DOCKER:/config:rw" -v "/home/user/:/storage:ro" -v "/home/user/ToDo/MakeMKV/output:/output:rw" --device /dev/sr0 --device /dev/sg0 --device /dev/sg1 jlesage/makemkv /opt/makemkv/bin/makemkvcon mkv disc:0 all /output

https://github.com/jlesage/docker-makemkv/issues/141

HandBrake

This will NOT work on a Raspberry Pi.

Use this in combination with ffmpeg or MakeMKV (as shown below) and FileBot to process your media through to your media server - like Emby or Plex..

I have changed the port from 5800 to 5802 because Jocelyn's other Docker image for MakeMKV uses the same port (so I move that one as well to 5801 - see above).

To make this work with your DVD drive (/dev/sr0) you need to have the second device (/dev/sg0) in order for it to work. I don't get it, but it works.

YouTube / DB Tech - How to install HandBrake in Docker

Blog / DB Tech - How to install HandBrake in Docker

Docker HandBrake by Jocelyn Le Sage

Docker Image by Jocelyn Le Sage

/root/docker/stacks/handbrake/docker-compose.yml 

version: '3'
services:
  handbrake:
    image: jlesage/handbrake
    container_name: handbrake
    ports:
      - "0.0.0.0:5802:5800"
    volumes:
      - "/home/paully:/storage:ro"
      - "/home/paully/ToDo/HandBrake/config:/config:rw"
      - "/home/paully/ToDo/HandBrake/watch:/watch:rw"
      - "/home/paully/ToDo/HandBrake/output:/output:rw"
    devices:
      - "/dev/sr0:/dev/sr0"
      - "/dev/sg0:/dev/sg0"
    environment:
      - USER_ID=1000
      - GROUP_ID=1000
      - TZ=Europe/London

Docker Process Output 

CONTAINER ID   IMAGE               COMMAND   CREATED        STATUS        PORTS                              NAMES
4a8b3106b00b   jlesage/handbrake   "/init"   40 hours ago   Up 40 hours   5900/tcp, 0.0.0.0:5802->5800/tcp   handbrake
89fe3ba8a31e   jlesage/makemkv     "/init"   40 hours ago   Up 40 hours   5900/tcp, 0.0.0.0:5801->5800/tcp   makemkv

Command Line 

docker run --rm -v "/home/paully/:/storage:ro" -v "/home/paully/ToDo/HandBrake/config:/config:rw" -v "/home/paully/ToDo/HandBrake/watch:/watch:rw" -v "/home/paully/ToDo/HandBrake/output:/output:rw" --device /dev/sr0 --device /dev/sg0 jlesage/handbrake /usr/bin/HandBrakeCLI --input "/output/file.mkv" --stop-at duration:120 --preset 'Fast 480p30' --non-anamorphic --encoder-preset slow --quality 22 --deinterlace --lapsharp --audio 1 --aencoder copy:ac3 --no-markers --output "/output/file.mp4"

docker run --rm -v "/home/paully/:/storage:ro" -v "/home/paully/ToDo/HandBrake/config:/config:rw" -v "/home/paully/ToDo/HandBrake/watch:/watch:rw" -v "/home/paully/ToDo/HandBrake/output:/output:rw" jlesage/handbrake /usr/bin/HandBrakeCLI --input "/storage/input.mkv" --preset 'Super HQ 2160p60 4K HEVC Surround' --encoder x265 --non-anamorphic --audio 1 --aencoder copy:eac3 --no-markers --output "/output/output.mkv"

FileBot

Setup

Create your directories for data volumes (https://github.com/jlesage/docker-filebot#data-volumes) ... 

mkdir -p ~/filebot/{config,output,watch}

The license file received via email can be saved on the host, into the configuration directory of the container (i.e. in the directory mapped to /config). Then, start or restart the container to have it automatically installed. NOTE: The license file is expected to have a .psm extension.

Usage

WORK IN PROGRESS

Rather than running all the time, we run the image ad-hoc with the rm option to delete the old container each time... 

docker run --rm --name=filebot -v ~/filebot/config:/config:rw -v $HOME:/storage:rw -v ~/filebot/output:/output:rw -v ~/filebot/watch:/watch:rw -e AMC_ACTION=test jlesage/filebot

https://github.com/jlesage/docker-filebot

https://github.com/filebot/filebot-docker

Automated Downloaderr

This takes the hassle out of going through the various web sites to find stuff and be bombarded with ads and pop-ups.

  1. FlareSolverr
  2. Prowlarr
  3. Radarr + Sonarr + Bazarr
  4. Transmission + NZBGet
  5. Tdarr
FlareSolverr > Prowlarr > Radarr + Sonarr + Bazarr > Transmission + NZBGet > Tdarr


HOW TO RESTART THE RRS IN ORDER ON PORTAINER OR OMV

Stacks > Click on each one > Stop > count to 10 > Start

  • WireGuard
  • FlareSolverr
  • Prowlarr
  • Radarr
  • Sonarr

ONE DAY WE WILL GET A SINGLE STACK WITH ALL THE RIGHT CONTAINERS STARTING IN THE RIGHT ORDER


https://hotio.dev/containers/autoscan/

ONE APP TO RULE THEM ALL

https://github.com/JagandeepBrar/LunaSea


GUIDE FOR DIRECTORY STRUCTURE

https://trash-guides.info/Hardlinks/How-to-setup-for/Docker/

This will enable you to automatically rename files but allow you to copy them to your actual Plex or Emby folders.

It is possible to do hard linking and let the rrrrs control all the files but I am not a fan of that.

This way, the files get renamed and moved to a 'halfway' house where you can check them and then simply move them to your desired location.

METHOD

Create the directories ... 

mkdir -p /path/to/data/{media,torrents,usenet}/{movies,music,tv}
mkdir -p /path/to/docker/appdata/{radarr,sonarr,bazarr,nzbget}

Change the ownership and permissions ... 

chown -R admin:users /path/to/data/ /path/to/docker/
find /path/to/data/ /path/to/docker/ -type d -exec chmod 0755 {} \;
find /path/to/data/ /path/to/docker/ -type d -exec chmod g+s {} \;

Finished directory structure ... 

/srv/dev-disk-by-uuid-7f81e7b6-1a05-4232-893b-f34c046b2bdb/data/
|-- media
|   |-- movies
|   |-- music
|   `-- tv
|-- torrents
|   |-- movies
|   |-- music
|   `-- tv
`-- usenet
    |-- completed
    |   |-- movies
    |   `-- tv
    |-- movies
    |-- music
    |-- nzb
    |   `-- Movie.Name.720p.nzb.queued
    |-- tmp
    `-- tv

PORTAINER STACK

This is from Open Media Vault (OMV) so the volume paths are long.

This works but needs the whole VPN thing added (which changes ports etc) but for now ...

Portainer > Stacks > Add Stack > Datarr 

version: "3.2"
services:
  prowlarr:
    container_name: prowlarr
    image: hotio/prowlarr:latest
    restart: unless-stopped
    logging:
      driver: json-file
    network_mode: bridge
    ports:
      - 9696:9696
    environment:
      - PUID=998
      - PGID=100
      - TZ=Europe/London
    volumes:
      - /etc/localtime:/etc/localtime:ro
      - /srv/dev-disk-by-uuid-7f81e7b6-1a05-4232-893b-f34c046b2bdb/docker/appdata/prowlarr:/config
      - /srv/dev-disk-by-uuid-7f81e7b6-1a05-4232-893b-f34c046b2bdb/data:/data
  radarr:
    container_name: radarr
    image: hotio/radarr:latest
    restart: unless-stopped
    logging:
      driver: json-file
    network_mode: bridge
    ports:
      - 7878:7878
    environment:
      - PUID=998
      - PGID=100
      - TZ=Europe/London
    volumes:
      - /etc/localtime:/etc/localtime:ro
      - /srv/dev-disk-by-uuid-7f81e7b6-1a05-4232-893b-f34c046b2bdb/docker/appdata/radarr:/config
      - /srv/dev-disk-by-uuid-7f81e7b6-1a05-4232-893b-f34c046b2bdb/data:/data
  sonarr:
    container_name: sonarr
    image: hotio/sonarr:latest
    restart: unless-stopped
    logging:
      driver: json-file
    network_mode: bridge
    ports:
      - 8989:8989
    environment:
      - PUID=998
      - PGID=100
      - TZ=Europe/London
    volumes:
      - /etc/localtime:/etc/localtime:ro
      - /srv/dev-disk-by-uuid-7f81e7b6-1a05-4232-893b-f34c046b2bdb/docker/appdata/sonarr:/config
      - /srv/dev-disk-by-uuid-7f81e7b6-1a05-4232-893b-f34c046b2bdb/data:/data
  bazarr:
    container_name: bazarr
    image: hotio/bazarr:latest
    restart: unless-stopped
    logging:
      driver: json-file
    network_mode: bridge
    ports:
      - 6767:6767
    environment:
      - PUID=998
      - PGID=100
      - TZ=Europe/London
    volumes:
      - /etc/localtime:/etc/localtime:ro
      - /srv/dev-disk-by-uuid-7f81e7b6-1a05-4232-893b-f34c046b2bdb/docker/appdata/bazarr:/config
      - /srv/dev-disk-by-uuid-7f81e7b6-1a05-4232-893b-f34c046b2bdb/data/media:/data/media
  nzbget:
    container_name: nzbget
    image: hotio/nzbget:latest
    restart: unless-stopped
    logging:
      driver: json-file
    network_mode: bridge
    ports:
      - 6789:6789
    environment:
      - PUID=998
      - PGID=100
      - TZ=Europe/London
    volumes:
      - /etc/localtime:/etc/localtime:ro
      - /srv/dev-disk-by-uuid-7f81e7b6-1a05-4232-893b-f34c046b2bdb/docker/appdata/nzbget:/config
      - /srv/dev-disk-by-uuid-7f81e7b6-1a05-4232-893b-f34c046b2bdb/data/usenet:/data/usenet:rw


OLD NOTES

Create a docker network which Jackett and Radarr share to talk to each other... 

sudo docker network create jackett-radarr

...then continue setting up the containers below.

FlareSolverr

FlareSolverr is a proxy server to bypass Cloudflare protection.

FlareSolverr starts a proxy server and it waits for user requests in an idle state using few resources. When some request arrives, it uses puppeteer with the stealth plugin to create a headless browser (Chrome). It opens the URL with user parameters and waits until the Cloudflare challenge is solved (or timeout). The HTML code and the cookies are sent back to the user, and those cookies can be used to bypass Cloudflare using other HTTP clients. 

Radarr > Jackett > FlareSolverr > Internet

https://github.com/FlareSolverr/FlareSolverr

https://hub.docker.com/r/flaresolverr/flaresolverr

Some indexers are protected by CloudFlare or similar services and Jackett is not able to solve the challenges. For these cases, FlareSolverr has been integrated into Jackett. This service is in charge of solving the challenges and configuring Jackett with the necessary cookies. Setting up this service is optional, most indexers don't need it.

Install FlareSolverr service using a Docker container, then configure FlareSolverr API URL in Jackett. For example: http://172.17.0.2:8191

Command line... 

docker run -d \
 --name=flaresolverr \
 -p 8191:8191 \
 -e LOG_LEVEL=info \
 --restart unless-stopped \
 ghcr.io/flaresolverr/flaresolverr:latest

Docker compose... 

---
version: "2.1"
services:
  flaresolverr:
    image: ghcr.io/flaresolverr/flaresolverr:latest
    container_name: flaresolverr
    environment:
      - PUID=1000
      - PGID=1000
      - LOG_LEVEL=${LOG_LEVEL:-info}
      - LOG_HTML=${LOG_HTML:-false}
      - CAPTCHA_SOLVER=${CAPTCHA_SOLVER:-none}
      - TZ=Europe/London
    ports:
      - "${PORT:-8191}:8191"
    restart: unless-stopped

Usage

To use it, you have to add a Proxy Server under Prowlarr > Settings > Indexers > Indexer Proxies

Testing 

curl -L -X POST 'http://localhost:8191/v1' -H 'Content-Type: application/json' --data-raw '{ "cmd": "request.get", "url":"https://www.paully.co.uk/", "maxTimeout": 60000 }'

{"status":"ok","message":"","startTimestamp":1651659265156,"endTimestamp":1651659269585,"version":"v2.2.4","solution":

Jackett

Jackett works as a proxy server: it translates queries from apps (Sonarr, SickRage, CouchPotato, Mylar, etc) into tracker-site-specific http queries, parses the html response, then sends results back to the requesting software. This allows for getting recent uploads (like RSS) and performing searches. Jackett is a single repository of maintained indexer scraping and translation logic - removing the burden from other apps.

So, this is where you build your list of web sites "with content you want" ;-)

https://fleet.linuxserver.io/image?name=linuxserver/jackett

https://docs.linuxserver.io/images/docker-jackett

https://hub.docker.com/r/linuxserver/jackett

https://github.com/Jackett/Jackett

/root/docker/stacks/docker-compose.yml 

---
version: "2.1"
services:
  jackett:
    image: ghcr.io/linuxserver/jackett
    container_name: jackett
    environment:
      - PUID=1000
      - PGID=1000
      - TZ=Europe/London
      - AUTO_UPDATE=true
    volumes:
      - ./data/config:/config
      - ./data/downloads:/downloads
    networks:
      - jackett-radarr
    ports:
      - 0.0.0.0:9117:9117
    restart: unless-stopped
networks:
  jackett-radarr:
    external: true

Prowlarr

An alternative to Jackett, and now the preferred application.

https://wiki.servarr.com/prowlarr

https://wiki.servarr.com/prowlarr/quick-start-guide

https://hub.docker.com/r/linuxserver/prowlarr

https://github.com/linuxserver/docker-prowlarr 

---
version: "2.1"
services:
  prowlarr:
    image: lscr.io/linuxserver/prowlarr:develop
    container_name: prowlarr
    environment:
      - PUID=1000
      - PGID=1000
      - TZ=Europe/London
    volumes:
      - /path/to/data:/config
    ports:
      - 9696:9696
    restart: unless-stopped

Radarr

Radarr is a movie collection manager for Usenet and BitTorrent users. It can monitor multiple RSS feeds for new movies and will interface with clients and indexers to grab, sort, and rename them. It can also be configured to automatically upgrade the quality of existing files in the library when a better quality format becomes available.

Radarr is the 'man-in-the-middle' to take lists from Jackett and pass them to Transmission to download.

Radarr is the web UI to search for "the content you want" ;-)

https://radarr.video/

https://docs.linuxserver.io/images/docker-radarr

https://github.com/linuxserver/docker-radarr

https://sasquatters.com/radarr-docker/

https://www.smarthomebeginner.com/install-radarr-using-docker/

https://trash-guides.info/Radarr/

https://discord.com/channels/264387956343570434/264388019585286144

Custom Scripts

So, you use Jackett as an Indexer of content, which answers questions from Radarr, which passes a good result to Transmission...

  1. Settings > Profiles > delete all but 'any' (and edit that to get rid of naff qualities at the bottom)
  2. Indexers > Add Indexer > Torznab > complete and TEST then SAVE
  3. Download Clients > Add Download Client > Transmission > complete and TEST and SAVE

/root/docker/stacks/docker-compose.yml 

---
version: "2.1"
services:
  radarr:
    image: ghcr.io/linuxserver/radarr
    container_name: radarr
    environment:
      - PUID=1000
      - PGID=1000
      - TZ=Europe/London
    volumes:
      - ./data/config:/config
      - ./data/downloads:/downloads
      - ./data/torrents:/torrents
    networks:
      - jackett-radarr
    ports:
      - 0.0.0.0:7878:7878
    restart: unless-stopped
networks:
  jackett-radarr:
    external: true

Sonarr

Sonarr (formerly NZBdrone) is a PVR for usenet and bittorrent users. It can monitor multiple RSS feeds for new episodes of your favorite shows and will grab, sort and rename them. It can also be configured to automatically upgrade the quality of files already downloaded when a better quality format becomes available.

https://sonarr.tv/

https://docs.linuxserver.io/images/docker-sonarr

Docker Compose using a WireGuard VPN container for internet ... 

---
version: "2.1"
services:
  sonarr:
    image: ghcr.io/linuxserver/sonarr
    container_name: sonarr
    network_mode: container:wireguard
    environment:
      - PUID=1000
      - PGID=1000
      - TZ=Europe/London
    volumes:
      - ./data/config:/config
      - ./data/downloads:/downloads
      - ./data/torrents:/torrents
    restart: "no"

UPDATE: 16 FEBRUARY 2023 / latest image is based on Ubuntu Jammy and not Alpine, which will cause a problem if your Docker version is below 20.10.10

https://docs.linuxserver.io/faq#jammy

To fix this, you can either upgrade your Docker (https://docs.docker.com/engine/install/ubuntu/#install-using-the-convenience-script) or add the following lines to your docker-compose.yml file ... 

security_opt:
  - seccomp=unconfined

Bazarr

https://www.bazarr.media/

Bazarr is a companion application to Sonarr and Radarr that manages and downloads subtitles based on your requirements.

Docker compose file ... 

---
version: "2.1"
services:
  bazarr:
    image: lscr.io/linuxserver/bazarr:latest
    container_name: bazarr
    environment:
      - PUID=1000
      - PGID=1000
      - TZ=Europe/London
    volumes:
      - /path/to/bazarr/config:/config
      - /path/to/movies:/movies #optional
      - /path/to/tv:/tv #optional
    ports:
      - 6767:6767
    restart: unless-stopped

NZBGet

NZBGet is a usenet downloader.

You will require the following 3 things at a basic level before you are able to use usenet:-

https://nzbget.net

https://hub.docker.com/r/linuxserver/nzbget

https://www.cogipas.com/nzbget-complete-how-to-guide/

The Web GUI can be found at <your-ip>:6789 and the default login details (change ASAP) are... 

username: nzbget
password: tegbzn6789

Docker Compose file... 

---
version: "2.1"
services:
  nzbget:
    image: lscr.io/linuxserver/nzbget:latest
    container_name: nzbget
    environment:
      - PUID=1000
      - PGID=1000
      - TZ=Europe/London
      - NZBGET_USER=nzbget
      - NZBGET_PASS=tegbzn6789
    volumes:
      - /path/to/data:/config
      - /path/to/downloads:/downloads
    ports:
      - 6789:6789
    restart: unless-stopped

Tdarr

Tdarr is a popular conditional transcoding application for processing large (or small) media libraries. The application comes in the form of a click-to-run web-app, which you run on your own device and access through a web browser.

Tdarr uses two popular transcoding applications under the hood: FFmpeg and HandBrake (which itself is built on top of FFmpeg).

Tdarr works in a distributed manner where you can use multiple devices to process your library together. It does this using 'Tdarr Nodes' which connect with a central server and pick up tasks so you can put all your spare devices to use.

Each Node can run multiple 'Tdarr Workers' in parallel to maximize the hardware usage % on that Node. For example, a single FFmpeg worker running on a 64 core CPU may only hit ~30% utilization. Running multiple Workers in parallel allows the CPU to hit 100% utilization, allowing you to process your library more quickly.

Home Page

Documentation

Using AMD GPU for Transcoding

Readarr

https://academy.pointtosource.com/containers/ebooks-calibre-readarr/

Unpackerr

Extracts downloads for Radarr, Sonarr, Lidarr, Readarr, and/or a Watch folder - Deletes extracted files after import.

https://github.com/Unpackerr/unpackerr

Docker Compose - https://github.com/Unpackerr/unpackerr/blob/main/examples/docker-compose.yml

Servarr (All-In-One)

This is a docker compose file which starts all the containers in the correct order. This is achieved by making the next service dependant on the previous service. 

# tdarr
# unpackerr
# readarr
# bazarr
# sonarr
# radarr
# sabnzbd
# prowlarr
# flaresolverr
# wireguard

services:

  tdarr:
    depends_on:
      - unpackerr
    container_name: tdarr
    image: ghcr.io/haveagitgat/tdarr:latest
    restart: unless-stopped
    network_mode: bridge
    ports:
      - 8265:8265 # webUI port
      - 8266:8266 # server port
    environment:
      - TZ=Europe/London
      - PUID=1000
      - PGID=1000
      - UMASK_SET=002
      - serverIP=0.0.0.0
      - serverPort=8266
      - webUIPort=8265
      - internalNode=true
      - inContainer=true
      - ffmpegVersion=6
      - nodeName=MyInternalNode
    volumes:
      - ./data/tdarr/server:/app/server
      - ./data/tdarr/configs:/app/configs
      - ./data/tdarr/logs:/app/logs
      - ./data/tdarr/transcode_cache:/temp
      - /home/user/data/media/movies:/input
      - /home/user/data/tdarr/movies/output:/output
      - /home/user/Emby:/Emby

  unpackerr:
    depends_on:
      - readarr
    image: golift/unpackerr
    container_name: unpackerr
    network_mode: container:wireguard
    volumes:
      # You need at least this one volume mapped so Unapckerr can find your files to extract.
      # Make sure this matches your Starr apps; the folder mount (/downloads or /data) should be identical.
      - /etc/localtime:/etc/localtime:ro
      - ./data/unpackerr/config:/config
      - /home/user/data:/data
    restart: "no"
    # Get the user:group correct so unpackerr can read and write to your files.
    user: 1000:1000
    # What you see below are defaults for this compose. You only need to modify things specific to your environment.
    environment:
      - TZ=Europe/London
      # General config
      - UN_DEBUG=false
      - UN_LOG_FILE=
      - UN_LOG_FILES=10
      - UN_LOG_FILE_MB=10
      - UN_INTERVAL=2m
      - UN_START_DELAY=1m
      - UN_RETRY_DELAY=5m
      - UN_MAX_RETRIES=3
      - UN_PARALLEL=1
      - UN_FILE_MODE=0644
      - UN_DIR_MODE=0755
      # Radarr Config
      - UN_RADARR_0_URL=http://172.21.0.2:7878
      - UN_RADARR_0_API_KEY=xxxxxxxxxxxxxxxxxxxxxxxxxx
      - UN_RADARR_0_PATHS_0=/downloads
      - UN_RADARR_0_PROTOCOLS=torrent
      - UN_RADARR_0_TIMEOUT=10s
      - UN_RADARR_0_DELETE_ORIG=false
      - UN_RADARR_0_DELETE_DELAY=5m
      # Sonarr Config
      - UN_SONARR_0_URL=http://172.21.0.2:8989
      - UN_SONARR_0_API_KEY=xxxxxxxxxxxxx
      - UN_SONARR_0_PATHS_0=/downloads
      - UN_SONARR_0_PROTOCOLS=torrent
      - UN_SONARR_0_TIMEOUT=10s
      - UN_SONARR_0_DELETE_ORIG=false
      - UN_SONARR_0_DELETE_DELAY=5m
      # Readarr Config
      - UN_READARR_0_URL=http://172.21.0.2:8787
      - UN_READARR_0_API_KEY=xxxxxxxxxxxxxxxxxxx
      - UN_READARR_0_PATHS_0=/downloads
      - UN_READARR_0_PROTOCOLS=torrent
      - UN_READARR_0_TIMEOUT=10s
      - UN_READARR_0_DELETE_ORIG=false
      - UN_READARR_0_DELETE_DELAY=5m
    security_opt:
      - no-new-privileges:true

  readarr:
    depends_on:
      - bazarr
    image: lscr.io/linuxserver/readarr:develop
    container_name: readarr
    network_mode: container:wireguard
    environment:
      - PUID=1000
      - PGID=1000
      - TZ=Europe/London
    volumes:
      - /etc/localtime:/etc/localtime:ro
      - ./data/readarr/config:/config
      - /home/user/data:/data
    # ports:
      # - 8787:8787
    restart: "no"

  bazarr:
    depends_on:
      - sonarr
    image: lscr.io/linuxserver/bazarr:latest
    container_name: bazarr
    network_mode: container:wireguard
    environment:
      - PUID=1000
      - PGID=1000
      - TZ=Europe/London
    volumes:
      - /etc/localtime:/etc/localtime:ro
      - ./data/bazarr/config:/config
      - /home/user/data:/data
    # ports:
      # - 6767:6767
    restart: "no"

  sonarr:
    depends_on:
      - radarr
    image: lscr.io/linuxserver/sonarr:latest
    container_name: sonarr
    network_mode: container:wireguard
    environment:
      - PUID=1000
      - PGID=1000
      - TZ=Europe/London
    volumes:
      - /etc/localtime:/etc/localtime:ro
      - ./data/sonarr/config:/config
      - /home/user/data:/data
    restart: "no"
    security_opt:
      - seccomp=unconfined

  radarr:
    depends_on:
      - sabnzbd
    image: lscr.io/linuxserver/radarr:latest
    container_name: radarr
    network_mode: container:wireguard
    environment:
      - PUID=1000
      - PGID=1000
      - TZ=Europe/London
    volumes:
      - /etc/localtime:/etc/localtime:ro
      - ./data/radarr/config:/config
      - /home/user/data:/data
    restart: "no"

  sabnzbd:
    depends_on:
      - prowlarr
    image: lscr.io/linuxserver/sabnzbd:latest
    container_name: sabnzbd
    network_mode: container:wireguard
    environment:
      - PUID=1000
      - PGID=1000
      - TZ=Europe/London
    volumes:
      - /etc/localtime:/etc/localtime:ro
      - ./data/sabnzbd/config:/config
      - /home/user/data:/data
    restart: "no"

  prowlarr:
    depends_on:
      - flaresolverr
    image: lscr.io/linuxserver/prowlarr:latest
    container_name: prowlarr
    network_mode: container:wireguard
    environment:
      - PUID=1000
      - PGID=1000
      - TZ=Europe/London
    volumes:
      - /etc/localtime:/etc/localtime:ro
      - ./data/prowlarr/config:/config
    restart: "no"

  flaresolverr:
    depends_on:
      - wireguard
    image: ghcr.io/flaresolverr/flaresolverr:latest
    container_name: flaresolverr
    network_mode: container:wireguard
    environment:
      - LOG_LEVEL=${LOG_LEVEL:-info}
      - LOG_HTML=${LOG_HTML:-false}
      - CAPTCHA_SOLVER=${CAPTCHA_SOLVER:-none}
      - TZ=Europe/London
    restart: "no"

  wireguard:
    image: ghcr.io/linuxserver/wireguard
    container_name: wireguard
    cap_add:
      - NET_ADMIN
      - SYS_MODULE
    environment:
      - PUID=1000
      - PGID=1000
      - TZ=Europe/London
      - SERVERURL=wireguard.domain.uk
      - SERVERPORT=51820
      - PEERDNS=auto
      - INTERNAL_SUBNET=10.6.0.0
      - ALLOWEDIPS=0.0.0.0/0
    volumes:
      - ./data/wireguard/config:/config
      - /lib/modules:/lib/modules
    ports:
      - 51820:51820/udp
      - 8191:8191/tcp # FlareSolverr
      - 9117:9117/tcp # Jackett
      - 9696:9696/tcp # Prowlarr
      - 9876:6789/tcp # NZBGet
      - 7878:7878/tcp # Radarr
      - 8989:8989/tcp # Sonarr
      - 6767:6767/tcp # Bazarr
      - 8787:8787/tcp # Readarr
    sysctls:
      - net.ipv4.conf.all.src_valid_mark=1
    restart: "no"
    networks:
      - wireguardvpn

networks:
  wireguardvpn:
    external: true

Calibre

eBook management and automation using Calibre, COPS or Calibre-Web, and Readarr.

Auto format conversion.

https://academy.pointtosource.com/containers/ebooks-calibre-readarr/

YouTube-DL

https://registry.hub.docker.com/search?q=youtube&sort=updated_at&order=desc

https://registry.hub.docker.com/r/mikenye/youtube-dl#quick-start

Nagios

Work in progress.

This is an old version of Nagios in the container image, so will look for a newer one.

/root/docker/stacks/nagios/docker-compose.yml 

version: '3'
services:
  nagios:
    image: jasonrivers/nagios
    container_name: nagios
    restart: unless-stopped
    ports:
      - 8181:80
    # volumes:
      # - ./data/etc/:/opt/nagios/etc/
    environment:
      - PUID=999
      - PGID=1000
      - TZ=Europe/London
      - NAGIOS_TIMEZONE=Europe/London

Start it with no volume mounts, then copy the etc directory to your host... 

cd /root/docker/stacks/nagios/
docker cp nagios:/opt/nagios/etc data/
chown -R 999:1000 data/

...then uncomment the # lines in the docker-compose file and restart the container.

Credentials

The default credentials for the web interface is nagiosadmin / nagios

To change the password, generate a new one by logging in to the container and running 'htpasswd'... 

docker exec -it nagios bash
htpasswd -n nagiosadmin
(copy the output)

...then editing the /opt/nagios/etc/htpasswd.users file and refreshing the admin web page.

https://github.com/ethnchao/docker-nagios

http://www.kraftinfosec.com/running-nagios-in-docker/

https://github.com/JasonRivers/Docker-Nagios

Tandoor Recipe Manager

The recipe manager that allows you to manage your ever growing collection of digital recipes.

https://docs.tandoor.dev/install/docker/

https://www.youtube.com/watch?v=7-nb3muJxI0

/root/docker/stacks/tandoor/docker-compose.yml 

version: "3"
services:
  db_recipes:
    container_name: tandoor_db
    restart: always
    image: postgres:11-alpine
    volumes:
      - ./data/postgresql:/var/lib/postgresql/data
    env_file:
      - ./.env
  web_recipes:
    container_name: tandoor_web
    image: vabene1111/recipes
    restart: always
    env_file:
      - ./.env
    volumes:
      - ./data/mediafiles:/opt/recipes/mediafiles
      - ./data/staticfiles:/opt/recipes/staticfiles
      - nginx_config:/opt/recipes/nginx/conf.d
    depends_on:
      - db_recipes
  nginx_recipes:
    container_name: tandoor_nginx
    image: nginx:mainline-alpine
    restart: always
    ports:
      - 80
    env_file:
      - ./.env
    depends_on:
      - web_recipes
    volumes:
      - ./data/mediafiles:/media
      - ./data/staticfiles:/static
      - nginx_config:/etc/nginx/conf.d:ro
volumes:
  nginx_config:
networks:
  default:
    external:
      name: nginx-proxy-manager

NextCloud

IMPORTANT


If you are receiving errors about PHP or issues with nginx or certificates:

  1. Switch your image to lscr.io/linuxserver/nextcloud:24.0.6-ls204 and start the container
  2. Execute docker exec -it nextcloud updater.phar repeatedly until there are no more updates (as of writing, Nextcloud 25 is the latest version)
  3. Switch your image to lscr.io/linuxserver/nextcloud (latest, no tag) and start the container
  4. Execute docker exec -it nextcloud mv /config/nginx/site-confs/default.conf /config/nginx/site-confs/default.conf.bak
  5. Execute docker exec -it nextcloud mv /config/nginx/nginx.conf /config/nginx/nginx.conf.bak
  6. Execute docker logs nextcloud and check for any other outdated configs, rename them with a .bak extension (like above)
  7. Restart the container
  8. Nextcloud should now be in a working state



Nextcloud gives you access to all your files wherever you are.

Create your container folders... 

mkdir -p /root/docker/stacks/nextcloud/data/{config,files}
chown -R 1000:1000 /root/docker/stacks/nextcloud/data/

/root/docker/stacks/nextcloud/docker-compose.yml 

version: "2.1"
services:
  nextcloud:
    image: ghcr.io/linuxserver/nextcloud
    container_name: nextcloud
    environment:
      - PUID=1000
      - PGID=1000
      - TZ=Europe/London
    volumes:
      - ./data/config:/config
      - ./data/files:/data
    ports:
      - 4443:443
    restart: unless-stopped

(I have changed the default port it listens on the local machine to 4443 but if you don't need any ports open then change the lines to:- 

    expose:
      - 443

...then use Nginx Proxy Manager to direct traffic to your NextCloud installation)

Now visit https://ip.address.of.host:4443

...and on that setup page, _untick_ the option for "Install recommended apps" which does not install Calendar, Contacts, Mail, Chat, etc.

Enjoy.

https://hub.docker.com/r/linuxserver/nextcloud

Command Line Admin OCC

https://docs.nextcloud.com/server/latest/admin_manual/configuration_server/occ_command.html

List Users

How do I list the users in NextCloud Docker? 

sudo docker exec -it nextcloud occ user:list

Reset Admin Password

How do you reset the admin user password in NextCloud Docker? 

sudo docker exec -it nextcloud occ user:resetpassword admin

https://docs.nextcloud.com/server/latest/admin_manual/configuration_user/reset_admin_password.html

Project Send

Self-hosted file sharing... small, simple, secure.

https://www.projectsend.org

https://docs.linuxserver.io/images/docker-projectsend

https://github.com/linuxserver/docker-projectsend

What they don't tell you in the docs is that you need a database backend - which is not in the docker compose file.

So, we just add a MariaDB database container to the stack!

Create your subdomain A record in DNS... 

cli53 rrcreate domain.uk 'send 300 A 123.45.678.90'

Create your Proxy Host in Ngnix Proxy Manager with an SSL... 

https://send.domain.uk

Create directories on the server for the Docker container files... 

sudo -i
mkdir -p /root/docker/stacks/projectsend/data/{config,db,files}
chown -R 1000:1000 /root/docker/stacks/projectsend/data/files

/root/docker/stacks/projectsend/docker-compose.yml 

version: "2.1"
services:
  projectsend:
    image: ghcr.io/linuxserver/projectsend
    container_name: projectsend
    environment:
      - PUID=1000
      - PGID=1000
      - TZ=Europe/London
      - MAX_UPLOAD=100
    volumes:
      - ./data/config:/config
      - ./data/files:/data
      - /etc/timezone:/etc/timezone:ro
    expose:
      - 80
    restart: unless-stopped
  projectsend-db:
    image: mariadb
    container_name: projectsend-db
    environment:
      TZ: Europe/London
      MYSQL_ROOT_PASSWORD: projectsend
      MYSQL_DATABASE: projectsend
      MYSQL_USER: projectsend
      MYSQL_PASSWORD: projectsend
    volumes:
      - ./data/db:/var/lib/mysql
      - /etc/timezone:/etc/timezone:ro
    restart: unless-stopped
networks:
  default:
    external:
      name: nginx-proxy-manager

Go to the secure web site URL and complete the installation, using the Docker container name for the 'Database hostname'. 

projectsend-db

Then, log in with your Admin username and password...

  1. Create Group 'Public' which is public.
  2. Create Group 'Customers' which is not public.
  3. Create Client 'Customer Name' which is assigned to the 'Customers' group.
  4. Upload some files and test both the Public and Client links.

Enjoy.

Troubleshooting

Change the Site URL

If you move host or domain name, you can log in to the DB container and change the 'base_uri'... 

docker exec -it projectsend-db bash
mysql -u root -p projectsend

MariaDB [projectsend]> 
MariaDB [projectsend]> select * from tbl_options where name = 'base_uri';
+----+----------+-------------------------+
| id | name     | value                   |
+----+----------+-------------------------+
|  1 | base_uri | https://send.domain.uk/ |
+----+----------+-------------------------+

MediaWiki

Installation

Create the docker compose file and use default volume. Go to your browser at http://localhost:8080 and finish setup. Download LocalSettings.php file and copy to it to the container filesystem, then copy the whole folder to the host filsystem... 

docker cp LocalSettings.php mediawiki:/var/www/html/
docker cp mediawiki:/var/www/html /root/docker/stacks/mediawiki/data/
chown -R www-data:www-data data/html
chmod o-w data/html
docker-compose down
(then edit your docker-compose.yml file so that local folders are used)
docker-compose up -d

Now, all the files are on your docker folder, ready to easily backup :-) 

mediawiki
`-- data
    |-- db
    `-- html

~/docker/mediawiki/docker-compose.yml 

version: '3'
services:
  mediawiki:
    image: mediawiki
    container_name: mediawiki
    restart: always
    ports:
      - 8080:80
    links:
      - database
    volumes:
      #- ./data/html:/var/www/html     <-- #2
      #- /var/www/html/images      <-- #1
    environment:
      - PUID=33
      - PGID=33
      - TZ=Europe/London
  database:
    image: mariadb
    container_name: mediawiki_db
    restart: always
    environment:
      MYSQL_DATABASE: my_wiki
      MYSQL_USER: wikiuser
      MYSQL_PASSWORD: example
      MYSQL_RANDOM_ROOT_PASSWORD: 'yes'
    volumes:
      - ./data/db:/var/lib/mysql

https://hub.docker.com/_/mediawiki

Tweaks

Change default skin to mobile responsive modern one... 

wfLoadSkin( 'Timeless' );
$wgDefaultSkin = "timeless";

Enable the new editing toolbar... 

wfLoadExtension( 'WikiEditor' );

Make the URL shorter... 

$wgScriptPath = "";
$wgScriptExtension = ".php";
$wgArticlePath = "/wiki/$1";
$wgUsePathInfo = true;

File uploads...

LocalSettings.php 

$wgEnableUploads = true;

https://www.mediawiki.org/wiki/Manual:Configuring_file_uploads

https://kindalame.com/2020/11/25/self-hosting-mediawiki-with-docker/

Importing

Pages

OLD SERVER

Generate the page dump in XML format... 

docker exec -it mediawiki bash
php maintenance/dumpBackup.php --current > pages.xml
exit

NEW SERVER

Import the pages... 

cp pages.xml ./data/html/
docker exec -it mediawiki bash
php maintenance/importDump.php < pages.xml
php maintenance/update.php
php maintenance/rebuildall.php
exit

https://www.hostknox.com/tutorials/mediawiki/pages/export-and-import#import-pages-via-ssh

Images

OLD SERVER

Generate the image dumps using dumpUploads.php, which creates a txt list of all image filenames in use... 

mkdir /tmp/workingBackupMediaFiles
php maintenance/dumpUploads.php \
   | sed 's~mwstore://local-backend/local-public~./images~' \
   | xargs cp -t /tmp/workingBackupMediaFiles
zip -r ~/Mediafiles.zip /tmp/workingBackupMediaFiles
rm -r /tmp/workingBackupMediaFiles

NEW SERVER

Unzip the files to your container filsystem... 

cd /root/docker/stacks/mediawiki
unzip Mediafiles.zip -d ./data/html/

Import the Images... 

docker exec -it mediawiki bash
php maintenance/importImages.php tmp/workingBackupMediaFiles
php maintenance/update.php
php maintenance/rebuildall.php
exit

https://stackoverflow.com/questions/1002258/exporting-and-importing-images-in-mediawiki

Kuma

A self-hosted monitoring tool like Uptime Robot and not as complicated as Nagios.

https://hub.docker.com/r/louislam/uptime-kuma

https://github.com/louislam/uptime-kuma

https://youtu.be/dIVf1nhT0mI

Kasm Containerized Apps and Desktops

Streaming containerized apps and desktops to end-users. The Workspaces platform provides enterprise-class orchestration, data loss prevention, and web streaming technology to enable the delivery of containerized workloads to your browser.

https://kasmweb.com

https://hub.docker.com/u/kasmweb

https://www.youtube.com/channel/UCgpv4MLH8diVlIiakCBu8eQ

SFTP

https://hub.docker.com/r/atmoz/sftp

Wordle

https://github.com/cwackerfuss/react-wordle

Ombi

Ombi allows you to host your own Emby Request and user management system. If you are sharing your Emby server with other users, allow them to request new content using an easy to manage interface! Manage all your requests for Movies and TV with ease, leave notes for the user and get notification when a user requests something. Allow your users to post issues against their requests so you know there is a problem with the audio etc. Even automatically send them weekly newsletters of new content that has been added to your Emby server :-)

https://hub.docker.com/r/linuxserver/ombi

Emby

https://fleet.linuxserver.io/image?name=linuxserver/emby

https://hub.docker.com/r/linuxserver/emby 

---
version: "2.1"
services:
  emby:
    image: lscr.io/linuxserver/emby
    container_name: emby
    environment:
      - PUID=998
      - PGID=100
      - TZ=Europe/London
    volumes:
      - /srv/dev-disk-by-uuid-7f81e7b6-1a05-4232-893b-f34c046b2bdb/Emby/Config:/config
      - /srv/dev-disk-by-uuid-7f81e7b6-1a05-4232-893b-f34c046b2bdb/Emby/TV:/data/tvshows
      - /srv/dev-disk-by-uuid-7f81e7b6-1a05-4232-893b-f34c046b2bdb/Emby/Movies:/data/movies
    ports:
      - 8096:8096
    restart: unless-stopped

Tdarr

Tdarr is a conditional based transcoding application for automating media library transcoding and remux management which uses cross-platform Tdarr Nodes which work together with Tdarr Server to process your files.

https://docs.tdarr.io/docs/installation/docker/run-compose 

version: "3.4"

services:

# server
  tdarr:
    container_name: tdarr
    image: ghcr.io/haveagitgat/tdarr:latest
    restart: unless-stopped
    network_mode: bridge
    ports:
      - 8265:8265 # webUI port
      - 8266:8266 # server port
      - 8267:8267 # Internal node port
      - 8268:8268 # Example extra node port
    environment:
      - TZ=Europe/London
      - PUID=${PUID}
      - PGID=${PGID}
      - UMASK_SET=002
      - serverIP=0.0.0.0
      - serverPort=8266
      - webUIPort=8265
      - internalNode=true
      - nodeID=MyInternalNode
    volumes:
      - /docker/tdarr/server:/app/server
      - /docker/tdarr/configs:/app/configs
      - /docker/tdarr/logs:/app/logs
      - /media:/media
      - /transcode_cache:/temp

# node example
  tdarr-node:
    container_name: tdarr-node
    image: ghcr.io/haveagitgat/tdarr_node:latest
    restart: unless-stopped
    network_mode: service:tdarr
    environment:
      - TZ=Europe/London
      - PUID=${PUID}
      - PGID=${PGID}
      - UMASK_SET=002
      - nodeID=MainNode
      - serverIP=0.0.0.0
      - serverPort=8266
    volumes:
      - /docker/tdarr/configs:/app/configs
      - /docker/tdarr/logs:/app/logs
      - /media:/media
      - /transcode_cache:/temp

Unifi Controller

https://github.com/linuxserver/docker-unifi-controller

If you want to fix a particular version of the controller then check the 'releases' at github and adjust your docker-compose.yml file accordingly.

e.g. linuxserver/unifi-controller:5.14.23-ls76 

version: "2.1"
services:
  unifi-controller:
    # image: linuxserver/unifi-controller:latest
    image: linuxserver/unifi-controller:5.14.23-ls76
    container_name: unifi-controller
    environment:
      - PUID=998
      - PGID=100
      # - MEM_LIMIT=1024 #optional
      # - MEM_STARTUP=1024 #optional
    volumes:
      - <path to data>:/config
    ports:
      - 8443:8443
      - 3478:3478/udp
      - 10001:10001/udp
      - 8080:8080
      - 1900:1900/udp #optional
      - 8843:8843 #optional
      - 8880:8880 #optional
      - 6789:6789 #optional
      - 5514:5514/udp #optional
    restart: unless-stopped

rPort

Rport helps you to manage your remote servers without the hassle of VPNs, chained SSH connections, jump-hosts, or the use of commercial tools like TeamViewer and its clones.

Rport acts as server and client establishing permanent or on-demand secure tunnels to devices inside protected intranets behind a firewall.

All operating systems provide secure and well-established mechanisms for remote management, being SSH and Remote Desktop the most widely used. Rport makes them accessible easily and securely.

https://rport.io/en/features

https://oss.rport.io/

https://github.com/cloudradar-monitoring/rport

https://hub.docker.com/r/acwhiteglint/rport

Paperless

Paperless is an application that indexes your scanned documents and allows you to easily search for documents and store metadata alongside your documents.

https://github.com/jonaswinkler/paperless-ng

https://paperless-ng.readthedocs.io/en/latest/setup.html#installation

Ansible

So, this is not installing Docker using Ansible... this is installing (or running) Ansible using Docker :-)

https://iceburn.medium.com/run-ansible-with-docker-9eb27d75285b

Pi Alert

WIFI / LAN intruder detector. Scan the devices connected to your WIFI / LAN and alert you the connection of unknown devices. It also warns the disconnection of "always connected" devices.

https://github.com/pucherot/Pi.Alert

Roundcube Webmail

https://hub.docker.com/search?q=roundcube

https://hub.docker.com/r/roundcubeorg/roundcubemail

https://github.com/roundcube/roundcubemail/wiki/Configuration 

docker run -e ROUNDCUBEMAIL_DEFAULT_HOST=ssl://mail.domain.co.uk -e ROUNDCUBEMAIL_DEFAULT_PORT=993 -e ROUNDCUBEMAIL_SMTP_SERVER=tls://mail.domain.co.uk -e ROUNDCUBEMAIL_SMTP_PORT=587 -e ROUNDCUBEMAIL_PLUGINS=archive,zipdownload,managesieve,mobile -p 8000:80 -d roundcube/roundcubemail

FAST SpeedTest

docker run -it --rm --name fast-cli mschirrmeister/fast-cli:latest

https://hub.docker.com/r/mschirrmeister/fast-cli

https://github.com/sindresorhus/fast-cli

Crypto Mining

XMRig

https://hub.docker.com/r/minerboy/xmrig

Go to the XMRig Wizard and generate config like this ... 

{
    "autosave": true,
    "cpu": true,
    "opencl": false,
    "cuda": false,
    "pools": [
        {
            "coin": "monero",
            "algo": "rx/0",
            "url": "stratum+tcp://randomxmonero.auto.nicehash.com:9200",
            "user": "NHbLd5exQeCGGyWnopVoLHLbzexKN5z8iq7p.NAS",
            "pass": "x",
            "tls": false,
            "keepalive": true,
            "nicehash": true
        }
    ]
}

Then save that file as config.json and run the following commands to start mining ... 

sudo -i
docker pull minerboy/xmrig
docker run --cap-add=SYS_ADMIN --cap-add=SYS_RAWIO --device=/dev/cpu --device=/dev/mem -v /lib/modules:/lib/modules -v /full/path/to/config.json:/etc/xmrig/config.json minerboy/xmrig:latest --cpu-max-threads-hint 50 --threads 2

Here is the same command as a lovely docker compose file ... 

version: '3.3'
services:
  xmrig:
      image: 'minerboy/xmrig:latest'
      container_name: xmrig
      devices:
        - /dev/cpu
        - /dev/mem
      volumes:
        - '/lib/modules:/lib/modules'
        - '/root/docker/stacks/xmrig/config.json:/etc/xmrig/config.json'
      command:
        - --cpu-max-threads-hint 50
        - --threads 2
      restart: "no"

BTCPay Server

https://docs.btcpayserver.org/Docker/ 

sudo -i
mkdir -p /root/docker
git clone https://github.com/btcpayserver/btcpayserver-docker
cd btcpayserver-docker
export BTCPAY_HOST="btcpay.mydomain.com"
export NBITCOIN_NETWORK="mainnet"
export BTCPAYGEN_CRYPTO1="btc"
export BTCPAYGEN_ADDITIONAL_FRAGMENTS="opt-save-storage-xs"
export BTCPAYGEN_REVERSEPROXY="nginx"
export BTCPAYGEN_LIGHTNING="clightning"
export BTCPAY_ENABLE_SSH=false
export REVERSEPROXY_HTTP_PORT=8080
export REVERSEPROXY_HTTPS_PORT=4443
. ./btcpay-setup.sh -i

Then, after 5 minutes, you can switch to the FastSync of the blockchain ... 

cd $BTCPAY_BASE_DIRECTORY/btcpayserver-docker
./btcpay-down.sh
cd $BTCPAY_BASE_DIRECTORY/btcpayserver-docker/contrib/FastSync
./load-utxo-set.sh
docker volume rm generated_bitcoin_wallet_datadir
cd $BTCPAY_BASE_DIRECTORY/btcpayserver-docker
./btcpay-up.sh
docker logs --tail -100 btcpayserver_bitcoind

Using Caddy Proxy Separately

Yes! I finally worked out how to do this!

You can use BTCPay Server with an existing Proxy like NginX or Caddy.

The trick is to tell the Docker BTCPay Server to disable its' own proxy offering and also disable the SSL termination.

These are the magic list of commands to use - setting the environment variables for the setup script ... 

BTCPAYGEN_REVERSEPROXY="none"
NOREVERSEPROXY_HTTP_PORT="3003"
BTCPAYGEN_EXCLUDE_FRAGMENTS="nginx-https"

So, your full command list is as follows ... 

sudo -i
mkdir -p /root/docker
git clone https://github.com/btcpayserver/btcpayserver-docker
cd btcpayserver-docker
export BTCPAY_HOST="btcpay.mydomain.com"
export NBITCOIN_NETWORK="mainnet"
export BTCPAYGEN_CRYPTO1="btc"
export BTCPAYGEN_ADDITIONAL_FRAGMENTS="opt-save-storage-xs"
export BTCPAYGEN_LIGHTNING="clightning"
export BTCPAY_ENABLE_SSH=false
export BTCPAYGEN_REVERSEPROXY="none"
export NOREVERSEPROXY_HTTP_PORT="3003"
export BTCPAYGEN_EXCLUDE_FRAGMENTS="nginx-https"
. ./btcpay-setup.sh -i

This should now give you these dockers with the ports used ... 

btcpayserver_bitcoind             8332-8333/tcp, 18332-18333/tcp, 18443-18444/tcp, 39388/tcp, 43782/tcp
btcpayserver_clightning_bitcoin   0.0.0.0:9735->9735/tcp, :::9735->9735/tcp, 9835/tcp, 0.0.0.0:32768->3010/tcp, [::]:32768->3010/tcp
generated-bitcoin_rtl-1           3000/tcp
generated_btcpayserver_1          0.0.0.0:3003->49392/tcp, [::]:3003->49392/tcp
generated_nbxplorer_1             32838/tcp
generated_postgres_1              5432/tcp
tor                               9050-9051/tcp
tor-gen

You will notice the crucial port 49392/tcp running in the btcpayserver container.

THIS IS WHAT YOU PROXY TO :)

You do not proxy to the host's 3003 port (as often mentioned in old instructions on web pages!)

This is the Caddyfile which uses static Let's Encrypt files I generated with certbot. You will also notice I have left my incorrect reverse_proxy line in for reference :)

So, Caddy proxies requests to the container name and container port ... 

btcpay.mydomain.com:443 {
       tls /ssl/certs/fullchain.pem /ssl/certs/key.pem
       #reverse_proxy 127.0.0.1:3003
       reverse_proxy generated_btcpayserver_1:49392
}

This is the Caddy docker-compose and you notice it's using the BTCPay Server 'generated_default' network - which is REALLY important! 

services:
  caddy:
    image: caddy:alpine
    container_name: caddy
    restart: unless-stopped
    ports:
      - 80:80
      - 443:443
      - 443:443/udp
    networks:
      - generated_default
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock
      - ./Caddyfile:/etc/caddy/Caddyfile
      - ./data:/data
      - ./config:/config
      - ./fullchain.pem:/ssl/certs/fullchain.pem:ro
      - ./key.pem:/ssl/certs/key.pem:ro
    environment:
      - TZ="Europe/London"
networks:
  generated_default:
    external: true

... and that's it! Enjoy.

Umbrel

# ssh into your umbrel 
ssh umbrel@umbrel.local    (or the IP of your Umbrel server)      
# Password is the same as Umbrel Web UI

# Edit the .env.app_proxy file
nano ~/umbrel/app-data/btcpay-server/.env.app_proxy

# Enter the following into the file
PROXY_TRUST_UPSTREAM=true

# Save using:
Control + X, then: Y, then: <enter>

# Restart BTCPayServer
~/umbrel/scripts/app restart btcpay-server

# You can also right-click on the GUI icon for BTCPay Server and choose 'Restart'

https://orange.surf/public-btcpay-umbrel-tailscale/

AI

Self-Hosted AI That's Actually Useful - Techno Tim

  • Open WebUI
  • Olama
  • SearXNG
  • Stable Diffusion + ComfyUI
  • Prompt Generator

Microsoft Windows

1) Watch ...

https://www.youtube.com/watch?v=3h1KtrL3CYQ

2) Install ...

https://github.com/dockur/windows

3) Tweak ...

https://christitus.com/windows-utility-improved/

https://www.youtube.com/watch?v=5_AaHXrelTE

Locust Web Site Testing Tool

Locust is an open source performance/load testing tool for HTTP and other protocols. Its developer-friendly approach lets you define your tests in regular Python code.

Locust tests can be run from command line or using its web-based UI. Throughput, response times and errors can be viewed in real time and/or exported for later analysis. 

services:
  master:
    image: locustio/locust
    container_name: locust_master
    restart: 'no'
    ports:
     - "8089:8089"
    volumes:
      - ./:/mnt/locust
    command: -f /mnt/locust/locustfile.py --master -H http://master:8089
  worker:
    image: locustio/locust
    container_name: locust_worker
    restart: 'no'
    volumes:
      - ./:/mnt/locust
    command: -f /mnt/locust/locustfile.py --worker --master-host master

Then connect to your server over SSH with this config to interact with the web gui ... 

Host    caddy-test-locust
        User ubuntu
        Port 22
        HostName myserverhostname
        LocalForward 9999 127.0.0.1:8089
        ProxyJump mysshbastionhost

Raspberry Pi

https://github.com/ptrsr/pi-ci

Swarm

Docker Swarm is a container orchestrator and a clustering and scheduling tool offered by Docker as a application mode, for managing multiples nodes, and it’s deployments like a whole single system. Swarm mode also exists natively for Docker Engine, the layer between the OS and container images. We generally use the Docker CLI to create a swarm, deploy application services to a swarm, and manage swarm behaviour. The activities of the cluster are controlled by a swarm manager, and machines that have joined the cluster are referred to as nodes. Docker Swarm lets you connect containers to multiple hosts similar to Kubernetes.

The docker swarm function recognises two different types of nodes, each with a different role within the docker swarm ecosystem:

Manager nodes: These contain the Swarm Manager, the process in charge of handling the commands in the Swarm mode and reconciling the desired state with the actual cluster state.

Worker nodes: In a docker swarm with numerous hosts, each worker node functions by receiving and executing the tasks that are allocated to it by manager nodes.

Docker Swarm Tutorial

On AWS, create your VPC and Security Groups ...




1) Initialise swarm on the Manager ... 

docker swarm init --advertise-addr 172.31.7.94

2) Join swarm on the Workers ... 

docker swarm join --token SWMTKN-1-5zy8ij0t240g4b9xxxxxxxxxxxxxxxxxxxxxxin829mxrsl-amipciivp9t0sbtjugtjrcqlf 172.31.7.94:2377

3) Deploy a Service ... 

docker service create --replicas 1 --name helloworld alpine ping docker.com

4) Check a Service ... 

docker service ls

5) Inspect a Service ... 

docker service ps helloworld

6) Scale up a Service ... 

docker service scale helloworld=5

7) Inspect a Service ... 

docker service ps helloworld

8) Scale down a Service ... 

docker service scale helloworld=1

9) Stop and remove a Service ... 

docker service scale helloworld=0
docker service rm helloworld

Help

DB Tech

https://www.youtube.com/channel/UCVy16RS5eEDh8anP8j94G2A

https://gist.github.com/dnburgess

Retrieved from ‘https://wiki.indie-it.com/index.php?title=Docker&oldid=3569’