Difference between revisions of "Docker"

From Indie IT Wiki
 
(154 intermediate revisions by 2 users not shown)
Line 22: Line 22:
  
 
[https://fleet.linuxserver.io/ LinuxServer]
 
[https://fleet.linuxserver.io/ LinuxServer]
 +
 +
== Useful Wiki ==
 +
 +
* https://wiki.servarr.com/
  
 
== Installation ==
 
== Installation ==
 +
 +
'''WINDOWS'''
 +
 +
# Make sure your computer supports Hardware Virtualisation by checking in the BIOS.
 +
# Install [https://docs.docker.com/desktop/install/windows-install/ Docker Desktop].
 +
# Install the [https://docs.microsoft.com/windows/wsl/wsl2-kernel Windows Subsystem for Linux Kernel Update].
 +
# Reboot.
 +
 +
'''LINUX'''
 +
 +
=== New All In One Official Method ===
 +
 +
sudo -i
 +
wget -qO- <nowiki>https://raw.githubusercontent.com/pi-hosted/pi-hosted/master/install_docker.sh</nowiki> | bash
 +
 +
which calls ...
 +
 +
https://get.docker.com/
  
 
=== Engine ===
 
=== Engine ===
Line 29: Line 51:
 
This will remove the old version of 'Docker' and install the new version 'Docker CE'...
 
This will remove the old version of 'Docker' and install the new version 'Docker CE'...
  
  sudo apt-get remove docker docker-engine docker.io
+
  sudo apt-get remove docker docker-engine docker.io containerd runc
  sudo apt-get -y install apt-transport-https ca-certificates curl software-properties-common
+
sudo apt-get update
  curl -fsSL <nowiki>https://download.docker.com/linux/ubuntu/gpg</nowiki> | sudo apt-key add -
+
  sudo apt-get -y install apt-transport-https ca-certificates curl gnupg software-properties-common
sudo apt-key fingerprint 0EBFCD88
+
sudo mkdir -p /etc/apt/keyrings
  sudo add-apt-repository "deb [arch=amd64] <nowiki>https://download.docker.com/linux/ubuntu</nowiki> $(lsb_release -cs) stable"
+
  curl -fsSL <nowiki>https://download.docker.com/linux/ubuntu/gpg</nowiki> | sudo gpg --dearmor -o /etc/apt/keyrings/docker.gpg
  sudo apt-get -y install docker-ce
+
  echo "deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.gpg] <nowiki>https://download.docker.com/linux/ubuntu</nowiki> $(lsb_release -cs) stable" | sudo tee /etc/apt/sources.list.d/docker.list > /dev/null
 +
sudo apt-get update
 +
  sudo apt-get -y install docker-ce docker-ce-cli containerd.io docker-compose-plugin
 +
sudo service docker start
 
  sudo docker run hello-world
 
  sudo docker run hello-world
  
 
https://docs.docker.com/install/linux/docker-ce/ubuntu/#install-docker-ce-1
 
https://docs.docker.com/install/linux/docker-ce/ubuntu/#install-docker-ce-1
  
=== Compose ===
+
==== Compose From Command Line ====
 +
 
 +
http://composerize.com - Turns docker run commands into docker-compose files!
  
curl -I -s "<nowiki>https://github.com/docker/compose/releases/latest</nowiki>" | grep 'location:' | sed 's/^.*[/]//'
+
Install composerize and convert your own commands locally.
  
Check the [https://github.com/docker/compose/releases/latest latest version] of 'Docker Compose' and edit the following command...
+
sudo apt instal npm
 +
sudo npm install composerize -g
 +
composerize docker run -p 80:80 -v /var/run/docker.sock:/tmp/docker.sock:ro --restart always --log-opt max-size=1g nginx
  
sudo curl -L "<nowiki>https://github.com/docker/compose/releases/download/1.29.2/docker-compose-$(uname -s)-$(uname -m)</nowiki>" -o /usr/local/bin/docker-compose
+
https://github.com/magicmark/composerize
sudo chmod +x /usr/local/bin/docker-compose
 
/usr/local/bin/docker-compose --version
 
  
https://docs.docker.com/compose/install/#install-compose
+
=== Using Ansible ===
 +
 
 +
<code>docker-install.yml</code>
 +
 
 +
- hosts: all
 +
 
 +
  become: yes
 +
  tasks:
 +
 
 +
  # Install Docker
 +
  # --
 +
  #
 +
  - name: install prerequisites
 +
    apt:
 +
      name:
 +
        - apt-transport-https
 +
        - ca-certificates
 +
        - curl
 +
        - gnupg-agent
 +
        - software-properties-common
 +
      update_cache: yes
 +
 
 +
  - name: add apt-key
 +
    apt_key:
 +
      url: <nowiki>https://download.docker.com/linux/ubuntu/gpg</nowiki>
 +
 
 +
  - name: add docker repo
 +
    apt_repository:
 +
      repo: deb <nowiki>https://download.docker.com/linux/ubuntu</nowiki> focal stable
 +
 
 +
  - name: install docker
 +
    apt:
 +
      name:
 +
        - docker-ce
 +
        - docker-ce-cli
 +
        - containerd.io
 +
      update_cache: yes
 +
 
 +
  - name: add user permissions
 +
    shell: "usermod -aG docker ubuntu"
 +
 
 +
  # Installs Docker SDK
 +
  # --
 +
  #  
 +
  - name: install python package manager
 +
    apt:
 +
      name: python3-pip
 +
 
 +
  - name: install python sdk
 +
    become_user: ubuntu
 +
    pip:
 +
      name:
 +
        - docker
 +
        - docker-compose
 +
 
 +
Then run Ansible using your playbook on your server host ...
 +
 
 +
ansible-playbook docker-install.yml -l 'myserver'
 +
 
 +
You can then deploy a container using Ansible as well. This will deploy Portainer ...
 +
 
 +
<code>docker_deploy-portainer.yml</code>
 +
 
 +
- hosts: all
 +
 
 +
  become: yes
 +
  become_user: ubuntu
 +
  tasks:
 +
 
 +
  # Create Portainer Volume
 +
  # --
 +
  #
 +
  - name: Create new Volume
 +
    community.docker.docker_volume:
 +
      name: portainer-data
 +
 
 +
  # Deploy Portainer
 +
  # --
 +
  # 
 +
  - name: Deploy Portainer
 +
    community.docker.docker_container:
 +
      name: portainer
 +
      image: "docker.io/portainer/portainer-ce"
 +
      ports:
 +
        - "9000:9000"
 +
        - "9443:9443"
 +
      volumes:
 +
        - /var/run/docker.sock:/var/run/docker.sock
 +
        - portainer-data:/data
 +
      restart_policy: always
 +
 
 +
... with this command ...
 +
 
 +
  ansible-playbook docker_deploy-portainer.yml -l 'myserver'
  
 
== Usage ==
 
== Usage ==
Line 129: Line 249:
 
  alias dcu='docker-compose up -d'
 
  alias dcu='docker-compose up -d'
 
  alias dps='docker ps'
 
  alias dps='docker ps'
 +
 +
=== Run Command In Docker Container ===
 +
 +
e.g.
 +
 +
List the mail queue for a running email server ...
 +
 +
docker exec -it mail.domain.co.uk-mailserver mailq
  
 
== Volumes ==
 
== Volumes ==
Line 164: Line 292:
  
 
== Networks ==
 
== Networks ==
 +
 +
=== Basic Usage ===
  
 
Create your network...
 
Create your network...
  
  docker network create existing-network
+
  docker network create networkname
  
 
Use it in your docker-compose.yml file...
 
Use it in your docker-compose.yml file...
Line 176: Line 306:
 
     restart: always
 
     restart: always
 
     networks:
 
     networks:
       - existing-network
+
       - networkname
 
  networks:
 
  networks:
   existing-network:
+
   networkname:
 
     external: true
 
     external: true
  
 
https://poopcode.com/join-to-an-existing-network-from-a-docker-container-in-docker-compose/
 
https://poopcode.com/join-to-an-existing-network-from-a-docker-container-in-docker-compose/
  
== Management ==
+
=== Advanced Usage ===
  
=== Cleaning Space ===
+
'''Static IP Address'''
  
Over the last month, a whopping 14Gb of space was being used by /var/lib/docker/overlay2/ and needed a way to safely remove unused data.
+
    networks:
 +
      traefik:
 +
        ipv4_address: 172.19.0.9
 +
      backend: null
  
Check your space usage...
+
[https://www.youtube.com/watch?v=vUyHGF1HMsw Force Docker Containers to use a VPN for their Network]
  
du -mcsh /var/lib/docker/overlay2
+
==== Block IP Address Using IPTables ====
14G    /var/lib/docker/overlay2
 
  
Check what Docker thinks is being used...
+
The key here is to make sure you use the -I or INSERT command for the rule so that it is FIRST in the chain.
  
docker system df
+
'''Block IP addresses from LITHUANIA'''
TYPE            TOTAL    ACTIVE    SIZE      RECLAIMABLE
 
Images          36        15        8.368GB  4.491GB (53%)
 
Containers      17        15        70.74MB  286B (0%)
 
Local Volumes  4        2        0B        0B
 
Build Cache    0        0        0B        0B
 
  
Clean...
+
iptables -I DOCKER-USER -i eth0 -s 141.98.10.0/24 -j DROP
 +
iptables -I DOCKER-USER -i eth0 -s 141.98.11.0/24 -j DROP
 +
iptables -L DOCKER-USER --line-numbers
 +
Chain DOCKER-USER (1 references)
 +
num  target    prot opt source              destination
 +
1    DROP      all  --  141.98.11.0/24      anywhere
 +
2    DROP      all  --  141.98.10.0/24      anywhere
 +
3    RETURN    all  --  anywhere            anywhere
  
docker system prune
+
== Docker Compose ==
docker image prune --all
 
  
Check again...
+
=== Restart Policy ===
  
du -mcsh /var/lib/docker/overlay2
+
The "no" option has quotes around it...
9.4G    /var/lib/docker/overlay2
 
 
docker system df
 
TYPE            TOTAL    ACTIVE    SIZE      RECLAIMABLE
 
Images          13        13        4.144GB  144MB (3%)
 
Containers      15        15        70.74MB  0B (0%)
 
Local Volumes  4        2        0B        0B
 
Build Cache    0        0        0B        0B
 
  
...job done.
+
restart: "no"
 +
restart: always
 +
restart: on-failure
 +
restart: unless-stopped
  
=== Portainer ===
+
== Management ==
  
https://www.portainer.io
+
=== Cleaning Pruning ===
  
https://github.com/portainer/portainer
+
sudo docker system df
 +
sudo docker system prune
 +
cd /etc/cron.daily
 +
sudo nano docker-prune
 +
 +
#!/bin/bash
 +
docker system prune -y
 +
 +
sudo chmod +x /etc/cron.daily/docker-prune
  
https://hub.docker.com/r/portainer/portainer-ce
+
https://alexgallacher.com/prune-unused-docker-images-automatically/
  
=== Monitoring ===
+
==== Delete All Stopped Docker Containers ====
  
'''CTop'''
+
docker rm $(docker ps --filter "status=exited" -q)
  
Press the Q key to stop it...
+
=== Updating with Docker Compose ===
  
  docker run -ti --name ctop --rm -v /var/run/docker.sock:/var/run/docker.sock wrfly/ctop:latest
+
  for d in ./*/ ; do (cd "$d" && sudo docker-compose pull && sudo docker-compose --compatibility up -d); done
  
'''Docker Stats'''
+
=== Logs Logging ===
  
docker stats
+
If you want to watch the logs in real time, then add the -f or --follow option to your command ...
  
=== Gotchas ===
+
docker logs nginx '''--follow'''
  
https://sosedoff.com/2016/10/05/docker-gotchas.html
+
After executing docker-compose up, list your running containers:
  
== Applications ==
+
docker ps
  
I have set up my docker containers in a master docker directory with sub-directories for each stack.
+
Copy the NAME of the given container and read its logs:
  
  docker
+
  docker logs NAME_OF_THE_CONTAINER -f
|-- backups
 
`-- stacks
 
    |-- bitwarden
 
    |  `-- bwdata
 
    |-- grafana
 
    |  `-- data
 
    |-- mailserver
 
    |  `-- data
 
    |-- nginx-proxy-manager
 
    |  `-- data
 
    `-- portainer
 
        `-- data
 
  
=== Backups ===
+
To only read the error logs:
  
https://github.com/alaub81/backup_docker_scripts
+
docker logs NAME_OF_THE_CONTAINER -f 1>/dev/null
  
=== Updates ===
+
To only read the access logs:
  
==== Tracking ====
+
docker logs NAME_OF_THE_CONTAINER -f 2>/dev/null
  
===== Watchtower =====
+
https://linuxhandbook.com/docker-logging/
 +
 
 +
To search or grep the logs:
  
A process for automating Docker container base image updates.
+
docker logs watchtower 2>&1 | grep 'msg="Found new'
  
With watchtower you can update the running version of your containerized app simply by pushing a new image to the Docker Hub or your own image registry. Watchtower will pull down your new image, gracefully shut down your existing container and restart it with the same options that were used when it was deployed initially.
+
=== Cleaning Space ===
  
'''First Time Run Once Check Only'''
+
Over the last month, a whopping 14Gb of space was being used by /var/lib/docker/overlay2/ and needed a way to safely remove unused data.
  
This will run and output if there are any updates them stop and remove itself...
+
Check your space usage...
  
  docker run --name watchtower -v /var/run/docker.sock:/var/run/docker.sock containrrr/watchtower --run-once --debug --monitor-only --rm
+
  du -mcsh /var/lib/docker/overlay2
 +
14G    /var/lib/docker/overlay2
  
'''Automated Scheduled Run Daily'''
+
Check what Docker thinks is being used...
  
This will start the container and schedule a check at 4am every day...
+
docker system df
 +
TYPE            TOTAL    ACTIVE    SIZE      RECLAIMABLE
 +
Images          36        15        8.368GB  4.491GB (53%)
 +
Containers      17        15        70.74MB  286B (0%)
 +
Local Volumes  4        2        0B        0B
 +
Build Cache    0        0        0B        0B
  
<code>~/watchtower/docker-compose.yml</code>
+
Clean...
  
  version: "3"
+
  docker system prune
  services:
+
  docker image prune --all
  watchtower:
 
    image: containrrr/watchtower
 
    container_name: watchtower
 
    restart: always
 
    volumes:
 
      - /var/run/docker.sock:/var/run/docker.sock
 
    environment:
 
      - TZ=${TZ}
 
      - WATCHTOWER_DEBUG=true
 
      - WATCHTOWER_MONITOR_ONLY=false
 
      - WATCHTOWER_CLEANUP=true
 
      - WATCHTOWER_LABEL_ENABLE=false
 
      - WATCHTOWER_NOTIFICATIONS=email
 
      - WATCHTOWER_NOTIFICATION_EMAIL_FROM=${EMAIL_FROM}
 
      - WATCHTOWER_NOTIFICATION_EMAIL_TO=${WATCHTOWER_EMAIL_TO}
 
      - WATCHTOWER_NOTIFICATION_EMAIL_SERVER=${SMTP_SERVER}
 
      - WATCHTOWER_NOTIFICATION_EMAIL_SERVER_PORT=${SMTP_PORT}
 
      - WATCHTOWER_NOTIFICATION_EMAIL_SERVER_USER=${SMTP_USER}
 
      - WATCHTOWER_NOTIFICATION_EMAIL_SERVER_PASSWORD=${SMTP_PASSWORD}
 
      - WATCHTOWER_SCHEDULE=0 0 4 * * *
 
  
https://containrrr.dev/watchtower/
+
Check again...
  
https://containrrr.dev/watchtower/arguments/#without_updating_containers
+
du -mcsh /var/lib/docker/overlay2
 
+
9.4G    /var/lib/docker/overlay2
https://github.com/containrrr/watchtower
+
 
+
docker system df
https://www.the-digital-life.com/watchtower/
+
TYPE            TOTAL    ACTIVE    SIZE      RECLAIMABLE
 +
Images          13        13        4.144GB  144MB (3%)
 +
Containers      15        15        70.74MB  0B (0%)
 +
Local Volumes  4        2        0B        0B
 +
Build Cache    0        0        0B        0B
 +
 
 +
...job done.
 +
 
 +
=== Dockge ===
 +
 
 +
Better than Portainer.
 +
 
 +
A fancy, easy-to-use and reactive docker 'compose.yaml' stack-oriented manager
 +
 
 +
https://github.com/louislam/dockge
  
https://www.youtube.com/watch?v=5lP_pdjcVMo
+
=== Portainer ===
  
==== Updating ====
+
https://www.portainer.io
  
You can either ask Watchtower to update the containers automatically for you, or do it manually.
+
https://github.com/portainer/portainer
  
Manually updating when using docker-compose...
+
[https://docs.portainer.io/v/ce-2.6/advanced/reset-admin Reset the Admin user Password]
  
cd /path/to/docker/stack/
+
==== Server ====
docker-compose stop
 
docker-compose pull
 
docker-compose start
 
  
=== Bitwarden ===
+
https://hub.docker.com/r/portainer/portainer-ce
  
<code>~/bitwardenrs/docker-compose.yml</code>
+
==== Agent ====
  
version: "2"
+
Portainer uses the Portainer Agent container to communicate with the Portainer Server instance and provide access to the node's resources. So, this is great for a small server (like a Raspberry Pi) where you don't need the full Portainer Server install.
services:
+
 
  bitwardenrs:
+
'''Deployment'''
    image: bitwardenrs/server:latest
+
 
    container_name: bitwardenrs
+
Run the following command to deploy the Portainer Agent:
    volumes:
+
 
      - ./data:/data/
+
docker run -d -p 9001:9001 --name portainer_agent --restart=always -v /var/run/docker.sock:/var/run/docker.sock -v /var/lib/docker/volumes:/var/lib/docker/volumes portainer/agent:2.11.0
    ports:
+
 
      - 8100:80
+
'''Adding your new environment'''
    restart: unless-stopped
+
 
    environment:
+
Once the agent has been installed you are ready to add the environment to your Portainer Server installation.
      - TZ=Europe/London
+
 
      #- SIGNUPS_ALLOWED=false
+
From the menu select Environments then click Add environment.
      #- INVITATIONS_ALLOWED=false
 
      #- WEB_VAULT_ENABLED=false
 
networks:
 
  default:
 
    external:
 
      name: nginx-proxy-manager
 
  
Uncomment the extra security # lines '''after''' you have signed up, imported your old vault and set up your phone app and browsers, etc.
+
From the Environment type section, select Agent. Since we have already installed the agent you can ignore the sample commands in the Information section.
  
  docker-compose down
+
  Name: my-raspberry-pi
  docker-compose up -d
+
  Environment URL: 192.168.0.106:9001
  
Check that the Bitwarden container environment has all the variables...
+
When you're ready, click Add environment.
  
docker exec -it bitwardenrs env | sort
+
Then, on the Portainer Home screen you select your new environment, or server running the Agent, and away you go!
+
 
HOME=/root
+
https://docs.portainer.io/v/ce-2.11/admin/environments/add/docker#method-2-connecting-via-the-portainer-agent
HOSTNAME=e5f327deb4dd
+
 
INVITATIONS_ALLOWED=false
+
=== Updating ===
PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
+
 
ROCKET_ENV=staging
+
==== Portainer ====
ROCKET_PORT=80
+
 
ROCKET_WORKERS=10
+
  Containers > Select > Stop > Recreate > Pull Latest Image > Start
SIGNUPS_ALLOWED=false
 
TERM=xterm
 
TZ=Europe/London
 
  WEB_VAULT_ENABLED=false
 
  
... and then refresh your web vault page to see it see "404: Not Found"  :-)
+
==== Watchtowerr ====
  
[https://hub.docker.com/r/bitwardenrs/server Bitwarden RS]
+
List updates ...
  
[http://wiki.indie-it.com/wiki/Bitwarden#Self_Hosting Bitwarden Official]
+
docker run --rm -v /var/run/docker.sock:/var/run/docker.sock containrrr/watchtower --run-once --log-format pretty --monitor-only
  
=== InfluxDB ===
+
Perform updates ...
  
You can have InfluxDB on its own but there is little point without something to view the stats so you might as well include InfluxDB in the Grafana stack and start both at the same time... see below :-)
+
docker run --rm -v /var/run/docker.sock:/var/run/docker.sock containrrr/watchtower --run-once --log-format pretty
  
=== Grafana ===
+
=== Backups ===
  
Here is a stack in docker-compose which starts both containers in their own network so they can talk to one another. I have exposed ports for InfluxDB and Grafana to the host so I can use them from the internet.
+
https://github.com/SavageSoftware/portainer-backup
  
Obviously, put your firewall in place and change the passwords below!
+
https://forum.openmediavault.org/index.php?thread/43235-backup-portainer-and-completely-uninstall-to-test-disaster-recovery/
  
<code>~/grafana/docker-compose.yml</code>
+
=== Monitoring ===
  
version: "3"
+
'''CTop'''
services:
+
 
  grafana:
+
Press the Q key to stop it...
    image: grafana/grafana
+
 
    container_name: grafana
+
docker run -ti --name ctop --rm -v /var/run/docker.sock:/var/run/docker.sock wrfly/ctop:latest
    restart: always
+
 
    networks:
+
'''Docker Stats'''
      - grafana-influxdb-network
+
 
    ports:
+
docker stats
      - 3000:3000
+
 
    volumes:
+
'''DeUnhealth'''
      - ./data/grafana:/var/lib/grafana
+
 
    environment:
+
Restart your unhealthy containers safely.
      - INFLUXDB_URL=<nowiki>http://influxdb:8086</nowiki>
+
 
    depends_on:
+
https://github.com/qdm12/deunhealth
      - influxdb
+
 
  influxdb:
+
https://www.youtube.com/watch?v=Oeo-mrtwRgE
    image: influxdb:1.8.4
+
 
    container_name: influxdb
+
'''Dozzle'''
    restart: always
+
 
    networks:
+
Dozzle is a small lightweight application with a web based interface to monitor Docker logs. It doesn’t store any log files. It is for live monitoring of your container logs only.
      - grafana-influxdb-network
+
 
    ports:
+
https://github.com/amir20/dozzle
      - 8086:8086
 
    volumes:
 
      - ./data/influxdb:/var/lib/influxdb
 
    environment:
 
      - INFLUXDB_DB=grafana
 
      - INFLUXDB_USER=grafana
 
      - INFLUXDB_USER_PASSWORD=password
 
      - INFLUXDB_ADMIN_ENABLED=true
 
      - INFLUXDB_ADMIN_USER=admin
 
      - INFLUXDB_ADMIN_PASSWORD=password
 
      - INFLUXDB_URL=<nowiki>http://influxdb:8086</nowiki>
 
networks:
 
  grafana-influxdb-network:
 
    external: true
 
  
After this, change your Telegraf configuration to point to the '''new host''' and change the database it uses to 'grafana'.
+
=== Gotchas ===
  
=== NGiNX Proxy Manager ===
+
https://sosedoff.com/2016/10/05/docker-gotchas.html
  
Provide users with an easy way to accomplish reverse proxying hosts with SSL termination that is so easy a monkey could do it.
+
== Applications ==
  
# Set up your host
+
I have set up my docker containers in a master docker directory with sub-directories for each stack.
# Add a proxy to point to the host (in Docker this will be the 'name' and the port)
 
# Go to <nowiki>http://yourhost</nowiki>
 
  
https://nginxproxymanager.com
+
docker
 +
|-- backups
 +
`-- stacks
 +
    |-- bitwarden
 +
    |  `-- bwdata
 +
    |-- grafana
 +
    |  `-- data
 +
    |-- mailserver
 +
    |  `-- data
 +
    |-- nginx-proxy-manager
 +
    |  `-- data
 +
    `-- portainer
 +
        `-- data
  
https://github.com/jc21/nginx-proxy-manager
+
=== Backups ===
 +
 
 +
https://github.com/alaub81/backup_docker_scripts
 +
 
 +
=== Updates ===
  
Create the Docker network...
+
==== Tracking ====
  
sudo -i
+
===== Watchtower =====
docker network create nginx-proxy-manager
 
  
<code>/root/stacks/nginx-proxy-manager/docker-compose.yml</code>
+
A process for automating Docker container base image updates.
  
  version: '3'
+
With watchtower you can update the running version of your containerized app simply by pushing a new image to the Docker Hub or your own image registry. Watchtower will pull down your new image, gracefully shut down your existing container and restart it with the same options that were used when it was deployed initially.
  services:
+
 
   app:
+
'''First Time Run Once Check Only'''
     image: 'jc21/nginx-proxy-manager:latest'
+
 
     container_name: nginx-proxy-manager_app
+
This will run and output if there are any updates them stop and remove itself...
 +
 
 +
docker run --name watchtower -v /var/run/docker.sock:/var/run/docker.sock containrrr/watchtower --run-once --debug --monitor-only --rm
 +
 
 +
'''Automated Scheduled Run Daily'''
 +
 
 +
This will start the container and schedule a check at 4am every day...
 +
 
 +
<code>~/watchtower/docker-compose.yml</code>
 +
 
 +
  version: "3"
 +
  services:
 +
   watchtower:
 +
     image: containrrr/watchtower
 +
     container_name: watchtower
 
     restart: always
 
     restart: always
     networks:
+
     volumes:
       - nginx-proxy-manager
+
       - /var/run/docker.sock:/var/run/docker.sock
    ports:
 
      - '80:80'
 
      - '81:81'
 
      - '443:443'
 
 
     environment:
 
     environment:
       TZ: "Europe/London"
+
       - TZ=${TZ}
       DB_MYSQL_HOST: "db"
+
       - WATCHTOWER_DEBUG=true
       DB_MYSQL_PORT: 3306
+
       - WATCHTOWER_MONITOR_ONLY=false
       DB_MYSQL_USER: "npm"
+
       - WATCHTOWER_CLEANUP=true
       DB_MYSQL_PASSWORD: "npm"
+
       - WATCHTOWER_LABEL_ENABLE=false
       DB_MYSQL_NAME: "npm"
+
       - WATCHTOWER_NOTIFICATIONS=email
    volumes:
+
      - WATCHTOWER_NOTIFICATION_EMAIL_FROM=${EMAIL_FROM}
       - ./data:/data
+
       - WATCHTOWER_NOTIFICATION_EMAIL_TO=${WATCHTOWER_EMAIL_TO}
       - ./data/letsencrypt:/etc/letsencrypt
+
       - WATCHTOWER_NOTIFICATION_EMAIL_SERVER=${SMTP_SERVER}
    depends_on:
+
       - WATCHTOWER_NOTIFICATION_EMAIL_SERVER_PORT=${SMTP_PORT}
       - db
+
      - WATCHTOWER_NOTIFICATION_EMAIL_SERVER_USER=${SMTP_USER}
  db:
+
      - WATCHTOWER_NOTIFICATION_EMAIL_SERVER_PASSWORD=${SMTP_PASSWORD}
    image: 'jc21/mariadb-aria:latest'
+
       - WATCHTOWER_SCHEDULE=0 0 4 * * *
    container_name: nginx-proxy-manager_db
+
 
    restart: always
+
https://containrrr.dev/watchtower/
    networks:
+
 
       - nginx-proxy-manager
+
https://containrrr.dev/watchtower/arguments/#without_updating_containers
    environment:
 
      TZ: "Europe/London"
 
      MYSQL_ROOT_PASSWORD: 'npm'
 
      MYSQL_DATABASE: 'npm'
 
      MYSQL_USER: 'npm'
 
      MYSQL_PASSWORD: 'npm'
 
    volumes:
 
      - ./data/mysql:/var/lib/mysql
 
networks:
 
  nginx-proxy-manager:
 
    external: true
 
  
'''Reset Password'''
+
https://github.com/containrrr/watchtower
  
docker exec -it nginx-proxy-manager_db sh
+
https://www.the-digital-life.com/watchtower/
mysql -u root -p npm
 
select * from user;
 
delete from user where id=1;
 
quit;
 
exit
 
  
'''Custom SSL Certificate'''
+
https://www.youtube.com/watch?v=5lP_pdjcVMo
  
You can add a custom SSL certificate to NPM by saving the 3 parts of the SSL from Let's Encrypt...
+
==== Updating ====
  
# privkey.pem
+
You can either ask Watchtower to update the containers automatically for you, or do it manually.
# cert.pem
 
# chain.pem
 
  
...and then uploading them to NPM.
+
Manually updating when using docker-compose...
 
 
'''Updating'''
 
  
 +
cd /path/to/docker/stack/
 +
docker-compose down
 
  docker-compose pull
 
  docker-compose pull
 
  docker-compose up -d
 
  docker-compose up -d
  
=== NGiNX ===
+
=== Bitwarden ===
  
'''Quick Container'''
+
<code>~/bitwardenrs/docker-compose.yml</code>
  
Run and delete everything afterwards (press CTRL+C to stop it)...
+
version: "2"
 +
services:
 +
  bitwardenrs:
 +
    image: bitwardenrs/server:latest
 +
    container_name: bitwardenrs
 +
    volumes:
 +
      - ./data:/data/
 +
    ports:
 +
      - 8100:80
 +
    restart: unless-stopped
 +
    environment:
 +
      - TZ=Europe/London
 +
      #- SIGNUPS_ALLOWED=false
 +
      #- INVITATIONS_ALLOWED=false
 +
      #- WEB_VAULT_ENABLED=false
 +
networks:
 +
  default:
 +
    external:
 +
      name: nginx-proxy-manager
  
docker run --rm --name test.domain.org-nginx -e VIRTUAL_HOST=test.domain.org nginx
+
Uncomment the extra security # lines '''after''' you have signed up, imported your old vault and set up your phone app and browsers, etc.
  
Run and detach and use a host folder to store the web pages and keep the container afterwards...
+
docker-compose down
 +
docker-compose up -d
  
docker run --name test.domain.org-nginx -e VIRTUAL_HOST=test.domain.org -v /some/content:/usr/share/nginx/html:ro -d nginx
+
Check that the Bitwarden container environment has all the variables...
  
Run and detach and connect to a specific network (like nginx-proxy-manager) and use a host folder to store the web pages and keep the container afterwards...
+
docker exec -it bitwardenrs env | sort
 +
 +
HOME=/root
 +
HOSTNAME=e5f327deb4dd
 +
INVITATIONS_ALLOWED=false
 +
PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
 +
ROCKET_ENV=staging
 +
ROCKET_PORT=80
 +
ROCKET_WORKERS=10
 +
SIGNUPS_ALLOWED=false
 +
TERM=xterm
 +
TZ=Europe/London
 +
WEB_VAULT_ENABLED=false
  
  docker run --name test.domain.org-nginx --network nginx-proxy-manager -e VIRTUAL_HOST=test.domain.org -v /some/content:/usr/share/nginx/html:ro -d nginx
+
... and then refresh your web vault page to see it see "404: Not Found"  :-)
  
Check the logs and always show them (like tail -f)...
+
[https://hub.docker.com/r/bitwardenrs/server Bitwarden RS]
  
  docker logs test.domain.org-nginx -f
+
[http://wiki.indie-it.com/wiki/Bitwarden#Self_Hosting Bitwarden Official]
  
<code>docker-compose.yml</code>
+
=== InfluxDB ===
  
version: "3"
+
You can have InfluxDB on its own but there is little point without something to view the stats so you might as well include InfluxDB in the Grafana stack and start both at the same time... see below :-)
services:
+
 
  nginx:
+
=== Grafana ===
    image: nginx
 
    container_name: nginx
 
    environment:
 
      - TZ=Europe/London
 
    volumes:
 
      - ./data/html:/usr/share/nginx/html:ro
 
      - /etc/timezone:/etc/timezone:ro
 
    expose:
 
      - 80
 
    restart: unless-stopped
 
  
 +
Here is a stack in docker-compose which starts both containers in their own network so they can talk to one another. I have exposed ports for InfluxDB and Grafana to the host so I can use them from the internet.
  
'''With PHP'''
+
Obviously, put your firewall in place and change the passwords below!
  
<code>./data/nginx/site.conf</code>
+
<code>~/grafana/docker-compose.yml</code>
  
  server {
+
  version: "3"
     server_name docker-demo.com;
+
services:
     root /var/www/html;
+
  grafana:
     index index.php index.html index.htm;
+
    image: grafana/grafana
    access_log /var/log/nginx/access.log;
+
    container_name: grafana
     error_log /var/log/nginx/error.log;
+
    restart: always
     location / {
+
     networks:
        try_files $uri $uri/ /index.php?$query_string;
+
      - grafana-influxdb-network
     }
+
     ports:
     # PHP-FPM Configuration Nginx
+
      - 3000:3000
     location ~ \.php$ {
+
     volumes:
        try_files $uri = 404;
+
      - ./data/grafana:/var/lib/grafana
        fastcgi_split_path_info ^(.+\.php)(/.+)$;
+
     environment:
        fastcgi_pass php:9000;
+
      - INFLUXDB_URL=<nowiki>http://influxdb:8086</nowiki>
        fastcgi_index index.php;
+
     depends_on:
        include fastcgi_params;
+
      - influxdb
        fastcgi_param REQUEST_URI $request_uri;
+
  influxdb:
        fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
+
    image: influxdb:1.8.4
        fastcgi_param PATH_INFO $fastcgi_path_info;
+
    container_name: influxdb
     }
+
     restart: always
}
+
     networks:
 +
      - grafana-influxdb-network
 +
     ports:
 +
      - 8086:8086
 +
    volumes:
 +
      - ./data/influxdb:/var/lib/influxdb
 +
    environment:
 +
      - INFLUXDB_DB=grafana
 +
      - INFLUXDB_USER=grafana
 +
      - INFLUXDB_USER_PASSWORD=password
 +
      - INFLUXDB_ADMIN_ENABLED=true
 +
      - INFLUXDB_ADMIN_USER=admin
 +
      - INFLUXDB_ADMIN_PASSWORD=password
 +
      - INFLUXDB_URL=<nowiki>http://influxdb:8086</nowiki>
 +
networks:
 +
  grafana-influxdb-network:
 +
     external: true
 +
 
 +
After this, change your Telegraf configuration to point to the '''new host''' and change the database it uses to 'grafana'.
  
<code>docker-compose.yml</code>
+
=== NGiNX Proxy Manager ===
  
version: "3"
+
Provide users with an easy way to accomplish reverse proxying hosts with SSL termination that is so easy a monkey could do it.
services:
+
 
  nginx:
+
# Set up your host
    image: nginx
+
# Add a proxy to point to the host (in Docker this will be the 'name' and the port)
    container_name: nginx
+
# Go to <nowiki>http://yourhost</nowiki>
    environment:
+
 
      - TZ=Europe/London
+
https://nginxproxymanager.com
    volumes:
 
      - ./data/html:/usr/share/nginx/html:ro
 
      - ./data/nginx:/etc/nginx/conf.d/
 
      - /etc/timezone:/etc/timezone:ro
 
    expose:
 
      - 80
 
    restart: unless-stopped
 
  php:
 
    image: php:7.2-fpm
 
    volumes:
 
      - ./data/html:/usr/share/nginx/html:ro
 
      - ./data/php:/usr/local/etc/php/php.ini
 
  
https://adoltech.com/blog/how-to-set-up-nginx-php-fpm-and-mysql-with-docker-compose/
+
https://github.com/jc21/nginx-proxy-manager
  
'''Load Balancer'''
+
Create the Docker network with a chosen subnet (used later for fixing container IP addresses)...
  
This is a simple exmaple test to show multiple backend servers answering web page requests.
+
sudo -i
 +
docker network create --subnet=172.20.0.0/16 nginx-proxy-manager
  
<code>docker-compose.yml</code>
+
<code>/root/stacks/nginx-proxy-manager/docker-compose.yml</code>
  
 
  version: '3'
 
  version: '3'
 
  services:
 
  services:
   # The load balancer
+
   db:
  nginx:
+
    image: 'jc21/mariadb-aria:latest'
     image: nginx:1.16.0-alpine
+
    container_name: nginx-proxy-manager_db
 +
    restart: always
 +
     networks:
 +
      nginx-proxy-manager:
 +
        ipv4_address: 172.20.0.2
 +
    environment:
 +
      TZ: "Europe/London"
 +
      MYSQL_ROOT_PASSWORD: 'npm'
 +
      MYSQL_DATABASE: 'npm'
 +
      MYSQL_USER: 'npm'
 +
      MYSQL_PASSWORD: 'npm'
 
     volumes:
 
     volumes:
       - ./nginx.conf:/etc/nginx/nginx.conf:ro
+
       - ./data/mysql:/var/lib/mysql
 +
  app:
 +
    image: 'jc21/nginx-proxy-manager:latest'
 +
    container_name: nginx-proxy-manager_app
 +
    restart: always
 +
    networks:
 +
      nginx-proxy-manager:
 +
        ipv4_address: 172.20.0.3
 
     ports:
 
     ports:
       - "80:80"
+
       - '80:80'
  # The web server1
+
      - '81:81'
  server1:
+
      - '443:443'
    image: nginx:1.16.0-alpine
+
    environment:
 +
      TZ: "Europe/London"
 +
      DB_MYSQL_HOST: "db"
 +
      DB_MYSQL_PORT: 3306
 +
      DB_MYSQL_USER: "npm"
 +
      DB_MYSQL_PASSWORD: "npm"
 +
      DB_MYSQL_NAME: "npm"
 
     volumes:
 
     volumes:
       - ./server1.html:/usr/share/nginx/html/index.html
+
       - ./data:/data
  # The web server2
+
      - ./data/letsencrypt:/etc/letsencrypt
  server2:
+
    depends_on:
    image: nginx:1.16.0-alpine
+
      - db
     volumes:
+
networks:
      - ./server2.html:/usr/share/nginx/html/index.html
+
  nginx-proxy-manager:
 +
     external: true
  
<code>nginx.conf</code>
+
'''Reset Password'''
  
  events {
+
  docker exec -it nginx-proxy-manager_db sh
    worker_connections 1024;
+
  mysql -u root -p npm
  }
+
  select * from user;
  http {
+
delete from user where id=1;
    upstream app_servers {    # Create an upstream for the web servers
+
quit;
        server server1:80;   # the first server
+
  exit
        server server2:80;   # the second server
 
    }
 
    server {
 
        listen 80;
 
        location / {
 
            proxy_pass        http://app_servers; # load balance the traffic
 
        }
 
    }
 
  }
 
  
https://omarghader.github.io/docker-compose-nginx-tutorial/
+
'''Custom SSL Certificate'''
  
'''Proxy'''
+
You can add a custom SSL certificate to NPM by saving the 3 parts of the SSL from Let's Encrypt...
  
This is very cool and allows you to run multiple web sites on-the-fly.
+
# privkey.pem
 +
# cert.pem
 +
# chain.pem
  
The container connects to the system docker socket and watches for new containers using the VIRTUAL_HOST environment variable.
+
...and then uploading them to NPM.
  
Start this, then add another container using the VIRTUAL_HOST variable and the proxy container will change its config file and reload nginx to serve the web site... automatically.
+
'''Updating'''
  
Incredible.
+
docker-compose pull
 +
docker-compose up -d
  
<code>~/nginx-proxy/docker-compose.yml</code>
+
=== NGiNX ===
  
version: "3"
+
'''Quick Container'''
services:
 
  nginx-proxy:
 
    image: jwilder/nginx-proxy
 
    container_name: nginx-proxy
 
    ports:
 
      - "80:80"
 
    volumes:
 
      - /var/run/docker.sock:/tmp/docker.sock:ro
 
networks:
 
  default:
 
    external:
 
      name: nginx-proxy
 
  
'''Normal'''
+
Run and delete everything afterwards (press CTRL+C to stop it)...
  
When using the nginx-proxy container above, you can just spin up a virtual web site using the standard 'nginx' docker image and link it to the 'nginx-proxy' network...
+
docker run --rm --name test.domain.org-nginx -e VIRTUAL_HOST=test.domain.org nginx
  
docker run -d --name nginx-website1.uk --expose 80 --net nginx-proxy -e VIRTUAL_HOST=website1.uk nginx
+
Run and detach and use a host folder to store the web pages and keep the container afterwards...
  
To use the host filesystem to store the web page files...
+
docker run --name test.domain.org-nginx -e VIRTUAL_HOST=test.domain.org -v /some/content:/usr/share/nginx/html:ro -d nginx
  
docker run -d --name nginx-website1.uk --expose 80 --net nginx-proxy -v /var/www/website1.uk/html:/usr/share/nginx/html:ro -e VIRTUAL_HOST=website1.uk nginx
+
Run and detach and connect to a specific network (like nginx-proxy-manager) and use a host folder to store the web pages and keep the container afterwards...
  
In Docker Compose, it will look like this...
+
  docker run --name test.domain.org-nginx --network nginx-proxy-manager -e VIRTUAL_HOST=test.domain.org -v /some/content:/usr/share/nginx/html:ro -d nginx
  
<code>~/nginx/docker-compose.yml</code>
+
Check the logs and always show them (like tail -f)...
 +
 
 +
  docker logs test.domain.org-nginx -f
 +
 
 +
<code>docker-compose.yml</code>
  
 
  version: "3"
 
  version: "3"
 
  services:
 
  services:
   nginx-website1.uk:
+
   nginx:
 
     image: nginx
 
     image: nginx
     container_name: nginx-website1.uk
+
     container_name: nginx
 +
    environment:
 +
      - TZ=Europe/London
 +
    volumes:
 +
      - ./data/html:/usr/share/nginx/html:ro
 +
      - /etc/timezone:/etc/timezone:ro
 
     expose:
 
     expose:
       - "80"
+
       - 80
     volumes:
+
     restart: unless-stopped
      - /var/www/website1.uk/html:/usr/share/nginx/html:ro
 
    environment:
 
      - VIRTUAL_HOST=website1.uk
 
networks:
 
  default:
 
    external:
 
      name: nginx-proxy
 
  
'''Multiple Virtual Host Web Sites'''
 
  
<code>~/nginx/docker-compose.yml</code>
+
'''With PHP'''
  
  version: "3"
+
<code>./data/nginx/site.conf</code>
 +
 
 +
server {
 +
    server_name docker-demo.com;
 +
    root /var/www/html;
 +
    index index.php index.html index.htm;
 +
    access_log /var/log/nginx/access.log;
 +
    error_log /var/log/nginx/error.log;
 +
    location / {
 +
        try_files $uri $uri/ /index.php?$query_string;
 +
    }
 +
    # PHP-FPM Configuration Nginx
 +
    location ~ \.php$ {
 +
        try_files $uri = 404;
 +
        fastcgi_split_path_info ^(.+\.php)(/.+)$;
 +
        fastcgi_pass php:9000;
 +
        fastcgi_index index.php;
 +
        include fastcgi_params;
 +
        fastcgi_param REQUEST_URI $request_uri;
 +
        fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
 +
        fastcgi_param PATH_INFO $fastcgi_path_info;
 +
    }
 +
}
 +
 
 +
<code>docker-compose.yml</code>
 +
 
 +
  version: "3"
 
  services:
 
  services:
   nginx-website1.uk:
+
   nginx:
 
     image: nginx
 
     image: nginx
     container_name: nginx-website1.uk
+
     container_name: nginx
     expose:
+
     environment:
       - "80"
+
       - TZ=Europe/London
 
     volumes:
 
     volumes:
       - /var/www/website1.uk/html:/usr/share/nginx/html:ro
+
       - ./data/html:/usr/share/nginx/html:ro
    environment:
+
       - ./data/nginx:/etc/nginx/conf.d/
       - VIRTUAL_HOST=website1.uk
+
      - /etc/timezone:/etc/timezone:ro
  nginx-website2.uk:
 
    image: nginx
 
    container_name: nginx-website2.uk
 
 
     expose:
 
     expose:
       - "80"
+
       - 80
 +
    restart: unless-stopped
 +
  php:
 +
    image: php:7.2-fpm
 
     volumes:
 
     volumes:
       - /var/www/website2.uk/html:/usr/share/nginx/html:ro
+
       - ./data/html:/usr/share/nginx/html:ro
    environment:
+
       - ./data/php:/usr/local/etc/php/php.ini
       - VIRTUAL_HOST=website2.uk
 
networks:
 
  default:
 
    external:
 
      name: nginx-proxy
 
  
'''Viewing Logs'''
+
https://adoltech.com/blog/how-to-set-up-nginx-php-fpm-and-mysql-with-docker-compose/
  
docker-compose logs nginx-website1.uk
+
'''With PERL'''
docker-compose logs nginx-website2.uk
 
  
'''Proxy Manager'''
+
This is a way to get the IP address of the visitor (REMOTE_ADDR) displayed...
  
This is a web front end to manage 'nginx-proxy', where you can choose containers and create SSL certificates etc.
+
# nginx.conf; mostly copied from defaults
 +
load_module "modules/ngx_http_perl_module.so";
 +
 +
user  nginx;
 +
worker_processes  auto;
 +
 +
error_log  /var/log/nginx/error.log notice;
 +
pid        /var/run/nginx.pid;
 +
 +
events {
 +
    worker_connections  1024;
 +
}
 +
 +
http {
 +
    #perl_modules /; # only needed the hello.pm isn't in @INC (e.g. dir specified below)
 +
    perl_modules /usr/lib/perl5/vendor_perl/x86_64-linux-thread-multi/;
 +
    perl_require hello.pm;
 +
 +
    server {
 +
        location / {
 +
            perl hello::handler;
 +
        }
 +
    }
 +
}
  
https://cyberhost.uk/npm-setup/
+
# hello.pm; put in a @INC dir
 +
package hello;
 +
use nginx;
 +
 +
sub handler {
 +
    my $r = shift;
 +
    $r->send_http_header("text/html");
 +
    return OK if $r->header_only;
 +
    $r->print($r->remote_addr);
 +
    return OK;
 +
}
 +
1;
  
'''Various'''
+
https://www.reddit.com/r/docker/comments/oabga4/run_perl_script_in_nginx_container/
  
https://hub.docker.com/_/nginx
+
'''Load Balancer'''
  
https://blog.ssdnodes.com/blog/host-multiple-websites-docker-nginx/
+
This is a simple exmaple test to show multiple backend servers answering web page requests.
  
https://github.com/nginx-proxy/nginx-proxy
+
<code>docker-compose.yml</code>
  
=== WordPress ===
+
version: '3'
 +
services:
 +
  # The load balancer
 +
  nginx:
 +
    image: nginx:1.16.0-alpine
 +
    volumes:
 +
      - ./nginx.conf:/etc/nginx/nginx.conf:ro
 +
    ports:
 +
      - "80:80"
 +
  # The web server1
 +
  server1:
 +
    image: nginx:1.16.0-alpine
 +
    volumes:
 +
      - ./server1.html:/usr/share/nginx/html/index.html
 +
  # The web server2
 +
  server2:
 +
    image: nginx:1.16.0-alpine
 +
    volumes:
 +
      - ./server2.html:/usr/share/nginx/html/index.html
  
https://hub.docker.com/_/wordpress/
+
<code>nginx.conf</code>
  
==== PHP File Uploads Fix ====
+
events {
 
+
    worker_connections 1024;
Create a new PHP configuration file, and name it docker-uploads.ini. Add the following configuration then save the changes.
+
}
 
+
http {
# Allow HTTP file uploads
+
    upstream app_servers {    # Create an upstream for the web servers
file_uploads = On
+
        server server1:80;    # the first server
+
        server server2:80;    # the second server
  # Maximum size of an uploaded file
+
    }
upload_max_filesize = 64M
+
    server {
+
        listen 80;
# Maximum size of form post data
+
        location / {
  post_max_size = 64M
+
            proxy_pass        http://app_servers; # load balance the traffic
 +
        }
 +
    }
 +
  }
  
Update the docker-compose.yml to bind the docker-uploads.ini to the wordpress container and then restart the WordPress container.
+
https://omarghader.github.io/docker-compose-nginx-tutorial/
  
volumes:
+
'''Proxy'''
  - ./data/config/docker-uploads.ini:/usr/local/etc/php/conf.d/docker-uploads.ini
 
  
==== WordPress Clone ====
+
This is very cool and allows you to run multiple web sites on-the-fly.
  
Create your A record in DNS using [http://wiki.indie-it.com/wiki/AWS_Route_53#cli53 AWS Route 53 CLI]...
+
The container connects to the system docker socket and watches for new containers using the VIRTUAL_HOST environment variable.
  
cli53 rrcreate domain.co.uk 'staging 300 A 123.456.78.910'
+
Start this, then add another container using the VIRTUAL_HOST variable and the proxy container will change its config file and reload nginx to serve the web site... automatically.
  
Create your docker folder for the cloned staging test web site...
+
Incredible.
  
mkdir -p ~/docker/stacks/staging.domain.co.uk/data/{db,html}
+
<code>~/nginx-proxy/docker-compose.yml</code>
 
 
Edit your docker compose file, with 2 containers, making sure you use the same network as your Nginx Proxy Manager...
 
 
 
<code>~/docker/stacks/staging.domain.co.uk/docker-compose.yml</code>
 
  
 
  version: "3"
 
  version: "3"
 
  services:
 
  services:
   staging.domain.co.uk-wordpress_db:
+
   nginx-proxy:
     image: mysql:5.7
+
     image: jwilder/nginx-proxy
     container_name: staging.domain.co.uk-wordpress_db
+
    container_name: nginx-proxy
 +
     ports:
 +
      - "80:80"
 
     volumes:
 
     volumes:
       - ./data/db:/var/lib/mysql
+
       - /var/run/docker.sock:/tmp/docker.sock:ro
    restart: always
+
  networks:
    environment:
+
   default:
      - TZ=Europe/London
+
     external:
      - MYSQL_ROOT_PASSWORD=siel6aiL
+
       name: nginx-proxy
      - MYSQL_DATABASE=dbname
+
 
      - MYSQL_USER=dbuser
+
'''Normal'''
      - MYSQL_PASSWORD=ru5BeoFa
 
  staging.domain.co.uk-wordpress:
 
    depends_on:
 
      - staging.domain.co.uk-wordpress_db
 
    image: wordpress:latest
 
    container_name: staging.domain.co.uk-wordpress
 
    volumes:
 
      - ./data/html:/var/www/html
 
    restart: always
 
    environment:
 
      - TZ=Europe/London
 
      - VIRTUAL_HOST=staging.domain.co.uk
 
      - WORDPRESS_DB_HOST=staging.domain.co.uk-wordpress_db:3306
 
      - WORDPRESS_DB_NAME=dbname
 
      - WORDPRESS_DB_USER=dbuser
 
      - WORDPRESS_DB_PASSWORD=ru5BeoFa
 
  networks:
 
   default:
 
     external:
 
       name: nginx-proxy-manager
 
  
Start containers with correct settings and credentials for existing line web site (so that the docker startup script sets up the MySQL permissions)...
+
When using the nginx-proxy container above, you can just spin up a virtual web site using the standard 'nginx' docker image and link it to the 'nginx-proxy' network...
  
  docker-compose up -d
+
  docker run -d --name nginx-website1.uk --expose 80 --net nginx-proxy -e VIRTUAL_HOST=website1.uk nginx
  
Check the logs to make sure all is well...
+
To use the host filesystem to store the web page files...
  
  docker logs staging.domain.co.uk-wordpress
+
  docker run -d --name nginx-website1.uk --expose 80 --net nginx-proxy -v /var/www/website1.uk/html:/usr/share/nginx/html:ro -e VIRTUAL_HOST=website1.uk nginx
docker logs staging.domain.co.uk-wordpress_db
 
  
Copy the WordPress files to the host folder and correct ownership...
+
In Docker Compose, it will look like this...
  
cp -av /path/to/backup_unzipped_wordpress/ ~/docker/stacks/staging.domain.co.uk/html/
+
<code>~/nginx/docker-compose.yml</code>
chown -R www-data:www-data ~/docker/stacks/staging.domain.co.uk/html/
 
  
Copy the sql file in to the running '''mysql''' container...
+
version: "3"
 +
services:
 +
  nginx-website1.uk:
 +
    image: nginx
 +
    container_name: nginx-website1.uk
 +
    expose:
 +
      - "80"
 +
    volumes:
 +
      - /var/www/website1.uk/html:/usr/share/nginx/html:ro
 +
    environment:
 +
      - VIRTUAL_HOST=website1.uk
 +
networks:
 +
  default:
 +
    external:
 +
      name: nginx-proxy
  
docker cp db_name.sql mysql_container_name:/tmp/
+
'''Multiple Virtual Host Web Sites'''
  
Log in to the database container...
+
<code>~/nginx/docker-compose.yml</code>
  
  docker exec -it mysql_container_name bash
+
  version: "3"
 
+
services:
Check and if necessary, change the timezone...
+
  nginx-website1.uk:
 
+
    image: nginx
date
+
    container_name: nginx-website1.uk
mv /etc/localtime /etc/localtime.backup
+
    expose:
ln -s /usr/share/zoneinfo/Europe/London /etc/localtime
+
      - "80"
  date
+
    volumes:
 +
      - /var/www/website1.uk/html:/usr/share/nginx/html:ro
 +
    environment:
 +
      - VIRTUAL_HOST=website1.uk
 +
  nginx-website2.uk:
 +
    image: nginx
 +
    container_name: nginx-website2.uk
 +
    expose:
 +
      - "80"
 +
    volumes:
 +
      - /var/www/website2.uk/html:/usr/share/nginx/html:ro
 +
    environment:
 +
      - VIRTUAL_HOST=website2.uk
 +
  networks:
 +
  default:
 +
    external:
 +
      name: nginx-proxy
  
Delete and create the database...
+
'''Viewing Logs'''
  
  mysql -u root -p -e "DROP DATABASE db_name; CREATE DATABASE db_name;"
+
  docker-compose logs nginx-website1.uk
 +
docker-compose logs nginx-website2.uk
  
Import the database from the sql file, check and exit out of the container...
+
'''Proxy Manager'''
  
mysql -u root -p mysql_db_name < /tmp/db_name.sql
+
This is a web front end to manage 'nginx-proxy', where you can choose containers and create SSL certificates etc.
mysql -u root -p -e "use db_name; show tables;"
 
rm /tmp/db_name.sql
 
exit
 
  
Edit the wp-config.php on your host server to match new DB_HOST and also add extra variables to be sure...
+
https://cyberhost.uk/npm-setup/
  
nano /path/to/docker/folder/html/wp-config.php
+
'''Various'''
define( 'WP_HOME', <nowiki>'http://staging.domain.co.uk'</nowiki> );
 
define( 'WP_SITEURL', <nowiki>'http://staging.domain.co.uk'</nowiki> );
 
  
Install [http://wiki.indie-it.com/wiki/WordPress#Install WordPress CLI] in the running container...
+
https://hub.docker.com/_/nginx
  
docker exec -it wordpress_container_name bash
+
https://blog.ssdnodes.com/blog/host-multiple-websites-docker-nginx/
  
Search and replace the original site url...
+
https://github.com/nginx-proxy/nginx-proxy
  
./wp --allow-root search-replace <nowiki>'http://www.domain.co.uk/' 'http://staging.domain.co.uk/'</nowiki> --dry-run
+
=== Typical LEMP ===
./wp --allow-root search-replace <nowiki>'http://www.domain.co.uk/' 'http://staging.domain.co.uk/'</nowiki>
 
  
Start your web browser and go to the test staging web site!
+
https://adoltech.com/blog/how-to-set-up-nginx-php-fpm-and-mysql-with-docker-compose/
  
==== WordPress CLI ====
+
=== WordPress ===
  
In your stack, set up the usual two DB + WordPress containers, then add a third services section for wp-cli...
+
https://hub.docker.com/_/wordpress/
  
  version: "3"
+
==== PHP File Uploads Fix ====
  services:
+
 
  www.domain.uk-wordpress_db:
+
Create a new PHP configuration file, and name it docker-uploads.ini. Add the following configuration then save the changes.
    image: mysql:5.7
+
 
    container_name: www.domain.uk-wordpress_db
+
  # Allow HTTP file uploads
    volumes:
+
file_uploads = On
      - ./data/db:/var/lib/mysql
+
    restart: always
+
# Maximum size of an uploaded file
    environment:
+
upload_max_filesize = 64M
      - MYSQL_ROOT_PASSWORD=password
+
      - MYSQL_DATABASE=wordpress
+
# Maximum size of form post data
      - MYSQL_USER=wordpress
+
  post_max_size = 64M
      - MYSQL_PASSWORD=password
+
 
  www.domain.uk-wordpress:
+
Update the docker-compose.yml to bind the docker-uploads.ini to the wordpress container and then restart the WordPress container.
    depends_on:
+
 
      - www.domain.uk-wordpress_db
+
volumes:
     image: wordpress:latest
+
  - ./data/config/docker-uploads.ini:/usr/local/etc/php/conf.d/docker-uploads.ini
     container_name: www.domain.uk-wordpress
+
 
 +
==== WordPress Clone ====
 +
 
 +
Create your A record in DNS using [http://wiki.indie-it.com/wiki/AWS_Route_53#cli53 AWS Route 53 CLI]...
 +
 
 +
cli53 rrcreate domain.co.uk 'staging 300 A 123.456.78.910'
 +
 
 +
Create your docker folder for the cloned staging test web site...
 +
 
 +
mkdir -p ~/docker/stacks/staging.domain.co.uk/data/{db,html}
 +
 
 +
Edit your docker compose file, with 2 containers, making sure you use the same network as your Nginx Proxy Manager...
 +
 
 +
<code>~/docker/stacks/staging.domain.co.uk/docker-compose.yml</code>
 +
 
 +
version: "3"
 +
services:
 +
  staging.domain.co.uk-wordpress_db:
 +
     image: mysql:5.7
 +
     container_name: staging.domain.co.uk-wordpress_db
 
     volumes:
 
     volumes:
       - ./data/html:/var/www/html
+
       - ./data/db:/var/lib/mysql
    expose:
 
      - 80
 
 
     restart: always
 
     restart: always
 
     environment:
 
     environment:
       - VIRTUAL_HOST=www.domain.uk
+
      - TZ=Europe/London
       - WORDPRESS_DB_HOST=www.domain.uk-wordpress_db:3306
+
      - MYSQL_ROOT_PASSWORD=changeme
       - WORDPRESS_DB_USER=wordpress
+
      - MYSQL_DATABASE=dbname
       - WORDPRESS_DB_PASSWORD=password
+
      - MYSQL_USER=dbuser
   '''www.domain.uk-wordpress-cli:'''
+
      - MYSQL_PASSWORD=changeme
 +
  staging.domain.co.uk-wordpress:
 +
    depends_on:
 +
      - staging.domain.co.uk-wordpress_db
 +
    image: wordpress:latest
 +
    container_name: staging.domain.co.uk-wordpress
 +
    volumes:
 +
      - ./data/html:/var/www/html
 +
    restart: always
 +
    environment:
 +
      - TZ=Europe/London
 +
       - VIRTUAL_HOST=staging.domain.co.uk
 +
       - WORDPRESS_DB_HOST=staging.domain.co.uk-wordpress_db:3306
 +
      - WORDPRESS_DB_NAME=dbname
 +
       - WORDPRESS_DB_USER=dbuser
 +
       - WORDPRESS_DB_PASSWORD=changeme
 +
   staging.domain.co.uk-wordpress-cli:
 
     image: wordpress:cli
 
     image: wordpress:cli
     container_name: www.domain.uk-wordpress-cli
+
     container_name: staging.domain.co.uk-wordpress-cli
 
     volumes:
 
     volumes:
 
       - ./data/html:/var/www/html
 
       - ./data/html:/var/www/html
 
     environment:
 
     environment:
 +
      - TZ=Europe/London
 
       - WP_CLI_CACHE_DIR=/tmp/
 
       - WP_CLI_CACHE_DIR=/tmp/
       - VIRTUAL_HOST=www.domain.uk
+
       - VIRTUAL_HOST=staging.domain.co.uk
       - WORDPRESS_DB_HOST=www.domain.uk-wordpress_db:3306
+
       - WORDPRESS_DB_HOST=staging.domain.co.uk-wordpress_db:3306
       - WORDPRESS_DB_USER=wordpress
+
      - WORDPRESS_DB_NAME=dbname
       - WORDPRESS_DB_PASSWORD=password
+
       - WORDPRESS_DB_USER=dbuser
 +
       - WORDPRESS_DB_PASSWORD=changeme
 
     working_dir: /var/www/html
 
     working_dir: /var/www/html
 
     user: "33:33"
 
     user: "33:33"
Line 936: Line 1,186:
 
       name: nginx-proxy-manager
 
       name: nginx-proxy-manager
  
...then start it all up.
+
Start containers with correct settings and credentials for existing live web site (so that the docker startup script sets up the MySQL permissions)...
  
 
  docker-compose up -d
 
  docker-compose up -d
  
Then, run your wp-cli commands (e.g. wp user list) on the end of a docker run command...
+
Check the logs to make sure all is well...
  
  docker-compose run --rm www.domain.uk-wordpress-cli '''wp --info'''
+
  docker logs staging.domain.co.uk-wordpress
docker-compose run --rm www.domain.uk-wordpress-cli '''wp cli version'''
+
  docker logs staging.domain.co.uk-wordpress_db
docker-compose run --rm www.domain.uk-wordpress-cli '''wp user list'''
 
  docker-compose run --rm www.domain.uk-wordpress-cli '''wp help theme'''
 
docker-compose run --rm www.domain.uk-wordpress-cli '''wp theme delete --all'''
 
  
=== Email Server (mailu) ===
+
Copy the WordPress files to the host folder and correct ownership...
  
Mailu is a simple yet full-featured mail server as a set of Docker images. It is free software (both as in free beer and as in free speech), open to suggestions and external contributions. The project aims at providing people with an easily setup, easily maintained and full-featured mail server while not shipping proprietary software nor unrelated features often found in popular groupware.
+
rsync -av /path/to/backup_unzipped_wordpress/ ~/docker/stacks/staging.domain.co.uk/html/
 +
chown -R www-data:www-data ~/docker/stacks/staging.domain.co.uk/html/
  
https://mailu.io/1.7/
+
Copy the sql file in to the running '''mysql''' container...
  
https://hub.docker.com/u/mailu
+
docker cp /path/to/backup_unzipped_wordpress/db_name.sql mysql_container_name:/tmp/
  
https://github.com/Mailu/Mailu
+
Log in to the database container...
  
=== Email Server (docker-mailserver) ===
+
docker exec -it mysql_container_name bash
  
https://github.com/docker-mailserver/docker-mailserver
+
Check and if necessary, change the timezone...
  
https://docker-mailserver.github.io/docker-mailserver/edge/config/security/ssl/#lets-encrypt-recommended
+
date
 
+
mv /etc/localtime /etc/localtime.backup
==== Postgrey ====
+
ln -s /usr/share/zoneinfo/Europe/London /etc/localtime
 +
date
  
[https://github.com/docker-mailserver/docker-mailserver/issues/1957 Local List]
+
Delete and create the database...
  
==== SpamAssassin ====
+
mysql -u root -p -e "DROP DATABASE db_name; CREATE DATABASE db_name;"
  
[https://docker-mailserver.github.io/docker-mailserver/edge/faq/#how-can-i-manage-my-custom-spamassassin-rules Custom Rules]
+
Import the database from the sql file, check and exit out of the container...
  
[https://github.com/docker-mailserver/docker-mailserver/issues/365 Bayes Database]
+
mysql -u root -p mysql_db_name < /tmp/db_name.sql
 +
mysql -u root -p -e "use db_name; show tables;"
 +
rm /tmp/db_name.sql
 +
exit
  
SpamAssassin is controlled by Amavis (a fork of MailScanner) with the user 'amavis'.
+
Edit the wp-config.php on your host server to match new DB_HOST and also add extra variables to be sure...
  
'''Show Bayes Database Stats'''
+
nano /path/to/docker/folder/html/wp-config.php
 +
define( 'WP_HOME', <nowiki>'http://staging.domain.co.uk'</nowiki> );
 +
define( 'WP_SITEURL', <nowiki>'http://staging.domain.co.uk'</nowiki> );
  
docker exec --user amavis mail.mydomain.org.uk-mailserver sa-learn --dump magic --dbpath /var/lib/amavis/.spamassassin
+
Install [http://wiki.indie-it.com/wiki/WordPress#Install WordPress CLI] in the running container...
  
'''Learn Ham'''
+
docker exec -it wordpress_container_name bash
  
docker exec --user amavis mail.mydomain.org.uk-mailserver sa-learn --ham --progress /var/mail/mydomain.org.uk/info/cur --dbpath /var/lib/amavis/.spamassassin
+
Search and replace the original site url...
  
==== Backups ====
+
./wp --allow-root search-replace <nowiki>'http://www.domain.co.uk/' 'http://staging.domain.co.uk/'</nowiki> --dry-run
 +
./wp --allow-root search-replace <nowiki>'http://www.domain.co.uk/' 'http://staging.domain.co.uk/'</nowiki>
  
[https://docker-mailserver.github.io/docker-mailserver/edge/faq/#what-about-backups Backups]
+
Start your web browser and go to the test staging web site!
  
==== Autodiscover ====
+
==== WordPress CLI ====
  
Create SRV and A record entries in your DNS for the services...
+
In your stack, set up the usual two DB + WordPress containers, then add a third services section for wp-cli...
 
 
$ORIGIN domain.org.uk.
 
@ 300 IN TXT "v=spf1 mx ~all; mailconf=<nowiki>https://autoconfig.domain.org.uk/mail/config-v1.1.xml</nowiki>"
 
_autodiscover._tcp 300 IN SRV 0 0 443 autodiscover.domain.org.uk.
 
_imap._tcp 300 IN SRV 0 0 0 .
 
_imaps._tcp 300 IN SRV 0 1 993 mail.domain.org.uk.
 
_ldap._tcp 300 IN SRV 0 0 636 mail.domain.org.uk.
 
_pop3._tcp 300 IN SRV 0 0 0 .
 
_pop3s._tcp 300 IN SRV 0 0 0 .
 
_submission._tcp 300 IN SRV 0 1 587 mail.domain.org.uk.
 
autoconfig 300 IN A 3.10.67.19
 
autodiscover 300 IN A 3.10.67.19
 
imap 300 IN CNAME mail
 
mail 300 IN A 3.10.67.19
 
smtp 300 IN CNAME mail
 
www 300 IN A 3.10.67.19
 
 
 
<code>docker-compose.yml</code>
 
  
 +
version: "3"
 
  services:
 
  services:
   mailserver-autodiscover:
+
   www.domain.uk-wordpress_db:
     image: monogramm/autodiscover-email-settings:latest
+
     image: mysql:5.7
     container_name: mail.domain.org.uk-mailserver-autodiscover
+
     container_name: www.domain.uk-wordpress_db
 +
    volumes:
 +
      - ./data/db:/var/lib/mysql
 +
    restart: always
 
     environment:
 
     environment:
       - COMPANY_NAME=My Company
+
       - MYSQL_ROOT_PASSWORD=password
       - SUPPORT_URL=<nowiki>https://autodiscover.domain.org.uk</nowiki>
+
       - MYSQL_DATABASE=wordpress
       - DOMAIN=domain.org.uk
+
      - MYSQL_USER=wordpress
       - IMAP_HOST=mail.domain.org.uk
+
      - MYSQL_PASSWORD=password
       - IMAP_PORT=993
+
  www.domain.uk-wordpress:
       - IMAP_SOCKET=SSL
+
    depends_on:
       - SMTP_HOST=mail.domain.org.uk
+
      - www.domain.uk-wordpress_db
       - SMTP_PORT=587
+
    image: wordpress:latest
       - SMTP_SOCKET=STARTTLS
+
    container_name: www.domain.uk-wordpress
     restart: unless-stopped
+
    volumes:
  networks:
+
      - ./data/html:/var/www/html
   default:
+
    expose:
 +
       - 80
 +
    restart: always
 +
    environment:
 +
      - VIRTUAL_HOST=www.domain.uk
 +
      - WORDPRESS_DB_HOST=www.domain.uk-wordpress_db:3306
 +
       - WORDPRESS_DB_NAME=wordpress
 +
      - WORDPRESS_DB_USER=wordpress
 +
      - WORDPRESS_DB_PASSWORD=password
 +
  '''www.domain.uk-wordpress-cli:'''
 +
    image: wordpress:cli
 +
    container_name: www.domain.uk-wordpress-cli
 +
    volumes:
 +
       - ./data/html:/var/www/html
 +
    environment:
 +
      - WP_CLI_CACHE_DIR=/tmp/
 +
       - VIRTUAL_HOST=www.domain.uk
 +
       - WORDPRESS_DB_HOST=www.domain.uk-wordpress_db:3306
 +
      - WORDPRESS_DB_NAME=wordpress
 +
       - WORDPRESS_DB_USER=wordpress
 +
       - WORDPRESS_DB_PASSWORD=password
 +
    working_dir: /var/www/html
 +
     user: "33:33"
 +
  networks:
 +
   default:
 
     external:
 
     external:
 
       name: nginx-proxy-manager
 
       name: nginx-proxy-manager
  
[https://docker-mailserver.github.io/docker-mailserver/edge/config/best-practices/autodiscover/ Autodiscover]
+
...then start it all up.
  
[https://hub.docker.com/r/monogramm/autodiscover-email-settings/ monogramm/autodiscover-email-settings]
+
docker-compose up -d
  
=== Internet Speedtest ===
+
Then, run your wp-cli commands (e.g. wp user list) on the end of a docker run command...
  
https://github.com/henrywhitaker3/Speedtest-Tracker
+
docker-compose run --rm www.domain.uk-wordpress-cli '''wp --info'''
 +
docker-compose run --rm www.domain.uk-wordpress-cli '''wp cli version'''
 +
docker-compose run --rm www.domain.uk-wordpress-cli '''wp user list'''
 +
docker-compose run --rm www.domain.uk-wordpress-cli '''wp help theme'''
 +
docker-compose run --rm www.domain.uk-wordpress-cli '''wp theme delete --all'''
  
=== Emby Media Server ===
+
==== SSL Behind A Reverse Proxy ====
  
https://emby.media/docker-server.html
+
https://wiki.indie-it.com/wiki/WordPress#SSL_When_Using_A_Reverse_Proxy
  
https://hub.docker.com/r/emby/embyserver
+
=== Email Server (mailu) ===
  
=== AWS CLI ===
+
Mailu is a simple yet full-featured mail server as a set of Docker images. It is free software (both as in free beer and as in free speech), open to suggestions and external contributions. The project aims at providing people with an easily setup, easily maintained and full-featured mail server while not shipping proprietary software nor unrelated features often found in popular groupware.
  
docker run --rm -it -v "/root/.aws:/root/.aws" amazon/aws-cli configure
+
https://mailu.io/1.7/
docker run --rm -it -v "/root/.aws:/root/.aws" amazon/aws-cli s3 ls
 
docker run --rm -it -v "/root/.aws:/root/.aws" amazon/aws-cli route53 list-hosted-zones
 
  
https://docs.aws.amazon.com/cli/latest/userguide/install-cliv2-docker.html
+
https://hub.docker.com/u/mailu
  
=== Let's Encrypt ===
+
https://github.com/Mailu/Mailu
  
Issue a wildcard certificate...
+
'''Postfix Admin'''
  
docker run -it --rm --name certbot -v "/usr/bin:/usr/bin" -v "/root/.aws:/root/.aws" -v "/etc/letsencrypt:/etc/letsencrypt" -v "/var/lib/letsencrypt:/var/lib/letsencrypt" certbot/dns-route53 certonly --dns-route53 --domain "example.com" --domain "*.example.com"
+
https://hub.docker.com/_/postfixadmin
  
Check your certificates...
+
=== Email Server (docker-mailserver) ===
  
docker run -it --rm --name certbot -v "/usr/bin:/usr/bin" -v "/root/.aws:/root/.aws" -v "/etc/letsencrypt:/etc/letsencrypt" -v "/var/lib/letsencrypt:/var/lib/letsencrypt" certbot/certbot certificates
+
https://github.com/docker-mailserver
  
https://certbot.eff.org/docs/install.html#running-with-docker
+
https://github.com/docker-mailserver/docker-mailserver
  
=== VPN ===
+
https://github.com/docker-mailserver/docker-mailserver-admin
  
==== OpenVPN ====
+
https://docker-mailserver.github.io/docker-mailserver/edge/config/security/ssl/#lets-encrypt-recommended
  
'''Server'''
+
==== Postscreen ====
 +
 
 +
Postscreen is an SMTP filter that blocks spambots (or zombie machines) away from the real Postfix smtpd daemon, so Postfix does not feel overloaded and can process legitimate emails more efficiently.
 +
 
 +
The example below shows a typical spambot attempt at accessing the SMTP service and being stopped...
 +
 
 +
Jun 24 10:42:26 mail postfix/postscreen[203907]: CONNECT from [212.70.149.56]:19452 to [172.23.0.2]:25
 +
Jun 24 10:42:26 mail postfix/dnsblog[386054]: addr 212.70.149.56 listed by domain b.barracudacentral.org as 127.0.0.2
 +
Jun 24 10:42:26 mail postfix/dnsblog[402550]: addr 212.70.149.56 listed by domain list.dnswl.org as 127.0.10.3
 +
Jun 24 10:42:26 mail postfix/dnsblog[407802]: addr 212.70.149.56 listed by domain bl.mailspike.net as 127.0.0.2
 +
Jun 24 10:42:26 mail postfix/dnsblog[386155]: addr 212.70.149.56 listed by domain psbl.surriel.com as 127.0.0.2
 +
Jun 24 10:42:29 mail postfix/postscreen[203907]: PREGREET 11 after 2.9 from [212.70.149.56]:19452: EHLO User\r\n
 +
Jun 24 10:42:29 mail postfix/postscreen[203907]: DISCONNECT [212.70.149.56]:19452
 +
 
 +
Postscreen is enabled by default but there are a few settings to tweak to get the best out of it.
 +
 
 +
Edit your <code>data/config/postfix-main.cf</code> file and add the following lines, making sure your Docker host IP is in bold...
  
https://hub.docker.com/r/linuxserver/openvpn-as
+
mynetworks = 127.0.0.0/8 [::1]/128 [fe80::]/64 172.19.0.2/32 '''172.19.0.1/32'''
 +
postscreen_greet_action = drop
 +
postscreen_pipelining_enable = yes
 +
postscreen_pipelining_action = drop
 +
postscreen_non_smtp_command_enable = yes
 +
postscreen_non_smtp_command_action = drop
 +
postscreen_bare_newline_enable = yes
 +
postscreen_bare_newline_action = drop
  
'''Client'''
+
[https://www.linuxbabe.com/mail-server/configure-postscreen-in-postfix-to-block-spambots Enable and Configure Postscreen in Postfix to Block Spambots]
  
https://hub.docker.com/r/dperson/openvpn-client
+
==== Postgrey ====
  
'''Routing Containers Through Container'''
+
[https://github.com/docker-mailserver/docker-mailserver/issues/1957 Local List]
  
sudo docker run -it --net=container:vpn -d some/docker-container
+
==== SpamAssassin ====
  
==== WireGuard ====
+
[https://docker-mailserver.github.io/docker-mailserver/edge/faq/#how-can-i-manage-my-custom-spamassassin-rules Custom Rules]
  
<code>docker-compose.yml</code>
+
[https://github.com/docker-mailserver/docker-mailserver/issues/365 Bayes Database]
  
version: "2.1"
+
SpamAssassin is controlled by Amavis (a fork of MailScanner) with the user 'amavis'.
services:
 
  wireguard:
 
    image: ghcr.io/linuxserver/wireguard
 
    container_name: wireguard
 
    cap_add:
 
      - NET_ADMIN
 
      - SYS_MODULE
 
    environment:
 
      - PUID=1000
 
      - PGID=1000
 
      - TZ=Europe/London
 
      - SERVERURL=wireguard.domain.uk
 
      - SERVERPORT=51820
 
      - PEERS=3
 
      - PEERDNS=auto
 
      - INTERNAL_SUBNET=10.13.13.0
 
      - ALLOWEDIPS=0.0.0.0/0
 
    volumes:
 
      - ./data/config:/config
 
      - /lib/modules:/lib/modules
 
    ports:
 
      - 51820:51820/udp
 
    sysctls:
 
      - net.ipv4.conf.all.src_valid_mark=1
 
    restart: unless-stopped
 
  
https://hub.docker.com/r/linuxserver/wireguard
+
'''Show Bayes Database Stats'''
  
'''Error: const struct ipv6_stub'''
+
docker exec --user amavis mail.mydomain.org.uk-mailserver sa-learn --dump magic --dbpath /var/lib/amavis/.spamassassin
  
If you receive an error in the container logs about not being able to compile the kernel module, then follow the instructions to compile the WireGuard kernel module and tools in your host system.
+
'''Learn Ham'''
  
https://github.com/linuxserver/docker-wireguard/issues/46#issuecomment-708278250
+
docker exec --user amavis mail.mydomain.org.uk-mailserver sa-learn --ham --progress /var/mail/mydomain.org.uk/info/cur --dbpath /var/lib/amavis/.spamassassin
  
=== ffmpeg ===
+
'''Backup and Restore from Existing Mail Server'''
  
https://registry.hub.docker.com/r/jrottenberg/ffmpeg
+
On the old server...
  
https://github.com/jrottenberg/ffmpeg
+
/bin/su -l -c '/usr/bin/sa-learn --backup > sa-learn_backup.txt' debian-spamd
 +
rsync -avP /var/lib/spamassassin/sa-learn_backup.txt user@mail.mydomain.org.uk:/tmp/
  
https://medium.com/coconut-stories/using-ffmpeg-with-docker-94523547f35c
+
On the new server...
  
https://github.com/linuxserver/docker-ffmpeg
+
docker cp /tmp/sa-learn_backup.txt mail.mydomain.org.uk-mailserver:/tmp/
 +
docker exec --user amavis mail.mydomain.org.uk-mailserver sa-learn --sync --dbpath /var/lib/amavis/.spamassassin
 +
docker exec --user amavis mail.mydomain.org.uk-mailserver sa-learn --clear --dbpath /var/lib/amavis/.spamassassin
 +
docker exec --user amavis mail.mydomain.org.uk-mailserver sa-learn --restore /tmp/sa-learn_backup.txt --dbpath /var/lib/amavis/.spamassassin
 +
docker exec --user amavis mail.mydomain.org.uk-mailserver sa-learn --sync --dbpath /var/lib/amavis/.spamassassin
 +
docker exec --user amavis mail.mydomain.org.uk-mailserver sa-learn --dump magic --dbpath /var/lib/amavis/.spamassassin
 +
 
 +
==== Fail2Ban ====
 +
 
 +
List jails...
 +
 
 +
docker exec -it mail.mydomain.org.uk-mailserver fail2ban-client status
 +
 +
Status
 +
|- Number of jail:  3
 +
`- Jail list:  dovecot, postfix, postfix-sasl
 +
 
 +
Manually ban IP address in named jail...
 +
 
 +
docker exec -it mail.mydomain.org.uk-mailserver fail2ban-client set postfix banip 212.70.149.56
  
=== MakeMKV ===
+
Check banned IPs...
  
'''This will NOT work on a Raspberry Pi.'''
+
docker exec -it mail.mydomain.org.uk-mailserver fail2ban-client status postfix
 +
 +
Status for the jail: postfix
 +
|- Filter
 +
|  |- Currently failed: 2
 +
|  |- Total failed:    2
 +
|  `- File list:        /var/log/mail.log
 +
`- Actions
 +
    |- Currently banned: 1
 +
    |- Total banned:    1
 +
    `- Banned IP list:  212.70.149.56
  
https://github.com/jlesage/docker-makemkv
+
https://www.the-lazy-dev.com/en/install-fail2ban-with-docker/
  
Use this in combination with [[FFmpeg_DVD|ffmpeg]] or [[HandBrake]] (as shown below) and [[FileBot]] to process your media through to your media server - like [[Emby]] or [[Plex]]..
+
==== Backups ====
  
MakeMKV > HandBrake > FileBot > Emby
+
[https://docker-mailserver.github.io/docker-mailserver/edge/faq/#what-about-backups Backups]
  
To make this work with your DVD drive (/dev/sr0) '''you need to have the second device''' (/dev/sg0) in order for it to work. I don't get it, but it works.
+
==== Autodiscover ====
  
<code>/root/docker/stacks/makemkv/docker-compose.yml</code>
+
Create SRV and A record entries in your DNS for the services...
  
  version: '3'
+
  $ORIGIN domain.org.uk.
  services:
+
  @ 300 IN TXT "v=spf1 mx ~all; mailconf=<nowiki>https://autoconfig.domain.org.uk/mail/config-v1.1.xml</nowiki>"
  makemkv:
+
_autodiscover._tcp 300 IN SRV 0 0 443 autodiscover.domain.org.uk.
    image: jlesage/makemkv
+
_imap._tcp 300 IN SRV 0 0 0 .
    container_name: makemkv
+
_imaps._tcp 300 IN SRV 0 1 993 mail.domain.org.uk.
    ports:
+
_ldap._tcp 300 IN SRV 0 0 636 mail.domain.org.uk.
      - "0.0.0.0:5801:5800"
+
_pop3._tcp 300 IN SRV 0 0 0 .
    volumes:
+
_pop3s._tcp 300 IN SRV 0 0 0 .
      - "/home/user/.MakeMKV_DOCKER:/config:rw"
+
_submission._tcp 300 IN SRV 0 1 587 mail.domain.org.uk.
      - "/home/user/:/storage:ro"
+
autoconfig 300 IN A 3.10.67.19
      - "/home/user/ToDo/MakeMKV/output:/output:rw"
+
autodiscover 300 IN A 3.10.67.19
    devices:
+
imap 300 IN CNAME mail
      - "/dev/sr0:/dev/sr0"
+
mail 300 IN A 3.10.67.19
      - "/dev/sg0:/dev/sg0"
+
smtp 300 IN CNAME mail
    environment:
+
www 300 IN A 3.10.67.19
      - USER_ID=1000
 
      - GROUP_ID=1000
 
      - TZ=Europe/London
 
      - MAKEMKV_KEY=your_licence_key
 
      - AUTO_DISC_RIPPER=1
 
  
'''Troubleshooting'''
+
<code>docker-compose.yml</code>
  
PROBLEM = "driver failed programming external connectivity on endpoint makemkv: Error starting userland proxy: listen tcp6 [::]:5800: socket: address family not supported by protocol."
+
services:
 +
  mailserver-autodiscover:
 +
    image: monogramm/autodiscover-email-settings:latest
 +
    container_name: mail.domain.org.uk-mailserver-autodiscover
 +
    environment:
 +
      - COMPANY_NAME=My Company
 +
      - SUPPORT_URL=<nowiki>https://autodiscover.domain.org.uk</nowiki>
 +
      - DOMAIN=domain.org.uk
 +
      - IMAP_HOST=mail.domain.org.uk
 +
      - IMAP_PORT=993
 +
      - IMAP_SOCKET=SSL
 +
      - SMTP_HOST=mail.domain.org.uk
 +
      - SMTP_PORT=587
 +
      - SMTP_SOCKET=STARTTLS
 +
    restart: unless-stopped
 +
networks:
 +
  default:
 +
    external:
 +
      name: nginx-proxy-manager
  
SOLUTION = Put 0.0.0.0:5801 in the published ports line of docker compose to restrict the network to IPv4.
+
[https://docker-mailserver.github.io/docker-mailserver/edge/config/best-practices/autodiscover/ Autodiscover]
  
'''Docker Process Output'''
+
[https://hub.docker.com/r/monogramm/autodiscover-email-settings/ monogramm/autodiscover-email-settings]
  
CONTAINER ID  IMAGE              COMMAND  CREATED        STATUS        PORTS                              NAMES
+
=== Internet Speedtest ===
4a8b3106b00b  jlesage/handbrake  "/init"  40 hours ago  Up 40 hours  5900/tcp, 0.0.0.0:5802->5800/tcp  handbrake
 
89fe3ba8a31e  jlesage/makemkv    "/init"  40 hours ago  Up 40 hours  5900/tcp, 0.0.0.0:5801->5800/tcp  makemkv
 
  
=== HandBrake ===
+
https://github.com/henrywhitaker3/Speedtest-Tracker
  
'''This will NOT work on a Raspberry Pi.'''
+
=== Emby Media Server ===
 +
 
 +
https://emby.media/docker-server.html
 +
 
 +
https://hub.docker.com/r/emby/embyserver
 +
 
 +
=== AWS CLI ===
  
Use this in combination with [[FFmpeg_DVD|ffmpeg]] or [[MakeMKV]] (as shown below) and [[FileBot]] to process your media through to your media server - like [[Emby]] or [[Plex]]..
+
docker run --rm -it -v "/root/.aws:/root/.aws" amazon/aws-cli configure
 +
docker run --rm -it -v "/root/.aws:/root/.aws" amazon/aws-cli s3 ls
 +
docker run --rm -it -v "/root/.aws:/root/.aws" amazon/aws-cli route53 list-hosted-zones
  
I have changed the port from 5800 to 5802 because Jocelyn's other Docker image for MakeMKV uses the same port (so I move that one as well to 5801 - see above).
+
https://docs.aws.amazon.com/cli/latest/userguide/install-cliv2-docker.html
  
To make this work with your DVD drive (/dev/sr0) '''you need to have the second device''' (/dev/sg0) in order for it to work. I don't get it, but it works.
+
=== Let's Encrypt ===
  
[https://www.youtube.com/watch?v=yKww_gg1fVI YouTube / DB Tech - How to install HandBrake in Docker]
+
Force RENEW a standalone certificate with the new preferred chain of "ISRG Root X1"
  
[https://dbtechreviews.com/2020/03/how-to-install-handbrake-in-openmediavault-and-docker/ Blog / DB Tech - How to install HandBrake in Docker]
+
docker run -it --rm --name certbot -v "/usr/bin:/usr/bin" -v "/root/.aws:/root/.aws" -v "/etc/letsencrypt:/etc/letsencrypt" -v "/var/lib/letsencrypt:/var/lib/letsencrypt" certbot/certbot --force-renewal --preferred-chain "ISRG Root X1" certonly --standalone --email me@mydomain.com --agree-tos -d www.mydomain.com
  
[https://github.com/jlesage/docker-handbrake Docker HandBrake by Jocelyn Le Sage]
+
Issue a wildcard certificate...
  
[https://hub.docker.com/r/jlesage/handbrake Docker Image by Jocelyn Le Sage]
+
docker run -it --rm --name certbot -v "/usr/bin:/usr/bin" -v "/root/.aws:/root/.aws" -v "/etc/letsencrypt:/etc/letsencrypt" -v "/var/lib/letsencrypt:/var/lib/letsencrypt" certbot/dns-route53 certonly --dns-route53 --domain "example.com" --domain "*.example.com"
  
<code>/root/docker/stacks/handbrake/docker-compose.yml</code>
+
Check your certificates...
  
  version: '3'
+
  docker run -it --rm --name certbot -v "/usr/bin:/usr/bin" -v "/root/.aws:/root/.aws" -v "/etc/letsencrypt:/etc/letsencrypt" -v "/var/lib/letsencrypt:/var/lib/letsencrypt" certbot/certbot certificates
services:
 
  handbrake:
 
    image: jlesage/handbrake
 
    container_name: handbrake
 
    ports:
 
      - "0.0.0.0:5802:5800"
 
    volumes:
 
      - "/home/user:/storage:ro"
 
      - "/home/user/ToDo/HandBrake/config:/config:rw"
 
      - "/home/user/ToDo/HandBrake/watch:/watch:rw"
 
      - "/home/user/ToDo/HandBrake/output:/output:rw"
 
    devices:
 
      - "/dev/sr0:/dev/sr0"
 
      - "/dev/sg0:/dev/sg0"
 
    environment:
 
      - USER_ID=1000
 
      - GROUP_ID=1000
 
      - TZ=Europe/London
 
  
'''Docker Process Output'''
+
Renew a certificate...
  
  CONTAINER ID  IMAGE              COMMAND  CREATED        STATUS        PORTS                              NAMES
+
  docker run -it --rm --name certbot -v "/usr/bin:/usr/bin" -v "/root/.aws:/root/.aws" -v "/etc/letsencrypt:/etc/letsencrypt" -v "/var/lib/letsencrypt:/var/lib/letsencrypt" certbot/dns-route53 renew
4a8b3106b00b  jlesage/handbrake  "/init"  40 hours ago  Up 40 hours  5900/tcp, 0.0.0.0:5802->5800/tcp  handbrake
 
89fe3ba8a31e  jlesage/makemkv    "/init"   40 hours ago  Up 40 hours  5900/tcp, 0.0.0.0:5801->5800/tcp  makemkv
 
  
=== Automated Downloader ===
+
If you have multiple profiles in your .aws/config then you will need to pass the AWS_PROFILE variable to the docker container...
  
This takes the hassle out of going through the various web sites to find stuff and be bombarded with ads and pop-ups.
+
docker run -it --rm --name certbot -v "/usr/bin:/usr/bin" -v "/root/.aws:/root/.aws" -v "/etc/letsencrypt:/etc/letsencrypt" -v "/var/lib/letsencrypt:/var/lib/letsencrypt" -e '''"AWS_PROFILE=certbot"''' certbot/dns-route53 renew
  
# Jackett
+
https://certbot.eff.org/docs/install.html#running-with-docker
# Radarr
 
# Transmission
 
  
Jackett > Radarr > Transmission
+
=== VPN ===
  
Create a docker network which Jackett and Radarr share to talk to each other...
+
==== Gluetun ====
  
sudo docker network create jackett-radarr
+
Gluetun VPN client
  
...then continue setting up the containers below.
+
Lightweight swiss-knife-like VPN client to tunnel to Cyberghost, ExpressVPN, FastestVPN, HideMyAss, IPVanish, IVPN, Mullvad, NordVPN, Perfect Privacy, Privado, Private Internet Access, PrivateVPN, ProtonVPN, PureVPN, Surfshark, TorGuard, VPNUnlimited, VyprVPN, WeVPN and Windscribe VPN servers using Go, OpenVPN or Wireguard, iptables, DNS over TLS, ShadowSocks and an HTTP proxy.
  
==== Jackett ====
+
[https://github.com/qdm12/gluetun Github]
  
Jackett works as a proxy server: it translates queries from apps (Sonarr, SickRage, CouchPotato, Mylar, etc) into tracker-site-specific http queries, parses the html response, then sends results back to the requesting software. This allows for getting recent uploads (like RSS) and performing searches. Jackett is a single repository of maintained indexer scraping and translation logic - removing the burden from other apps.
+
[https://github.com/qdm12/gluetun/wiki/Connect-a-container-to-gluetun Connect a container to Gluetun]
  
'''So, this is where you build your list of web sites "with content you want" ;-)'''
+
==== OpenVPN ====
  
https://fleet.linuxserver.io/image?name=linuxserver/jackett
+
'''Server'''
  
https://docs.linuxserver.io/images/docker-jackett
+
https://hub.docker.com/r/linuxserver/openvpn-as
  
https://hub.docker.com/r/linuxserver/jackett
+
'''Client'''
 +
 
 +
https://hub.docker.com/r/dperson/openvpn-client
 +
 
 +
'''Routing Containers Through Container'''
 +
 
 +
sudo docker run -it --net=container:vpn -d some/docker-container
 +
 
 +
'''OpenVPN-PiHole'''
 +
 
 +
https://github.com/Simonwep/openvpn-pihole
 +
 
 +
==== WireHole ====
 +
 
 +
WireHole is a combination of WireGuard, PiHole, and Unbound in a docker-compose project with the intent of enabling users to quickly and easily create and deploy a personally managed full or split-tunnel WireGuard VPN with ad blocking capabilities (via Pihole), and DNS caching with additional privacy options (via Unbound).
 +
 
 +
https://github.com/IAmStoxe/wirehole
 +
 
 +
To view a QR code, run this ...
 +
 
 +
docker exec -it wireguard /app/show-peer 1
 +
 
 +
==== WireGuard ====
  
https://github.com/Jackett/Jackett
+
'''Use [https://wiki.indie-it.com/wiki/Docker#WireHole WireHole] instead!'''
  
<code>/root/docker/stacks/docker-compose.yml</code>
+
<code>docker-compose.yml</code>
  
 
  version: "2.1"
 
  version: "2.1"
 
  services:
 
  services:
   jackett:
+
   wireguard:
     image: ghcr.io/linuxserver/jackett
+
     image: ghcr.io/linuxserver/wireguard
     container_name: jackett
+
     container_name: wireguard
 +
    cap_add:
 +
      - NET_ADMIN
 +
      - SYS_MODULE
 
     environment:
 
     environment:
 
       - PUID=1000
 
       - PUID=1000
 
       - PGID=1000
 
       - PGID=1000
 
       - TZ=Europe/London
 
       - TZ=Europe/London
       - AUTO_UPDATE=true
+
       - SERVERURL=wireguard.domain.uk
 +
      - SERVERPORT=51820
 +
      - PEERS=3
 +
      - PEERDNS=auto
 +
      - INTERNAL_SUBNET=10.13.13.0
 +
      - ALLOWEDIPS=0.0.0.0/0
 
     volumes:
 
     volumes:
 
       - ./data/config:/config
 
       - ./data/config:/config
       - ./data/downloads:/downloads
+
       - /lib/modules:/lib/modules
    networks:
 
      - jackett-radarr
 
 
     ports:
 
     ports:
       - 0.0.0.0:9117:9117
+
       - 51820:51820/udp
 +
    sysctls:
 +
      - net.ipv4.conf.all.src_valid_mark=1
 
     restart: unless-stopped
 
     restart: unless-stopped
networks:
 
  jackett-radarr:
 
    external: true
 
  
==== Radarr ====
+
https://hub.docker.com/r/linuxserver/wireguard
  
Radarr is a movie collection manager for Usenet and BitTorrent users. It can monitor multiple RSS feeds for new movies and will interface with clients and indexers to grab, sort, and rename them. It can also be configured to automatically upgrade the quality of existing files in the library when a better quality format becomes available.
+
'''To show the QR code'''
  
'''Radarr is the 'man-in-the-middle' to take lists from Jackett and pass them to Transmission to download.'''
+
docker exec -it wireguard /app/show-peer 1
 +
docker exec -it wireguard /app/show-peer 2
 +
docker exec -it wireguard /app/show-peer 3
  
Radarr is the web UI to search for "the content you want" ;-)
+
'''Error: const struct ipv6_stub'''
  
https://radarr.video/
+
If you receive an error in the container logs about not being able to compile the kernel module, then follow the instructions to compile the WireGuard kernel module and tools in your host system.
  
https://docs.linuxserver.io/images/docker-radarr
+
https://github.com/linuxserver/docker-wireguard/issues/46#issuecomment-708278250
  
https://github.com/linuxserver/docker-radarr
+
'''[https://www.youtube.com/watch?v=vUyHGF1HMsw Force Docker Containers to use a VPN for connection]'''
  
https://sasquatters.com/radarr-docker/
+
=== ffmpeg ===
 +
 
 +
docker pull jrottenberg/ffmpeg
 +
docker run jrottenberg/ffmpeg -h
 +
docker run jrottenberg/ffmpeg -i /path/to/input.mkv -stats $ffmpeg_options - > out.mp4
 +
docker run -v $(pwd):$(pwd) -w $(pwd) jrottenberg/ffmpeg -y -i input.mkv -t 00:00:05.00 -vf scale=-1:360 output.mp4
 +
 
 +
https://registry.hub.docker.com/r/jrottenberg/ffmpeg
 +
 
 +
https://github.com/jrottenberg/ffmpeg
 +
 
 +
https://medium.com/coconut-stories/using-ffmpeg-with-docker-94523547f35c
 +
 
 +
https://github.com/linuxserver/docker-ffmpeg
 +
 
 +
=== MediaInfo ===
 +
 
 +
Install ...
 +
 
 +
sudo docker pull jlesage/mediainfo
 +
 
 +
Run ...
  
https://www.smarthomebeginner.com/install-radarr-using-docker/
+
docker run --rm --name=mediainfo -e USER_ID=$(id -u) -e GROUP_ID=$(id -g) -v $(pwd):$(pwd):ro jlesage/mediainfo su-exec "$(id -u):$(id -g)" /usr/bin/mediainfo --help
  
https://trash-guides.info/Radarr/
+
https://github.com/jlesage/docker-mediainfo
  
So, you use Jackett as an Indexer of content, which answers questions from Radarr, which passes a good result to Transmission...
+
=== MKV Toolnix ===
  
# Settings > Profiles > delete all but 'any' (and edit that to get rid of naff qualities at the bottom)
+
Install ...
# Indexers > Add Indexer > Torznab > complete and TEST then SAVE
 
# Download Clients > Add Download Client > Transmission > complete and TEST and SAVE
 
  
<code>/root/docker/stacks/docker-compose.yml</code>
+
sudo docker pull jlesage/mkvtoolnix
  
version: "2.1"
+
Run ...
services:
 
  radarr:
 
    image: ghcr.io/linuxserver/radarr
 
    container_name: radarr
 
    environment:
 
      - PUID=1000
 
      - PGID=1000
 
      - TZ=Europe/London
 
    volumes:
 
      - ./data/config:/config
 
      - ./data/downloads:/downloads
 
      - ./data/torrents:/torrents
 
    networks:
 
      - jackett-radarr
 
    ports:
 
      - 0.0.0.0:7878:7878
 
    restart: unless-stopped
 
networks:
 
  jackett-radarr:
 
    external: true
 
  
==== NZBGet ====
+
'''mkvextract'''
  
Nzbget is a usenet downloader. That's all there is to say on that... even the official home page keeps things quiet.
+
docker run --rm --name=mkvextract -e USER_ID=$(id -u) -e GROUP_ID=$(id -g) -v $(pwd):$(pwd):rw jlesage/mkvtoolnix su-exec "$(id -u):$(id -g)" /usr/bin/mkvextract "$(pwd)/filename.mkv" tracks 3:"$(pwd)/filename.eng.srt"
  
https://nzbget.net
+
'''mkvpropedit'''
  
https://hub.docker.com/r/linuxserver/nzbget
+
docker run --rm --name=mkvextract -e USER_ID=$(id -u) -e GROUP_ID=$(id -g) -v $(pwd):$(pwd):rw jlesage/mkvtoolnix su-exec "$(id -u):$(id -g)" /usr/bin/mkvpropedit "$(pwd)/filename.mkv" --edit track:a1 --set language=eng
  
Webui can be found at <your-ip>:6789 and the default login details (change ASAP) are...
+
https://github.com/jlesage/docker-mkvtoolnix
  
username: nzbget
+
=== MakeMKV ===
password: tegbzn6789
 
  
=== Nagios ===
+
'''This will NOT work on a Raspberry Pi.'''
  
Work in progress.
+
https://github.com/jlesage/docker-makemkv
  
This is an old version of Nagios in the container image, so will look for a newer one.
+
Use this in combination with [[FFmpeg_DVD|ffmpeg]] or [[HandBrake]] (as shown below) and [[FileBot]] to process your media through to your media server - like [[Emby]] or [[Plex]]..
  
<code>/root/docker/stacks/nagios/docker-compose.yml</code>
+
MakeMKV > HandBrake > FileBot > Emby
  
  version: '3'
+
To make this work with your DVD drive (/dev/sr0) '''you need to have the second device''' (/dev/sg0) in order for it to work. I don't get it, but it works.
 +
 
 +
<code>/root/docker/stacks/makemkv/docker-compose.yml</code>
 +
 
 +
  version: '3'
 
  services:
 
  services:
   nagios:
+
   makemkv:
     image: jasonrivers/nagios
+
     image: jlesage/makemkv
     container_name: nagios
+
     container_name: makemkv
    restart: unless-stopped
 
 
     ports:
 
     ports:
       - 8181:80
+
       - "0.0.0.0:5801:5800"
     # volumes:
+
     volumes:
       # - ./data/etc/:/opt/nagios/etc/
+
       - "/home/user/.MakeMKV_DOCKER:/config:rw"
 +
      - "/home/user/:/storage:ro"
 +
      - "/home/user/ToDo/MakeMKV/output:/output:rw"
 +
    devices:
 +
      - "/dev/sr0:/dev/sr0"
 +
      - "/dev/sg0:/dev/sg0"
 
     environment:
 
     environment:
       - PUID=999
+
       - USER_ID=1000
       - PGID=1000
+
       - GROUP_ID=1000
 
       - TZ=Europe/London
 
       - TZ=Europe/London
       - NAGIOS_TIMEZONE=Europe/London
+
       - MAKEMKV_KEY=your_licence_key
 +
      - AUTO_DISC_RIPPER=1
  
Start it with no volume mounts, then copy the etc directory to your host...
+
'''Troubleshooting'''
  
cd /root/docker/stacks/nagios/
+
PROBLEM = "driver failed programming external connectivity on endpoint makemkv: Error starting userland proxy: listen tcp6 [::]:5800: socket: address family not supported by protocol."
docker cp nagios:/opt/nagios/etc data/
 
chown -R 999:1000 data/
 
  
...then uncomment the # lines in the docker-compose file and restart the container.
+
SOLUTION = Put 0.0.0.0:5801 in the published ports line of docker compose to restrict the network to IPv4.
  
'''Credentials'''
+
'''Docker Process Output'''
  
The default credentials for the web interface is nagiosadmin / nagios
+
CONTAINER ID  IMAGE              COMMAND  CREATED        STATUS        PORTS                              NAMES
 +
4a8b3106b00b  jlesage/handbrake  "/init"  40 hours ago  Up 40 hours  5900/tcp, 0.0.0.0:5802->5800/tcp  handbrake
 +
89fe3ba8a31e  jlesage/makemkv    "/init"  40 hours ago  Up 40 hours  5900/tcp, 0.0.0.0:5801->5800/tcp  makemkv
  
To change the password, generate a new one by logging in to the container and running 'htpasswd'...
+
'''Command Line'''
  
  docker exec -it nagios bash
+
  docker run --rm -v "/home/user/.MakeMKV_DOCKER:/config:rw" -v "/home/user/:/storage:ro" -v "/home/user/ToDo/MakeMKV/output:/output:rw" --device /dev/sr0 --device /dev/sg0 --device /dev/sg1 jlesage/makemkv /opt/makemkv/bin/makemkvcon mkv disc:0 all /output
htpasswd -n nagiosadmin
 
(copy the output)
 
  
...then editing the <code>/opt/nagios/etc/htpasswd.users</code> file and refreshing the admin web page.
+
https://github.com/jlesage/docker-makemkv/issues/141
  
https://github.com/ethnchao/docker-nagios
+
=== HandBrake ===
  
http://www.kraftinfosec.com/running-nagios-in-docker/
+
'''This will NOT work on a Raspberry Pi.'''
  
https://github.com/JasonRivers/Docker-Nagios
+
Use this in combination with [[FFmpeg_DVD|ffmpeg]] or [[MakeMKV]] (as shown below) and [[FileBot]] to process your media through to your media server - like [[Emby]] or [[Plex]]..
  
=== Tandoor Recipe Manager ===
+
I have changed the port from 5800 to 5802 because Jocelyn's other Docker image for MakeMKV uses the same port (so I move that one as well to 5801 - see above).
  
The recipe manager that allows you to manage your ever growing collection of digital recipes.
+
To make this work with your DVD drive (/dev/sr0) '''you need to have the second device''' (/dev/sg0) in order for it to work. I don't get it, but it works.
  
https://docs.tandoor.dev/install/docker/
+
[https://www.youtube.com/watch?v=yKww_gg1fVI YouTube / DB Tech - How to install HandBrake in Docker]
 +
 
 +
[https://dbtechreviews.com/2020/03/how-to-install-handbrake-in-openmediavault-and-docker/ Blog / DB Tech - How to install HandBrake in Docker]
 +
 
 +
[https://github.com/jlesage/docker-handbrake Docker HandBrake by Jocelyn Le Sage]
  
https://www.youtube.com/watch?v=7-nb3muJxI0
+
[https://hub.docker.com/r/jlesage/handbrake Docker Image by Jocelyn Le Sage]
  
<code>/root/docker/stacks/tandoor/docker-compose.yml</code>
+
<code>/root/docker/stacks/handbrake/docker-compose.yml</code>
  
  version: "3"
+
  version: '3'
 
  services:
 
  services:
   db_recipes:
+
   handbrake:
     container_name: tandoor_db
+
    image: jlesage/handbrake
     restart: always
+
     container_name: handbrake
    image: postgres:11-alpine
+
     ports:
 +
      - "0.0.0.0:5802:5800"
 
     volumes:
 
     volumes:
       - ./data/postgresql:/var/lib/postgresql/data
+
       - "/home/paully:/storage:ro"
    env_file:
+
       - "/home/paully/ToDo/HandBrake/config:/config:rw"
       - ./.env
+
       - "/home/paully/ToDo/HandBrake/watch:/watch:rw"
  web_recipes:
+
       - "/home/paully/ToDo/HandBrake/output:/output:rw"
    container_name: tandoor_web
+
    devices:
    image: vabene1111/recipes
+
       - "/dev/sr0:/dev/sr0"
    restart: always
+
       - "/dev/sg0:/dev/sg0"
    env_file:
+
     environment:
       - ./.env
+
       - USER_ID=1000
    volumes:
+
      - GROUP_ID=1000
       - ./data/mediafiles:/opt/recipes/mediafiles
+
      - TZ=Europe/London
       - ./data/staticfiles:/opt/recipes/staticfiles
+
 
       - nginx_config:/opt/recipes/nginx/conf.d
+
'''Docker Process Output'''
     depends_on:
+
 
       - db_recipes
+
CONTAINER ID  IMAGE              COMMAND  CREATED       STATUS        PORTS                              NAMES
  nginx_recipes:
+
4a8b3106b00b  jlesage/handbrake  "/init"  40 hours ago  Up 40 hours  5900/tcp, 0.0.0.0:5802->5800/tcp  handbrake
    container_name: tandoor_nginx
+
89fe3ba8a31e  jlesage/makemkv    "/init"  40 hours ago  Up 40 hours  5900/tcp, 0.0.0.0:5801->5800/tcp  makemkv
    image: nginx:mainline-alpine
+
 
    restart: always
+
'''Command Line'''
    ports:
+
 
       - 80
+
docker run --rm -v "/home/paully/:/storage:ro" -v "/home/paully/ToDo/HandBrake/config:/config:rw" -v "/home/paully/ToDo/HandBrake/watch:/watch:rw" -v "/home/paully/ToDo/HandBrake/output:/output:rw" --device /dev/sr0 --device /dev/sg0 jlesage/handbrake /usr/bin/HandBrakeCLI --input "/output/file.mkv" --stop-at duration:120 --preset 'Fast 480p30' --non-anamorphic --encoder-preset slow --quality 22 --deinterlace --lapsharp --audio 1 --aencoder copy:ac3 --no-markers --output "/output/file.mp4"
    env_file:
+
 
      - ./.env
+
=== FileBot ===
    depends_on:
+
 
      - web_recipes
+
==== Setup ====
    volumes:
+
 
      - ./data/mediafiles:/media
+
Create your directories for data volumes (https://github.com/jlesage/docker-filebot#data-volumes) ...
      - ./data/staticfiles:/static
+
 
      - nginx_config:/etc/nginx/conf.d:ro
+
  mkdir -p ~/filebot/{config,output,watch}
volumes:
+
 
  nginx_config:
+
The license file received via email can be saved on the host, into the configuration directory of the container (i.e. in the directory mapped to /config). Then, start or restart the container to have it automatically installed. NOTE: The license file is expected to have a .psm extension.
  networks:
+
 
  default:
+
==== Usage ====
    external:
 
      name: nginx-proxy-manager
 
  
=== NextCloud ===
+
WORK IN PROGRESS
  
Nextcloud gives you access to all your files wherever you are.
+
Rather than running all the time, we run the image ad-hoc with the rm option to delete the old container each time...
  
Create your container folders...
+
docker run --rm --name=filebot -v ~/filebot/config:/config:rw -v $HOME:/storage:rw -v ~/filebot/output:/output:rw -v ~/filebot/watch:/watch:rw -e AMC_ACTION=test jlesage/filebot
  
mkdir -p /root/docker/stacks/nextcloud/data/{config,files}
+
https://github.com/jlesage/docker-filebot
chown -R 1000:1000 /root/docker/stacks/nextcloud/data/
 
  
<code>/root/docker/stacks/nextcloud/docker-compose.yml</code>
+
=== Automated Downloaderr ===
  
version: "2.1"
+
This takes the hassle out of going through the various web sites to find stuff and be bombarded with ads and pop-ups.
services:
 
  nextcloud:
 
    image: ghcr.io/linuxserver/nextcloud
 
    container_name: nextcloud
 
    environment:
 
      - PUID=1000
 
      - PGID=1000
 
      - TZ=Europe/London
 
    volumes:
 
      - ./data/config:/config
 
      - ./data/files:/data
 
    ports:
 
      - 4443:443
 
    restart: unless-stopped
 
  
(I have changed the default port it listens on the local machine to 4443 but if you don't need any ports open then change the lines to:-
+
# FlareSolverr
 +
# Prowlarr
 +
# Radarr + Sonarr + Bazarr
 +
# Transmission + NZBGet
 +
# Tdarr
  
    expose:
+
FlareSolverr > Prowlarr > Radarr + Sonarr + Bazarr > Transmission + NZBGet > Tdarr
      - 443
 
  
...then use Nginx Proxy Manager to direct traffic to your NextCloud installation)
 
  
Now visit https://ip.address.of.host:4443
+
'''HOW TO RESTART THE RRS IN ORDER ON PORTAINER OR OMV'''
  
'''...and on that setup page, untick the option for "Install recommended apps" which does not install Calendar, Contacts, Mail, Chat, etc.'''
+
Stacks > Click on each one > Stop > count to 10 > Start
  
Enjoy.
+
* WireGuard
 +
* FlareSolverr
 +
* Prowlarr
 +
* Radarr
 +
* Sonarr
  
https://hub.docker.com/r/linuxserver/nextcloud
+
ONE DAY WE WILL GET A SINGLE STACK WITH ALL THE RIGHT CONTAINERS STARTING IN THE RIGHT ORDER
  
=== Project Send ===
 
  
Self-hosted file sharing... small, simple, secure.
+
https://hotio.dev/containers/autoscan/
  
https://www.projectsend.org
+
'''ONE APP TO RULE THEM ALL'''
  
https://docs.linuxserver.io/images/docker-projectsend
+
https://github.com/JagandeepBrar/LunaSea
  
https://github.com/linuxserver/docker-projectsend
 
  
'''What they don't tell you in the docs is that you need a database backend - which is not in the docker compose file.'''
+
'''GUIDE FOR DIRECTORY STRUCTURE'''
  
So, we just add a MariaDB database container to the stack!
+
https://trash-guides.info/Hardlinks/How-to-setup-for/Docker/
  
Create your subdomain A record in DNS...
+
This will enable you to automatically rename files but allow you to copy them to your actual Plex or Emby folders.
  
cli53 rrcreate domain.uk 'send 300 A 123.45.678.90'
+
It is possible to do hard linking and let the rrrrs control all the files but I am not a fan of that.
  
Create your Proxy Host in Ngnix Proxy Manager with an SSL...
+
This way, the files get renamed and moved to a 'halfway' house where you can check them and then simply move them to your desired location.
  
  <nowiki>https://send.domain.uk</nowiki>
+
'''METHOD'''
 +
 
 +
Create the directories ...
 +
 
 +
mkdir -p /path/to/data/{media,torrents,usenet}/{movies,music,tv}
 +
mkdir -p /path/to/docker/appdata/{radarr,sonarr,bazarr,nzbget}
 +
 
 +
Change the ownership and permissions ...
 +
 
 +
  chown -R admin:users /path/to/data/ /path/to/docker/
 +
find /path/to/data/ /path/to/docker/ -type d -exec chmod 0755 {} \;
 +
find /path/to/data/ /path/to/docker/ -type d -exec chmod g+s {} \;
 +
 
 +
Finished directory structure ...
 +
 
 +
/srv/dev-disk-by-uuid-7f81e7b6-1a05-4232-893b-f34c046b2bdb/data/
 +
|-- media
 +
|  |-- movies
 +
|  |-- music
 +
|  `-- tv
 +
|-- torrents
 +
|  |-- movies
 +
|  |-- music
 +
|  `-- tv
 +
`-- usenet
 +
    |-- completed
 +
    |  |-- movies
 +
    |  `-- tv
 +
    |-- movies
 +
    |-- music
 +
    |-- nzb
 +
    |  `-- Movie.Name.720p.nzb.queued
 +
    |-- tmp
 +
    `-- tv
 +
 
 +
'''PORTAINER STACK'''
  
Create directories on the server for the Docker container files...
+
This is from Open Media Vault (OMV) so the volume paths are long.
  
sudo -i
+
This works but needs the whole VPN thing added (which changes ports etc) but for now ...
mkdir -p /root/docker/stacks/projectsend/data/{config,db,files}
 
chown -R 1000:1000 /root/docker/stacks/projectsend/data/files
 
  
<code>/root/docker/stacks/projectsend/docker-compose.yml</code>
+
Portainer > Stacks > Add Stack > Datarr
  
  version: "2.1"
+
  version: "3.2"
 
  services:
 
  services:
   projectsend:
+
   prowlarr:
     image: ghcr.io/linuxserver/projectsend
+
    container_name: prowlarr
     container_name: projectsend
+
     image: hotio/prowlarr:latest
 +
    restart: unless-stopped
 +
    logging:
 +
      driver: json-file
 +
    network_mode: bridge
 +
     ports:
 +
      - 9696:9696
 
     environment:
 
     environment:
       - PUID=1000
+
       - PUID=998
       - PGID=1000
+
       - PGID=100
 
       - TZ=Europe/London
 
       - TZ=Europe/London
      - MAX_UPLOAD=100
 
 
     volumes:
 
     volumes:
       - ./data/config:/config
+
       - /etc/localtime:/etc/localtime:ro
       - ./data/files:/data
+
       - /srv/dev-disk-by-uuid-7f81e7b6-1a05-4232-893b-f34c046b2bdb/docker/appdata/prowlarr:/config
       - /etc/timezone:/etc/timezone:ro
+
       - /srv/dev-disk-by-uuid-7f81e7b6-1a05-4232-893b-f34c046b2bdb/data:/data
     expose:
+
  radarr:
      - 80
+
     container_name: radarr
 +
    image: hotio/radarr:latest
 
     restart: unless-stopped
 
     restart: unless-stopped
   projectsend-db:
+
    logging:
     image: mariadb
+
      driver: json-file
     container_name: projectsend-db
+
    network_mode: bridge
 +
    ports:
 +
      - 7878:7878
 +
    environment:
 +
      - PUID=998
 +
      - PGID=100
 +
      - TZ=Europe/London
 +
    volumes:
 +
      - /etc/localtime:/etc/localtime:ro
 +
      - /srv/dev-disk-by-uuid-7f81e7b6-1a05-4232-893b-f34c046b2bdb/docker/appdata/radarr:/config
 +
      - /srv/dev-disk-by-uuid-7f81e7b6-1a05-4232-893b-f34c046b2bdb/data:/data
 +
   sonarr:
 +
    container_name: sonarr
 +
    image: hotio/sonarr:latest
 +
    restart: unless-stopped
 +
    logging:
 +
      driver: json-file
 +
     network_mode: bridge
 +
     ports:
 +
      - 8989:8989
 
     environment:
 
     environment:
       TZ: Europe/London
+
       - PUID=998
      MYSQL_ROOT_PASSWORD: projectsend
+
      - PGID=100
      MYSQL_DATABASE: projectsend
+
      - TZ=Europe/London
      MYSQL_USER: projectsend
 
      MYSQL_PASSWORD: projectsend
 
 
     volumes:
 
     volumes:
       - ./data/db:/var/lib/mysql
+
       - /etc/localtime:/etc/localtime:ro
       - /etc/timezone:/etc/timezone:ro
+
      - /srv/dev-disk-by-uuid-7f81e7b6-1a05-4232-893b-f34c046b2bdb/docker/appdata/sonarr:/config
 +
       - /srv/dev-disk-by-uuid-7f81e7b6-1a05-4232-893b-f34c046b2bdb/data:/data
 +
  bazarr:
 +
    container_name: bazarr
 +
    image: hotio/bazarr:latest
 
     restart: unless-stopped
 
     restart: unless-stopped
networks:
+
    logging:
   default:
+
      driver: json-file
     external:
+
    network_mode: bridge
       name: nginx-proxy-manager
+
    ports:
 
+
      - 6767:6767
Go to the secure web site URL and complete the installation, using the Docker container name for the 'Database hostname'.
+
    environment:
 
+
      - PUID=998
projectsend-db
+
      - PGID=100
 +
      - TZ=Europe/London
 +
    volumes:
 +
      - /etc/localtime:/etc/localtime:ro
 +
      - /srv/dev-disk-by-uuid-7f81e7b6-1a05-4232-893b-f34c046b2bdb/docker/appdata/bazarr:/config
 +
      - /srv/dev-disk-by-uuid-7f81e7b6-1a05-4232-893b-f34c046b2bdb/data/media:/data/media
 +
   nzbget:
 +
     container_name: nzbget
 +
    image: hotio/nzbget:latest
 +
    restart: unless-stopped
 +
    logging:
 +
       driver: json-file
 +
    network_mode: bridge
 +
    ports:
 +
      - 6789:6789
 +
    environment:
 +
      - PUID=998
 +
      - PGID=100
 +
      - TZ=Europe/London
 +
    volumes:
 +
      - /etc/localtime:/etc/localtime:ro
 +
      - /srv/dev-disk-by-uuid-7f81e7b6-1a05-4232-893b-f34c046b2bdb/docker/appdata/nzbget:/config
 +
      - /srv/dev-disk-by-uuid-7f81e7b6-1a05-4232-893b-f34c046b2bdb/data/usenet:/data/usenet:rw
  
Then, log in with your Admin username and password...
 
  
# Create Group 'Public' which is public.
+
OLD NOTES
# Create Group 'Customers' which is not public.
 
# Create Client 'Customer Name' which is assigned to the 'Customers' group.
 
# Upload some files and test both the Public and Client links.
 
  
Enjoy.
+
Create a docker network which Jackett and Radarr share to talk to each other...
  
==== Troubleshooting ====
+
sudo docker network create jackett-radarr
  
'''Change the Site URL'''
+
...then continue setting up the containers below.
  
If you move host or domain name, you can log in to the DB container and change the 'base_uri'...
+
==== FlareSolverr ====
  
docker exec -it projectsend-db bash
+
FlareSolverr is a proxy server to bypass Cloudflare protection.
mysql -u root -p projectsend
 
 
MariaDB [projectsend]>
 
MariaDB [projectsend]> select * from tbl_options where name = 'base_uri';
 
+----+----------+-------------------------+
 
| id | name    | value                  |
 
+----+----------+-------------------------+
 
|  1 | base_uri | <nowiki>https://send.domain.uk/</nowiki> |
 
+----+----------+-------------------------+
 
  
=== MediaWiki ===
+
FlareSolverr starts a proxy server and it waits for user requests in an idle state using few resources. When some request arrives, it uses puppeteer with the stealth plugin to create a headless browser (Chrome). It opens the URL with user parameters and waits until the Cloudflare challenge is solved (or timeout). The HTML code and the cookies are sent back to the user, and those cookies can be used to bypass Cloudflare using other HTTP clients.
  
==== Installation ====
+
Radarr > Jackett > FlareSolverr > Internet
  
Create the docker compose file and use default volume. Go to your browser at http://localhost:8080 and finish setup. Download LocalSettings.php file and copy to it to the container filesystem, then copy the whole folder to the host filsystem...
+
https://github.com/FlareSolverr/FlareSolverr
 +
 
 +
https://hub.docker.com/r/flaresolverr/flaresolverr
 +
 
 +
Some indexers are protected by CloudFlare or similar services and Jackett is not able to solve the challenges. For these cases, FlareSolverr has been integrated into Jackett. This service is in charge of solving the challenges and configuring Jackett with the necessary cookies. Setting up this service is optional, most indexers don't need it.
  
docker cp LocalSettings.php mediawiki:/var/www/html/
+
Install FlareSolverr service using a Docker container, then configure '''FlareSolverr API URL''' in Jackett. For example: http://172.17.0.2:8191
docker cp mediawiki:/var/www/html /root/docker/stacks/mediawiki/data/
 
chown -R www-data:www-data data/html
 
chmod o-w data/html
 
docker-compose down
 
(then edit your docker-compose.yml file so that local folders are used)
 
docker-compose up -d
 
  
Now, all the files are on your docker folder, ready to easily backup :-)
+
Command line...
  
  mediawiki
+
  docker run -d \
`-- data
+
  --name=flaresolverr \
    |-- db
+
  -p 8191:8191 \
    `-- html
+
  -e LOG_LEVEL=info \
 +
  --restart unless-stopped \
 +
  ghcr.io/flaresolverr/flaresolverr:latest
  
<code>~/docker/mediawiki/docker-compose.yml</code>
+
Docker compose...
  
  version: '3'
+
---
 +
  version: "2.1"
 
  services:
 
  services:
   mediawiki:
+
   flaresolverr:
     image: mediawiki
+
     image: ghcr.io/flaresolverr/flaresolverr:latest
     container_name: mediawiki
+
     container_name: flaresolverr
     restart: always
+
     environment:
 +
      - PUID=1000
 +
      - PGID=1000
 +
      - LOG_LEVEL=${LOG_LEVEL:-info}
 +
      - LOG_HTML=${LOG_HTML:-false}
 +
      - CAPTCHA_SOLVER=${CAPTCHA_SOLVER:-none}
 +
      - TZ=Europe/London
 
     ports:
 
     ports:
       - 8080:80
+
       - "${PORT:-8191}:8191"
    links:
+
     restart: unless-stopped
      - database
 
    volumes:
 
      #- ./data/html:/var/www/html    <-- '''#2'''
 
      #- /var/www/html/images      <-- '''#1'''
 
    environment:
 
      - PUID=33
 
      - PGID=33
 
      - TZ=Europe/London
 
  database:
 
    image: mariadb
 
    container_name: mediawiki_db
 
     restart: always
 
    environment:
 
      MYSQL_DATABASE: my_wiki
 
      MYSQL_USER: wikiuser
 
      MYSQL_PASSWORD: example
 
      MYSQL_RANDOM_ROOT_PASSWORD: 'yes'
 
    volumes:
 
      - ./data/db:/var/lib/mysql
 
  
https://hub.docker.com/_/mediawiki
+
'''Usage'''
  
==== Tweaks ====
+
To use it, you have to add a Proxy Server under Prowlarr > Settings > Indexers > Indexer Proxies
  
Change default skin to mobile responsive modern one...
+
'''Testing'''
  
  wfLoadSkin( 'Timeless' );
+
  curl -L -X POST '<nowiki>http://localhost:8191/v1</nowiki>' -H 'Content-Type: application/json' --data-raw '{ "cmd": "request.get", "url":"<nowiki>https://www.paully.co.uk/</nowiki>", "maxTimeout": 60000 }'
  $wgDefaultSkin = "timeless";
+
 +
  {"status":"ok","message":"","startTimestamp":1651659265156,"endTimestamp":1651659269585,"version":"v2.2.4","solution":
  
Enable the new editing toolbar...
+
==== Jackett ====
  
wfLoadExtension( 'WikiEditor' );
+
Jackett works as a proxy server: it translates queries from apps (Sonarr, SickRage, CouchPotato, Mylar, etc) into tracker-site-specific http queries, parses the html response, then sends results back to the requesting software. This allows for getting recent uploads (like RSS) and performing searches. Jackett is a single repository of maintained indexer scraping and translation logic - removing the burden from other apps.
  
Make the URL shorter...
+
'''So, this is where you build your list of web sites "with content you want" ;-)'''
  
$wgScriptPath = "";
+
https://fleet.linuxserver.io/image?name=linuxserver/jackett
$wgScriptExtension = ".php";
 
$wgArticlePath = "/wiki/$1";
 
$wgUsePathInfo = true;
 
  
File uploads...
+
https://docs.linuxserver.io/images/docker-jackett
  
<code>LocalSettings.php</code>
+
https://hub.docker.com/r/linuxserver/jackett
  
$wgEnableUploads = true;
+
https://github.com/Jackett/Jackett
  
https://www.mediawiki.org/wiki/Manual:Configuring_file_uploads
+
<code>/root/docker/stacks/docker-compose.yml</code>
  
https://kindalame.com/2020/11/25/self-hosting-mediawiki-with-docker/
+
---
 
+
version: "2.1"
==== Importing ====
+
services:
 
+
  jackett:
'''Pages'''
+
    image: ghcr.io/linuxserver/jackett
 
+
    container_name: jackett
OLD SERVER
+
    environment:
 +
      - PUID=1000
 +
      - PGID=1000
 +
      - TZ=Europe/London
 +
      - AUTO_UPDATE=true
 +
    volumes:
 +
      - ./data/config:/config
 +
      - ./data/downloads:/downloads
 +
    networks:
 +
      - jackett-radarr
 +
    ports:
 +
      - 0.0.0.0:9117:9117
 +
    restart: unless-stopped
 +
networks:
 +
  jackett-radarr:
 +
    external: true
  
Generate the page dump in XML format...
+
==== Prowlarr ====
  
docker exec -it mediawiki bash
+
An alternative to Jackett, and now the '''preferred''' application.
php maintenance/dumpBackup.php --current > pages.xml
 
exit
 
  
NEW SERVER
+
https://wiki.servarr.com/prowlarr
  
Import the pages...
+
https://wiki.servarr.com/prowlarr/quick-start-guide
  
cp pages.xml ./data/html/
+
https://hub.docker.com/r/linuxserver/prowlarr
docker exec -it mediawiki bash
 
php maintenance/importDump.php < pages.xml
 
php maintenance/update.php
 
php maintenance/rebuildall.php
 
exit
 
  
https://www.hostknox.com/tutorials/mediawiki/pages/export-and-import#import-pages-via-ssh
+
https://github.com/linuxserver/docker-prowlarr
  
'''Images'''
+
---
 +
version: "2.1"
 +
services:
 +
  prowlarr:
 +
    image: lscr.io/linuxserver/prowlarr:develop
 +
    container_name: prowlarr
 +
    environment:
 +
      - PUID=1000
 +
      - PGID=1000
 +
      - TZ=Europe/London
 +
    volumes:
 +
      - /path/to/data:/config
 +
    ports:
 +
      - 9696:9696
 +
    restart: unless-stopped
  
OLD SERVER
+
==== Radarr ====
  
Generate the image dumps using dumpUploads.php, which creates a txt list of all image filenames in use...
+
Radarr is a movie collection manager for Usenet and BitTorrent users. It can monitor multiple RSS feeds for new movies and will interface with clients and indexers to grab, sort, and rename them. It can also be configured to automatically upgrade the quality of existing files in the library when a better quality format becomes available.
  
mkdir /tmp/workingBackupMediaFiles
+
'''Radarr is the 'man-in-the-middle' to take lists from Jackett and pass them to Transmission to download.'''
php maintenance/dumpUploads.php \
 
    | sed 's~mwstore://local-backend/local-public~./images~' \
 
    | xargs cp -t /tmp/workingBackupMediaFiles
 
zip -r ~/Mediafiles.zip /tmp/workingBackupMediaFiles
 
rm -r /tmp/workingBackupMediaFiles
 
  
NEW SERVER
+
Radarr is the web UI to search for "the content you want" ;-)
  
Unzip the files to your container filsystem...
+
https://radarr.video/
  
cd /root/docker/stacks/mediawiki
+
https://docs.linuxserver.io/images/docker-radarr
unzip Mediafiles.zip -d ./data/html/
 
  
Import the Images...
+
https://github.com/linuxserver/docker-radarr
  
  docker exec -it mediawiki bash
+
https://sasquatters.com/radarr-docker/
  php maintenance/importImages.php tmp/workingBackupMediaFiles
+
 
  php maintenance/update.php
+
https://www.smarthomebeginner.com/install-radarr-using-docker/
  php maintenance/rebuildall.php
+
 
  exit
+
https://trash-guides.info/Radarr/
 +
 
 +
https://discord.com/channels/264387956343570434/264388019585286144
 +
 
 +
'''[https://wiki.servarr.com/radarr/custom-scripts Custom Scripts]'''
 +
 
 +
So, you use Jackett as an Indexer of content, which answers questions from Radarr, which passes a good result to Transmission...
 +
 
 +
# Settings > Profiles > delete all but 'any' (and edit that to get rid of naff qualities at the bottom)
 +
# Indexers > Add Indexer > Torznab > complete and TEST then SAVE
 +
# Download Clients > Add Download Client > Transmission > complete and TEST and SAVE
 +
 
 +
<code>/root/docker/stacks/docker-compose.yml</code>
 +
 
 +
---
 +
version: "2.1"
 +
services:
 +
  radarr:
 +
    image: ghcr.io/linuxserver/radarr
 +
    container_name: radarr
 +
    environment:
 +
      - PUID=1000
 +
      - PGID=1000
 +
      - TZ=Europe/London
 +
    volumes:
 +
      - ./data/config:/config
 +
      - ./data/downloads:/downloads
 +
      - ./data/torrents:/torrents
 +
    networks:
 +
      - jackett-radarr
 +
    ports:
 +
      - 0.0.0.0:7878:7878
 +
    restart: unless-stopped
 +
networks:
 +
  jackett-radarr:
 +
    external: true
 +
 
 +
==== Sonarr ====
 +
 
 +
Sonarr (formerly NZBdrone) is a PVR for usenet and bittorrent users. It can monitor multiple RSS feeds for new episodes of your favorite shows and will grab, sort and rename them. It can also be configured to automatically upgrade the quality of files already downloaded when a better quality format becomes available.
 +
 
 +
https://sonarr.tv/
 +
 
 +
https://docs.linuxserver.io/images/docker-sonarr
 +
 
 +
Docker Compose using a WireGuard VPN container for internet ...
 +
 
 +
---
 +
version: "2.1"
 +
services:
 +
  sonarr:
 +
    image: ghcr.io/linuxserver/sonarr
 +
    container_name: sonarr
 +
    network_mode: container:wireguard
 +
    environment:
 +
      - PUID=1000
 +
      - PGID=1000
 +
      - TZ=Europe/London
 +
    volumes:
 +
      - ./data/config:/config
 +
      - ./data/downloads:/downloads
 +
      - ./data/torrents:/torrents
 +
    restart: "no"
 +
 
 +
'''UPDATE: 16 FEBRUARY 2023 / latest image is based on Ubuntu Jammy and not Alpine, which will cause a problem if your Docker version is below 20.10.10'''
 +
 
 +
https://docs.linuxserver.io/faq#jammy
 +
 
 +
To fix this, you can either upgrade your Docker (https://docs.docker.com/engine/install/ubuntu/#install-using-the-convenience-script) or add the following lines to your docker-compose.yml file ...
 +
 
 +
security_opt:
 +
  - seccomp=unconfined
 +
 
 +
==== Bazarr ====
 +
 
 +
https://www.bazarr.media/
 +
 
 +
Bazarr is a companion application to Sonarr and Radarr that manages and downloads subtitles based on your requirements.
 +
 
 +
Docker compose file ...
 +
 
 +
---
 +
version: "2.1"
 +
services:
 +
  bazarr:
 +
    image: lscr.io/linuxserver/bazarr:latest
 +
    container_name: bazarr
 +
    environment:
 +
      - PUID=1000
 +
      - PGID=1000
 +
      - TZ=Europe/London
 +
    volumes:
 +
      - /path/to/bazarr/config:/config
 +
      - /path/to/movies:/movies #optional
 +
      - /path/to/tv:/tv #optional
 +
    ports:
 +
      - 6767:6767
 +
    restart: unless-stopped
 +
 
 +
==== NZBGet ====
 +
 
 +
NZBGet is a usenet downloader.
 +
 
 +
You will require the following 3 things at a basic level before you are able to use usenet:-
 +
 
 +
* A usenet provider (Reddit provides a detailed [https://www.reddit.com/r/usenet/wiki/providers#wiki_usenet_services_map Provider Map] for usenet)
 +
* An NZB indexer ([https://www.nzbgeek.info/ NZBGeek])
 +
* A usenet client ([https://nzbget.net/ NZBget])
 +
 
 +
https://nzbget.net
 +
 
 +
https://hub.docker.com/r/linuxserver/nzbget
 +
 
 +
https://www.cogipas.com/nzbget-complete-how-to-guide/
 +
 
 +
The Web GUI can be found at <your-ip>:6789 and the default login details (change ASAP) are...
 +
 
 +
username: nzbget
 +
password: tegbzn6789
 +
 
 +
Docker Compose file...
 +
 
 +
---
 +
version: "2.1"
 +
services:
 +
  nzbget:
 +
    image: lscr.io/linuxserver/nzbget:latest
 +
    container_name: nzbget
 +
    environment:
 +
      - PUID=1000
 +
      - PGID=1000
 +
      - TZ=Europe/London
 +
      - NZBGET_USER=nzbget
 +
      - NZBGET_PASS=tegbzn6789
 +
    volumes:
 +
      - /path/to/data:/config
 +
      - /path/to/downloads:/downloads
 +
    ports:
 +
      - 6789:6789
 +
    restart: unless-stopped
 +
 
 +
==== Tdarr ====
 +
 
 +
Tdarr is a popular conditional transcoding application for processing large (or small) media libraries. The application comes in the form of a click-to-run web-app, which you run on your own device and access through a web browser.
 +
 
 +
Tdarr uses two popular transcoding applications under the hood: FFmpeg and HandBrake (which itself is built on top of FFmpeg).
 +
 
 +
Tdarr works in a distributed manner where you can use multiple devices to process your library together. It does this using 'Tdarr Nodes' which connect with a central server and pick up tasks so you can put all your spare devices to use.
 +
 
 +
Each Node can run multiple 'Tdarr Workers' in parallel to maximize the hardware usage % on that Node. For example, a single FFmpeg worker running on a 64 core CPU may only hit ~30% utilization. Running multiple Workers in parallel allows the CPU to hit 100% utilization, allowing you to process your library more quickly.
 +
 
 +
[https://www.tdarr.io Home Page]
 +
 
 +
[https://docs.tdarr.io Documentation]
 +
 
 +
[https://www.reddit.com/r/Tdarr/comments/qsppw6/using_amd_vcn_for_large_h264_to_h265_transcode/ Using AMD GPU for Transcoding]
 +
 
 +
==== Readarr ====
 +
 
 +
https://academy.pointtosource.com/containers/ebooks-calibre-readarr/
 +
 
 +
==== Unpackerr ====
 +
 
 +
Extracts downloads for Radarr, Sonarr, Lidarr, Readarr, and/or a Watch folder - Deletes extracted files after import.
 +
 
 +
https://github.com/Unpackerr/unpackerr
 +
 
 +
Docker Compose - https://github.com/Unpackerr/unpackerr/blob/main/examples/docker-compose.yml
 +
 
 +
=== Calibre ===
 +
 
 +
eBook management and automation using Calibre, COPS or Calibre-Web, and Readarr.
 +
 
 +
Auto format conversion.
 +
 
 +
https://academy.pointtosource.com/containers/ebooks-calibre-readarr/
 +
 
 +
=== YouTube-DL ===
 +
 
 +
https://registry.hub.docker.com/search?q=youtube&sort=updated_at&order=desc
 +
 
 +
https://registry.hub.docker.com/r/mikenye/youtube-dl#quick-start
 +
 
 +
=== Nagios ===
 +
 
 +
Work in progress.
 +
 
 +
This is an old version of Nagios in the container image, so will look for a newer one.
 +
 
 +
<code>/root/docker/stacks/nagios/docker-compose.yml</code>
 +
 
 +
version: '3'
 +
services:
 +
  nagios:
 +
    image: jasonrivers/nagios
 +
    container_name: nagios
 +
    restart: unless-stopped
 +
    ports:
 +
      - 8181:80
 +
    # volumes:
 +
      # - ./data/etc/:/opt/nagios/etc/
 +
    environment:
 +
      - PUID=999
 +
      - PGID=1000
 +
      - TZ=Europe/London
 +
      - NAGIOS_TIMEZONE=Europe/London
 +
 
 +
Start it with no volume mounts, then copy the etc directory to your host...
 +
 
 +
cd /root/docker/stacks/nagios/
 +
docker cp nagios:/opt/nagios/etc data/
 +
chown -R 999:1000 data/
 +
 
 +
...then uncomment the # lines in the docker-compose file and restart the container.
 +
 
 +
'''Credentials'''
 +
 
 +
The default credentials for the web interface is nagiosadmin / nagios
 +
 
 +
To change the password, generate a new one by logging in to the container and running 'htpasswd'...
 +
 
 +
docker exec -it nagios bash
 +
htpasswd -n nagiosadmin
 +
(copy the output)
 +
 
 +
...then editing the <code>/opt/nagios/etc/htpasswd.users</code> file and refreshing the admin web page.
 +
 
 +
https://github.com/ethnchao/docker-nagios
 +
 
 +
http://www.kraftinfosec.com/running-nagios-in-docker/
 +
 
 +
https://github.com/JasonRivers/Docker-Nagios
 +
 
 +
=== Tandoor Recipe Manager ===
 +
 
 +
The recipe manager that allows you to manage your ever growing collection of digital recipes.
 +
 
 +
https://docs.tandoor.dev/install/docker/
 +
 
 +
https://www.youtube.com/watch?v=7-nb3muJxI0
 +
 
 +
<code>/root/docker/stacks/tandoor/docker-compose.yml</code>
 +
 
 +
version: "3"
 +
services:
 +
  db_recipes:
 +
    container_name: tandoor_db
 +
    restart: always
 +
    image: postgres:11-alpine
 +
    volumes:
 +
      - ./data/postgresql:/var/lib/postgresql/data
 +
    env_file:
 +
      - ./.env
 +
  web_recipes:
 +
    container_name: tandoor_web
 +
    image: vabene1111/recipes
 +
    restart: always
 +
    env_file:
 +
      - ./.env
 +
    volumes:
 +
      - ./data/mediafiles:/opt/recipes/mediafiles
 +
      - ./data/staticfiles:/opt/recipes/staticfiles
 +
      - nginx_config:/opt/recipes/nginx/conf.d
 +
    depends_on:
 +
      - db_recipes
 +
  nginx_recipes:
 +
    container_name: tandoor_nginx
 +
    image: nginx:mainline-alpine
 +
    restart: always
 +
    ports:
 +
      - 80
 +
    env_file:
 +
      - ./.env
 +
    depends_on:
 +
      - web_recipes
 +
    volumes:
 +
      - ./data/mediafiles:/media
 +
      - ./data/staticfiles:/static
 +
      - nginx_config:/etc/nginx/conf.d:ro
 +
volumes:
 +
  nginx_config:
 +
networks:
 +
  default:
 +
    external:
 +
      name: nginx-proxy-manager
 +
 
 +
=== NextCloud ===
 +
 
 +
IMPORTANT
 +
 
 +
 
 +
 
 +
If you are receiving errors about PHP or issues with nginx or certificates:
 +
 
 +
# Switch your image to '''lscr.io/linuxserver/nextcloud:24.0.6-ls204''' and start the container
 +
# Execute '''docker exec -it nextcloud updater.phar''' repeatedly until there are no more updates (as of writing, Nextcloud 25 is the latest version)
 +
# Switch your image to '''lscr.io/linuxserver/nextcloud''' (latest, no tag) and start the container
 +
# Execute '''docker exec -it nextcloud mv /config/nginx/site-confs/default.conf /config/nginx/site-confs/default.conf.bak'''
 +
# Execute '''docker exec -it nextcloud mv /config/nginx/nginx.conf /config/nginx/nginx.conf.bak'''
 +
# Execute <code>docker logs nextcloud</code> and check for any other outdated configs, rename them with a .bak extension (like above)
 +
# Restart the container
 +
# Nextcloud should now be in a working state
 +
 
 +
 
 +
 
 +
 
 +
Nextcloud gives you access to all your files wherever you are.
 +
 
 +
Create your container folders...
 +
 
 +
mkdir -p /root/docker/stacks/nextcloud/data/{config,files}
 +
chown -R 1000:1000 /root/docker/stacks/nextcloud/data/
 +
 
 +
<code>/root/docker/stacks/nextcloud/docker-compose.yml</code>
 +
 
 +
version: "2.1"
 +
services:
 +
  nextcloud:
 +
    image: ghcr.io/linuxserver/nextcloud
 +
    container_name: nextcloud
 +
    environment:
 +
      - PUID=1000
 +
      - PGID=1000
 +
      - TZ=Europe/London
 +
    volumes:
 +
      - ./data/config:/config
 +
      - ./data/files:/data
 +
    ports:
 +
      - 4443:443
 +
    restart: unless-stopped
 +
 
 +
(I have changed the default port it listens on the local machine to 4443 but if you don't need any ports open then change the lines to:-
 +
 
 +
    expose:
 +
      - 443
 +
 
 +
...then use Nginx Proxy Manager to direct traffic to your NextCloud installation)
 +
 
 +
Now visit https://ip.address.of.host:4443
 +
 
 +
'''...and on that setup page, _untick_ the option for "Install recommended apps" which does not install Calendar, Contacts, Mail, Chat, etc.'''
 +
 
 +
Enjoy.
 +
 
 +
https://hub.docker.com/r/linuxserver/nextcloud
 +
 
 +
==== Command Line Admin OCC ====
 +
 
 +
https://docs.nextcloud.com/server/latest/admin_manual/configuration_server/occ_command.html
 +
 
 +
'''List Users'''
 +
 
 +
How do I list the users in NextCloud Docker?
 +
 
 +
sudo docker exec -it nextcloud occ user:list
 +
 
 +
'''Reset Admin Password'''
 +
 
 +
How do you reset the admin user password in NextCloud Docker?
 +
 
 +
sudo docker exec -it nextcloud occ user:resetpassword admin
 +
 
 +
https://docs.nextcloud.com/server/latest/admin_manual/configuration_user/reset_admin_password.html
 +
 
 +
=== Project Send ===
 +
 
 +
Self-hosted file sharing... small, simple, secure.
 +
 
 +
https://www.projectsend.org
 +
 
 +
https://docs.linuxserver.io/images/docker-projectsend
 +
 
 +
https://github.com/linuxserver/docker-projectsend
 +
 
 +
'''What they don't tell you in the docs is that you need a database backend - which is not in the docker compose file.'''
 +
 
 +
So, we just add a MariaDB database container to the stack!
 +
 
 +
Create your subdomain A record in DNS...
 +
 
 +
cli53 rrcreate domain.uk 'send 300 A 123.45.678.90'
 +
 
 +
Create your Proxy Host in Ngnix Proxy Manager with an SSL...
 +
 
 +
<nowiki>https://send.domain.uk</nowiki>
 +
 
 +
Create directories on the server for the Docker container files...
 +
 
 +
sudo -i
 +
mkdir -p /root/docker/stacks/projectsend/data/{config,db,files}
 +
chown -R 1000:1000 /root/docker/stacks/projectsend/data/files
 +
 
 +
<code>/root/docker/stacks/projectsend/docker-compose.yml</code>
 +
 
 +
version: "2.1"
 +
services:
 +
  projectsend:
 +
    image: ghcr.io/linuxserver/projectsend
 +
    container_name: projectsend
 +
    environment:
 +
      - PUID=1000
 +
      - PGID=1000
 +
      - TZ=Europe/London
 +
      - MAX_UPLOAD=100
 +
    volumes:
 +
      - ./data/config:/config
 +
      - ./data/files:/data
 +
      - /etc/timezone:/etc/timezone:ro
 +
    expose:
 +
      - 80
 +
    restart: unless-stopped
 +
  projectsend-db:
 +
    image: mariadb
 +
    container_name: projectsend-db
 +
    environment:
 +
      TZ: Europe/London
 +
      MYSQL_ROOT_PASSWORD: projectsend
 +
      MYSQL_DATABASE: projectsend
 +
      MYSQL_USER: projectsend
 +
      MYSQL_PASSWORD: projectsend
 +
    volumes:
 +
      - ./data/db:/var/lib/mysql
 +
      - /etc/timezone:/etc/timezone:ro
 +
    restart: unless-stopped
 +
networks:
 +
  default:
 +
    external:
 +
      name: nginx-proxy-manager
 +
 
 +
Go to the secure web site URL and complete the installation, using the Docker container name for the 'Database hostname'.
 +
 
 +
projectsend-db
 +
 
 +
Then, log in with your Admin username and password...
 +
 
 +
# Create Group 'Public' which is public.
 +
# Create Group 'Customers' which is not public.
 +
# Create Client 'Customer Name' which is assigned to the 'Customers' group.
 +
# Upload some files and test both the Public and Client links.
 +
 
 +
Enjoy.
 +
 
 +
==== Troubleshooting ====
 +
 
 +
'''Change the Site URL'''
 +
 
 +
If you move host or domain name, you can log in to the DB container and change the 'base_uri'...
 +
 
 +
docker exec -it projectsend-db bash
 +
mysql -u root -p projectsend
 +
 +
MariaDB [projectsend]>
 +
MariaDB [projectsend]> select * from tbl_options where name = 'base_uri';
 +
+----+----------+-------------------------+
 +
| id | name    | value                  |
 +
+----+----------+-------------------------+
 +
|  1 | base_uri | <nowiki>https://send.domain.uk/</nowiki> |
 +
+----+----------+-------------------------+
 +
 
 +
=== MediaWiki ===
 +
 
 +
==== Installation ====
 +
 
 +
Create the docker compose file and use default volume. Go to your browser at http://localhost:8080 and finish setup. Download LocalSettings.php file and copy to it to the container filesystem, then copy the whole folder to the host filsystem...
 +
 
 +
docker cp LocalSettings.php mediawiki:/var/www/html/
 +
docker cp mediawiki:/var/www/html /root/docker/stacks/mediawiki/data/
 +
chown -R www-data:www-data data/html
 +
chmod o-w data/html
 +
docker-compose down
 +
(then edit your docker-compose.yml file so that local folders are used)
 +
docker-compose up -d
 +
 
 +
Now, all the files are on your docker folder, ready to easily backup :-)
 +
 
 +
mediawiki
 +
`-- data
 +
    |-- db
 +
    `-- html
 +
 
 +
<code>~/docker/mediawiki/docker-compose.yml</code>
 +
 
 +
version: '3'
 +
services:
 +
  mediawiki:
 +
    image: mediawiki
 +
    container_name: mediawiki
 +
    restart: always
 +
    ports:
 +
      - 8080:80
 +
    links:
 +
      - database
 +
    volumes:
 +
      #- ./data/html:/var/www/html    <-- '''#2'''
 +
      #- /var/www/html/images      <-- '''#1'''
 +
    environment:
 +
      - PUID=33
 +
      - PGID=33
 +
      - TZ=Europe/London
 +
  database:
 +
    image: mariadb
 +
    container_name: mediawiki_db
 +
    restart: always
 +
    environment:
 +
      MYSQL_DATABASE: my_wiki
 +
      MYSQL_USER: wikiuser
 +
      MYSQL_PASSWORD: example
 +
      MYSQL_RANDOM_ROOT_PASSWORD: 'yes'
 +
    volumes:
 +
      - ./data/db:/var/lib/mysql
 +
 
 +
https://hub.docker.com/_/mediawiki
 +
 
 +
==== Tweaks ====
 +
 
 +
Change default skin to mobile responsive modern one...
 +
 
 +
wfLoadSkin( 'Timeless' );
 +
$wgDefaultSkin = "timeless";
 +
 
 +
Enable the new editing toolbar...
 +
 
 +
wfLoadExtension( 'WikiEditor' );
 +
 
 +
Make the URL shorter...
 +
 
 +
$wgScriptPath = "";
 +
$wgScriptExtension = ".php";
 +
$wgArticlePath = "/wiki/$1";
 +
$wgUsePathInfo = true;
 +
 
 +
File uploads...
 +
 
 +
<code>LocalSettings.php</code>
 +
 
 +
$wgEnableUploads = true;
 +
 
 +
https://www.mediawiki.org/wiki/Manual:Configuring_file_uploads
 +
 
 +
https://kindalame.com/2020/11/25/self-hosting-mediawiki-with-docker/
 +
 
 +
==== Importing ====
 +
 
 +
'''Pages'''
 +
 
 +
OLD SERVER
 +
 
 +
Generate the page dump in XML format...
 +
 
 +
docker exec -it mediawiki bash
 +
php maintenance/dumpBackup.php --current > pages.xml
 +
exit
 +
 
 +
NEW SERVER
 +
 
 +
Import the pages...
 +
 
 +
cp pages.xml ./data/html/
 +
docker exec -it mediawiki bash
 +
php maintenance/importDump.php < pages.xml
 +
php maintenance/update.php
 +
php maintenance/rebuildall.php
 +
exit
 +
 
 +
https://www.hostknox.com/tutorials/mediawiki/pages/export-and-import#import-pages-via-ssh
 +
 
 +
'''Images'''
 +
 
 +
OLD SERVER
 +
 
 +
Generate the image dumps using dumpUploads.php, which creates a txt list of all image filenames in use...
 +
 
 +
mkdir /tmp/workingBackupMediaFiles
 +
php maintenance/dumpUploads.php \
 +
    | sed 's~mwstore://local-backend/local-public~./images~' \
 +
    | xargs cp -t /tmp/workingBackupMediaFiles
 +
zip -r ~/Mediafiles.zip /tmp/workingBackupMediaFiles
 +
rm -r /tmp/workingBackupMediaFiles
 +
 
 +
NEW SERVER
 +
 
 +
Unzip the files to your container filsystem...
 +
 
 +
cd /root/docker/stacks/mediawiki
 +
unzip Mediafiles.zip -d ./data/html/
 +
 
 +
Import the Images...
 +
 
 +
  docker exec -it mediawiki bash
 +
  php maintenance/importImages.php tmp/workingBackupMediaFiles
 +
  php maintenance/update.php
 +
  php maintenance/rebuildall.php
 +
  exit
  
 
https://stackoverflow.com/questions/1002258/exporting-and-importing-images-in-mediawiki
 
https://stackoverflow.com/questions/1002258/exporting-and-importing-images-in-mediawiki
 +
 +
=== Kuma ===
 +
 +
A self-hosted monitoring tool like Uptime Robot and not as complicated as Nagios.
 +
 +
https://hub.docker.com/r/louislam/uptime-kuma
 +
 +
https://github.com/louislam/uptime-kuma
 +
 +
https://youtu.be/dIVf1nhT0mI
 +
 +
=== Kasm Containerized Apps and Desktops ===
 +
 +
Streaming containerized apps and desktops to end-users. The Workspaces platform provides enterprise-class orchestration, data loss prevention, and web streaming technology to enable the delivery of containerized workloads to your browser.
 +
 +
https://kasmweb.com
 +
 +
https://hub.docker.com/u/kasmweb
 +
 +
https://www.youtube.com/channel/UCgpv4MLH8diVlIiakCBu8eQ
 +
 +
=== SFTP ===
 +
 +
https://hub.docker.com/r/atmoz/sftp
 +
 +
=== Wordle ===
 +
 +
https://github.com/cwackerfuss/react-wordle
 +
 +
=== Ombi ===
 +
 +
Ombi allows you to host your own Emby Request and user management system. If you are sharing your Emby server with other users, allow them to request new content using an easy to manage interface! Manage all your requests for Movies and TV with ease, leave notes for the user and get notification when a user requests something. Allow your users to post issues against their requests so you know there is a problem with the audio etc. Even automatically send them weekly newsletters of new content that has been added to your Emby server :-)
 +
 +
https://hub.docker.com/r/linuxserver/ombi
 +
 +
=== Emby ===
 +
 +
https://fleet.linuxserver.io/image?name=linuxserver/emby
 +
 +
https://hub.docker.com/r/linuxserver/emby
 +
 +
---
 +
version: "2.1"
 +
services:
 +
  emby:
 +
    image: lscr.io/linuxserver/emby
 +
    container_name: emby
 +
    environment:
 +
      - PUID=998
 +
      - PGID=100
 +
      - TZ=Europe/London
 +
    volumes:
 +
      - /srv/dev-disk-by-uuid-7f81e7b6-1a05-4232-893b-f34c046b2bdb/Emby/Config:/config
 +
      - /srv/dev-disk-by-uuid-7f81e7b6-1a05-4232-893b-f34c046b2bdb/Emby/TV:/data/tvshows
 +
      - /srv/dev-disk-by-uuid-7f81e7b6-1a05-4232-893b-f34c046b2bdb/Emby/Movies:/data/movies
 +
    ports:
 +
      - 8096:8096
 +
    restart: unless-stopped
 +
 +
=== Tdarr ===
 +
 +
Tdarr is a conditional based transcoding application for automating media library transcoding and remux management which uses cross-platform Tdarr Nodes which work together with Tdarr Server to process your files.
 +
 +
https://docs.tdarr.io/docs/installation/docker/run-compose
 +
 +
version: "3.4"
 +
 +
services:
 +
 +
# server
 +
  tdarr:
 +
    container_name: tdarr
 +
    image: ghcr.io/haveagitgat/tdarr:latest
 +
    restart: unless-stopped
 +
    network_mode: bridge
 +
    ports:
 +
      - 8265:8265 # webUI port
 +
      - 8266:8266 # server port
 +
      - 8267:8267 # Internal node port
 +
      - 8268:8268 # Example extra node port
 +
    environment:
 +
      - TZ=Europe/London
 +
      - PUID=${PUID}
 +
      - PGID=${PGID}
 +
      - UMASK_SET=002
 +
      - serverIP=0.0.0.0
 +
      - serverPort=8266
 +
      - webUIPort=8265
 +
      - internalNode=true
 +
      - nodeID=MyInternalNode
 +
    volumes:
 +
      - /docker/tdarr/server:/app/server
 +
      - /docker/tdarr/configs:/app/configs
 +
      - /docker/tdarr/logs:/app/logs
 +
      - /media:/media
 +
      - /transcode_cache:/temp
 +
 +
# node example
 +
  tdarr-node:
 +
    container_name: tdarr-node
 +
    image: ghcr.io/haveagitgat/tdarr_node:latest
 +
    restart: unless-stopped
 +
    network_mode: service:tdarr
 +
    environment:
 +
      - TZ=Europe/London
 +
      - PUID=${PUID}
 +
      - PGID=${PGID}
 +
      - UMASK_SET=002
 +
      - nodeID=MainNode
 +
      - serverIP=0.0.0.0
 +
      - serverPort=8266
 +
    volumes:
 +
      - /docker/tdarr/configs:/app/configs
 +
      - /docker/tdarr/logs:/app/logs
 +
      - /media:/media
 +
      - /transcode_cache:/temp
 +
 +
=== Unifi Controller ===
 +
 +
https://github.com/linuxserver/docker-unifi-controller
 +
 +
If you want to fix a particular version of the controller then check the '[https://github.com/linuxserver/docker-unifi-controller/releases?q=%E2%80%9C5.14.23%E2%80%9D&expanded=true releases]' at github and adjust your docker-compose.yml file accordingly.
 +
 +
e.g. linuxserver/unifi-controller:'''5.14.23-ls76'''
 +
 +
version: "2.1"
 +
services:
 +
  unifi-controller:
 +
    # image: linuxserver/unifi-controller:latest
 +
    image: linuxserver/unifi-controller:'''5.14.23-ls76'''
 +
    container_name: unifi-controller
 +
    environment:
 +
      - PUID=998
 +
      - PGID=100
 +
      # - MEM_LIMIT=1024 #optional
 +
      # - MEM_STARTUP=1024 #optional
 +
    volumes:
 +
      - <path to data>:/config
 +
    ports:
 +
      - 8443:8443
 +
      - 3478:3478/udp
 +
      - 10001:10001/udp
 +
      - 8080:8080
 +
      - 1900:1900/udp #optional
 +
      - 8843:8843 #optional
 +
      - 8880:8880 #optional
 +
      - 6789:6789 #optional
 +
      - 5514:5514/udp #optional
 +
    restart: unless-stopped
 +
 +
=== rPort ===
 +
 +
Rport helps you to manage your remote servers without the hassle of VPNs, chained SSH connections, jump-hosts, or the use of commercial tools like TeamViewer and its clones.
 +
 +
Rport acts as server and client establishing permanent or on-demand secure tunnels to devices inside protected intranets behind a firewall.
 +
 +
All operating systems provide secure and well-established mechanisms for remote management, being SSH and Remote Desktop the most widely used. Rport makes them accessible easily and securely.
 +
 +
https://rport.io/en/features
 +
 +
https://oss.rport.io/
 +
 +
https://github.com/cloudradar-monitoring/rport
 +
 +
https://hub.docker.com/r/acwhiteglint/rport
 +
 +
=== Paperless ===
 +
 +
Paperless is an application that indexes your scanned documents and allows you to easily search for documents and store metadata alongside your documents.
 +
 +
https://github.com/jonaswinkler/paperless-ng
 +
 +
https://paperless-ng.readthedocs.io/en/latest/setup.html#installation
 +
 +
=== Ansible ===
 +
 +
So, this is not installing Docker using Ansible... this is installing (or running) Ansible using Docker :-)
 +
 +
https://iceburn.medium.com/run-ansible-with-docker-9eb27d75285b
 +
 +
=== Pi Alert ===
 +
 +
WIFI / LAN intruder detector. Scan the devices connected to your WIFI / LAN and alert you the connection of unknown devices. It also warns the disconnection of "always connected" devices.
 +
 +
https://github.com/pucherot/Pi.Alert
 +
 +
=== Roundcube Webmail ===
 +
 +
https://hub.docker.com/search?q=roundcube
 +
 +
https://hub.docker.com/r/roundcubeorg/roundcubemail
 +
 +
https://github.com/roundcube/roundcubemail/wiki/Configuration
 +
 +
docker run -e ROUNDCUBEMAIL_DEFAULT_HOST=ssl://mail.domain.co.uk -e ROUNDCUBEMAIL_DEFAULT_PORT=993 -e ROUNDCUBEMAIL_SMTP_SERVER=tls://mail.domain.co.uk -e ROUNDCUBEMAIL_SMTP_PORT=587 -e ROUNDCUBEMAIL_PLUGINS=archive,zipdownload,managesieve,mobile -p 8000:80 -d roundcube/roundcubemail
 +
 +
=== FAST SpeedTest ===
 +
 +
docker run -it --rm --name fast-cli mschirrmeister/fast-cli:latest
 +
 +
https://hub.docker.com/r/mschirrmeister/fast-cli
 +
 +
https://github.com/sindresorhus/fast-cli
 +
 +
=== Crypto Mining ===
 +
 +
==== XMRig ====
 +
 +
https://hub.docker.com/r/minerboy/xmrig
 +
 +
Go to the [https://xmrig.com/wizard XMRig Wizard] and generate config like this ...
 +
 +
{
 +
    "autosave": true,
 +
    "cpu": true,
 +
    "opencl": false,
 +
    "cuda": false,
 +
    "pools": [
 +
        {
 +
            "coin": "monero",
 +
            "algo": "rx/0",
 +
            "url": "stratum+tcp://randomxmonero.auto.nicehash.com:9200",
 +
            "user": "NHbLd5exQeCGGyWnopVoLHLbzexKN5z8iq7p.NAS",
 +
            "pass": "x",
 +
            "tls": false,
 +
            "keepalive": true,
 +
            "nicehash": true
 +
        }
 +
    ]
 +
}
 +
 +
Then save that file as '''config.json''' and run the following commands to start mining ...
 +
 +
sudo -i
 +
docker pull minerboy/xmrig
 +
docker run --cap-add=SYS_ADMIN --cap-add=SYS_RAWIO --device=/dev/cpu --device=/dev/mem -v /lib/modules:/lib/modules -v /full/path/to/config.json:/etc/xmrig/config.json minerboy/xmrig:latest --cpu-max-threads-hint 50 --threads 2
 +
 +
Here is the same command as a lovely docker compose file ...
 +
 +
version: '3.3'
 +
services:
 +
  xmrig:
 +
      image: 'minerboy/xmrig:latest'
 +
      container_name: xmrig
 +
      devices:
 +
        - /dev/cpu
 +
        - /dev/mem
 +
      volumes:
 +
        - '/lib/modules:/lib/modules'
 +
        - '/root/docker/stacks/xmrig/config.json:/etc/xmrig/config.json'
 +
      command:
 +
        - --cpu-max-threads-hint 50
 +
        - --threads 2
 +
      restart: "no"
  
 
== Help ==
 
== Help ==

Latest revision as of 13:43, 30 December 2023

Introduction

Docker vs Virtual Machines

Docker is a set of platform as a service products that use OS-level virtualization to deliver software in packages called containers. Containers are isolated from one another and bundle their own software, libraries and configuration files but they can communicate with each other through well-defined channels.

https://www.docker.com - Official Web Site.

https://hub.docker.com - Official Repository of Container Images.

It was originally developed for programmers to test their software but has now become a fully fledged answer to running servers in mission critical situations.

Each container has a mini operating system plus the software needed to run the program you want, and no more.

All of the 'hard work' for a piece of software has been 'done for you' and the end result is starting a program with one command line.

For example, the WordPress image contains the LAP part of LAMP (Linux + Apache + PHP) all configured and running.

Images

Docker Hub

LinuxServer

Useful Wiki

Installation

WINDOWS

  1. Make sure your computer supports Hardware Virtualisation by checking in the BIOS.
  2. Install Docker Desktop.
  3. Install the Windows Subsystem for Linux Kernel Update.
  4. Reboot.

LINUX

New All In One Official Method

sudo -i
wget -qO- https://raw.githubusercontent.com/pi-hosted/pi-hosted/master/install_docker.sh | bash

which calls ...

https://get.docker.com/

Engine

This will remove the old version of 'Docker' and install the new version 'Docker CE'...

sudo apt-get remove docker docker-engine docker.io containerd runc
sudo apt-get update
sudo apt-get -y install apt-transport-https ca-certificates curl gnupg software-properties-common
sudo mkdir -p /etc/apt/keyrings
curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo gpg --dearmor -o /etc/apt/keyrings/docker.gpg
echo "deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.gpg] https://download.docker.com/linux/ubuntu $(lsb_release -cs) stable" | sudo tee /etc/apt/sources.list.d/docker.list > /dev/null
sudo apt-get update
sudo apt-get -y install docker-ce docker-ce-cli containerd.io docker-compose-plugin
sudo service docker start
sudo docker run hello-world

https://docs.docker.com/install/linux/docker-ce/ubuntu/#install-docker-ce-1

Compose From Command Line

http://composerize.com - Turns docker run commands into docker-compose files!

Install composerize and convert your own commands locally.

sudo apt instal npm
sudo npm install composerize -g
composerize docker run -p 80:80 -v /var/run/docker.sock:/tmp/docker.sock:ro --restart always --log-opt max-size=1g nginx

https://github.com/magicmark/composerize

Using Ansible

docker-install.yml

- hosts: all
  
  become: yes
  tasks:
  
  # Install Docker
  # --
  # 
  - name: install prerequisites
    apt:
      name:
        - apt-transport-https
        - ca-certificates 
        - curl 
        - gnupg-agent
        - software-properties-common
      update_cache: yes
  
  - name: add apt-key
    apt_key:
      url: https://download.docker.com/linux/ubuntu/gpg
  
  - name: add docker repo
    apt_repository:
      repo: deb https://download.docker.com/linux/ubuntu focal stable
  
  - name: install docker 
    apt:
      name: 
        - docker-ce
        - docker-ce-cli
        - containerd.io
      update_cache: yes
  
  - name: add user permissions
    shell: "usermod -aG docker ubuntu"
  
  # Installs Docker SDK
  # --
  # 
  - name: install python package manager
    apt:
      name: python3-pip
  
  - name: install python sdk
    become_user: ubuntu
    pip:
      name:
        - docker
        - docker-compose

Then run Ansible using your playbook on your server host ...

ansible-playbook docker-install.yml -l 'myserver'

You can then deploy a container using Ansible as well. This will deploy Portainer ...

docker_deploy-portainer.yml

- hosts: all
  
  become: yes
  become_user: ubuntu
  tasks:
  
  # Create Portainer Volume
  # --
  # 
  - name: Create new Volume
    community.docker.docker_volume:
      name: portainer-data
  
  # Deploy Portainer
  # --
  #   
  - name: Deploy Portainer
    community.docker.docker_container:
      name: portainer
      image: "docker.io/portainer/portainer-ce"
      ports:
        - "9000:9000"
        - "9443:9443"
      volumes:
        - /var/run/docker.sock:/var/run/docker.sock
        - portainer-data:/data
      restart_policy: always

... with this command ...

 ansible-playbook docker_deploy-portainer.yml -l 'myserver'

Usage

Statistics

docker stats
docker stats --no-stream

System information

docker system info

Run container

docker run hello-world

List containers

docker container ls
docker container ls -a

List container processes

docker ps
docker ps --format "table {{.ID}}\t{{.Names}}\t{{.Image}}\t{{.Ports}}\t{{.Status}}"

List container names

docker ps --format '{{.Names}}'
docker ps -a | awk '{print $NF}'

List volumes

docker volume ls
docker volume ls -f dangling=true

List networks

docker network ls

Information about container

docker container inspect container_name or id

Stop container

docker stop container_name

Delete container

docker rm container_name

Delete volumes

docker volume rm volume_name

Delete all unused volumes

docker volume prune

Delete all unused networks

docker network prune

Prune everything unused

docker system prune

Upgrade a stack

docker-compose pull
docker-compose up -d

BASH Aliases for use with Docker commands

alias dcd='docker-compose down'
alias dcr='docker-compose restart'
alias dcu='docker-compose up -d'
alias dps='docker ps'

Run Command In Docker Container

e.g.

List the mail queue for a running email server ...

docker exec -it mail.domain.co.uk-mailserver mailq

Volumes

Multiple Containers

Use volumes which are bind mounted from the host filesystem between multiple containers.

First, create the volume bind mounted to the folder...

docker volume create --driver local --opt type=none --opt device=/path/to/folder --opt o=bind volume_name

Then, use it in your docker compose file...

services:
  ftp.domain.uk-nginx:
    image: nginx
    container_name: ftp.domain.uk-nginx
    expose:
      - "80"
    volumes:
      - ./data/etc/nginx:/etc/nginx
      - ftp.domain.uk:/usr/share/nginx:ro
    environment:
      - VIRTUAL_HOST=ftp.domain.uk
networks:
  default:
    external:
      name: nginx-proxy-manager
volumes:
  ftp.domain.uk:
    external: true

Using volumes in Docker Compose

Networks

Basic Usage

Create your network...

docker network create networkname

Use it in your docker-compose.yml file...

services:
  service_name:
    image: image_name:latest
    restart: always
    networks:
      - networkname
networks:
  networkname:
    external: true

https://poopcode.com/join-to-an-existing-network-from-a-docker-container-in-docker-compose/

Advanced Usage

Static IP Address

   networks:
     traefik:
       ipv4_address: 172.19.0.9
     backend: null

Force Docker Containers to use a VPN for their Network

Block IP Address Using IPTables

The key here is to make sure you use the -I or INSERT command for the rule so that it is FIRST in the chain.

Block IP addresses from LITHUANIA

iptables -I DOCKER-USER -i eth0 -s 141.98.10.0/24 -j DROP
iptables -I DOCKER-USER -i eth0 -s 141.98.11.0/24 -j DROP
iptables -L DOCKER-USER --line-numbers
Chain DOCKER-USER (1 references)
num  target     prot opt source               destination
1    DROP       all  --  141.98.11.0/24       anywhere
2    DROP       all  --  141.98.10.0/24       anywhere
3    RETURN     all  --  anywhere             anywhere

Docker Compose

Restart Policy

The "no" option has quotes around it...

restart: "no"
restart: always
restart: on-failure
restart: unless-stopped

Management

Cleaning Pruning

sudo docker system df
sudo docker system prune
cd /etc/cron.daily
sudo nano docker-prune

#!/bin/bash
docker system prune -y

sudo chmod +x /etc/cron.daily/docker-prune

https://alexgallacher.com/prune-unused-docker-images-automatically/

Delete All Stopped Docker Containers

docker rm $(docker ps --filter "status=exited" -q)

Updating with Docker Compose

for d in ./*/ ; do (cd "$d" && sudo docker-compose pull && sudo docker-compose --compatibility up -d); done

Logs Logging

If you want to watch the logs in real time, then add the -f or --follow option to your command ...

docker logs nginx --follow

After executing docker-compose up, list your running containers:

docker ps

Copy the NAME of the given container and read its logs:

docker logs NAME_OF_THE_CONTAINER -f

To only read the error logs:

docker logs NAME_OF_THE_CONTAINER -f 1>/dev/null

To only read the access logs:

docker logs NAME_OF_THE_CONTAINER -f 2>/dev/null

https://linuxhandbook.com/docker-logging/

To search or grep the logs:

docker logs watchtower 2>&1 | grep 'msg="Found new'

Cleaning Space

Over the last month, a whopping 14Gb of space was being used by /var/lib/docker/overlay2/ and needed a way to safely remove unused data.

Check your space usage...

du -mcsh /var/lib/docker/overlay2
14G     /var/lib/docker/overlay2

Check what Docker thinks is being used...

docker system df
TYPE            TOTAL     ACTIVE    SIZE      RECLAIMABLE
Images          36        15        8.368GB   4.491GB (53%)
Containers      17        15        70.74MB   286B (0%)
Local Volumes   4         2         0B        0B
Build Cache     0         0         0B        0B

Clean...

docker system prune
docker image prune --all

Check again...

du -mcsh /var/lib/docker/overlay2
9.4G    /var/lib/docker/overlay2

docker system df
TYPE            TOTAL     ACTIVE    SIZE      RECLAIMABLE
Images          13        13        4.144GB   144MB (3%)
Containers      15        15        70.74MB   0B (0%)
Local Volumes   4         2         0B        0B
Build Cache     0         0         0B        0B

...job done.

Dockge

Better than Portainer.

A fancy, easy-to-use and reactive docker 'compose.yaml' stack-oriented manager

https://github.com/louislam/dockge

Portainer

https://www.portainer.io

https://github.com/portainer/portainer

Reset the Admin user Password

Server

https://hub.docker.com/r/portainer/portainer-ce

Agent

Portainer uses the Portainer Agent container to communicate with the Portainer Server instance and provide access to the node's resources. So, this is great for a small server (like a Raspberry Pi) where you don't need the full Portainer Server install.

Deployment

Run the following command to deploy the Portainer Agent:

docker run -d -p 9001:9001 --name portainer_agent --restart=always -v /var/run/docker.sock:/var/run/docker.sock -v /var/lib/docker/volumes:/var/lib/docker/volumes portainer/agent:2.11.0

Adding your new environment

Once the agent has been installed you are ready to add the environment to your Portainer Server installation.

From the menu select Environments then click Add environment.

From the Environment type section, select Agent. Since we have already installed the agent you can ignore the sample commands in the Information section.

Name: my-raspberry-pi
Environment URL: 192.168.0.106:9001

When you're ready, click Add environment.

Then, on the Portainer Home screen you select your new environment, or server running the Agent, and away you go!

https://docs.portainer.io/v/ce-2.11/admin/environments/add/docker#method-2-connecting-via-the-portainer-agent

Updating

Portainer

Containers > Select > Stop > Recreate > Pull Latest Image > Start

Watchtowerr

List updates ...

docker run --rm -v /var/run/docker.sock:/var/run/docker.sock containrrr/watchtower --run-once --log-format pretty --monitor-only

Perform updates ...

docker run --rm -v /var/run/docker.sock:/var/run/docker.sock containrrr/watchtower --run-once --log-format pretty

Backups

https://github.com/SavageSoftware/portainer-backup

https://forum.openmediavault.org/index.php?thread/43235-backup-portainer-and-completely-uninstall-to-test-disaster-recovery/

Monitoring

CTop

Press the Q key to stop it...

docker run -ti --name ctop --rm -v /var/run/docker.sock:/var/run/docker.sock wrfly/ctop:latest

Docker Stats

docker stats

DeUnhealth

Restart your unhealthy containers safely.

https://github.com/qdm12/deunhealth

https://www.youtube.com/watch?v=Oeo-mrtwRgE

Dozzle

Dozzle is a small lightweight application with a web based interface to monitor Docker logs. It doesn’t store any log files. It is for live monitoring of your container logs only.

https://github.com/amir20/dozzle

Gotchas

https://sosedoff.com/2016/10/05/docker-gotchas.html

Applications

I have set up my docker containers in a master docker directory with sub-directories for each stack.

docker
|-- backups
`-- stacks
    |-- bitwarden
    |   `-- bwdata
    |-- grafana
    |   `-- data
    |-- mailserver
    |   `-- data
    |-- nginx-proxy-manager
    |   `-- data
    `-- portainer
        `-- data

Backups

https://github.com/alaub81/backup_docker_scripts

Updates

Tracking

Watchtower

A process for automating Docker container base image updates.

With watchtower you can update the running version of your containerized app simply by pushing a new image to the Docker Hub or your own image registry. Watchtower will pull down your new image, gracefully shut down your existing container and restart it with the same options that were used when it was deployed initially.

First Time Run Once Check Only

This will run and output if there are any updates them stop and remove itself...

docker run --name watchtower -v /var/run/docker.sock:/var/run/docker.sock containrrr/watchtower --run-once --debug --monitor-only --rm

Automated Scheduled Run Daily

This will start the container and schedule a check at 4am every day...

~/watchtower/docker-compose.yml

version: "3"
services:
  watchtower:
    image: containrrr/watchtower
    container_name: watchtower
    restart: always
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock
    environment:
      - TZ=${TZ}
      - WATCHTOWER_DEBUG=true
      - WATCHTOWER_MONITOR_ONLY=false
      - WATCHTOWER_CLEANUP=true
      - WATCHTOWER_LABEL_ENABLE=false
      - WATCHTOWER_NOTIFICATIONS=email
      - WATCHTOWER_NOTIFICATION_EMAIL_FROM=${EMAIL_FROM}
      - WATCHTOWER_NOTIFICATION_EMAIL_TO=${WATCHTOWER_EMAIL_TO}
      - WATCHTOWER_NOTIFICATION_EMAIL_SERVER=${SMTP_SERVER}
      - WATCHTOWER_NOTIFICATION_EMAIL_SERVER_PORT=${SMTP_PORT}
      - WATCHTOWER_NOTIFICATION_EMAIL_SERVER_USER=${SMTP_USER}
      - WATCHTOWER_NOTIFICATION_EMAIL_SERVER_PASSWORD=${SMTP_PASSWORD}
      - WATCHTOWER_SCHEDULE=0 0 4 * * *

https://containrrr.dev/watchtower/

https://containrrr.dev/watchtower/arguments/#without_updating_containers

https://github.com/containrrr/watchtower

https://www.the-digital-life.com/watchtower/

https://www.youtube.com/watch?v=5lP_pdjcVMo

Updating

You can either ask Watchtower to update the containers automatically for you, or do it manually.

Manually updating when using docker-compose...

cd /path/to/docker/stack/
docker-compose down
docker-compose pull
docker-compose up -d

Bitwarden

~/bitwardenrs/docker-compose.yml

version: "2"
services:
  bitwardenrs:
    image: bitwardenrs/server:latest
    container_name: bitwardenrs
    volumes:
      - ./data:/data/
    ports:
      - 8100:80
    restart: unless-stopped
    environment:
      - TZ=Europe/London
      #- SIGNUPS_ALLOWED=false
      #- INVITATIONS_ALLOWED=false
      #- WEB_VAULT_ENABLED=false
networks:
  default:
    external:
      name: nginx-proxy-manager

Uncomment the extra security # lines after you have signed up, imported your old vault and set up your phone app and browsers, etc.

docker-compose down
docker-compose up -d

Check that the Bitwarden container environment has all the variables...

docker exec -it bitwardenrs env | sort

HOME=/root
HOSTNAME=e5f327deb4dd
INVITATIONS_ALLOWED=false
PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
ROCKET_ENV=staging
ROCKET_PORT=80
ROCKET_WORKERS=10
SIGNUPS_ALLOWED=false
TERM=xterm
TZ=Europe/London
WEB_VAULT_ENABLED=false

... and then refresh your web vault page to see it see "404: Not Found" :-)

Bitwarden RS

Bitwarden Official

InfluxDB

You can have InfluxDB on its own but there is little point without something to view the stats so you might as well include InfluxDB in the Grafana stack and start both at the same time... see below :-)

Grafana

Here is a stack in docker-compose which starts both containers in their own network so they can talk to one another. I have exposed ports for InfluxDB and Grafana to the host so I can use them from the internet.

Obviously, put your firewall in place and change the passwords below!

~/grafana/docker-compose.yml

version: "3"
services:
  grafana:
    image: grafana/grafana
    container_name: grafana
    restart: always
    networks:
      - grafana-influxdb-network
    ports:
      - 3000:3000
    volumes:
      - ./data/grafana:/var/lib/grafana
    environment:
      - INFLUXDB_URL=http://influxdb:8086
    depends_on:
      - influxdb
  influxdb:
    image: influxdb:1.8.4
    container_name: influxdb
    restart: always
    networks:
      - grafana-influxdb-network
    ports:
      - 8086:8086
    volumes:
      - ./data/influxdb:/var/lib/influxdb
    environment:
      - INFLUXDB_DB=grafana
      - INFLUXDB_USER=grafana
      - INFLUXDB_USER_PASSWORD=password
      - INFLUXDB_ADMIN_ENABLED=true
      - INFLUXDB_ADMIN_USER=admin 
      - INFLUXDB_ADMIN_PASSWORD=password 
      - INFLUXDB_URL=http://influxdb:8086
networks:
  grafana-influxdb-network:
    external: true

After this, change your Telegraf configuration to point to the new host and change the database it uses to 'grafana'.

NGiNX Proxy Manager

Provide users with an easy way to accomplish reverse proxying hosts with SSL termination that is so easy a monkey could do it.

  1. Set up your host
  2. Add a proxy to point to the host (in Docker this will be the 'name' and the port)
  3. Go to http://yourhost

https://nginxproxymanager.com

https://github.com/jc21/nginx-proxy-manager

Create the Docker network with a chosen subnet (used later for fixing container IP addresses)...

sudo -i
docker network create --subnet=172.20.0.0/16 nginx-proxy-manager 

/root/stacks/nginx-proxy-manager/docker-compose.yml

version: '3'
services:
  db:
    image: 'jc21/mariadb-aria:latest'
    container_name: nginx-proxy-manager_db
    restart: always
    networks:
      nginx-proxy-manager:
        ipv4_address: 172.20.0.2
    environment:
      TZ: "Europe/London"
      MYSQL_ROOT_PASSWORD: 'npm'
      MYSQL_DATABASE: 'npm'
      MYSQL_USER: 'npm'
      MYSQL_PASSWORD: 'npm'
    volumes:
      - ./data/mysql:/var/lib/mysql
  app:
    image: 'jc21/nginx-proxy-manager:latest'
    container_name: nginx-proxy-manager_app
    restart: always
    networks:
      nginx-proxy-manager:
        ipv4_address: 172.20.0.3
    ports:
      - '80:80'
      - '81:81'
      - '443:443'
    environment:
      TZ: "Europe/London"
      DB_MYSQL_HOST: "db"
      DB_MYSQL_PORT: 3306
      DB_MYSQL_USER: "npm"
      DB_MYSQL_PASSWORD: "npm"
      DB_MYSQL_NAME: "npm"
    volumes:
      - ./data:/data
      - ./data/letsencrypt:/etc/letsencrypt
    depends_on:
      - db
networks:
  nginx-proxy-manager:
    external: true

Reset Password

docker exec -it nginx-proxy-manager_db sh
mysql -u root -p npm
select * from user;
delete from user where id=1;
quit;
exit

Custom SSL Certificate

You can add a custom SSL certificate to NPM by saving the 3 parts of the SSL from Let's Encrypt...

  1. privkey.pem
  2. cert.pem
  3. chain.pem

...and then uploading them to NPM.

Updating

docker-compose pull
docker-compose up -d

NGiNX

Quick Container

Run and delete everything afterwards (press CTRL+C to stop it)...

docker run --rm --name test.domain.org-nginx -e VIRTUAL_HOST=test.domain.org nginx

Run and detach and use a host folder to store the web pages and keep the container afterwards...

docker run --name test.domain.org-nginx -e VIRTUAL_HOST=test.domain.org -v /some/content:/usr/share/nginx/html:ro -d nginx

Run and detach and connect to a specific network (like nginx-proxy-manager) and use a host folder to store the web pages and keep the container afterwards...

 docker run --name test.domain.org-nginx --network nginx-proxy-manager -e VIRTUAL_HOST=test.domain.org -v /some/content:/usr/share/nginx/html:ro -d nginx

Check the logs and always show them (like tail -f)...

 docker logs test.domain.org-nginx -f

docker-compose.yml

version: "3"
services:
  nginx:
    image: nginx
    container_name: nginx
    environment:
      - TZ=Europe/London
    volumes:
      - ./data/html:/usr/share/nginx/html:ro
      - /etc/timezone:/etc/timezone:ro
    expose:
      - 80
    restart: unless-stopped


With PHP

./data/nginx/site.conf

server {
    server_name docker-demo.com;
    root /var/www/html;
    index index.php index.html index.htm;
    access_log /var/log/nginx/access.log;
    error_log /var/log/nginx/error.log;
    location / {
        try_files $uri $uri/ /index.php?$query_string;
    }
    # PHP-FPM Configuration Nginx
    location ~ \.php$ {
        try_files $uri = 404;
        fastcgi_split_path_info ^(.+\.php)(/.+)$;
        fastcgi_pass php:9000;
        fastcgi_index index.php;
        include fastcgi_params;
        fastcgi_param REQUEST_URI $request_uri;
        fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
        fastcgi_param PATH_INFO $fastcgi_path_info;
    }
}

docker-compose.yml

version: "3"
services:
  nginx:
    image: nginx
    container_name: nginx
    environment:
      - TZ=Europe/London
    volumes:
      - ./data/html:/usr/share/nginx/html:ro
      - ./data/nginx:/etc/nginx/conf.d/
      - /etc/timezone:/etc/timezone:ro
    expose:
      - 80
    restart: unless-stopped
  php:
    image: php:7.2-fpm
    volumes:
      - ./data/html:/usr/share/nginx/html:ro
      - ./data/php:/usr/local/etc/php/php.ini

https://adoltech.com/blog/how-to-set-up-nginx-php-fpm-and-mysql-with-docker-compose/

With PERL

This is a way to get the IP address of the visitor (REMOTE_ADDR) displayed...

# nginx.conf; mostly copied from defaults
load_module "modules/ngx_http_perl_module.so";

user  nginx;
worker_processes  auto;

error_log  /var/log/nginx/error.log notice;
pid        /var/run/nginx.pid;

events {
    worker_connections  1024;
}

http {
    #perl_modules /; # only needed the hello.pm isn't in @INC (e.g. dir specified below)
    perl_modules /usr/lib/perl5/vendor_perl/x86_64-linux-thread-multi/;
    perl_require hello.pm;

    server {
        location / {
            perl hello::handler;
        }
    }
}
# hello.pm; put in a @INC dir
package hello;
use nginx;

sub handler {
    my $r = shift;
    $r->send_http_header("text/html");
    return OK if $r->header_only;
    $r->print($r->remote_addr);
    return OK;
}
1;

https://www.reddit.com/r/docker/comments/oabga4/run_perl_script_in_nginx_container/

Load Balancer

This is a simple exmaple test to show multiple backend servers answering web page requests.

docker-compose.yml

version: '3'
services:
  # The load balancer
  nginx:
    image: nginx:1.16.0-alpine
    volumes:
      - ./nginx.conf:/etc/nginx/nginx.conf:ro
    ports:
      - "80:80"
  # The web server1
  server1:
    image: nginx:1.16.0-alpine
    volumes:
      - ./server1.html:/usr/share/nginx/html/index.html
  # The web server2
  server2:
    image: nginx:1.16.0-alpine
    volumes:
      - ./server2.html:/usr/share/nginx/html/index.html

nginx.conf

events {
    worker_connections 1024;
}
http {
    upstream app_servers {    # Create an upstream for the web servers
        server server1:80;    # the first server
        server server2:80;    # the second server
    }
    server {
        listen 80;
        location / {
            proxy_pass         http://app_servers;  # load balance the traffic
        }
    }
}

https://omarghader.github.io/docker-compose-nginx-tutorial/

Proxy

This is very cool and allows you to run multiple web sites on-the-fly.

The container connects to the system docker socket and watches for new containers using the VIRTUAL_HOST environment variable.

Start this, then add another container using the VIRTUAL_HOST variable and the proxy container will change its config file and reload nginx to serve the web site... automatically.

Incredible.

~/nginx-proxy/docker-compose.yml

version: "3"
services:
  nginx-proxy:
    image: jwilder/nginx-proxy
    container_name: nginx-proxy
    ports:
      - "80:80"
    volumes:
      - /var/run/docker.sock:/tmp/docker.sock:ro
networks:
  default:
    external:
      name: nginx-proxy

Normal

When using the nginx-proxy container above, you can just spin up a virtual web site using the standard 'nginx' docker image and link it to the 'nginx-proxy' network...

docker run -d --name nginx-website1.uk --expose 80 --net nginx-proxy -e VIRTUAL_HOST=website1.uk nginx

To use the host filesystem to store the web page files...

docker run -d --name nginx-website1.uk --expose 80 --net nginx-proxy -v /var/www/website1.uk/html:/usr/share/nginx/html:ro -e VIRTUAL_HOST=website1.uk nginx

In Docker Compose, it will look like this...

~/nginx/docker-compose.yml

version: "3"
services:
  nginx-website1.uk:
    image: nginx
    container_name: nginx-website1.uk
    expose:
      - "80"
    volumes:
      - /var/www/website1.uk/html:/usr/share/nginx/html:ro
    environment:
      - VIRTUAL_HOST=website1.uk
networks:
  default:
    external:
      name: nginx-proxy

Multiple Virtual Host Web Sites

~/nginx/docker-compose.yml

version: "3"
services:
  nginx-website1.uk:
    image: nginx
    container_name: nginx-website1.uk
    expose:
      - "80"
    volumes:
      - /var/www/website1.uk/html:/usr/share/nginx/html:ro
    environment:
      - VIRTUAL_HOST=website1.uk
  nginx-website2.uk:
    image: nginx
    container_name: nginx-website2.uk
    expose:
      - "80"
    volumes:
      - /var/www/website2.uk/html:/usr/share/nginx/html:ro
    environment:
      - VIRTUAL_HOST=website2.uk
networks:
  default:
    external:
      name: nginx-proxy

Viewing Logs

docker-compose logs nginx-website1.uk
docker-compose logs nginx-website2.uk

Proxy Manager

This is a web front end to manage 'nginx-proxy', where you can choose containers and create SSL certificates etc.

https://cyberhost.uk/npm-setup/

Various

https://hub.docker.com/_/nginx

https://blog.ssdnodes.com/blog/host-multiple-websites-docker-nginx/

https://github.com/nginx-proxy/nginx-proxy

Typical LEMP

https://adoltech.com/blog/how-to-set-up-nginx-php-fpm-and-mysql-with-docker-compose/

WordPress

https://hub.docker.com/_/wordpress/

PHP File Uploads Fix

Create a new PHP configuration file, and name it docker-uploads.ini. Add the following configuration then save the changes.

# Allow HTTP file uploads
file_uploads = On

# Maximum size of an uploaded file
upload_max_filesize = 64M

# Maximum size of form post data
post_max_size = 64M

Update the docker-compose.yml to bind the docker-uploads.ini to the wordpress container and then restart the WordPress container.

volumes:
  - ./data/config/docker-uploads.ini:/usr/local/etc/php/conf.d/docker-uploads.ini

WordPress Clone

Create your A record in DNS using AWS Route 53 CLI...

cli53 rrcreate domain.co.uk 'staging 300 A 123.456.78.910'

Create your docker folder for the cloned staging test web site...

mkdir -p ~/docker/stacks/staging.domain.co.uk/data/{db,html}

Edit your docker compose file, with 2 containers, making sure you use the same network as your Nginx Proxy Manager...

~/docker/stacks/staging.domain.co.uk/docker-compose.yml

version: "3"
services:
  staging.domain.co.uk-wordpress_db:
    image: mysql:5.7
    container_name: staging.domain.co.uk-wordpress_db
    volumes:
      - ./data/db:/var/lib/mysql
    restart: always
    environment:
      - TZ=Europe/London
      - MYSQL_ROOT_PASSWORD=changeme
      - MYSQL_DATABASE=dbname
      - MYSQL_USER=dbuser
      - MYSQL_PASSWORD=changeme
  staging.domain.co.uk-wordpress:
    depends_on:
      - staging.domain.co.uk-wordpress_db
    image: wordpress:latest
    container_name: staging.domain.co.uk-wordpress
    volumes:
      - ./data/html:/var/www/html
    restart: always
    environment:
      - TZ=Europe/London
      - VIRTUAL_HOST=staging.domain.co.uk
      - WORDPRESS_DB_HOST=staging.domain.co.uk-wordpress_db:3306
      - WORDPRESS_DB_NAME=dbname
      - WORDPRESS_DB_USER=dbuser
      - WORDPRESS_DB_PASSWORD=changeme
  staging.domain.co.uk-wordpress-cli:
    image: wordpress:cli
    container_name: staging.domain.co.uk-wordpress-cli
    volumes:
      - ./data/html:/var/www/html
    environment:
      - TZ=Europe/London
      - WP_CLI_CACHE_DIR=/tmp/
      - VIRTUAL_HOST=staging.domain.co.uk
      - WORDPRESS_DB_HOST=staging.domain.co.uk-wordpress_db:3306
      - WORDPRESS_DB_NAME=dbname
      - WORDPRESS_DB_USER=dbuser
      - WORDPRESS_DB_PASSWORD=changeme
    working_dir: /var/www/html
    user: "33:33"
networks:
  default:
    external:
      name: nginx-proxy-manager

Start containers with correct settings and credentials for existing live web site (so that the docker startup script sets up the MySQL permissions)...

docker-compose up -d

Check the logs to make sure all is well...

docker logs staging.domain.co.uk-wordpress
docker logs staging.domain.co.uk-wordpress_db

Copy the WordPress files to the host folder and correct ownership...

rsync -av /path/to/backup_unzipped_wordpress/ ~/docker/stacks/staging.domain.co.uk/html/
chown -R www-data:www-data ~/docker/stacks/staging.domain.co.uk/html/

Copy the sql file in to the running mysql container...

docker cp /path/to/backup_unzipped_wordpress/db_name.sql mysql_container_name:/tmp/

Log in to the database container...

docker exec -it mysql_container_name bash

Check and if necessary, change the timezone...

date
mv /etc/localtime /etc/localtime.backup
ln -s /usr/share/zoneinfo/Europe/London /etc/localtime
date

Delete and create the database...

mysql -u root -p -e "DROP DATABASE db_name; CREATE DATABASE db_name;"

Import the database from the sql file, check and exit out of the container...

mysql -u root -p mysql_db_name < /tmp/db_name.sql
mysql -u root -p -e "use db_name; show tables;"
rm /tmp/db_name.sql
exit

Edit the wp-config.php on your host server to match new DB_HOST and also add extra variables to be sure...

nano /path/to/docker/folder/html/wp-config.php
define( 'WP_HOME', 'http://staging.domain.co.uk' );
define( 'WP_SITEURL', 'http://staging.domain.co.uk' );

Install WordPress CLI in the running container...

docker exec -it wordpress_container_name bash

Search and replace the original site url...

./wp --allow-root search-replace 'http://www.domain.co.uk/' 'http://staging.domain.co.uk/' --dry-run
./wp --allow-root search-replace 'http://www.domain.co.uk/' 'http://staging.domain.co.uk/'

Start your web browser and go to the test staging web site!

WordPress CLI

In your stack, set up the usual two DB + WordPress containers, then add a third services section for wp-cli...

version: "3"
services:
  www.domain.uk-wordpress_db:
    image: mysql:5.7
    container_name: www.domain.uk-wordpress_db
    volumes:
      - ./data/db:/var/lib/mysql
    restart: always
    environment:
      - MYSQL_ROOT_PASSWORD=password
      - MYSQL_DATABASE=wordpress
      - MYSQL_USER=wordpress
      - MYSQL_PASSWORD=password
  www.domain.uk-wordpress:
    depends_on:
      - www.domain.uk-wordpress_db
    image: wordpress:latest
    container_name: www.domain.uk-wordpress
    volumes:
      - ./data/html:/var/www/html
    expose:
      - 80
    restart: always
    environment:
      - VIRTUAL_HOST=www.domain.uk
      - WORDPRESS_DB_HOST=www.domain.uk-wordpress_db:3306
      - WORDPRESS_DB_NAME=wordpress
      - WORDPRESS_DB_USER=wordpress
      - WORDPRESS_DB_PASSWORD=password
  www.domain.uk-wordpress-cli:
    image: wordpress:cli
    container_name: www.domain.uk-wordpress-cli
    volumes:
      - ./data/html:/var/www/html
    environment:
      - WP_CLI_CACHE_DIR=/tmp/
      - VIRTUAL_HOST=www.domain.uk
      - WORDPRESS_DB_HOST=www.domain.uk-wordpress_db:3306
      - WORDPRESS_DB_NAME=wordpress
      - WORDPRESS_DB_USER=wordpress
      - WORDPRESS_DB_PASSWORD=password
    working_dir: /var/www/html
    user: "33:33"
networks:
  default:
    external:
      name: nginx-proxy-manager

...then start it all up.

docker-compose up -d

Then, run your wp-cli commands (e.g. wp user list) on the end of a docker run command...

docker-compose run --rm www.domain.uk-wordpress-cli wp --info
docker-compose run --rm www.domain.uk-wordpress-cli wp cli version
docker-compose run --rm www.domain.uk-wordpress-cli wp user list
docker-compose run --rm www.domain.uk-wordpress-cli wp help theme
docker-compose run --rm www.domain.uk-wordpress-cli wp theme delete --all

SSL Behind A Reverse Proxy

https://wiki.indie-it.com/wiki/WordPress#SSL_When_Using_A_Reverse_Proxy

Email Server (mailu)

Mailu is a simple yet full-featured mail server as a set of Docker images. It is free software (both as in free beer and as in free speech), open to suggestions and external contributions. The project aims at providing people with an easily setup, easily maintained and full-featured mail server while not shipping proprietary software nor unrelated features often found in popular groupware.

https://mailu.io/1.7/

https://hub.docker.com/u/mailu

https://github.com/Mailu/Mailu

Postfix Admin

https://hub.docker.com/_/postfixadmin

Email Server (docker-mailserver)

https://github.com/docker-mailserver

https://github.com/docker-mailserver/docker-mailserver

https://github.com/docker-mailserver/docker-mailserver-admin

https://docker-mailserver.github.io/docker-mailserver/edge/config/security/ssl/#lets-encrypt-recommended

Postscreen

Postscreen is an SMTP filter that blocks spambots (or zombie machines) away from the real Postfix smtpd daemon, so Postfix does not feel overloaded and can process legitimate emails more efficiently.

The example below shows a typical spambot attempt at accessing the SMTP service and being stopped...

Jun 24 10:42:26 mail postfix/postscreen[203907]: CONNECT from [212.70.149.56]:19452 to [172.23.0.2]:25
Jun 24 10:42:26 mail postfix/dnsblog[386054]: addr 212.70.149.56 listed by domain b.barracudacentral.org as 127.0.0.2
Jun 24 10:42:26 mail postfix/dnsblog[402550]: addr 212.70.149.56 listed by domain list.dnswl.org as 127.0.10.3
Jun 24 10:42:26 mail postfix/dnsblog[407802]: addr 212.70.149.56 listed by domain bl.mailspike.net as 127.0.0.2
Jun 24 10:42:26 mail postfix/dnsblog[386155]: addr 212.70.149.56 listed by domain psbl.surriel.com as 127.0.0.2
Jun 24 10:42:29 mail postfix/postscreen[203907]: PREGREET 11 after 2.9 from [212.70.149.56]:19452: EHLO User\r\n
Jun 24 10:42:29 mail postfix/postscreen[203907]: DISCONNECT [212.70.149.56]:19452

Postscreen is enabled by default but there are a few settings to tweak to get the best out of it.

Edit your data/config/postfix-main.cf file and add the following lines, making sure your Docker host IP is in bold...

mynetworks = 127.0.0.0/8 [::1]/128 [fe80::]/64 172.19.0.2/32 172.19.0.1/32
postscreen_greet_action = drop
postscreen_pipelining_enable = yes
postscreen_pipelining_action = drop
postscreen_non_smtp_command_enable = yes
postscreen_non_smtp_command_action = drop
postscreen_bare_newline_enable = yes 
postscreen_bare_newline_action = drop

Enable and Configure Postscreen in Postfix to Block Spambots

Postgrey

Local List

SpamAssassin

Custom Rules

Bayes Database

SpamAssassin is controlled by Amavis (a fork of MailScanner) with the user 'amavis'.

Show Bayes Database Stats

docker exec --user amavis mail.mydomain.org.uk-mailserver sa-learn --dump magic --dbpath /var/lib/amavis/.spamassassin

Learn Ham

docker exec --user amavis mail.mydomain.org.uk-mailserver sa-learn --ham --progress /var/mail/mydomain.org.uk/info/cur --dbpath /var/lib/amavis/.spamassassin

Backup and Restore from Existing Mail Server

On the old server...

/bin/su -l -c '/usr/bin/sa-learn --backup > sa-learn_backup.txt' debian-spamd
rsync -avP /var/lib/spamassassin/sa-learn_backup.txt user@mail.mydomain.org.uk:/tmp/

On the new server...

docker cp /tmp/sa-learn_backup.txt mail.mydomain.org.uk-mailserver:/tmp/
docker exec --user amavis mail.mydomain.org.uk-mailserver sa-learn --sync --dbpath /var/lib/amavis/.spamassassin
docker exec --user amavis mail.mydomain.org.uk-mailserver sa-learn --clear --dbpath /var/lib/amavis/.spamassassin
docker exec --user amavis mail.mydomain.org.uk-mailserver sa-learn --restore /tmp/sa-learn_backup.txt --dbpath /var/lib/amavis/.spamassassin
docker exec --user amavis mail.mydomain.org.uk-mailserver sa-learn --sync --dbpath /var/lib/amavis/.spamassassin
docker exec --user amavis mail.mydomain.org.uk-mailserver sa-learn --dump magic --dbpath /var/lib/amavis/.spamassassin

Fail2Ban

List jails...

docker exec -it mail.mydomain.org.uk-mailserver fail2ban-client status

Status
|- Number of jail:   3
`- Jail list:   dovecot, postfix, postfix-sasl

Manually ban IP address in named jail...

docker exec -it mail.mydomain.org.uk-mailserver fail2ban-client set postfix banip 212.70.149.56

Check banned IPs...

docker exec -it mail.mydomain.org.uk-mailserver fail2ban-client status postfix

Status for the jail: postfix
|- Filter
|  |- Currently failed: 2
|  |- Total failed:     2
|  `- File list:        /var/log/mail.log
`- Actions
   |- Currently banned: 1
   |- Total banned:     1
   `- Banned IP list:   212.70.149.56

https://www.the-lazy-dev.com/en/install-fail2ban-with-docker/

Backups

Backups

Autodiscover

Create SRV and A record entries in your DNS for the services...

$ORIGIN domain.org.uk.
@	300	IN	TXT	"v=spf1 mx ~all; mailconf=https://autoconfig.domain.org.uk/mail/config-v1.1.xml"
_autodiscover._tcp	300	IN	SRV	0 0 443 autodiscover.domain.org.uk.
_imap._tcp	300	IN	SRV	0 0 0 .
_imaps._tcp	300	IN	SRV	0 1 993 mail.domain.org.uk.
_ldap._tcp	300	IN	SRV	0 0 636 mail.domain.org.uk.
_pop3._tcp	300	IN	SRV	0 0 0 .
_pop3s._tcp	300	IN	SRV	0 0 0 .
_submission._tcp	300	IN	SRV	0 1 587 mail.domain.org.uk.
autoconfig	300	IN	A	3.10.67.19
autodiscover	300	IN	A	3.10.67.19
imap	300	IN	CNAME	mail
mail	300	IN	A	3.10.67.19
smtp	300	IN	CNAME	mail
www	300	IN	A	3.10.67.19

docker-compose.yml

services:
  mailserver-autodiscover:
    image: monogramm/autodiscover-email-settings:latest
    container_name: mail.domain.org.uk-mailserver-autodiscover
    environment:
      - COMPANY_NAME=My Company
      - SUPPORT_URL=https://autodiscover.domain.org.uk
      - DOMAIN=domain.org.uk
      - IMAP_HOST=mail.domain.org.uk
      - IMAP_PORT=993
      - IMAP_SOCKET=SSL
      - SMTP_HOST=mail.domain.org.uk
      - SMTP_PORT=587
      - SMTP_SOCKET=STARTTLS
    restart: unless-stopped
networks:
  default:
    external:
      name: nginx-proxy-manager

Autodiscover

monogramm/autodiscover-email-settings

Internet Speedtest

https://github.com/henrywhitaker3/Speedtest-Tracker

Emby Media Server

https://emby.media/docker-server.html

https://hub.docker.com/r/emby/embyserver

AWS CLI

docker run --rm -it -v "/root/.aws:/root/.aws" amazon/aws-cli configure
docker run --rm -it -v "/root/.aws:/root/.aws" amazon/aws-cli s3 ls
docker run --rm -it -v "/root/.aws:/root/.aws" amazon/aws-cli route53 list-hosted-zones

https://docs.aws.amazon.com/cli/latest/userguide/install-cliv2-docker.html

Let's Encrypt

Force RENEW a standalone certificate with the new preferred chain of "ISRG Root X1"

docker run -it --rm --name certbot -v "/usr/bin:/usr/bin" -v "/root/.aws:/root/.aws" -v "/etc/letsencrypt:/etc/letsencrypt" -v "/var/lib/letsencrypt:/var/lib/letsencrypt" certbot/certbot --force-renewal --preferred-chain "ISRG Root X1" certonly --standalone --email me@mydomain.com --agree-tos -d www.mydomain.com

Issue a wildcard certificate...

docker run -it --rm --name certbot -v "/usr/bin:/usr/bin" -v "/root/.aws:/root/.aws" -v "/etc/letsencrypt:/etc/letsencrypt" -v "/var/lib/letsencrypt:/var/lib/letsencrypt" certbot/dns-route53 certonly --dns-route53 --domain "example.com" --domain "*.example.com"

Check your certificates...

docker run -it --rm --name certbot -v "/usr/bin:/usr/bin" -v "/root/.aws:/root/.aws" -v "/etc/letsencrypt:/etc/letsencrypt" -v "/var/lib/letsencrypt:/var/lib/letsencrypt" certbot/certbot certificates

Renew a certificate...

docker run -it --rm --name certbot -v "/usr/bin:/usr/bin" -v "/root/.aws:/root/.aws" -v "/etc/letsencrypt:/etc/letsencrypt" -v "/var/lib/letsencrypt:/var/lib/letsencrypt" certbot/dns-route53 renew

If you have multiple profiles in your .aws/config then you will need to pass the AWS_PROFILE variable to the docker container...

docker run -it --rm --name certbot -v "/usr/bin:/usr/bin" -v "/root/.aws:/root/.aws" -v "/etc/letsencrypt:/etc/letsencrypt" -v "/var/lib/letsencrypt:/var/lib/letsencrypt" -e "AWS_PROFILE=certbot" certbot/dns-route53 renew

https://certbot.eff.org/docs/install.html#running-with-docker

VPN

Gluetun

Gluetun VPN client

Lightweight swiss-knife-like VPN client to tunnel to Cyberghost, ExpressVPN, FastestVPN, HideMyAss, IPVanish, IVPN, Mullvad, NordVPN, Perfect Privacy, Privado, Private Internet Access, PrivateVPN, ProtonVPN, PureVPN, Surfshark, TorGuard, VPNUnlimited, VyprVPN, WeVPN and Windscribe VPN servers using Go, OpenVPN or Wireguard, iptables, DNS over TLS, ShadowSocks and an HTTP proxy.

Github

Connect a container to Gluetun

OpenVPN

Server

https://hub.docker.com/r/linuxserver/openvpn-as

Client

https://hub.docker.com/r/dperson/openvpn-client

Routing Containers Through Container

sudo docker run -it --net=container:vpn -d some/docker-container

OpenVPN-PiHole

https://github.com/Simonwep/openvpn-pihole

WireHole

WireHole is a combination of WireGuard, PiHole, and Unbound in a docker-compose project with the intent of enabling users to quickly and easily create and deploy a personally managed full or split-tunnel WireGuard VPN with ad blocking capabilities (via Pihole), and DNS caching with additional privacy options (via Unbound).

https://github.com/IAmStoxe/wirehole

To view a QR code, run this ...

docker exec -it wireguard /app/show-peer 1

WireGuard

Use WireHole instead!

docker-compose.yml

version: "2.1"
services:
  wireguard:
    image: ghcr.io/linuxserver/wireguard
    container_name: wireguard
    cap_add:
      - NET_ADMIN
      - SYS_MODULE
    environment:
      - PUID=1000
      - PGID=1000
      - TZ=Europe/London
      - SERVERURL=wireguard.domain.uk
      - SERVERPORT=51820
      - PEERS=3
      - PEERDNS=auto
      - INTERNAL_SUBNET=10.13.13.0
      - ALLOWEDIPS=0.0.0.0/0
    volumes:
      - ./data/config:/config
      - /lib/modules:/lib/modules
    ports:
      - 51820:51820/udp
    sysctls:
      - net.ipv4.conf.all.src_valid_mark=1
    restart: unless-stopped

https://hub.docker.com/r/linuxserver/wireguard

To show the QR code

docker exec -it wireguard /app/show-peer 1
docker exec -it wireguard /app/show-peer 2
docker exec -it wireguard /app/show-peer 3

Error: const struct ipv6_stub

If you receive an error in the container logs about not being able to compile the kernel module, then follow the instructions to compile the WireGuard kernel module and tools in your host system.

https://github.com/linuxserver/docker-wireguard/issues/46#issuecomment-708278250

Force Docker Containers to use a VPN for connection

ffmpeg

docker pull jrottenberg/ffmpeg
docker run jrottenberg/ffmpeg -h
docker run jrottenberg/ffmpeg -i /path/to/input.mkv -stats $ffmpeg_options - > out.mp4
docker run -v $(pwd):$(pwd) -w $(pwd) jrottenberg/ffmpeg -y -i input.mkv -t 00:00:05.00 -vf scale=-1:360 output.mp4

https://registry.hub.docker.com/r/jrottenberg/ffmpeg

https://github.com/jrottenberg/ffmpeg

https://medium.com/coconut-stories/using-ffmpeg-with-docker-94523547f35c

https://github.com/linuxserver/docker-ffmpeg

MediaInfo

Install ...

sudo docker pull jlesage/mediainfo

Run ...

docker run --rm --name=mediainfo -e USER_ID=$(id -u) -e GROUP_ID=$(id -g) -v $(pwd):$(pwd):ro jlesage/mediainfo su-exec "$(id -u):$(id -g)" /usr/bin/mediainfo --help

https://github.com/jlesage/docker-mediainfo

MKV Toolnix

Install ...

sudo docker pull jlesage/mkvtoolnix

Run ...

mkvextract

docker run --rm --name=mkvextract -e USER_ID=$(id -u) -e GROUP_ID=$(id -g) -v $(pwd):$(pwd):rw jlesage/mkvtoolnix su-exec "$(id -u):$(id -g)" /usr/bin/mkvextract "$(pwd)/filename.mkv" tracks 3:"$(pwd)/filename.eng.srt"

mkvpropedit

docker run --rm --name=mkvextract -e USER_ID=$(id -u) -e GROUP_ID=$(id -g) -v $(pwd):$(pwd):rw jlesage/mkvtoolnix su-exec "$(id -u):$(id -g)" /usr/bin/mkvpropedit "$(pwd)/filename.mkv" --edit track:a1 --set language=eng

https://github.com/jlesage/docker-mkvtoolnix

MakeMKV

This will NOT work on a Raspberry Pi.

https://github.com/jlesage/docker-makemkv

Use this in combination with ffmpeg or HandBrake (as shown below) and FileBot to process your media through to your media server - like Emby or Plex..

MakeMKV > HandBrake > FileBot > Emby

To make this work with your DVD drive (/dev/sr0) you need to have the second device (/dev/sg0) in order for it to work. I don't get it, but it works.

/root/docker/stacks/makemkv/docker-compose.yml

version: '3'
services:
  makemkv:
    image: jlesage/makemkv
    container_name: makemkv
    ports:
      - "0.0.0.0:5801:5800"
    volumes:
      - "/home/user/.MakeMKV_DOCKER:/config:rw"
      - "/home/user/:/storage:ro"
      - "/home/user/ToDo/MakeMKV/output:/output:rw"
    devices:
      - "/dev/sr0:/dev/sr0"
      - "/dev/sg0:/dev/sg0"
    environment:
      - USER_ID=1000
      - GROUP_ID=1000
      - TZ=Europe/London
      - MAKEMKV_KEY=your_licence_key
      - AUTO_DISC_RIPPER=1

Troubleshooting

PROBLEM = "driver failed programming external connectivity on endpoint makemkv: Error starting userland proxy: listen tcp6 [::]:5800: socket: address family not supported by protocol."

SOLUTION = Put 0.0.0.0:5801 in the published ports line of docker compose to restrict the network to IPv4.

Docker Process Output

CONTAINER ID   IMAGE               COMMAND   CREATED        STATUS        PORTS                              NAMES
4a8b3106b00b   jlesage/handbrake   "/init"   40 hours ago   Up 40 hours   5900/tcp, 0.0.0.0:5802->5800/tcp   handbrake
89fe3ba8a31e   jlesage/makemkv     "/init"   40 hours ago   Up 40 hours   5900/tcp, 0.0.0.0:5801->5800/tcp   makemkv

Command Line

docker run --rm -v "/home/user/.MakeMKV_DOCKER:/config:rw" -v "/home/user/:/storage:ro" -v "/home/user/ToDo/MakeMKV/output:/output:rw" --device /dev/sr0 --device /dev/sg0 --device /dev/sg1 jlesage/makemkv /opt/makemkv/bin/makemkvcon mkv disc:0 all /output

https://github.com/jlesage/docker-makemkv/issues/141

HandBrake

This will NOT work on a Raspberry Pi.

Use this in combination with ffmpeg or MakeMKV (as shown below) and FileBot to process your media through to your media server - like Emby or Plex..

I have changed the port from 5800 to 5802 because Jocelyn's other Docker image for MakeMKV uses the same port (so I move that one as well to 5801 - see above).

To make this work with your DVD drive (/dev/sr0) you need to have the second device (/dev/sg0) in order for it to work. I don't get it, but it works.

YouTube / DB Tech - How to install HandBrake in Docker

Blog / DB Tech - How to install HandBrake in Docker

Docker HandBrake by Jocelyn Le Sage

Docker Image by Jocelyn Le Sage

/root/docker/stacks/handbrake/docker-compose.yml

version: '3'
services:
  handbrake:
    image: jlesage/handbrake
    container_name: handbrake
    ports:
      - "0.0.0.0:5802:5800"
    volumes:
      - "/home/paully:/storage:ro"
      - "/home/paully/ToDo/HandBrake/config:/config:rw"
      - "/home/paully/ToDo/HandBrake/watch:/watch:rw"
      - "/home/paully/ToDo/HandBrake/output:/output:rw"
    devices:
      - "/dev/sr0:/dev/sr0"
      - "/dev/sg0:/dev/sg0"
    environment:
      - USER_ID=1000
      - GROUP_ID=1000
      - TZ=Europe/London

Docker Process Output

CONTAINER ID   IMAGE               COMMAND   CREATED        STATUS        PORTS                              NAMES
4a8b3106b00b   jlesage/handbrake   "/init"   40 hours ago   Up 40 hours   5900/tcp, 0.0.0.0:5802->5800/tcp   handbrake
89fe3ba8a31e   jlesage/makemkv     "/init"   40 hours ago   Up 40 hours   5900/tcp, 0.0.0.0:5801->5800/tcp   makemkv

Command Line

docker run --rm -v "/home/paully/:/storage:ro" -v "/home/paully/ToDo/HandBrake/config:/config:rw" -v "/home/paully/ToDo/HandBrake/watch:/watch:rw" -v "/home/paully/ToDo/HandBrake/output:/output:rw" --device /dev/sr0 --device /dev/sg0 jlesage/handbrake /usr/bin/HandBrakeCLI --input "/output/file.mkv" --stop-at duration:120 --preset 'Fast 480p30' --non-anamorphic --encoder-preset slow --quality 22 --deinterlace --lapsharp --audio 1 --aencoder copy:ac3 --no-markers --output "/output/file.mp4"

FileBot

Setup

Create your directories for data volumes (https://github.com/jlesage/docker-filebot#data-volumes) ...

mkdir -p ~/filebot/{config,output,watch}

The license file received via email can be saved on the host, into the configuration directory of the container (i.e. in the directory mapped to /config). Then, start or restart the container to have it automatically installed. NOTE: The license file is expected to have a .psm extension.

Usage

WORK IN PROGRESS

Rather than running all the time, we run the image ad-hoc with the rm option to delete the old container each time...

docker run --rm --name=filebot -v ~/filebot/config:/config:rw -v $HOME:/storage:rw -v ~/filebot/output:/output:rw -v ~/filebot/watch:/watch:rw -e AMC_ACTION=test jlesage/filebot

https://github.com/jlesage/docker-filebot

Automated Downloaderr

This takes the hassle out of going through the various web sites to find stuff and be bombarded with ads and pop-ups.

  1. FlareSolverr
  2. Prowlarr
  3. Radarr + Sonarr + Bazarr
  4. Transmission + NZBGet
  5. Tdarr
FlareSolverr > Prowlarr > Radarr + Sonarr + Bazarr > Transmission + NZBGet > Tdarr


HOW TO RESTART THE RRS IN ORDER ON PORTAINER OR OMV

Stacks > Click on each one > Stop > count to 10 > Start

  • WireGuard
  • FlareSolverr
  • Prowlarr
  • Radarr
  • Sonarr

ONE DAY WE WILL GET A SINGLE STACK WITH ALL THE RIGHT CONTAINERS STARTING IN THE RIGHT ORDER


https://hotio.dev/containers/autoscan/

ONE APP TO RULE THEM ALL

https://github.com/JagandeepBrar/LunaSea


GUIDE FOR DIRECTORY STRUCTURE

https://trash-guides.info/Hardlinks/How-to-setup-for/Docker/

This will enable you to automatically rename files but allow you to copy them to your actual Plex or Emby folders.

It is possible to do hard linking and let the rrrrs control all the files but I am not a fan of that.

This way, the files get renamed and moved to a 'halfway' house where you can check them and then simply move them to your desired location.

METHOD

Create the directories ...

mkdir -p /path/to/data/{media,torrents,usenet}/{movies,music,tv}
mkdir -p /path/to/docker/appdata/{radarr,sonarr,bazarr,nzbget}

Change the ownership and permissions ...

chown -R admin:users /path/to/data/ /path/to/docker/
find /path/to/data/ /path/to/docker/ -type d -exec chmod 0755 {} \;
find /path/to/data/ /path/to/docker/ -type d -exec chmod g+s {} \;

Finished directory structure ...

/srv/dev-disk-by-uuid-7f81e7b6-1a05-4232-893b-f34c046b2bdb/data/
|-- media
|   |-- movies
|   |-- music
|   `-- tv
|-- torrents
|   |-- movies
|   |-- music
|   `-- tv
`-- usenet
    |-- completed
    |   |-- movies
    |   `-- tv
    |-- movies
    |-- music
    |-- nzb
    |   `-- Movie.Name.720p.nzb.queued
    |-- tmp
    `-- tv

PORTAINER STACK

This is from Open Media Vault (OMV) so the volume paths are long.

This works but needs the whole VPN thing added (which changes ports etc) but for now ...

Portainer > Stacks > Add Stack > Datarr

version: "3.2"
services:
  prowlarr:
    container_name: prowlarr
    image: hotio/prowlarr:latest
    restart: unless-stopped
    logging:
      driver: json-file
    network_mode: bridge
    ports:
      - 9696:9696
    environment:
      - PUID=998
      - PGID=100
      - TZ=Europe/London
    volumes:
      - /etc/localtime:/etc/localtime:ro
      - /srv/dev-disk-by-uuid-7f81e7b6-1a05-4232-893b-f34c046b2bdb/docker/appdata/prowlarr:/config
      - /srv/dev-disk-by-uuid-7f81e7b6-1a05-4232-893b-f34c046b2bdb/data:/data
  radarr:
    container_name: radarr
    image: hotio/radarr:latest
    restart: unless-stopped
    logging:
      driver: json-file
    network_mode: bridge
    ports:
      - 7878:7878
    environment:
      - PUID=998
      - PGID=100
      - TZ=Europe/London
    volumes:
      - /etc/localtime:/etc/localtime:ro
      - /srv/dev-disk-by-uuid-7f81e7b6-1a05-4232-893b-f34c046b2bdb/docker/appdata/radarr:/config
      - /srv/dev-disk-by-uuid-7f81e7b6-1a05-4232-893b-f34c046b2bdb/data:/data
  sonarr:
    container_name: sonarr
    image: hotio/sonarr:latest
    restart: unless-stopped
    logging:
      driver: json-file
    network_mode: bridge
    ports:
      - 8989:8989
    environment:
      - PUID=998
      - PGID=100
      - TZ=Europe/London
    volumes:
      - /etc/localtime:/etc/localtime:ro
      - /srv/dev-disk-by-uuid-7f81e7b6-1a05-4232-893b-f34c046b2bdb/docker/appdata/sonarr:/config
      - /srv/dev-disk-by-uuid-7f81e7b6-1a05-4232-893b-f34c046b2bdb/data:/data
  bazarr:
    container_name: bazarr
    image: hotio/bazarr:latest
    restart: unless-stopped
    logging:
      driver: json-file
    network_mode: bridge
    ports:
      - 6767:6767
    environment:
      - PUID=998
      - PGID=100
      - TZ=Europe/London
    volumes:
      - /etc/localtime:/etc/localtime:ro
      - /srv/dev-disk-by-uuid-7f81e7b6-1a05-4232-893b-f34c046b2bdb/docker/appdata/bazarr:/config
      - /srv/dev-disk-by-uuid-7f81e7b6-1a05-4232-893b-f34c046b2bdb/data/media:/data/media
  nzbget:
    container_name: nzbget
    image: hotio/nzbget:latest
    restart: unless-stopped
    logging:
      driver: json-file
    network_mode: bridge
    ports:
      - 6789:6789
    environment:
      - PUID=998
      - PGID=100
      - TZ=Europe/London
    volumes:
      - /etc/localtime:/etc/localtime:ro
      - /srv/dev-disk-by-uuid-7f81e7b6-1a05-4232-893b-f34c046b2bdb/docker/appdata/nzbget:/config
      - /srv/dev-disk-by-uuid-7f81e7b6-1a05-4232-893b-f34c046b2bdb/data/usenet:/data/usenet:rw


OLD NOTES

Create a docker network which Jackett and Radarr share to talk to each other...

sudo docker network create jackett-radarr

...then continue setting up the containers below.

FlareSolverr

FlareSolverr is a proxy server to bypass Cloudflare protection.

FlareSolverr starts a proxy server and it waits for user requests in an idle state using few resources. When some request arrives, it uses puppeteer with the stealth plugin to create a headless browser (Chrome). It opens the URL with user parameters and waits until the Cloudflare challenge is solved (or timeout). The HTML code and the cookies are sent back to the user, and those cookies can be used to bypass Cloudflare using other HTTP clients.

Radarr > Jackett > FlareSolverr > Internet

https://github.com/FlareSolverr/FlareSolverr

https://hub.docker.com/r/flaresolverr/flaresolverr

Some indexers are protected by CloudFlare or similar services and Jackett is not able to solve the challenges. For these cases, FlareSolverr has been integrated into Jackett. This service is in charge of solving the challenges and configuring Jackett with the necessary cookies. Setting up this service is optional, most indexers don't need it.

Install FlareSolverr service using a Docker container, then configure FlareSolverr API URL in Jackett. For example: http://172.17.0.2:8191

Command line...

docker run -d \
 --name=flaresolverr \
 -p 8191:8191 \
 -e LOG_LEVEL=info \
 --restart unless-stopped \
 ghcr.io/flaresolverr/flaresolverr:latest

Docker compose...

---
version: "2.1"
services:
  flaresolverr:
    image: ghcr.io/flaresolverr/flaresolverr:latest
    container_name: flaresolverr
    environment:
      - PUID=1000
      - PGID=1000
      - LOG_LEVEL=${LOG_LEVEL:-info}
      - LOG_HTML=${LOG_HTML:-false}
      - CAPTCHA_SOLVER=${CAPTCHA_SOLVER:-none}
      - TZ=Europe/London
    ports:
      - "${PORT:-8191}:8191"
    restart: unless-stopped

Usage

To use it, you have to add a Proxy Server under Prowlarr > Settings > Indexers > Indexer Proxies

Testing

curl -L -X POST 'http://localhost:8191/v1' -H 'Content-Type: application/json' --data-raw '{ "cmd": "request.get", "url":"https://www.paully.co.uk/", "maxTimeout": 60000 }'

{"status":"ok","message":"","startTimestamp":1651659265156,"endTimestamp":1651659269585,"version":"v2.2.4","solution":

Jackett

Jackett works as a proxy server: it translates queries from apps (Sonarr, SickRage, CouchPotato, Mylar, etc) into tracker-site-specific http queries, parses the html response, then sends results back to the requesting software. This allows for getting recent uploads (like RSS) and performing searches. Jackett is a single repository of maintained indexer scraping and translation logic - removing the burden from other apps.

So, this is where you build your list of web sites "with content you want" ;-)

https://fleet.linuxserver.io/image?name=linuxserver/jackett

https://docs.linuxserver.io/images/docker-jackett

https://hub.docker.com/r/linuxserver/jackett

https://github.com/Jackett/Jackett

/root/docker/stacks/docker-compose.yml

---
version: "2.1"
services:
  jackett:
    image: ghcr.io/linuxserver/jackett
    container_name: jackett
    environment:
      - PUID=1000
      - PGID=1000
      - TZ=Europe/London
      - AUTO_UPDATE=true
    volumes:
      - ./data/config:/config
      - ./data/downloads:/downloads
    networks:
      - jackett-radarr
    ports:
      - 0.0.0.0:9117:9117
    restart: unless-stopped
networks:
  jackett-radarr:
    external: true

Prowlarr

An alternative to Jackett, and now the preferred application.

https://wiki.servarr.com/prowlarr

https://wiki.servarr.com/prowlarr/quick-start-guide

https://hub.docker.com/r/linuxserver/prowlarr

https://github.com/linuxserver/docker-prowlarr

---
version: "2.1"
services:
  prowlarr:
    image: lscr.io/linuxserver/prowlarr:develop
    container_name: prowlarr
    environment:
      - PUID=1000
      - PGID=1000
      - TZ=Europe/London
    volumes:
      - /path/to/data:/config
    ports:
      - 9696:9696
    restart: unless-stopped

Radarr

Radarr is a movie collection manager for Usenet and BitTorrent users. It can monitor multiple RSS feeds for new movies and will interface with clients and indexers to grab, sort, and rename them. It can also be configured to automatically upgrade the quality of existing files in the library when a better quality format becomes available.

Radarr is the 'man-in-the-middle' to take lists from Jackett and pass them to Transmission to download.

Radarr is the web UI to search for "the content you want" ;-)

https://radarr.video/

https://docs.linuxserver.io/images/docker-radarr

https://github.com/linuxserver/docker-radarr

https://sasquatters.com/radarr-docker/

https://www.smarthomebeginner.com/install-radarr-using-docker/

https://trash-guides.info/Radarr/

https://discord.com/channels/264387956343570434/264388019585286144

Custom Scripts

So, you use Jackett as an Indexer of content, which answers questions from Radarr, which passes a good result to Transmission...

  1. Settings > Profiles > delete all but 'any' (and edit that to get rid of naff qualities at the bottom)
  2. Indexers > Add Indexer > Torznab > complete and TEST then SAVE
  3. Download Clients > Add Download Client > Transmission > complete and TEST and SAVE

/root/docker/stacks/docker-compose.yml

---
version: "2.1"
services:
  radarr:
    image: ghcr.io/linuxserver/radarr
    container_name: radarr
    environment:
      - PUID=1000
      - PGID=1000
      - TZ=Europe/London
    volumes:
      - ./data/config:/config
      - ./data/downloads:/downloads
      - ./data/torrents:/torrents
    networks:
      - jackett-radarr
    ports:
      - 0.0.0.0:7878:7878
    restart: unless-stopped
networks:
  jackett-radarr:
    external: true

Sonarr

Sonarr (formerly NZBdrone) is a PVR for usenet and bittorrent users. It can monitor multiple RSS feeds for new episodes of your favorite shows and will grab, sort and rename them. It can also be configured to automatically upgrade the quality of files already downloaded when a better quality format becomes available.

https://sonarr.tv/

https://docs.linuxserver.io/images/docker-sonarr

Docker Compose using a WireGuard VPN container for internet ...

---
version: "2.1"
services:
  sonarr:
    image: ghcr.io/linuxserver/sonarr
    container_name: sonarr
    network_mode: container:wireguard
    environment:
      - PUID=1000
      - PGID=1000
      - TZ=Europe/London
    volumes:
      - ./data/config:/config
      - ./data/downloads:/downloads
      - ./data/torrents:/torrents
    restart: "no"

UPDATE: 16 FEBRUARY 2023 / latest image is based on Ubuntu Jammy and not Alpine, which will cause a problem if your Docker version is below 20.10.10

https://docs.linuxserver.io/faq#jammy

To fix this, you can either upgrade your Docker (https://docs.docker.com/engine/install/ubuntu/#install-using-the-convenience-script) or add the following lines to your docker-compose.yml file ...

security_opt:
  - seccomp=unconfined

Bazarr

https://www.bazarr.media/

Bazarr is a companion application to Sonarr and Radarr that manages and downloads subtitles based on your requirements.

Docker compose file ...

---
version: "2.1"
services:
  bazarr:
    image: lscr.io/linuxserver/bazarr:latest
    container_name: bazarr
    environment:
      - PUID=1000
      - PGID=1000
      - TZ=Europe/London
    volumes:
      - /path/to/bazarr/config:/config
      - /path/to/movies:/movies #optional
      - /path/to/tv:/tv #optional
    ports:
      - 6767:6767
    restart: unless-stopped

NZBGet

NZBGet is a usenet downloader.

You will require the following 3 things at a basic level before you are able to use usenet:-

https://nzbget.net

https://hub.docker.com/r/linuxserver/nzbget

https://www.cogipas.com/nzbget-complete-how-to-guide/

The Web GUI can be found at <your-ip>:6789 and the default login details (change ASAP) are...

username: nzbget
password: tegbzn6789

Docker Compose file...

---
version: "2.1"
services:
  nzbget:
    image: lscr.io/linuxserver/nzbget:latest
    container_name: nzbget
    environment:
      - PUID=1000
      - PGID=1000
      - TZ=Europe/London
      - NZBGET_USER=nzbget
      - NZBGET_PASS=tegbzn6789
    volumes:
      - /path/to/data:/config
      - /path/to/downloads:/downloads
    ports:
      - 6789:6789
    restart: unless-stopped

Tdarr

Tdarr is a popular conditional transcoding application for processing large (or small) media libraries. The application comes in the form of a click-to-run web-app, which you run on your own device and access through a web browser.

Tdarr uses two popular transcoding applications under the hood: FFmpeg and HandBrake (which itself is built on top of FFmpeg).

Tdarr works in a distributed manner where you can use multiple devices to process your library together. It does this using 'Tdarr Nodes' which connect with a central server and pick up tasks so you can put all your spare devices to use.

Each Node can run multiple 'Tdarr Workers' in parallel to maximize the hardware usage % on that Node. For example, a single FFmpeg worker running on a 64 core CPU may only hit ~30% utilization. Running multiple Workers in parallel allows the CPU to hit 100% utilization, allowing you to process your library more quickly.

Home Page

Documentation

Using AMD GPU for Transcoding

Readarr

https://academy.pointtosource.com/containers/ebooks-calibre-readarr/

Unpackerr

Extracts downloads for Radarr, Sonarr, Lidarr, Readarr, and/or a Watch folder - Deletes extracted files after import.

https://github.com/Unpackerr/unpackerr

Docker Compose - https://github.com/Unpackerr/unpackerr/blob/main/examples/docker-compose.yml

Calibre

eBook management and automation using Calibre, COPS or Calibre-Web, and Readarr.

Auto format conversion.

https://academy.pointtosource.com/containers/ebooks-calibre-readarr/

YouTube-DL

https://registry.hub.docker.com/search?q=youtube&sort=updated_at&order=desc

https://registry.hub.docker.com/r/mikenye/youtube-dl#quick-start

Nagios

Work in progress.

This is an old version of Nagios in the container image, so will look for a newer one.

/root/docker/stacks/nagios/docker-compose.yml

version: '3'
services:
  nagios:
    image: jasonrivers/nagios
    container_name: nagios
    restart: unless-stopped
    ports:
      - 8181:80
    # volumes:
      # - ./data/etc/:/opt/nagios/etc/
    environment:
      - PUID=999
      - PGID=1000
      - TZ=Europe/London
      - NAGIOS_TIMEZONE=Europe/London

Start it with no volume mounts, then copy the etc directory to your host...

cd /root/docker/stacks/nagios/
docker cp nagios:/opt/nagios/etc data/
chown -R 999:1000 data/

...then uncomment the # lines in the docker-compose file and restart the container.

Credentials

The default credentials for the web interface is nagiosadmin / nagios

To change the password, generate a new one by logging in to the container and running 'htpasswd'...

docker exec -it nagios bash
htpasswd -n nagiosadmin
(copy the output)

...then editing the /opt/nagios/etc/htpasswd.users file and refreshing the admin web page.

https://github.com/ethnchao/docker-nagios

http://www.kraftinfosec.com/running-nagios-in-docker/

https://github.com/JasonRivers/Docker-Nagios

Tandoor Recipe Manager

The recipe manager that allows you to manage your ever growing collection of digital recipes.

https://docs.tandoor.dev/install/docker/

https://www.youtube.com/watch?v=7-nb3muJxI0

/root/docker/stacks/tandoor/docker-compose.yml

version: "3"
services:
  db_recipes:
    container_name: tandoor_db
    restart: always
    image: postgres:11-alpine
    volumes:
      - ./data/postgresql:/var/lib/postgresql/data
    env_file:
      - ./.env
  web_recipes:
    container_name: tandoor_web
    image: vabene1111/recipes
    restart: always
    env_file:
      - ./.env
    volumes:
      - ./data/mediafiles:/opt/recipes/mediafiles
      - ./data/staticfiles:/opt/recipes/staticfiles
      - nginx_config:/opt/recipes/nginx/conf.d
    depends_on:
      - db_recipes
  nginx_recipes:
    container_name: tandoor_nginx
    image: nginx:mainline-alpine
    restart: always
    ports:
      - 80
    env_file:
      - ./.env
    depends_on:
      - web_recipes
    volumes:
      - ./data/mediafiles:/media
      - ./data/staticfiles:/static
      - nginx_config:/etc/nginx/conf.d:ro
volumes:
  nginx_config:
networks:
  default:
    external:
      name: nginx-proxy-manager

NextCloud

IMPORTANT


If you are receiving errors about PHP or issues with nginx or certificates:

  1. Switch your image to lscr.io/linuxserver/nextcloud:24.0.6-ls204 and start the container
  2. Execute docker exec -it nextcloud updater.phar repeatedly until there are no more updates (as of writing, Nextcloud 25 is the latest version)
  3. Switch your image to lscr.io/linuxserver/nextcloud (latest, no tag) and start the container
  4. Execute docker exec -it nextcloud mv /config/nginx/site-confs/default.conf /config/nginx/site-confs/default.conf.bak
  5. Execute docker exec -it nextcloud mv /config/nginx/nginx.conf /config/nginx/nginx.conf.bak
  6. Execute docker logs nextcloud and check for any other outdated configs, rename them with a .bak extension (like above)
  7. Restart the container
  8. Nextcloud should now be in a working state



Nextcloud gives you access to all your files wherever you are.

Create your container folders...

mkdir -p /root/docker/stacks/nextcloud/data/{config,files}
chown -R 1000:1000 /root/docker/stacks/nextcloud/data/

/root/docker/stacks/nextcloud/docker-compose.yml

version: "2.1"
services:
  nextcloud:
    image: ghcr.io/linuxserver/nextcloud
    container_name: nextcloud
    environment:
      - PUID=1000
      - PGID=1000
      - TZ=Europe/London
    volumes:
      - ./data/config:/config
      - ./data/files:/data
    ports:
      - 4443:443
    restart: unless-stopped

(I have changed the default port it listens on the local machine to 4443 but if you don't need any ports open then change the lines to:-

    expose:
      - 443

...then use Nginx Proxy Manager to direct traffic to your NextCloud installation)

Now visit https://ip.address.of.host:4443

...and on that setup page, _untick_ the option for "Install recommended apps" which does not install Calendar, Contacts, Mail, Chat, etc.

Enjoy.

https://hub.docker.com/r/linuxserver/nextcloud

Command Line Admin OCC

https://docs.nextcloud.com/server/latest/admin_manual/configuration_server/occ_command.html

List Users

How do I list the users in NextCloud Docker?

sudo docker exec -it nextcloud occ user:list

Reset Admin Password

How do you reset the admin user password in NextCloud Docker?

sudo docker exec -it nextcloud occ user:resetpassword admin

https://docs.nextcloud.com/server/latest/admin_manual/configuration_user/reset_admin_password.html

Project Send

Self-hosted file sharing... small, simple, secure.

https://www.projectsend.org

https://docs.linuxserver.io/images/docker-projectsend

https://github.com/linuxserver/docker-projectsend

What they don't tell you in the docs is that you need a database backend - which is not in the docker compose file.

So, we just add a MariaDB database container to the stack!

Create your subdomain A record in DNS...

cli53 rrcreate domain.uk 'send 300 A 123.45.678.90'

Create your Proxy Host in Ngnix Proxy Manager with an SSL...

https://send.domain.uk

Create directories on the server for the Docker container files...

sudo -i
mkdir -p /root/docker/stacks/projectsend/data/{config,db,files}
chown -R 1000:1000 /root/docker/stacks/projectsend/data/files

/root/docker/stacks/projectsend/docker-compose.yml

version: "2.1"
services:
  projectsend:
    image: ghcr.io/linuxserver/projectsend
    container_name: projectsend
    environment:
      - PUID=1000
      - PGID=1000
      - TZ=Europe/London
      - MAX_UPLOAD=100
    volumes:
      - ./data/config:/config
      - ./data/files:/data
      - /etc/timezone:/etc/timezone:ro
    expose:
      - 80
    restart: unless-stopped
  projectsend-db:
    image: mariadb
    container_name: projectsend-db
    environment:
      TZ: Europe/London
      MYSQL_ROOT_PASSWORD: projectsend
      MYSQL_DATABASE: projectsend
      MYSQL_USER: projectsend
      MYSQL_PASSWORD: projectsend
    volumes:
      - ./data/db:/var/lib/mysql
      - /etc/timezone:/etc/timezone:ro
    restart: unless-stopped
networks:
  default:
    external:
      name: nginx-proxy-manager

Go to the secure web site URL and complete the installation, using the Docker container name for the 'Database hostname'.

projectsend-db

Then, log in with your Admin username and password...

  1. Create Group 'Public' which is public.
  2. Create Group 'Customers' which is not public.
  3. Create Client 'Customer Name' which is assigned to the 'Customers' group.
  4. Upload some files and test both the Public and Client links.

Enjoy.

Troubleshooting

Change the Site URL

If you move host or domain name, you can log in to the DB container and change the 'base_uri'...

docker exec -it projectsend-db bash
mysql -u root -p projectsend

MariaDB [projectsend]> 
MariaDB [projectsend]> select * from tbl_options where name = 'base_uri';
+----+----------+-------------------------+
| id | name     | value                   |
+----+----------+-------------------------+
|  1 | base_uri | https://send.domain.uk/ |
+----+----------+-------------------------+

MediaWiki

Installation

Create the docker compose file and use default volume. Go to your browser at http://localhost:8080 and finish setup. Download LocalSettings.php file and copy to it to the container filesystem, then copy the whole folder to the host filsystem...

docker cp LocalSettings.php mediawiki:/var/www/html/
docker cp mediawiki:/var/www/html /root/docker/stacks/mediawiki/data/
chown -R www-data:www-data data/html
chmod o-w data/html
docker-compose down
(then edit your docker-compose.yml file so that local folders are used)
docker-compose up -d

Now, all the files are on your docker folder, ready to easily backup :-)

mediawiki
`-- data
    |-- db
    `-- html

~/docker/mediawiki/docker-compose.yml

version: '3'
services:
  mediawiki:
    image: mediawiki
    container_name: mediawiki
    restart: always
    ports:
      - 8080:80
    links:
      - database
    volumes:
      #- ./data/html:/var/www/html     <-- #2
      #- /var/www/html/images      <-- #1
    environment:
      - PUID=33
      - PGID=33
      - TZ=Europe/London
  database:
    image: mariadb
    container_name: mediawiki_db
    restart: always
    environment:
      MYSQL_DATABASE: my_wiki
      MYSQL_USER: wikiuser
      MYSQL_PASSWORD: example
      MYSQL_RANDOM_ROOT_PASSWORD: 'yes'
    volumes:
      - ./data/db:/var/lib/mysql

https://hub.docker.com/_/mediawiki

Tweaks

Change default skin to mobile responsive modern one...

wfLoadSkin( 'Timeless' );
$wgDefaultSkin = "timeless";

Enable the new editing toolbar...

wfLoadExtension( 'WikiEditor' );

Make the URL shorter...

$wgScriptPath = "";
$wgScriptExtension = ".php";
$wgArticlePath = "/wiki/$1";
$wgUsePathInfo = true;

File uploads...

LocalSettings.php

$wgEnableUploads = true;

https://www.mediawiki.org/wiki/Manual:Configuring_file_uploads

https://kindalame.com/2020/11/25/self-hosting-mediawiki-with-docker/

Importing

Pages

OLD SERVER

Generate the page dump in XML format...

docker exec -it mediawiki bash
php maintenance/dumpBackup.php --current > pages.xml
exit

NEW SERVER

Import the pages...

cp pages.xml ./data/html/
docker exec -it mediawiki bash
php maintenance/importDump.php < pages.xml
php maintenance/update.php
php maintenance/rebuildall.php
exit

https://www.hostknox.com/tutorials/mediawiki/pages/export-and-import#import-pages-via-ssh

Images

OLD SERVER

Generate the image dumps using dumpUploads.php, which creates a txt list of all image filenames in use...

mkdir /tmp/workingBackupMediaFiles
php maintenance/dumpUploads.php \
   | sed 's~mwstore://local-backend/local-public~./images~' \
   | xargs cp -t /tmp/workingBackupMediaFiles
zip -r ~/Mediafiles.zip /tmp/workingBackupMediaFiles
rm -r /tmp/workingBackupMediaFiles

NEW SERVER

Unzip the files to your container filsystem...

cd /root/docker/stacks/mediawiki
unzip Mediafiles.zip -d ./data/html/

Import the Images...

docker exec -it mediawiki bash
php maintenance/importImages.php tmp/workingBackupMediaFiles
php maintenance/update.php
php maintenance/rebuildall.php
exit

https://stackoverflow.com/questions/1002258/exporting-and-importing-images-in-mediawiki

Kuma

A self-hosted monitoring tool like Uptime Robot and not as complicated as Nagios.

https://hub.docker.com/r/louislam/uptime-kuma

https://github.com/louislam/uptime-kuma

https://youtu.be/dIVf1nhT0mI

Kasm Containerized Apps and Desktops

Streaming containerized apps and desktops to end-users. The Workspaces platform provides enterprise-class orchestration, data loss prevention, and web streaming technology to enable the delivery of containerized workloads to your browser.

https://kasmweb.com

https://hub.docker.com/u/kasmweb

https://www.youtube.com/channel/UCgpv4MLH8diVlIiakCBu8eQ

SFTP

https://hub.docker.com/r/atmoz/sftp

Wordle

https://github.com/cwackerfuss/react-wordle

Ombi

Ombi allows you to host your own Emby Request and user management system. If you are sharing your Emby server with other users, allow them to request new content using an easy to manage interface! Manage all your requests for Movies and TV with ease, leave notes for the user and get notification when a user requests something. Allow your users to post issues against their requests so you know there is a problem with the audio etc. Even automatically send them weekly newsletters of new content that has been added to your Emby server :-)

https://hub.docker.com/r/linuxserver/ombi

Emby

https://fleet.linuxserver.io/image?name=linuxserver/emby

https://hub.docker.com/r/linuxserver/emby

---
version: "2.1"
services:
  emby:
    image: lscr.io/linuxserver/emby
    container_name: emby
    environment:
      - PUID=998
      - PGID=100
      - TZ=Europe/London
    volumes:
      - /srv/dev-disk-by-uuid-7f81e7b6-1a05-4232-893b-f34c046b2bdb/Emby/Config:/config
      - /srv/dev-disk-by-uuid-7f81e7b6-1a05-4232-893b-f34c046b2bdb/Emby/TV:/data/tvshows
      - /srv/dev-disk-by-uuid-7f81e7b6-1a05-4232-893b-f34c046b2bdb/Emby/Movies:/data/movies
    ports:
      - 8096:8096
    restart: unless-stopped

Tdarr

Tdarr is a conditional based transcoding application for automating media library transcoding and remux management which uses cross-platform Tdarr Nodes which work together with Tdarr Server to process your files.

https://docs.tdarr.io/docs/installation/docker/run-compose

version: "3.4"

services:

# server
  tdarr:
    container_name: tdarr
    image: ghcr.io/haveagitgat/tdarr:latest
    restart: unless-stopped
    network_mode: bridge
    ports:
      - 8265:8265 # webUI port
      - 8266:8266 # server port
      - 8267:8267 # Internal node port
      - 8268:8268 # Example extra node port
    environment:
      - TZ=Europe/London
      - PUID=${PUID}
      - PGID=${PGID}
      - UMASK_SET=002
      - serverIP=0.0.0.0
      - serverPort=8266
      - webUIPort=8265
      - internalNode=true
      - nodeID=MyInternalNode
    volumes:
      - /docker/tdarr/server:/app/server
      - /docker/tdarr/configs:/app/configs
      - /docker/tdarr/logs:/app/logs
      - /media:/media
      - /transcode_cache:/temp

# node example
  tdarr-node:
    container_name: tdarr-node
    image: ghcr.io/haveagitgat/tdarr_node:latest
    restart: unless-stopped
    network_mode: service:tdarr
    environment:
      - TZ=Europe/London
      - PUID=${PUID}
      - PGID=${PGID}
      - UMASK_SET=002
      - nodeID=MainNode
      - serverIP=0.0.0.0
      - serverPort=8266
    volumes:
      - /docker/tdarr/configs:/app/configs
      - /docker/tdarr/logs:/app/logs
      - /media:/media
      - /transcode_cache:/temp

Unifi Controller

https://github.com/linuxserver/docker-unifi-controller

If you want to fix a particular version of the controller then check the 'releases' at github and adjust your docker-compose.yml file accordingly.

e.g. linuxserver/unifi-controller:5.14.23-ls76

version: "2.1"
services:
  unifi-controller:
    # image: linuxserver/unifi-controller:latest
    image: linuxserver/unifi-controller:5.14.23-ls76
    container_name: unifi-controller
    environment:
      - PUID=998
      - PGID=100
      # - MEM_LIMIT=1024 #optional
      # - MEM_STARTUP=1024 #optional
    volumes:
      - <path to data>:/config
    ports:
      - 8443:8443
      - 3478:3478/udp
      - 10001:10001/udp
      - 8080:8080
      - 1900:1900/udp #optional
      - 8843:8843 #optional
      - 8880:8880 #optional
      - 6789:6789 #optional
      - 5514:5514/udp #optional
    restart: unless-stopped

rPort

Rport helps you to manage your remote servers without the hassle of VPNs, chained SSH connections, jump-hosts, or the use of commercial tools like TeamViewer and its clones.

Rport acts as server and client establishing permanent or on-demand secure tunnels to devices inside protected intranets behind a firewall.

All operating systems provide secure and well-established mechanisms for remote management, being SSH and Remote Desktop the most widely used. Rport makes them accessible easily and securely.

https://rport.io/en/features

https://oss.rport.io/

https://github.com/cloudradar-monitoring/rport

https://hub.docker.com/r/acwhiteglint/rport

Paperless

Paperless is an application that indexes your scanned documents and allows you to easily search for documents and store metadata alongside your documents.

https://github.com/jonaswinkler/paperless-ng

https://paperless-ng.readthedocs.io/en/latest/setup.html#installation

Ansible

So, this is not installing Docker using Ansible... this is installing (or running) Ansible using Docker :-)

https://iceburn.medium.com/run-ansible-with-docker-9eb27d75285b

Pi Alert

WIFI / LAN intruder detector. Scan the devices connected to your WIFI / LAN and alert you the connection of unknown devices. It also warns the disconnection of "always connected" devices.

https://github.com/pucherot/Pi.Alert

Roundcube Webmail

https://hub.docker.com/search?q=roundcube

https://hub.docker.com/r/roundcubeorg/roundcubemail

https://github.com/roundcube/roundcubemail/wiki/Configuration

docker run -e ROUNDCUBEMAIL_DEFAULT_HOST=ssl://mail.domain.co.uk -e ROUNDCUBEMAIL_DEFAULT_PORT=993 -e ROUNDCUBEMAIL_SMTP_SERVER=tls://mail.domain.co.uk -e ROUNDCUBEMAIL_SMTP_PORT=587 -e ROUNDCUBEMAIL_PLUGINS=archive,zipdownload,managesieve,mobile -p 8000:80 -d roundcube/roundcubemail

FAST SpeedTest

docker run -it --rm --name fast-cli mschirrmeister/fast-cli:latest

https://hub.docker.com/r/mschirrmeister/fast-cli

https://github.com/sindresorhus/fast-cli

Crypto Mining

XMRig

https://hub.docker.com/r/minerboy/xmrig

Go to the XMRig Wizard and generate config like this ...

{
    "autosave": true,
    "cpu": true,
    "opencl": false,
    "cuda": false,
    "pools": [
        {
            "coin": "monero",
            "algo": "rx/0",
            "url": "stratum+tcp://randomxmonero.auto.nicehash.com:9200",
            "user": "NHbLd5exQeCGGyWnopVoLHLbzexKN5z8iq7p.NAS",
            "pass": "x",
            "tls": false,
            "keepalive": true,
            "nicehash": true
        }
    ]
}

Then save that file as config.json and run the following commands to start mining ...

sudo -i
docker pull minerboy/xmrig
docker run --cap-add=SYS_ADMIN --cap-add=SYS_RAWIO --device=/dev/cpu --device=/dev/mem -v /lib/modules:/lib/modules -v /full/path/to/config.json:/etc/xmrig/config.json minerboy/xmrig:latest --cpu-max-threads-hint 50 --threads 2

Here is the same command as a lovely docker compose file ...

version: '3.3'
services:
  xmrig:
      image: 'minerboy/xmrig:latest'
      container_name: xmrig
      devices:
        - /dev/cpu
        - /dev/mem
      volumes:
        - '/lib/modules:/lib/modules'
        - '/root/docker/stacks/xmrig/config.json:/etc/xmrig/config.json'
      command:
        - --cpu-max-threads-hint 50
        - --threads 2
      restart: "no"

Help

DB Tech

https://www.youtube.com/channel/UCVy16RS5eEDh8anP8j94G2A

https://gist.github.com/dnburgess