Docker

Introduction

Docker is a set of platform as a service products that use OS-level virtualization to deliver software in packages called containers. Containers are isolated from one another and bundle their own software, libraries and configuration files but they can communicate with each other through well-defined channels.

https://www.docker.com - Official Web Site.

https://hub.docker.com - Official Repository of Container Images.

It was originally developed for programmers to test their software but has now become a fully fledged answer to running servers in mission critical situations.

Each container has a mini operating system plus the software needed to run the program you want, and no more.

All of the 'hard work' for a piece of software has been 'done for you' and the end result is starting a program with one command line.

For example, the WordPress image contains the LAP part of LAMP (Linux + Apache + PHP) all configured and running.

Images

Docker Hub

LinuxServer

Useful Wiki

https://wiki.servarr.com/

Installation

WINDOWS

Make sure your computer supports Hardware Virtualisation by checking in the BIOS.
Install Docker Desktop.
Install the Windows Subsystem for Linux Kernel Update.
Reboot.

LINUX

New All In One Official Method

sudo -i
wget -qO- https://raw.githubusercontent.com/pi-hosted/pi-hosted/master/install_docker.sh | bash

which calls ...

https://get.docker.com/

Engine

This will remove the old version of 'Docker' and install the new version 'Docker CE'...

sudo apt-get remove docker docker-engine docker.io containerd runc
sudo apt-get update
sudo apt-get -y install apt-transport-https ca-certificates curl gnupg software-properties-common
sudo mkdir -p /etc/apt/keyrings
curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo gpg --dearmor -o /etc/apt/keyrings/docker.gpg
echo "deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.gpg] https://download.docker.com/linux/ubuntu $(lsb_release -cs) stable" | sudo tee /etc/apt/sources.list.d/docker.list > /dev/null
sudo apt-get update
sudo apt-get -y install docker-ce docker-ce-cli containerd.io docker-compose-plugin
sudo service docker start
sudo docker run hello-world

https://docs.docker.com/install/linux/docker-ce/ubuntu/#install-docker-ce-1

Compose From Command Line

http://composerize.com - Turns docker run commands into docker-compose files!

Install composerize and convert your own commands locally.

sudo apt instal npm
sudo npm install composerize -g
composerize docker run -p 80:80 -v /var/run/docker.sock:/tmp/docker.sock:ro --restart always --log-opt max-size=1g nginx

https://github.com/magicmark/composerize

Using Ansible

docker-install.yml

- hosts: all
  
  become: yes
  tasks:
  
  # Install Docker
  # --
  # 
  - name: install prerequisites
    apt:
      name:
        - apt-transport-https
        - ca-certificates 
        - curl 
        - gnupg-agent
        - software-properties-common
      update_cache: yes
  
  - name: add apt-key
    apt_key:
      url: https://download.docker.com/linux/ubuntu/gpg
  
  - name: add docker repo
    apt_repository:
      repo: deb https://download.docker.com/linux/ubuntu focal stable
  
  - name: install docker 
    apt:
      name: 
        - docker-ce
        - docker-ce-cli
        - containerd.io
      update_cache: yes
  
  - name: add user permissions
    shell: "usermod -aG docker ubuntu"
  
  # Installs Docker SDK
  # --
  # 
  - name: install python package manager
    apt:
      name: python3-pip
  
  - name: install python sdk
    become_user: ubuntu
    pip:
      name:
        - docker
        - docker-compose

Then run Ansible using your playbook on your server host ...

ansible-playbook docker-install.yml -l 'myserver'

You can then deploy a container using Ansible as well. This will deploy Portainer ...

docker_deploy-portainer.yml

- hosts: all
  
  become: yes
  become_user: ubuntu
  tasks:
  
  # Create Portainer Volume
  # --
  # 
  - name: Create new Volume
    community.docker.docker_volume:
      name: portainer-data
  
  # Deploy Portainer
  # --
  #   
  - name: Deploy Portainer
    community.docker.docker_container:
      name: portainer
      image: "docker.io/portainer/portainer-ce"
      ports:
        - "9000:9000"
        - "9443:9443"
      volumes:
        - /var/run/docker.sock:/var/run/docker.sock
        - portainer-data:/data
      restart_policy: always

... with this command ...

 ansible-playbook docker_deploy-portainer.yml -l 'myserver'

Usage

Statistics

docker stats
docker stats --no-stream

System information

docker system info

Run container

docker run hello-world

List containers

docker container ls
docker container ls -a

List container processes

docker ps
docker ps --format "table {{.ID}}\t{{.Names}}\t{{.Image}}\t{{.Ports}}\t{{.Status}}"

List container names

docker ps --format '{{.Names}}'
docker ps -a | awk '{print $NF}'

List volumes

docker volume ls
docker volume ls -f dangling=true

List networks

docker network ls

Information about container

docker container inspect container_name or id

Stop container

docker stop container_name

Delete container

docker rm container_name

Delete volumes

docker volume rm volume_name

Delete all unused volumes

docker volume prune

Delete all unused networks

docker network prune

Prune everything unused

docker system prune

Upgrade a stack

docker-compose pull
docker-compose up -d

BASH Aliases for use with Docker commands

alias dcd='docker-compose down'
alias dcr='docker-compose restart'
alias dcu='docker-compose up -d'
alias dps='docker ps'

Run Command In Docker Container

e.g.

List the mail queue for a running email server ...

docker exec -it mail.domain.co.uk-mailserver mailq

Volumes

Multiple Containers

Use volumes which are bind mounted from the host filesystem between multiple containers.

First, create the volume bind mounted to the folder...

docker volume create --driver local --opt type=none --opt device=/path/to/folder --opt o=bind volume_name

Then, use it in your docker compose file...

services:
  ftp.domain.uk-nginx:
    image: nginx
    container_name: ftp.domain.uk-nginx
    expose:
      - "80"
    volumes:
      - ./data/etc/nginx:/etc/nginx
      - ftp.domain.uk:/usr/share/nginx:ro
    environment:
      - VIRTUAL_HOST=ftp.domain.uk
networks:
  default:
    external:
      name: nginx-proxy-manager
volumes:
  ftp.domain.uk:
    external: true

Using volumes in Docker Compose

Networks

Basic Usage

Create your network...

docker network create networkname

Use it in your docker-compose.yml file...

services:
  service_name:
    image: image_name:latest
    restart: always
    networks:
      - networkname
networks:
  networkname:
    external: true

https://poopcode.com/join-to-an-existing-network-from-a-docker-container-in-docker-compose/

Advanced Usage

Static IP Address

   networks:
     traefik:
       ipv4_address: 172.19.0.9
     backend: null

Force Docker Containers to use a VPN for their Network

Block IP Address Using IPTables

The key here is to make sure you use the -I or INSERT command for the rule so that it is FIRST in the chain.

Block IP addresses from LITHUANIA

iptables -I DOCKER-USER -i eth0 -s 141.98.10.0/24 -j DROP
iptables -I DOCKER-USER -i eth0 -s 141.98.11.0/24 -j DROP
iptables -L DOCKER-USER --line-numbers
Chain DOCKER-USER (1 references)
num  target     prot opt source               destination
1    DROP       all  --  141.98.11.0/24       anywhere
2    DROP       all  --  141.98.10.0/24       anywhere
3    RETURN     all  --  anywhere             anywhere

Docker Compose

Restart Policy

The "no" option has quotes around it...

restart: "no"
restart: always
restart: on-failure
restart: unless-stopped

Management

Cleaning Pruning

sudo docker system df
sudo docker system prune
cd /etc/cron.daily
sudo nano docker-prune

#!/bin/bash
docker system prune -y

sudo chmod +x /etc/cron.daily/docker-prune

https://alexgallacher.com/prune-unused-docker-images-automatically/

Delete All Stopped Docker Containers

docker rm $(docker ps --filter "status=exited" -q)

Updating with Docker Compose

for d in ./*/ ; do (cd "$d" && sudo docker-compose pull && sudo docker-compose --compatibility up -d); done

Logs Logging

If you want to watch the logs in real time, then add the -f or --follow option to your command ...

docker logs nginx --follow

After executing docker-compose up, list your running containers:

docker ps

Copy the NAME of the given container and read its logs:

docker logs NAME_OF_THE_CONTAINER -f

To only read the error logs:

docker logs NAME_OF_THE_CONTAINER -f 1>/dev/null

To only read the access logs:

docker logs NAME_OF_THE_CONTAINER -f 2>/dev/null

https://linuxhandbook.com/docker-logging/

To search or grep the logs:

docker logs watchtower 2>&1 | grep 'msg="Found new'

Cleaning Space

Over the last month, a whopping 14Gb of space was being used by /var/lib/docker/overlay2/ and needed a way to safely remove unused data.

Check your space usage...

du -mcsh /var/lib/docker/overlay2
14G     /var/lib/docker/overlay2

Check what Docker thinks is being used...

docker system df
TYPE            TOTAL     ACTIVE    SIZE      RECLAIMABLE
Images          36        15        8.368GB   4.491GB (53%)
Containers      17        15        70.74MB   286B (0%)
Local Volumes   4         2         0B        0B
Build Cache     0         0         0B        0B

Clean...

docker system prune
docker image prune --all

Check again...

du -mcsh /var/lib/docker/overlay2
9.4G    /var/lib/docker/overlay2

docker system df
TYPE            TOTAL     ACTIVE    SIZE      RECLAIMABLE
Images          13        13        4.144GB   144MB (3%)
Containers      15        15        70.74MB   0B (0%)
Local Volumes   4         2         0B        0B
Build Cache     0         0         0B        0B

...job done.

Dockge

Better than Portainer.

A fancy, easy-to-use and reactive docker 'compose.yaml' stack-oriented manager

https://github.com/louislam/dockge

Portainer

https://www.portainer.io

https://github.com/portainer/portainer

Reset the Admin user Password

Server

https://hub.docker.com/r/portainer/portainer-ce

Agent

Portainer uses the Portainer Agent container to communicate with the Portainer Server instance and provide access to the node's resources. So, this is great for a small server (like a Raspberry Pi) where you don't need the full Portainer Server install.

Deployment

Run the following command to deploy the Portainer Agent:

docker run -d -p 9001:9001 --name portainer_agent --restart=always -v /var/run/docker.sock:/var/run/docker.sock -v /var/lib/docker/volumes:/var/lib/docker/volumes portainer/agent:2.11.0

Adding your new environment

Once the agent has been installed you are ready to add the environment to your Portainer Server installation.

From the menu select Environments then click Add environment.

From the Environment type section, select Agent. Since we have already installed the agent you can ignore the sample commands in the Information section.

Name: my-raspberry-pi
Environment URL: 192.168.0.106:9001

When you're ready, click Add environment.

Then, on the Portainer Home screen you select your new environment, or server running the Agent, and away you go!

https://docs.portainer.io/v/ce-2.11/admin/environments/add/docker#method-2-connecting-via-the-portainer-agent

Updating

Portainer > Containers > Select > Stop > Recreate > Pull Latest Image > Start

Backups

https://github.com/SavageSoftware/portainer-backup

https://forum.openmediavault.org/index.php?thread/43235-backup-portainer-and-completely-uninstall-to-test-disaster-recovery/

Monitoring

CTop

Press the Q key to stop it...

docker run -ti --name ctop --rm -v /var/run/docker.sock:/var/run/docker.sock wrfly/ctop:latest

Docker Stats

docker stats

DeUnhealth

Restart your unhealthy containers safely.

https://github.com/qdm12/deunhealth

https://www.youtube.com/watch?v=Oeo-mrtwRgE

Dozzle

Dozzle is a small lightweight application with a web based interface to monitor Docker logs. It doesn’t store any log files. It is for live monitoring of your container logs only.

https://github.com/amir20/dozzle

Gotchas

https://sosedoff.com/2016/10/05/docker-gotchas.html

Applications

I have set up my docker containers in a master docker directory with sub-directories for each stack.

docker
|-- backups
`-- stacks
    |-- bitwarden
    |   `-- bwdata
    |-- grafana
    |   `-- data
    |-- mailserver
    |   `-- data
    |-- nginx-proxy-manager
    |   `-- data
    `-- portainer
        `-- data

Backups

https://github.com/alaub81/backup_docker_scripts

Updates

Tracking

Watchtower

A process for automating Docker container base image updates.

With watchtower you can update the running version of your containerized app simply by pushing a new image to the Docker Hub or your own image registry. Watchtower will pull down your new image, gracefully shut down your existing container and restart it with the same options that were used when it was deployed initially.

First Time Run Once Check Only

This will run and output if there are any updates them stop and remove itself...

docker run --name watchtower -v /var/run/docker.sock:/var/run/docker.sock containrrr/watchtower --run-once --debug --monitor-only --rm

Automated Scheduled Run Daily

This will start the container and schedule a check at 4am every day...

~/watchtower/docker-compose.yml

version: "3"
services:
  watchtower:
    image: containrrr/watchtower
    container_name: watchtower
    restart: always
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock
    environment:
      - TZ=${TZ}
      - WATCHTOWER_DEBUG=true
      - WATCHTOWER_MONITOR_ONLY=false
      - WATCHTOWER_CLEANUP=true
      - WATCHTOWER_LABEL_ENABLE=false
      - WATCHTOWER_NOTIFICATIONS=email
      - WATCHTOWER_NOTIFICATION_EMAIL_FROM=${EMAIL_FROM}
      - WATCHTOWER_NOTIFICATION_EMAIL_TO=${WATCHTOWER_EMAIL_TO}
      - WATCHTOWER_NOTIFICATION_EMAIL_SERVER=${SMTP_SERVER}
      - WATCHTOWER_NOTIFICATION_EMAIL_SERVER_PORT=${SMTP_PORT}
      - WATCHTOWER_NOTIFICATION_EMAIL_SERVER_USER=${SMTP_USER}
      - WATCHTOWER_NOTIFICATION_EMAIL_SERVER_PASSWORD=${SMTP_PASSWORD}
      - WATCHTOWER_SCHEDULE=0 0 4 * * *

https://containrrr.dev/watchtower/

https://containrrr.dev/watchtower/arguments/#without_updating_containers

https://github.com/containrrr/watchtower

https://www.the-digital-life.com/watchtower/

https://www.youtube.com/watch?v=5lP_pdjcVMo

Updating

You can either ask Watchtower to update the containers automatically for you, or do it manually.

Manually updating when using docker-compose...

cd /path/to/docker/stack/
docker-compose down
docker-compose pull
docker-compose up -d

Bitwarden

~/bitwardenrs/docker-compose.yml

version: "2"
services:
  bitwardenrs:
    image: bitwardenrs/server:latest
    container_name: bitwardenrs
    volumes:
      - ./data:/data/
    ports:
      - 8100:80
    restart: unless-stopped
    environment:
      - TZ=Europe/London
      #- SIGNUPS_ALLOWED=false
      #- INVITATIONS_ALLOWED=false
      #- WEB_VAULT_ENABLED=false
networks:
  default:
    external:
      name: nginx-proxy-manager

Uncomment the extra security # lines after you have signed up, imported your old vault and set up your phone app and browsers, etc.

docker-compose down
docker-compose up -d

Check that the Bitwarden container environment has all the variables...

docker exec -it bitwardenrs env | sort

HOME=/root
HOSTNAME=e5f327deb4dd
INVITATIONS_ALLOWED=false
PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
ROCKET_ENV=staging
ROCKET_PORT=80
ROCKET_WORKERS=10
SIGNUPS_ALLOWED=false
TERM=xterm
TZ=Europe/London
WEB_VAULT_ENABLED=false

... and then refresh your web vault page to see it see "404: Not Found" :-)

Bitwarden RS

Bitwarden Official

InfluxDB

You can have InfluxDB on its own but there is little point without something to view the stats so you might as well include InfluxDB in the Grafana stack and start both at the same time... see below :-)

Grafana

Here is a stack in docker-compose which starts both containers in their own network so they can talk to one another. I have exposed ports for InfluxDB and Grafana to the host so I can use them from the internet.

Obviously, put your firewall in place and change the passwords below!

~/grafana/docker-compose.yml

version: "3"
services:
  grafana:
    image: grafana/grafana
    container_name: grafana
    restart: always
    networks:
      - grafana-influxdb-network
    ports:
      - 3000:3000
    volumes:
      - ./data/grafana:/var/lib/grafana
    environment:
      - INFLUXDB_URL=http://influxdb:8086
    depends_on:
      - influxdb
  influxdb:
    image: influxdb:1.8.4
    container_name: influxdb
    restart: always
    networks:
      - grafana-influxdb-network
    ports:
      - 8086:8086
    volumes:
      - ./data/influxdb:/var/lib/influxdb
    environment:
      - INFLUXDB_DB=grafana
      - INFLUXDB_USER=grafana
      - INFLUXDB_USER_PASSWORD=password
      - INFLUXDB_ADMIN_ENABLED=true
      - INFLUXDB_ADMIN_USER=admin 
      - INFLUXDB_ADMIN_PASSWORD=password 
      - INFLUXDB_URL=http://influxdb:8086
networks:
  grafana-influxdb-network:
    external: true

After this, change your Telegraf configuration to point to the new host and change the database it uses to 'grafana'.

NGiNX Proxy Manager

Provide users with an easy way to accomplish reverse proxying hosts with SSL termination that is so easy a monkey could do it.

Set up your host
Add a proxy to point to the host (in Docker this will be the 'name' and the port)
Go to http://yourhost

https://nginxproxymanager.com

https://github.com/jc21/nginx-proxy-manager

Create the Docker network with a chosen subnet (used later for fixing container IP addresses)...

sudo -i
docker network create --subnet=172.20.0.0/16 nginx-proxy-manager

/root/stacks/nginx-proxy-manager/docker-compose.yml

version: '3'
services:
  db:
    image: 'jc21/mariadb-aria:latest'
    container_name: nginx-proxy-manager_db
    restart: always
    networks:
      nginx-proxy-manager:
        ipv4_address: 172.20.0.2
    environment:
      TZ: "Europe/London"
      MYSQL_ROOT_PASSWORD: 'npm'
      MYSQL_DATABASE: 'npm'
      MYSQL_USER: 'npm'
      MYSQL_PASSWORD: 'npm'
    volumes:
      - ./data/mysql:/var/lib/mysql
  app:
    image: 'jc21/nginx-proxy-manager:latest'
    container_name: nginx-proxy-manager_app
    restart: always
    networks:
      nginx-proxy-manager:
        ipv4_address: 172.20.0.3
    ports:
      - '80:80'
      - '81:81'
      - '443:443'
    environment:
      TZ: "Europe/London"
      DB_MYSQL_HOST: "db"
      DB_MYSQL_PORT: 3306
      DB_MYSQL_USER: "npm"
      DB_MYSQL_PASSWORD: "npm"
      DB_MYSQL_NAME: "npm"
    volumes:
      - ./data:/data
      - ./data/letsencrypt:/etc/letsencrypt
    depends_on:
      - db
networks:
  nginx-proxy-manager:
    external: true

Reset Password

docker exec -it nginx-proxy-manager_db sh
mysql -u root -p npm
select * from user;
delete from user where id=1;
quit;
exit

Custom SSL Certificate

You can add a custom SSL certificate to NPM by saving the 3 parts of the SSL from Let's Encrypt...

privkey.pem
cert.pem
chain.pem

...and then uploading them to NPM.

Updating

docker-compose pull
docker-compose up -d

NGiNX

Quick Container

Run and delete everything afterwards (press CTRL+C to stop it)...

docker run --rm --name test.domain.org-nginx -e VIRTUAL_HOST=test.domain.org nginx

Run and detach and use a host folder to store the web pages and keep the container afterwards...

docker run --name test.domain.org-nginx -e VIRTUAL_HOST=test.domain.org -v /some/content:/usr/share/nginx/html:ro -d nginx

Run and detach and connect to a specific network (like nginx-proxy-manager) and use a host folder to store the web pages and keep the container afterwards...

 docker run --name test.domain.org-nginx --network nginx-proxy-manager -e VIRTUAL_HOST=test.domain.org -v /some/content:/usr/share/nginx/html:ro -d nginx

Check the logs and always show them (like tail -f)...

 docker logs test.domain.org-nginx -f

docker-compose.yml

version: "3"
services:
  nginx:
    image: nginx
    container_name: nginx
    environment:
      - TZ=Europe/London
    volumes:
      - ./data/html:/usr/share/nginx/html:ro
      - /etc/timezone:/etc/timezone:ro
    expose:
      - 80
    restart: unless-stopped

With PHP

./data/nginx/site.conf

server {
    server_name docker-demo.com;
    root /var/www/html;
    index index.php index.html index.htm;
    access_log /var/log/nginx/access.log;
    error_log /var/log/nginx/error.log;
    location / {
        try_files $uri $uri/ /index.php?$query_string;
    }
    # PHP-FPM Configuration Nginx
    location ~ \.php$ {
        try_files $uri = 404;
        fastcgi_split_path_info ^(.+\.php)(/.+)$;
        fastcgi_pass php:9000;
        fastcgi_index index.php;
        include fastcgi_params;
        fastcgi_param REQUEST_URI $request_uri;
        fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
        fastcgi_param PATH_INFO $fastcgi_path_info;
    }
}

docker-compose.yml

version: "3"
services:
  nginx:
    image: nginx
    container_name: nginx
    environment:
      - TZ=Europe/London
    volumes:
      - ./data/html:/usr/share/nginx/html:ro
      - ./data/nginx:/etc/nginx/conf.d/
      - /etc/timezone:/etc/timezone:ro
    expose:
      - 80
    restart: unless-stopped
  php:
    image: php:7.2-fpm
    volumes:
      - ./data/html:/usr/share/nginx/html:ro
      - ./data/php:/usr/local/etc/php/php.ini

https://adoltech.com/blog/how-to-set-up-nginx-php-fpm-and-mysql-with-docker-compose/

With PERL

This is a way to get the IP address of the visitor (REMOTE_ADDR) displayed...

# nginx.conf; mostly copied from defaults
load_module "modules/ngx_http_perl_module.so";

user  nginx;
worker_processes  auto;

error_log  /var/log/nginx/error.log notice;
pid        /var/run/nginx.pid;

events {
    worker_connections  1024;
}

http {
    #perl_modules /; # only needed the hello.pm isn't in @INC (e.g. dir specified below)
    perl_modules /usr/lib/perl5/vendor_perl/x86_64-linux-thread-multi/;
    perl_require hello.pm;

    server {
        location / {
            perl hello::handler;
        }
    }
}

# hello.pm; put in a @INC dir
package hello;
use nginx;

sub handler {
    my $r = shift;
    $r->send_http_header("text/html");
    return OK if $r->header_only;
    $r->print($r->remote_addr);
    return OK;
}
1;

https://www.reddit.com/r/docker/comments/oabga4/run_perl_script_in_nginx_container/

Load Balancer

This is a simple exmaple test to show multiple backend servers answering web page requests.

docker-compose.yml

version: '3'
services:
  # The load balancer
  nginx:
    image: nginx:1.16.0-alpine
    volumes:
      - ./nginx.conf:/etc/nginx/nginx.conf:ro
    ports:
      - "80:80"
  # The web server1
  server1:
    image: nginx:1.16.0-alpine
    volumes:
      - ./server1.html:/usr/share/nginx/html/index.html
  # The web server2
  server2:
    image: nginx:1.16.0-alpine
    volumes:
      - ./server2.html:/usr/share/nginx/html/index.html

nginx.conf

events {
    worker_connections 1024;
}
http {
    upstream app_servers {    # Create an upstream for the web servers
        server server1:80;    # the first server
        server server2:80;    # the second server
    }
    server {
        listen 80;
        location / {
            proxy_pass         http://app_servers;  # load balance the traffic
        }
    }
}

https://omarghader.github.io/docker-compose-nginx-tutorial/

Proxy

This is very cool and allows you to run multiple web sites on-the-fly.

The container connects to the system docker socket and watches for new containers using the VIRTUAL_HOST environment variable.

Start this, then add another container using the VIRTUAL_HOST variable and the proxy container will change its config file and reload nginx to serve the web site... automatically.

Incredible.

~/nginx-proxy/docker-compose.yml

version: "3"
services:
  nginx-proxy:
    image: jwilder/nginx-proxy
    container_name: nginx-proxy
    ports:
      - "80:80"
    volumes:
      - /var/run/docker.sock:/tmp/docker.sock:ro
networks:
  default:
    external:
      name: nginx-proxy

Normal

When using the nginx-proxy container above, you can just spin up a virtual web site using the standard 'nginx' docker image and link it to the 'nginx-proxy' network...

docker run -d --name nginx-website1.uk --expose 80 --net nginx-proxy -e VIRTUAL_HOST=website1.uk nginx

To use the host filesystem to store the web page files...

docker run -d --name nginx-website1.uk --expose 80 --net nginx-proxy -v /var/www/website1.uk/html:/usr/share/nginx/html:ro -e VIRTUAL_HOST=website1.uk nginx

In Docker Compose, it will look like this...

~/nginx/docker-compose.yml

version: "3"
services:
  nginx-website1.uk:
    image: nginx
    container_name: nginx-website1.uk
    expose:
      - "80"
    volumes:
      - /var/www/website1.uk/html:/usr/share/nginx/html:ro
    environment:
      - VIRTUAL_HOST=website1.uk
networks:
  default:
    external:
      name: nginx-proxy

Multiple Virtual Host Web Sites

~/nginx/docker-compose.yml

version: "3"
services:
  nginx-website1.uk:
    image: nginx
    container_name: nginx-website1.uk
    expose:
      - "80"
    volumes:
      - /var/www/website1.uk/html:/usr/share/nginx/html:ro
    environment:
      - VIRTUAL_HOST=website1.uk
  nginx-website2.uk:
    image: nginx
    container_name: nginx-website2.uk
    expose:
      - "80"
    volumes:
      - /var/www/website2.uk/html:/usr/share/nginx/html:ro
    environment:
      - VIRTUAL_HOST=website2.uk
networks:
  default:
    external:
      name: nginx-proxy

Viewing Logs

docker-compose logs nginx-website1.uk
docker-compose logs nginx-website2.uk

Proxy Manager

This is a web front end to manage 'nginx-proxy', where you can choose containers and create SSL certificates etc.

https://cyberhost.uk/npm-setup/

Various

https://hub.docker.com/_/nginx

https://blog.ssdnodes.com/blog/host-multiple-websites-docker-nginx/

https://github.com/nginx-proxy/nginx-proxy

Typical LEMP

https://adoltech.com/blog/how-to-set-up-nginx-php-fpm-and-mysql-with-docker-compose/

WordPress

https://hub.docker.com/_/wordpress/

PHP File Uploads Fix

Create a new PHP configuration file, and name it docker-uploads.ini. Add the following configuration then save the changes.

# Allow HTTP file uploads
file_uploads = On

# Maximum size of an uploaded file
upload_max_filesize = 64M

# Maximum size of form post data
post_max_size = 64M

Update the docker-compose.yml to bind the docker-uploads.ini to the wordpress container and then restart the WordPress container.

volumes:
  - ./data/config/docker-uploads.ini:/usr/local/etc/php/conf.d/docker-uploads.ini

WordPress Clone

Create your A record in DNS using AWS Route 53 CLI...

cli53 rrcreate domain.co.uk 'staging 300 A 123.456.78.910'

Create your docker folder for the cloned staging test web site...

mkdir -p ~/docker/stacks/staging.domain.co.uk/data/{db,html}

Edit your docker compose file, with 2 containers, making sure you use the same network as your Nginx Proxy Manager...

~/docker/stacks/staging.domain.co.uk/docker-compose.yml

version: "3"
services:
  staging.domain.co.uk-wordpress_db:
    image: mysql:5.7
    container_name: staging.domain.co.uk-wordpress_db
    volumes:
      - ./data/db:/var/lib/mysql
    restart: always
    environment:
      - TZ=Europe/London
      - MYSQL_ROOT_PASSWORD=changeme
      - MYSQL_DATABASE=dbname
      - MYSQL_USER=dbuser
      - MYSQL_PASSWORD=changeme
  staging.domain.co.uk-wordpress:
    depends_on:
      - staging.domain.co.uk-wordpress_db
    image: wordpress:latest
    container_name: staging.domain.co.uk-wordpress
    volumes:
      - ./data/html:/var/www/html
    restart: always
    environment:
      - TZ=Europe/London
      - VIRTUAL_HOST=staging.domain.co.uk
      - WORDPRESS_DB_HOST=staging.domain.co.uk-wordpress_db:3306
      - WORDPRESS_DB_NAME=dbname
      - WORDPRESS_DB_USER=dbuser
      - WORDPRESS_DB_PASSWORD=changeme
  staging.domain.co.uk-wordpress-cli:
    image: wordpress:cli
    container_name: staging.domain.co.uk-wordpress-cli
    volumes:
      - ./data/html:/var/www/html
    environment:
      - TZ=Europe/London
      - WP_CLI_CACHE_DIR=/tmp/
      - VIRTUAL_HOST=staging.domain.co.uk
      - WORDPRESS_DB_HOST=staging.domain.co.uk-wordpress_db:3306
      - WORDPRESS_DB_NAME=dbname
      - WORDPRESS_DB_USER=dbuser
      - WORDPRESS_DB_PASSWORD=changeme
    working_dir: /var/www/html
    user: "33:33"
networks:
  default:
    external:
      name: nginx-proxy-manager

Start containers with correct settings and credentials for existing live web site (so that the docker startup script sets up the MySQL permissions)...

docker-compose up -d

Check the logs to make sure all is well...

docker logs staging.domain.co.uk-wordpress
docker logs staging.domain.co.uk-wordpress_db

Copy the WordPress files to the host folder and correct ownership...

rsync -av /path/to/backup_unzipped_wordpress/ ~/docker/stacks/staging.domain.co.uk/html/
chown -R www-data:www-data ~/docker/stacks/staging.domain.co.uk/html/

Copy the sql file in to the running mysql container...

docker cp /path/to/backup_unzipped_wordpress/db_name.sql mysql_container_name:/tmp/

Log in to the database container...

docker exec -it mysql_container_name bash

Check and if necessary, change the timezone...

date
mv /etc/localtime /etc/localtime.backup
ln -s /usr/share/zoneinfo/Europe/London /etc/localtime
date

Delete and create the database...

mysql -u root -p -e "DROP DATABASE db_name; CREATE DATABASE db_name;"

Import the database from the sql file, check and exit out of the container...

mysql -u root -p mysql_db_name < /tmp/db_name.sql
mysql -u root -p -e "use db_name; show tables;"
rm /tmp/db_name.sql
exit

Edit the wp-config.php on your host server to match new DB_HOST and also add extra variables to be sure...

nano /path/to/docker/folder/html/wp-config.php
define( 'WP_HOME', 'http://staging.domain.co.uk' );
define( 'WP_SITEURL', 'http://staging.domain.co.uk' );

Install WordPress CLI in the running container...

docker exec -it wordpress_container_name bash

Search and replace the original site url...

./wp --allow-root search-replace 'http://www.domain.co.uk/' 'http://staging.domain.co.uk/' --dry-run
./wp --allow-root search-replace 'http://www.domain.co.uk/' 'http://staging.domain.co.uk/'

Start your web browser and go to the test staging web site!

WordPress CLI

In your stack, set up the usual two DB + WordPress containers, then add a third services section for wp-cli...

version: "3"
services:
  www.domain.uk-wordpress_db:
    image: mysql:5.7
    container_name: www.domain.uk-wordpress_db
    volumes:
      - ./data/db:/var/lib/mysql
    restart: always
    environment:
      - MYSQL_ROOT_PASSWORD=password
      - MYSQL_DATABASE=wordpress
      - MYSQL_USER=wordpress
      - MYSQL_PASSWORD=password
  www.domain.uk-wordpress:
    depends_on:
      - www.domain.uk-wordpress_db
    image: wordpress:latest
    container_name: www.domain.uk-wordpress
    volumes:
      - ./data/html:/var/www/html
    expose:
      - 80
    restart: always
    environment:
      - VIRTUAL_HOST=www.domain.uk
      - WORDPRESS_DB_HOST=www.domain.uk-wordpress_db:3306
      - WORDPRESS_DB_NAME=wordpress
      - WORDPRESS_DB_USER=wordpress
      - WORDPRESS_DB_PASSWORD=password
  www.domain.uk-wordpress-cli:
    image: wordpress:cli
    container_name: www.domain.uk-wordpress-cli
    volumes:
      - ./data/html:/var/www/html
    environment:
      - WP_CLI_CACHE_DIR=/tmp/
      - VIRTUAL_HOST=www.domain.uk
      - WORDPRESS_DB_HOST=www.domain.uk-wordpress_db:3306
      - WORDPRESS_DB_NAME=wordpress
      - WORDPRESS_DB_USER=wordpress
      - WORDPRESS_DB_PASSWORD=password
    working_dir: /var/www/html
    user: "33:33"
networks:
  default:
    external:
      name: nginx-proxy-manager

...then start it all up.

docker-compose up -d

Then, run your wp-cli commands (e.g. wp user list) on the end of a docker run command...

docker-compose run --rm www.domain.uk-wordpress-cli wp --info
docker-compose run --rm www.domain.uk-wordpress-cli wp cli version
docker-compose run --rm www.domain.uk-wordpress-cli wp user list
docker-compose run --rm www.domain.uk-wordpress-cli wp help theme
docker-compose run --rm www.domain.uk-wordpress-cli wp theme delete --all

SSL Behind A Reverse Proxy

https://wiki.indie-it.com/wiki/WordPress#SSL_When_Using_A_Reverse_Proxy

Email Server (mailu)

Mailu is a simple yet full-featured mail server as a set of Docker images. It is free software (both as in free beer and as in free speech), open to suggestions and external contributions. The project aims at providing people with an easily setup, easily maintained and full-featured mail server while not shipping proprietary software nor unrelated features often found in popular groupware.

https://mailu.io/1.7/

https://hub.docker.com/u/mailu

https://github.com/Mailu/Mailu

Postfix Admin

https://hub.docker.com/_/postfixadmin

Email Server (docker-mailserver)

https://github.com/docker-mailserver

https://github.com/docker-mailserver/docker-mailserver

https://github.com/docker-mailserver/docker-mailserver-admin

https://docker-mailserver.github.io/docker-mailserver/edge/config/security/ssl/#lets-encrypt-recommended

Postscreen

Postscreen is an SMTP filter that blocks spambots (or zombie machines) away from the real Postfix smtpd daemon, so Postfix does not feel overloaded and can process legitimate emails more efficiently.

The example below shows a typical spambot attempt at accessing the SMTP service and being stopped...

Jun 24 10:42:26 mail postfix/postscreen[203907]: CONNECT from [212.70.149.56]:19452 to [172.23.0.2]:25
Jun 24 10:42:26 mail postfix/dnsblog[386054]: addr 212.70.149.56 listed by domain b.barracudacentral.org as 127.0.0.2
Jun 24 10:42:26 mail postfix/dnsblog[402550]: addr 212.70.149.56 listed by domain list.dnswl.org as 127.0.10.3
Jun 24 10:42:26 mail postfix/dnsblog[407802]: addr 212.70.149.56 listed by domain bl.mailspike.net as 127.0.0.2
Jun 24 10:42:26 mail postfix/dnsblog[386155]: addr 212.70.149.56 listed by domain psbl.surriel.com as 127.0.0.2
Jun 24 10:42:29 mail postfix/postscreen[203907]: PREGREET 11 after 2.9 from [212.70.149.56]:19452: EHLO User\r\n
Jun 24 10:42:29 mail postfix/postscreen[203907]: DISCONNECT [212.70.149.56]:19452

Postscreen is enabled by default but there are a few settings to tweak to get the best out of it.

Edit your data/config/postfix-main.cf file and add the following lines, making sure your Docker host IP is in bold...

mynetworks = 127.0.0.0/8 [::1]/128 [fe80::]/64 172.19.0.2/32 172.19.0.1/32
postscreen_greet_action = drop
postscreen_pipelining_enable = yes
postscreen_pipelining_action = drop
postscreen_non_smtp_command_enable = yes
postscreen_non_smtp_command_action = drop
postscreen_bare_newline_enable = yes 
postscreen_bare_newline_action = drop

Enable and Configure Postscreen in Postfix to Block Spambots

Postgrey

Local List

SpamAssassin

Custom Rules

Bayes Database

SpamAssassin is controlled by Amavis (a fork of MailScanner) with the user 'amavis'.

Show Bayes Database Stats

docker exec --user amavis mail.mydomain.org.uk-mailserver sa-learn --dump magic --dbpath /var/lib/amavis/.spamassassin

Learn Ham

docker exec --user amavis mail.mydomain.org.uk-mailserver sa-learn --ham --progress /var/mail/mydomain.org.uk/info/cur --dbpath /var/lib/amavis/.spamassassin

Backup and Restore from Existing Mail Server

On the old server...

/bin/su -l -c '/usr/bin/sa-learn --backup > sa-learn_backup.txt' debian-spamd
rsync -avP /var/lib/spamassassin/sa-learn_backup.txt user@mail.mydomain.org.uk:/tmp/

On the new server...

docker cp /tmp/sa-learn_backup.txt mail.mydomain.org.uk-mailserver:/tmp/
docker exec --user amavis mail.mydomain.org.uk-mailserver sa-learn --sync --dbpath /var/lib/amavis/.spamassassin
docker exec --user amavis mail.mydomain.org.uk-mailserver sa-learn --clear --dbpath /var/lib/amavis/.spamassassin
docker exec --user amavis mail.mydomain.org.uk-mailserver sa-learn --restore /tmp/sa-learn_backup.txt --dbpath /var/lib/amavis/.spamassassin
docker exec --user amavis mail.mydomain.org.uk-mailserver sa-learn --sync --dbpath /var/lib/amavis/.spamassassin
docker exec --user amavis mail.mydomain.org.uk-mailserver sa-learn --dump magic --dbpath /var/lib/amavis/.spamassassin

Fail2Ban

List jails...

docker exec -it mail.mydomain.org.uk-mailserver fail2ban-client status

Status
|- Number of jail:   3
`- Jail list:   dovecot, postfix, postfix-sasl

Manually ban IP address in named jail...

docker exec -it mail.mydomain.org.uk-mailserver fail2ban-client set postfix banip 212.70.149.56

Check banned IPs...

docker exec -it mail.mydomain.org.uk-mailserver fail2ban-client status postfix

Status for the jail: postfix
|- Filter
|  |- Currently failed: 2
|  |- Total failed:     2
|  `- File list:        /var/log/mail.log
`- Actions
   |- Currently banned: 1
   |- Total banned:     1
   `- Banned IP list:   212.70.149.56

https://www.the-lazy-dev.com/en/install-fail2ban-with-docker/

Backups

Autodiscover

Create SRV and A record entries in your DNS for the services...

$ORIGIN domain.org.uk.
@	300	IN	TXT	"v=spf1 mx ~all; mailconf=https://autoconfig.domain.org.uk/mail/config-v1.1.xml"
_autodiscover._tcp	300	IN	SRV	0 0 443 autodiscover.domain.org.uk.
_imap._tcp	300	IN	SRV	0 0 0 .
_imaps._tcp	300	IN	SRV	0 1 993 mail.domain.org.uk.
_ldap._tcp	300	IN	SRV	0 0 636 mail.domain.org.uk.
_pop3._tcp	300	IN	SRV	0 0 0 .
_pop3s._tcp	300	IN	SRV	0 0 0 .
_submission._tcp	300	IN	SRV	0 1 587 mail.domain.org.uk.
autoconfig	300	IN	A	3.10.67.19
autodiscover	300	IN	A	3.10.67.19
imap	300	IN	CNAME	mail
mail	300	IN	A	3.10.67.19
smtp	300	IN	CNAME	mail
www	300	IN	A	3.10.67.19

docker-compose.yml

services:
  mailserver-autodiscover:
    image: monogramm/autodiscover-email-settings:latest
    container_name: mail.domain.org.uk-mailserver-autodiscover
    environment:
      - COMPANY_NAME=My Company
      - SUPPORT_URL=https://autodiscover.domain.org.uk
      - DOMAIN=domain.org.uk
      - IMAP_HOST=mail.domain.org.uk
      - IMAP_PORT=993
      - IMAP_SOCKET=SSL
      - SMTP_HOST=mail.domain.org.uk
      - SMTP_PORT=587
      - SMTP_SOCKET=STARTTLS
    restart: unless-stopped
networks:
  default:
    external:
      name: nginx-proxy-manager

Autodiscover

monogramm/autodiscover-email-settings

Internet Speedtest

https://github.com/henrywhitaker3/Speedtest-Tracker

Emby Media Server

https://emby.media/docker-server.html

https://hub.docker.com/r/emby/embyserver

AWS CLI

docker run --rm -it -v "/root/.aws:/root/.aws" amazon/aws-cli configure
docker run --rm -it -v "/root/.aws:/root/.aws" amazon/aws-cli s3 ls
docker run --rm -it -v "/root/.aws:/root/.aws" amazon/aws-cli route53 list-hosted-zones

https://docs.aws.amazon.com/cli/latest/userguide/install-cliv2-docker.html

Let's Encrypt

Force RENEW a standalone certificate with the new preferred chain of "ISRG Root X1"

docker run -it --rm --name certbot -v "/usr/bin:/usr/bin" -v "/root/.aws:/root/.aws" -v "/etc/letsencrypt:/etc/letsencrypt" -v "/var/lib/letsencrypt:/var/lib/letsencrypt" certbot/certbot --force-renewal --preferred-chain "ISRG Root X1" certonly --standalone --email me@mydomain.com --agree-tos -d www.mydomain.com

Issue a wildcard certificate...

docker run -it --rm --name certbot -v "/usr/bin:/usr/bin" -v "/root/.aws:/root/.aws" -v "/etc/letsencrypt:/etc/letsencrypt" -v "/var/lib/letsencrypt:/var/lib/letsencrypt" certbot/dns-route53 certonly --dns-route53 --domain "example.com" --domain "*.example.com"

Check your certificates...

docker run -it --rm --name certbot -v "/usr/bin:/usr/bin" -v "/root/.aws:/root/.aws" -v "/etc/letsencrypt:/etc/letsencrypt" -v "/var/lib/letsencrypt:/var/lib/letsencrypt" certbot/certbot certificates

Renew a certificate...

docker run -it --rm --name certbot -v "/usr/bin:/usr/bin" -v "/root/.aws:/root/.aws" -v "/etc/letsencrypt:/etc/letsencrypt" -v "/var/lib/letsencrypt:/var/lib/letsencrypt" certbot/dns-route53 renew

If you have multiple profiles in your .aws/config then you will need to pass the AWS_PROFILE variable to the docker container...

docker run -it --rm --name certbot -v "/usr/bin:/usr/bin" -v "/root/.aws:/root/.aws" -v "/etc/letsencrypt:/etc/letsencrypt" -v "/var/lib/letsencrypt:/var/lib/letsencrypt" -e "AWS_PROFILE=certbot" certbot/dns-route53 renew

https://certbot.eff.org/docs/install.html#running-with-docker

VPN

Gluetun

Gluetun VPN client

Lightweight swiss-knife-like VPN client to tunnel to Cyberghost, ExpressVPN, FastestVPN, HideMyAss, IPVanish, IVPN, Mullvad, NordVPN, Perfect Privacy, Privado, Private Internet Access, PrivateVPN, ProtonVPN, PureVPN, Surfshark, TorGuard, VPNUnlimited, VyprVPN, WeVPN and Windscribe VPN servers using Go, OpenVPN or Wireguard, iptables, DNS over TLS, ShadowSocks and an HTTP proxy.

Github

Connect a container to Gluetun

OpenVPN

Server

https://hub.docker.com/r/linuxserver/openvpn-as

Client

https://hub.docker.com/r/dperson/openvpn-client

Routing Containers Through Container

sudo docker run -it --net=container:vpn -d some/docker-container

OpenVPN-PiHole

https://github.com/Simonwep/openvpn-pihole

WireHole

WireHole is a combination of WireGuard, PiHole, and Unbound in a docker-compose project with the intent of enabling users to quickly and easily create and deploy a personally managed full or split-tunnel WireGuard VPN with ad blocking capabilities (via Pihole), and DNS caching with additional privacy options (via Unbound).

https://github.com/IAmStoxe/wirehole

To view a QR code, run this ...

docker exec -it wireguard /app/show-peer 1

WireGuard

Use WireHole instead!

docker-compose.yml

version: "2.1"
services:
  wireguard:
    image: ghcr.io/linuxserver/wireguard
    container_name: wireguard
    cap_add:
      - NET_ADMIN
      - SYS_MODULE
    environment:
      - PUID=1000
      - PGID=1000
      - TZ=Europe/London
      - SERVERURL=wireguard.domain.uk
      - SERVERPORT=51820
      - PEERS=3
      - PEERDNS=auto
      - INTERNAL_SUBNET=10.13.13.0
      - ALLOWEDIPS=0.0.0.0/0
    volumes:
      - ./data/config:/config
      - /lib/modules:/lib/modules
    ports:
      - 51820:51820/udp
    sysctls:
      - net.ipv4.conf.all.src_valid_mark=1
    restart: unless-stopped

https://hub.docker.com/r/linuxserver/wireguard

To show the QR code

docker exec -it wireguard /app/show-peer 1
docker exec -it wireguard /app/show-peer 2
docker exec -it wireguard /app/show-peer 3

Error: const struct ipv6_stub

If you receive an error in the container logs about not being able to compile the kernel module, then follow the instructions to compile the WireGuard kernel module and tools in your host system.

https://github.com/linuxserver/docker-wireguard/issues/46#issuecomment-708278250

Force Docker Containers to use a VPN for connection

ffmpeg

docker pull jrottenberg/ffmpeg
docker run jrottenberg/ffmpeg -h
docker run jrottenberg/ffmpeg -i /path/to/input.mkv -stats $ffmpeg_options - > out.mp4
docker run -v $(pwd):$(pwd) -w $(pwd) jrottenberg/ffmpeg -y -i input.mkv -t 00:00:05.00 -vf scale=-1:360 output.mp4

https://registry.hub.docker.com/r/jrottenberg/ffmpeg

https://github.com/jrottenberg/ffmpeg

https://medium.com/coconut-stories/using-ffmpeg-with-docker-94523547f35c

https://github.com/linuxserver/docker-ffmpeg

MediaInfo

Install ...

sudo docker pull jlesage/mediainfo

Run ...

docker run --rm --name=mediainfo -e USER_ID=$(id -u) -e GROUP_ID=$(id -g) -v $(pwd):$(pwd):ro jlesage/mediainfo su-exec "$(id -u):$(id -g)" /usr/bin/mediainfo --help

https://github.com/jlesage/docker-mediainfo

MKV Toolnix

Install ...

sudo docker pull jlesage/mkvtoolnix

Run ...

mkvextract

docker run --rm --name=mkvextract -e USER_ID=$(id -u) -e GROUP_ID=$(id -g) -v $(pwd):$(pwd):rw jlesage/mkvtoolnix su-exec "$(id -u):$(id -g)" /usr/bin/mkvextract "$(pwd)/filename.mkv" tracks 3:"$(pwd)/filename.eng.srt"

mkvpropedit

docker run --rm --name=mkvextract -e USER_ID=$(id -u) -e GROUP_ID=$(id -g) -v $(pwd):$(pwd):rw jlesage/mkvtoolnix su-exec "$(id -u):$(id -g)" /usr/bin/mkvpropedit "$(pwd)/filename.mkv" --edit track:a1 --set language=eng

https://github.com/jlesage/docker-mkvtoolnix

MakeMKV

This will NOT work on a Raspberry Pi.

https://github.com/jlesage/docker-makemkv

Use this in combination with ffmpeg or HandBrake (as shown below) and FileBot to process your media through to your media server - like Emby or Plex..

MakeMKV > HandBrake > FileBot > Emby

To make this work with your DVD drive (/dev/sr0) you need to have the second device (/dev/sg0) in order for it to work. I don't get it, but it works.

/root/docker/stacks/makemkv/docker-compose.yml

version: '3'
services:
  makemkv:
    image: jlesage/makemkv
    container_name: makemkv
    ports:
      - "0.0.0.0:5801:5800"
    volumes:
      - "/home/user/.MakeMKV_DOCKER:/config:rw"
      - "/home/user/:/storage:ro"
      - "/home/user/ToDo/MakeMKV/output:/output:rw"
    devices:
      - "/dev/sr0:/dev/sr0"
      - "/dev/sg0:/dev/sg0"
    environment:
      - USER_ID=1000
      - GROUP_ID=1000
      - TZ=Europe/London
      - MAKEMKV_KEY=your_licence_key
      - AUTO_DISC_RIPPER=1

Troubleshooting

PROBLEM = "driver failed programming external connectivity on endpoint makemkv: Error starting userland proxy: listen tcp6 [::]:5800: socket: address family not supported by protocol."

SOLUTION = Put 0.0.0.0:5801 in the published ports line of docker compose to restrict the network to IPv4.

Docker Process Output

CONTAINER ID   IMAGE               COMMAND   CREATED        STATUS        PORTS                              NAMES
4a8b3106b00b   jlesage/handbrake   "/init"   40 hours ago   Up 40 hours   5900/tcp, 0.0.0.0:5802->5800/tcp   handbrake
89fe3ba8a31e   jlesage/makemkv     "/init"   40 hours ago   Up 40 hours   5900/tcp, 0.0.0.0:5801->5800/tcp   makemkv

Command Line

docker run --rm -v "/home/user/.MakeMKV_DOCKER:/config:rw" -v "/home/user/:/storage:ro" -v "/home/user/ToDo/MakeMKV/output:/output:rw" --device /dev/sr0 --device /dev/sg0 --device /dev/sg1 jlesage/makemkv /opt/makemkv/bin/makemkvcon mkv disc:0 all /output

https://github.com/jlesage/docker-makemkv/issues/141

HandBrake

This will NOT work on a Raspberry Pi.

Use this in combination with ffmpeg or MakeMKV (as shown below) and FileBot to process your media through to your media server - like Emby or Plex..

I have changed the port from 5800 to 5802 because Jocelyn's other Docker image for MakeMKV uses the same port (so I move that one as well to 5801 - see above).

To make this work with your DVD drive (/dev/sr0) you need to have the second device (/dev/sg0) in order for it to work. I don't get it, but it works.

YouTube / DB Tech - How to install HandBrake in Docker

Blog / DB Tech - How to install HandBrake in Docker

Docker HandBrake by Jocelyn Le Sage

Docker Image by Jocelyn Le Sage

/root/docker/stacks/handbrake/docker-compose.yml

version: '3'
services:
  handbrake:
    image: jlesage/handbrake
    container_name: handbrake
    ports:
      - "0.0.0.0:5802:5800"
    volumes:
      - "/home/paully:/storage:ro"
      - "/home/paully/ToDo/HandBrake/config:/config:rw"
      - "/home/paully/ToDo/HandBrake/watch:/watch:rw"
      - "/home/paully/ToDo/HandBrake/output:/output:rw"
    devices:
      - "/dev/sr0:/dev/sr0"
      - "/dev/sg0:/dev/sg0"
    environment:
      - USER_ID=1000
      - GROUP_ID=1000
      - TZ=Europe/London

Docker Process Output

CONTAINER ID   IMAGE               COMMAND   CREATED        STATUS        PORTS                              NAMES
4a8b3106b00b   jlesage/handbrake   "/init"   40 hours ago   Up 40 hours   5900/tcp, 0.0.0.0:5802->5800/tcp   handbrake
89fe3ba8a31e   jlesage/makemkv     "/init"   40 hours ago   Up 40 hours   5900/tcp, 0.0.0.0:5801->5800/tcp   makemkv

Command Line

docker run --rm -v "/home/paully/:/storage:ro" -v "/home/paully/ToDo/HandBrake/config:/config:rw" -v "/home/paully/ToDo/HandBrake/watch:/watch:rw" -v "/home/paully/ToDo/HandBrake/output:/output:rw" --device /dev/sr0 --device /dev/sg0 jlesage/handbrake /usr/bin/HandBrakeCLI --input "/output/file.mkv" --stop-at duration:120 --preset 'Fast 480p30' --non-anamorphic --encoder-preset slow --quality 22 --deinterlace --lapsharp --audio 1 --aencoder copy:ac3 --no-markers --output "/output/file.mp4"

FileBot

Setup

Create your directories for data volumes (https://github.com/jlesage/docker-filebot#data-volumes) ...

mkdir -p ~/filebot/{config,output,watch}

The license file received via email can be saved on the host, into the configuration directory of the container (i.e. in the directory mapped to /config). Then, start or restart the container to have it automatically installed. NOTE: The license file is expected to have a .psm extension.

Usage

WORK IN PROGRESS

Rather than running all the time, we run the image ad-hoc with the rm option to delete the old container each time...

docker run --rm --name=filebot -v ~/filebot/config:/config:rw -v $HOME:/storage:rw -v ~/filebot/output:/output:rw -v ~/filebot/watch:/watch:rw -e AMC_ACTION=test jlesage/filebot

https://github.com/jlesage/docker-filebot

Automated Downloaderr

This takes the hassle out of going through the various web sites to find stuff and be bombarded with ads and pop-ups.

FlareSolverr
Prowlarr
Radarr + Sonarr + Bazarr
Transmission + NZBGet
Tdarr

FlareSolverr > Prowlarr > Radarr + Sonarr + Bazarr > Transmission + NZBGet > Tdarr

HOW TO RESTART THE RRS IN ORDER ON PORTAINER OR OMV

Stacks > Click on each one > Stop > count to 10 > Start

WireGuard
FlareSolverr
Prowlarr
Radarr
Sonarr

ONE DAY WE WILL GET A SINGLE STACK WITH ALL THE RIGHT CONTAINERS STARTING IN THE RIGHT ORDER

https://hotio.dev/containers/autoscan/

ONE APP TO RULE THEM ALL

https://github.com/JagandeepBrar/LunaSea

GUIDE FOR DIRECTORY STRUCTURE

https://trash-guides.info/Hardlinks/How-to-setup-for/Docker/

This will enable you to automatically rename files but allow you to copy them to your actual Plex or Emby folders.

It is possible to do hard linking and let the rrrrs control all the files but I am not a fan of that.

This way, the files get renamed and moved to a 'halfway' house where you can check them and then simply move them to your desired location.

METHOD

Create the directories ...

mkdir -p /path/to/data/{media,torrents,usenet}/{movies,music,tv}
mkdir -p /path/to/docker/appdata/{radarr,sonarr,bazarr,nzbget}

Change the ownership and permissions ...

chown -R admin:users /path/to/data/ /path/to/docker/
find /path/to/data/ /path/to/docker/ -type d -exec chmod 0755 {} \;
find /path/to/data/ /path/to/docker/ -type d -exec chmod g+s {} \;

Finished directory structure ...

/srv/dev-disk-by-uuid-7f81e7b6-1a05-4232-893b-f34c046b2bdb/data/
|-- media
|   |-- movies
|   |-- music
|   `-- tv
|-- torrents
|   |-- movies
|   |-- music
|   `-- tv
`-- usenet
    |-- completed
    |   |-- movies
    |   `-- tv
    |-- movies
    |-- music
    |-- nzb
    |   `-- Movie.Name.720p.nzb.queued
    |-- tmp
    `-- tv

PORTAINER STACK

This is from Open Media Vault (OMV) so the volume paths are long.

This works but needs the whole VPN thing added (which changes ports etc) but for now ...

Portainer > Stacks > Add Stack > Datarr

version: "3.2"
services:
  prowlarr:
    container_name: prowlarr
    image: hotio/prowlarr:latest
    restart: unless-stopped
    logging:
      driver: json-file
    network_mode: bridge
    ports:
      - 9696:9696
    environment:
      - PUID=998
      - PGID=100
      - TZ=Europe/London
    volumes:
      - /etc/localtime:/etc/localtime:ro
      - /srv/dev-disk-by-uuid-7f81e7b6-1a05-4232-893b-f34c046b2bdb/docker/appdata/prowlarr:/config
      - /srv/dev-disk-by-uuid-7f81e7b6-1a05-4232-893b-f34c046b2bdb/data:/data
  radarr:
    container_name: radarr
    image: hotio/radarr:latest
    restart: unless-stopped
    logging:
      driver: json-file
    network_mode: bridge
    ports:
      - 7878:7878
    environment:
      - PUID=998
      - PGID=100
      - TZ=Europe/London
    volumes:
      - /etc/localtime:/etc/localtime:ro
      - /srv/dev-disk-by-uuid-7f81e7b6-1a05-4232-893b-f34c046b2bdb/docker/appdata/radarr:/config
      - /srv/dev-disk-by-uuid-7f81e7b6-1a05-4232-893b-f34c046b2bdb/data:/data
  sonarr:
    container_name: sonarr
    image: hotio/sonarr:latest
    restart: unless-stopped
    logging:
      driver: json-file
    network_mode: bridge
    ports:
      - 8989:8989
    environment:
      - PUID=998
      - PGID=100
      - TZ=Europe/London
    volumes:
      - /etc/localtime:/etc/localtime:ro
      - /srv/dev-disk-by-uuid-7f81e7b6-1a05-4232-893b-f34c046b2bdb/docker/appdata/sonarr:/config
      - /srv/dev-disk-by-uuid-7f81e7b6-1a05-4232-893b-f34c046b2bdb/data:/data
  bazarr:
    container_name: bazarr
    image: hotio/bazarr:latest
    restart: unless-stopped
    logging:
      driver: json-file
    network_mode: bridge
    ports:
      - 6767:6767
    environment:
      - PUID=998
      - PGID=100
      - TZ=Europe/London
    volumes:
      - /etc/localtime:/etc/localtime:ro
      - /srv/dev-disk-by-uuid-7f81e7b6-1a05-4232-893b-f34c046b2bdb/docker/appdata/bazarr:/config
      - /srv/dev-disk-by-uuid-7f81e7b6-1a05-4232-893b-f34c046b2bdb/data/media:/data/media
  nzbget:
    container_name: nzbget
    image: hotio/nzbget:latest
    restart: unless-stopped
    logging:
      driver: json-file
    network_mode: bridge
    ports:
      - 6789:6789
    environment:
      - PUID=998
      - PGID=100
      - TZ=Europe/London
    volumes:
      - /etc/localtime:/etc/localtime:ro
      - /srv/dev-disk-by-uuid-7f81e7b6-1a05-4232-893b-f34c046b2bdb/docker/appdata/nzbget:/config
      - /srv/dev-disk-by-uuid-7f81e7b6-1a05-4232-893b-f34c046b2bdb/data/usenet:/data/usenet:rw

OLD NOTES

Create a docker network which Jackett and Radarr share to talk to each other...

sudo docker network create jackett-radarr

...then continue setting up the containers below.

FlareSolverr

FlareSolverr is a proxy server to bypass Cloudflare protection.

FlareSolverr starts a proxy server and it waits for user requests in an idle state using few resources. When some request arrives, it uses puppeteer with the stealth plugin to create a headless browser (Chrome). It opens the URL with user parameters and waits until the Cloudflare challenge is solved (or timeout). The HTML code and the cookies are sent back to the user, and those cookies can be used to bypass Cloudflare using other HTTP clients.

Radarr > Jackett > FlareSolverr > Internet

https://github.com/FlareSolverr/FlareSolverr

https://hub.docker.com/r/flaresolverr/flaresolverr

Some indexers are protected by CloudFlare or similar services and Jackett is not able to solve the challenges. For these cases, FlareSolverr has been integrated into Jackett. This service is in charge of solving the challenges and configuring Jackett with the necessary cookies. Setting up this service is optional, most indexers don't need it.

Install FlareSolverr service using a Docker container, then configure FlareSolverr API URL in Jackett. For example: http://172.17.0.2:8191

Command line...

docker run -d \
 --name=flaresolverr \
 -p 8191:8191 \
 -e LOG_LEVEL=info \
 --restart unless-stopped \
 ghcr.io/flaresolverr/flaresolverr:latest

Docker compose...

---
version: "2.1"
services:
  flaresolverr:
    image: ghcr.io/flaresolverr/flaresolverr:latest
    container_name: flaresolverr
    environment:
      - PUID=1000
      - PGID=1000
      - LOG_LEVEL=${LOG_LEVEL:-info}
      - LOG_HTML=${LOG_HTML:-false}
      - CAPTCHA_SOLVER=${CAPTCHA_SOLVER:-none}
      - TZ=Europe/London
    ports:
      - "${PORT:-8191}:8191"
    restart: unless-stopped

Usage

To use it, you have to add a Proxy Server under Prowlarr > Settings > Indexers > Indexer Proxies

Testing

curl -L -X POST 'http://localhost:8191/v1' -H 'Content-Type: application/json' --data-raw '{ "cmd": "request.get", "url":"https://www.paully.co.uk/", "maxTimeout": 60000 }'

{"status":"ok","message":"","startTimestamp":1651659265156,"endTimestamp":1651659269585,"version":"v2.2.4","solution":

Jackett

Jackett works as a proxy server: it translates queries from apps (Sonarr, SickRage, CouchPotato, Mylar, etc) into tracker-site-specific http queries, parses the html response, then sends results back to the requesting software. This allows for getting recent uploads (like RSS) and performing searches. Jackett is a single repository of maintained indexer scraping and translation logic - removing the burden from other apps.

So, this is where you build your list of web sites "with content you want" ;-)

https://fleet.linuxserver.io/image?name=linuxserver/jackett

https://docs.linuxserver.io/images/docker-jackett

https://hub.docker.com/r/linuxserver/jackett

https://github.com/Jackett/Jackett

/root/docker/stacks/docker-compose.yml

---
version: "2.1"
services:
  jackett:
    image: ghcr.io/linuxserver/jackett
    container_name: jackett
    environment:
      - PUID=1000
      - PGID=1000
      - TZ=Europe/London
      - AUTO_UPDATE=true
    volumes:
      - ./data/config:/config
      - ./data/downloads:/downloads
    networks:
      - jackett-radarr
    ports:
      - 0.0.0.0:9117:9117
    restart: unless-stopped
networks:
  jackett-radarr:
    external: true

Prowlarr

An alternative to Jackett, and now the preferred application.

https://wiki.servarr.com/prowlarr

https://wiki.servarr.com/prowlarr/quick-start-guide

https://hub.docker.com/r/linuxserver/prowlarr

https://github.com/linuxserver/docker-prowlarr

---
version: "2.1"
services:
  prowlarr:
    image: lscr.io/linuxserver/prowlarr:develop
    container_name: prowlarr
    environment:
      - PUID=1000
      - PGID=1000
      - TZ=Europe/London
    volumes:
      - /path/to/data:/config
    ports:
      - 9696:9696
    restart: unless-stopped

Radarr

Radarr is a movie collection manager for Usenet and BitTorrent users. It can monitor multiple RSS feeds for new movies and will interface with clients and indexers to grab, sort, and rename them. It can also be configured to automatically upgrade the quality of existing files in the library when a better quality format becomes available.

Radarr is the 'man-in-the-middle' to take lists from Jackett and pass them to Transmission to download.

Radarr is the web UI to search for "the content you want" ;-)

https://radarr.video/

https://docs.linuxserver.io/images/docker-radarr

https://github.com/linuxserver/docker-radarr

https://sasquatters.com/radarr-docker/

https://www.smarthomebeginner.com/install-radarr-using-docker/

https://trash-guides.info/Radarr/

https://discord.com/channels/264387956343570434/264388019585286144

Custom Scripts

So, you use Jackett as an Indexer of content, which answers questions from Radarr, which passes a good result to Transmission...

Settings > Profiles > delete all but 'any' (and edit that to get rid of naff qualities at the bottom)
Indexers > Add Indexer > Torznab > complete and TEST then SAVE
Download Clients > Add Download Client > Transmission > complete and TEST and SAVE

/root/docker/stacks/docker-compose.yml

---
version: "2.1"
services:
  radarr:
    image: ghcr.io/linuxserver/radarr
    container_name: radarr
    environment:
      - PUID=1000
      - PGID=1000
      - TZ=Europe/London
    volumes:
      - ./data/config:/config
      - ./data/downloads:/downloads
      - ./data/torrents:/torrents
    networks:
      - jackett-radarr
    ports:
      - 0.0.0.0:7878:7878
    restart: unless-stopped
networks:
  jackett-radarr:
    external: true

Sonarr

Sonarr (formerly NZBdrone) is a PVR for usenet and bittorrent users. It can monitor multiple RSS feeds for new episodes of your favorite shows and will grab, sort and rename them. It can also be configured to automatically upgrade the quality of files already downloaded when a better quality format becomes available.

https://sonarr.tv/

https://docs.linuxserver.io/images/docker-sonarr

Docker Compose using a WireGuard VPN container for internet ...

---
version: "2.1"
services:
  sonarr:
    image: ghcr.io/linuxserver/sonarr
    container_name: sonarr
    network_mode: container:wireguard
    environment:
      - PUID=1000
      - PGID=1000
      - TZ=Europe/London
    volumes:
      - ./data/config:/config
      - ./data/downloads:/downloads
      - ./data/torrents:/torrents
    restart: "no"

UPDATE: 16 FEBRUARY 2023 / latest image is based on Ubuntu Jammy and not Alpine, which will cause a problem if your Docker version is below 20.10.10

https://docs.linuxserver.io/faq#jammy

To fix this, you can either upgrade your Docker (https://docs.docker.com/engine/install/ubuntu/#install-using-the-convenience-script) or add the following lines to your docker-compose.yml file ...

security_opt:
  - seccomp=unconfined

Bazarr

https://www.bazarr.media/

Bazarr is a companion application to Sonarr and Radarr that manages and downloads subtitles based on your requirements.

Docker compose file ...

---
version: "2.1"
services:
  bazarr:
    image: lscr.io/linuxserver/bazarr:latest
    container_name: bazarr
    environment:
      - PUID=1000
      - PGID=1000
      - TZ=Europe/London
    volumes:
      - /path/to/bazarr/config:/config
      - /path/to/movies:/movies #optional
      - /path/to/tv:/tv #optional
    ports:
      - 6767:6767
    restart: unless-stopped

NZBGet

NZBGet is a usenet downloader.

You will require the following 3 things at a basic level before you are able to use usenet:-

A usenet provider (Reddit provides a detailed Provider Map for usenet)
An NZB indexer (NZBGeek)
A usenet client (NZBget)

https://nzbget.net

https://hub.docker.com/r/linuxserver/nzbget

https://www.cogipas.com/nzbget-complete-how-to-guide/

The Web GUI can be found at <your-ip>:6789 and the default login details (change ASAP) are...

username: nzbget
password: tegbzn6789

Docker Compose file...

---
version: "2.1"
services:
  nzbget:
    image: lscr.io/linuxserver/nzbget:latest
    container_name: nzbget
    environment:
      - PUID=1000
      - PGID=1000
      - TZ=Europe/London
      - NZBGET_USER=nzbget
      - NZBGET_PASS=tegbzn6789
    volumes:
      - /path/to/data:/config
      - /path/to/downloads:/downloads
    ports:
      - 6789:6789
    restart: unless-stopped

Tdarr

Tdarr is a popular conditional transcoding application for processing large (or small) media libraries. The application comes in the form of a click-to-run web-app, which you run on your own device and access through a web browser.

Tdarr uses two popular transcoding applications under the hood: FFmpeg and HandBrake (which itself is built on top of FFmpeg).

Tdarr works in a distributed manner where you can use multiple devices to process your library together. It does this using 'Tdarr Nodes' which connect with a central server and pick up tasks so you can put all your spare devices to use.

Each Node can run multiple 'Tdarr Workers' in parallel to maximize the hardware usage % on that Node. For example, a single FFmpeg worker running on a 64 core CPU may only hit ~30% utilization. Running multiple Workers in parallel allows the CPU to hit 100% utilization, allowing you to process your library more quickly.

Home Page

Documentation

Using AMD GPU for Transcoding

Readarr

https://academy.pointtosource.com/containers/ebooks-calibre-readarr/

Unpackerr

Extracts downloads for Radarr, Sonarr, Lidarr, Readarr, and/or a Watch folder - Deletes extracted files after import.

https://github.com/Unpackerr/unpackerr

Docker Compose - https://github.com/Unpackerr/unpackerr/blob/main/examples/docker-compose.yml

Calibre

eBook management and automation using Calibre, COPS or Calibre-Web, and Readarr.

Auto format conversion.

https://academy.pointtosource.com/containers/ebooks-calibre-readarr/

YouTube-DL

https://registry.hub.docker.com/search?q=youtube&sort=updated_at&order=desc

https://registry.hub.docker.com/r/mikenye/youtube-dl#quick-start

Nagios

Work in progress.

This is an old version of Nagios in the container image, so will look for a newer one.

/root/docker/stacks/nagios/docker-compose.yml

version: '3'
services:
  nagios:
    image: jasonrivers/nagios
    container_name: nagios
    restart: unless-stopped
    ports:
      - 8181:80
    # volumes:
      # - ./data/etc/:/opt/nagios/etc/
    environment:
      - PUID=999
      - PGID=1000
      - TZ=Europe/London
      - NAGIOS_TIMEZONE=Europe/London

Start it with no volume mounts, then copy the etc directory to your host...

cd /root/docker/stacks/nagios/
docker cp nagios:/opt/nagios/etc data/
chown -R 999:1000 data/

...then uncomment the # lines in the docker-compose file and restart the container.

Credentials

The default credentials for the web interface is nagiosadmin / nagios

To change the password, generate a new one by logging in to the container and running 'htpasswd'...

docker exec -it nagios bash
htpasswd -n nagiosadmin
(copy the output)

...then editing the /opt/nagios/etc/htpasswd.users file and refreshing the admin web page.

https://github.com/ethnchao/docker-nagios

http://www.kraftinfosec.com/running-nagios-in-docker/

https://github.com/JasonRivers/Docker-Nagios

Tandoor Recipe Manager

The recipe manager that allows you to manage your ever growing collection of digital recipes.

https://docs.tandoor.dev/install/docker/

https://www.youtube.com/watch?v=7-nb3muJxI0

/root/docker/stacks/tandoor/docker-compose.yml

version: "3"
services:
  db_recipes:
    container_name: tandoor_db
    restart: always
    image: postgres:11-alpine
    volumes:
      - ./data/postgresql:/var/lib/postgresql/data
    env_file:
      - ./.env
  web_recipes:
    container_name: tandoor_web
    image: vabene1111/recipes
    restart: always
    env_file:
      - ./.env
    volumes:
      - ./data/mediafiles:/opt/recipes/mediafiles
      - ./data/staticfiles:/opt/recipes/staticfiles
      - nginx_config:/opt/recipes/nginx/conf.d
    depends_on:
      - db_recipes
  nginx_recipes:
    container_name: tandoor_nginx
    image: nginx:mainline-alpine
    restart: always
    ports:
      - 80
    env_file:
      - ./.env
    depends_on:
      - web_recipes
    volumes:
      - ./data/mediafiles:/media
      - ./data/staticfiles:/static
      - nginx_config:/etc/nginx/conf.d:ro
volumes:
  nginx_config:
networks:
  default:
    external:
      name: nginx-proxy-manager

NextCloud

IMPORTANT

If you are receiving errors about PHP or issues with nginx or certificates:

Switch your image to lscr.io/linuxserver/nextcloud:24.0.6-ls204 and start the container
Execute docker exec -it nextcloud updater.phar repeatedly until there are no more updates (as of writing, Nextcloud 25 is the latest version)
Switch your image to lscr.io/linuxserver/nextcloud (latest, no tag) and start the container
Execute docker exec -it nextcloud mv /config/nginx/site-confs/default.conf /config/nginx/site-confs/default.conf.bak
Execute docker exec -it nextcloud mv /config/nginx/nginx.conf /config/nginx/nginx.conf.bak
Execute docker logs nextcloud and check for any other outdated configs, rename them with a .bak extension (like above)
Restart the container
Nextcloud should now be in a working state

Nextcloud gives you access to all your files wherever you are.

Create your container folders...

mkdir -p /root/docker/stacks/nextcloud/data/{config,files}
chown -R 1000:1000 /root/docker/stacks/nextcloud/data/

/root/docker/stacks/nextcloud/docker-compose.yml

version: "2.1"
services:
  nextcloud:
    image: ghcr.io/linuxserver/nextcloud
    container_name: nextcloud
    environment:
      - PUID=1000
      - PGID=1000
      - TZ=Europe/London
    volumes:
      - ./data/config:/config
      - ./data/files:/data
    ports:
      - 4443:443
    restart: unless-stopped

(I have changed the default port it listens on the local machine to 4443 but if you don't need any ports open then change the lines to:-

    expose:
      - 443

...then use Nginx Proxy Manager to direct traffic to your NextCloud installation)

Now visit https://ip.address.of.host:4443

...and on that setup page, _untick_ the option for "Install recommended apps" which does not install Calendar, Contacts, Mail, Chat, etc.

Enjoy.

https://hub.docker.com/r/linuxserver/nextcloud

Command Line Admin OCC

https://docs.nextcloud.com/server/latest/admin_manual/configuration_server/occ_command.html

List Users

How do I list the users in NextCloud Docker?

sudo docker exec -it nextcloud occ user:list

Reset Admin Password

How do you reset the admin user password in NextCloud Docker?

sudo docker exec -it nextcloud occ user:resetpassword admin

https://docs.nextcloud.com/server/latest/admin_manual/configuration_user/reset_admin_password.html

Project Send

Self-hosted file sharing... small, simple, secure.

https://www.projectsend.org

https://docs.linuxserver.io/images/docker-projectsend

https://github.com/linuxserver/docker-projectsend

What they don't tell you in the docs is that you need a database backend - which is not in the docker compose file.

So, we just add a MariaDB database container to the stack!

Create your subdomain A record in DNS...

cli53 rrcreate domain.uk 'send 300 A 123.45.678.90'

Create your Proxy Host in Ngnix Proxy Manager with an SSL...

https://send.domain.uk

Create directories on the server for the Docker container files...

sudo -i
mkdir -p /root/docker/stacks/projectsend/data/{config,db,files}
chown -R 1000:1000 /root/docker/stacks/projectsend/data/files

/root/docker/stacks/projectsend/docker-compose.yml

version: "2.1"
services:
  projectsend:
    image: ghcr.io/linuxserver/projectsend
    container_name: projectsend
    environment:
      - PUID=1000
      - PGID=1000
      - TZ=Europe/London
      - MAX_UPLOAD=100
    volumes:
      - ./data/config:/config
      - ./data/files:/data
      - /etc/timezone:/etc/timezone:ro
    expose:
      - 80
    restart: unless-stopped
  projectsend-db:
    image: mariadb
    container_name: projectsend-db
    environment:
      TZ: Europe/London
      MYSQL_ROOT_PASSWORD: projectsend
      MYSQL_DATABASE: projectsend
      MYSQL_USER: projectsend
      MYSQL_PASSWORD: projectsend
    volumes:
      - ./data/db:/var/lib/mysql
      - /etc/timezone:/etc/timezone:ro
    restart: unless-stopped
networks:
  default:
    external:
      name: nginx-proxy-manager

Go to the secure web site URL and complete the installation, using the Docker container name for the 'Database hostname'.

projectsend-db

Then, log in with your Admin username and password...

Create Group 'Public' which is public.
Create Group 'Customers' which is not public.
Create Client 'Customer Name' which is assigned to the 'Customers' group.
Upload some files and test both the Public and Client links.

Enjoy.

Troubleshooting

Change the Site URL

If you move host or domain name, you can log in to the DB container and change the 'base_uri'...

docker exec -it projectsend-db bash
mysql -u root -p projectsend

MariaDB [projectsend]> 
MariaDB [projectsend]> select * from tbl_options where name = 'base_uri';
+----+----------+-------------------------+
| id | name     | value                   |
+----+----------+-------------------------+
|  1 | base_uri | https://send.domain.uk/ |
+----+----------+-------------------------+

MediaWiki

Installation

Create the docker compose file and use default volume. Go to your browser at http://localhost:8080 and finish setup. Download LocalSettings.php file and copy to it to the container filesystem, then copy the whole folder to the host filsystem...

docker cp LocalSettings.php mediawiki:/var/www/html/
docker cp mediawiki:/var/www/html /root/docker/stacks/mediawiki/data/
chown -R www-data:www-data data/html
chmod o-w data/html
docker-compose down
(then edit your docker-compose.yml file so that local folders are used)
docker-compose up -d

Now, all the files are on your docker folder, ready to easily backup :-)

mediawiki
`-- data
    |-- db
    `-- html

~/docker/mediawiki/docker-compose.yml

version: '3'
services:
  mediawiki:
    image: mediawiki
    container_name: mediawiki
    restart: always
    ports:
      - 8080:80
    links:
      - database
    volumes:
      #- ./data/html:/var/www/html     <-- #2
      #- /var/www/html/images      <-- #1
    environment:
      - PUID=33
      - PGID=33
      - TZ=Europe/London
  database:
    image: mariadb
    container_name: mediawiki_db
    restart: always
    environment:
      MYSQL_DATABASE: my_wiki
      MYSQL_USER: wikiuser
      MYSQL_PASSWORD: example
      MYSQL_RANDOM_ROOT_PASSWORD: 'yes'
    volumes:
      - ./data/db:/var/lib/mysql

https://hub.docker.com/_/mediawiki

Tweaks

Change default skin to mobile responsive modern one...

wfLoadSkin( 'Timeless' );
$wgDefaultSkin = "timeless";

Enable the new editing toolbar...

wfLoadExtension( 'WikiEditor' );

Make the URL shorter...

$wgScriptPath = "";
$wgScriptExtension = ".php";
$wgArticlePath = "/wiki/$1";
$wgUsePathInfo = true;

File uploads...

LocalSettings.php

$wgEnableUploads = true;

https://www.mediawiki.org/wiki/Manual:Configuring_file_uploads

https://kindalame.com/2020/11/25/self-hosting-mediawiki-with-docker/

Importing

Pages

OLD SERVER

Generate the page dump in XML format...

docker exec -it mediawiki bash
php maintenance/dumpBackup.php --current > pages.xml
exit

NEW SERVER

Import the pages...

cp pages.xml ./data/html/
docker exec -it mediawiki bash
php maintenance/importDump.php < pages.xml
php maintenance/update.php
php maintenance/rebuildall.php
exit

https://www.hostknox.com/tutorials/mediawiki/pages/export-and-import#import-pages-via-ssh

Images

OLD SERVER

Generate the image dumps using dumpUploads.php, which creates a txt list of all image filenames in use...

mkdir /tmp/workingBackupMediaFiles
php maintenance/dumpUploads.php \
   | sed 's~mwstore://local-backend/local-public~./images~' \
   | xargs cp -t /tmp/workingBackupMediaFiles
zip -r ~/Mediafiles.zip /tmp/workingBackupMediaFiles
rm -r /tmp/workingBackupMediaFiles

NEW SERVER

Unzip the files to your container filsystem...

cd /root/docker/stacks/mediawiki
unzip Mediafiles.zip -d ./data/html/

Import the Images...

docker exec -it mediawiki bash
php maintenance/importImages.php tmp/workingBackupMediaFiles
php maintenance/update.php
php maintenance/rebuildall.php
exit

https://stackoverflow.com/questions/1002258/exporting-and-importing-images-in-mediawiki

Kuma

A self-hosted monitoring tool like Uptime Robot and not as complicated as Nagios.

https://hub.docker.com/r/louislam/uptime-kuma

https://github.com/louislam/uptime-kuma

https://youtu.be/dIVf1nhT0mI

Kasm Containerized Apps and Desktops

Streaming containerized apps and desktops to end-users. The Workspaces platform provides enterprise-class orchestration, data loss prevention, and web streaming technology to enable the delivery of containerized workloads to your browser.

https://kasmweb.com

https://hub.docker.com/u/kasmweb

https://www.youtube.com/channel/UCgpv4MLH8diVlIiakCBu8eQ

SFTP

https://hub.docker.com/r/atmoz/sftp

Wordle

https://github.com/cwackerfuss/react-wordle

Ombi

Ombi allows you to host your own Emby Request and user management system. If you are sharing your Emby server with other users, allow them to request new content using an easy to manage interface! Manage all your requests for Movies and TV with ease, leave notes for the user and get notification when a user requests something. Allow your users to post issues against their requests so you know there is a problem with the audio etc. Even automatically send them weekly newsletters of new content that has been added to your Emby server :-)

https://hub.docker.com/r/linuxserver/ombi

Emby

https://fleet.linuxserver.io/image?name=linuxserver/emby

https://hub.docker.com/r/linuxserver/emby

---
version: "2.1"
services:
  emby:
    image: lscr.io/linuxserver/emby
    container_name: emby
    environment:
      - PUID=998
      - PGID=100
      - TZ=Europe/London
    volumes:
      - /srv/dev-disk-by-uuid-7f81e7b6-1a05-4232-893b-f34c046b2bdb/Emby/Config:/config
      - /srv/dev-disk-by-uuid-7f81e7b6-1a05-4232-893b-f34c046b2bdb/Emby/TV:/data/tvshows
      - /srv/dev-disk-by-uuid-7f81e7b6-1a05-4232-893b-f34c046b2bdb/Emby/Movies:/data/movies
    ports:
      - 8096:8096
    restart: unless-stopped

Tdarr

Tdarr is a conditional based transcoding application for automating media library transcoding and remux management which uses cross-platform Tdarr Nodes which work together with Tdarr Server to process your files.

https://docs.tdarr.io/docs/installation/docker/run-compose

version: "3.4"

services:

# server
  tdarr:
    container_name: tdarr
    image: ghcr.io/haveagitgat/tdarr:latest
    restart: unless-stopped
    network_mode: bridge
    ports:
      - 8265:8265 # webUI port
      - 8266:8266 # server port
      - 8267:8267 # Internal node port
      - 8268:8268 # Example extra node port
    environment:
      - TZ=Europe/London
      - PUID=${PUID}
      - PGID=${PGID}
      - UMASK_SET=002
      - serverIP=0.0.0.0
      - serverPort=8266
      - webUIPort=8265
      - internalNode=true
      - nodeID=MyInternalNode
    volumes:
      - /docker/tdarr/server:/app/server
      - /docker/tdarr/configs:/app/configs
      - /docker/tdarr/logs:/app/logs
      - /media:/media
      - /transcode_cache:/temp

# node example
  tdarr-node:
    container_name: tdarr-node
    image: ghcr.io/haveagitgat/tdarr_node:latest
    restart: unless-stopped
    network_mode: service:tdarr
    environment:
      - TZ=Europe/London
      - PUID=${PUID}
      - PGID=${PGID}
      - UMASK_SET=002
      - nodeID=MainNode
      - serverIP=0.0.0.0
      - serverPort=8266
    volumes:
      - /docker/tdarr/configs:/app/configs
      - /docker/tdarr/logs:/app/logs
      - /media:/media
      - /transcode_cache:/temp

Unifi Controller

https://github.com/linuxserver/docker-unifi-controller

If you want to fix a particular version of the controller then check the 'releases' at github and adjust your docker-compose.yml file accordingly.

e.g. linuxserver/unifi-controller:5.14.23-ls76

version: "2.1"
services:
  unifi-controller:
    # image: linuxserver/unifi-controller:latest
    image: linuxserver/unifi-controller:5.14.23-ls76
    container_name: unifi-controller
    environment:
      - PUID=998
      - PGID=100
      # - MEM_LIMIT=1024 #optional
      # - MEM_STARTUP=1024 #optional
    volumes:
      - <path to data>:/config
    ports:
      - 8443:8443
      - 3478:3478/udp
      - 10001:10001/udp
      - 8080:8080
      - 1900:1900/udp #optional
      - 8843:8843 #optional
      - 8880:8880 #optional
      - 6789:6789 #optional
      - 5514:5514/udp #optional
    restart: unless-stopped

rPort

Rport helps you to manage your remote servers without the hassle of VPNs, chained SSH connections, jump-hosts, or the use of commercial tools like TeamViewer and its clones.

Rport acts as server and client establishing permanent or on-demand secure tunnels to devices inside protected intranets behind a firewall.

All operating systems provide secure and well-established mechanisms for remote management, being SSH and Remote Desktop the most widely used. Rport makes them accessible easily and securely.

https://rport.io/en/features

https://oss.rport.io/

https://github.com/cloudradar-monitoring/rport

https://hub.docker.com/r/acwhiteglint/rport

Paperless

Paperless is an application that indexes your scanned documents and allows you to easily search for documents and store metadata alongside your documents.

https://github.com/jonaswinkler/paperless-ng

https://paperless-ng.readthedocs.io/en/latest/setup.html#installation

Ansible

So, this is not installing Docker using Ansible... this is installing (or running) Ansible using Docker :-)

https://iceburn.medium.com/run-ansible-with-docker-9eb27d75285b

Pi Alert

WIFI / LAN intruder detector. Scan the devices connected to your WIFI / LAN and alert you the connection of unknown devices. It also warns the disconnection of "always connected" devices.

https://github.com/pucherot/Pi.Alert

Roundcube Webmail

https://hub.docker.com/search?q=roundcube

https://hub.docker.com/r/roundcubeorg/roundcubemail

https://github.com/roundcube/roundcubemail/wiki/Configuration

docker run -e ROUNDCUBEMAIL_DEFAULT_HOST=ssl://mail.domain.co.uk -e ROUNDCUBEMAIL_DEFAULT_PORT=993 -e ROUNDCUBEMAIL_SMTP_SERVER=tls://mail.domain.co.uk -e ROUNDCUBEMAIL_SMTP_PORT=587 -e ROUNDCUBEMAIL_PLUGINS=archive,zipdownload,managesieve,mobile -p 8000:80 -d roundcube/roundcubemail

FAST SpeedTest

docker run -it --rm --name fast-cli mschirrmeister/fast-cli:latest

https://hub.docker.com/r/mschirrmeister/fast-cli

https://github.com/sindresorhus/fast-cli

Crypto Mining

XMRig

https://hub.docker.com/r/minerboy/xmrig

Go to the XMRig Wizard and generate config like this ...

{
    "autosave": true,
    "cpu": true,
    "opencl": false,
    "cuda": false,
    "pools": [
        {
            "coin": "monero",
            "algo": "rx/0",
            "url": "stratum+tcp://randomxmonero.auto.nicehash.com:9200",
            "user": "NHbLd5exQeCGGyWnopVoLHLbzexKN5z8iq7p.NAS",
            "pass": "x",
            "tls": false,
            "keepalive": true,
            "nicehash": true
        }
    ]
}

Then save that file as config.json and run the following commands to start mining ...

sudo -i
docker pull minerboy/xmrig
docker run --cap-add=SYS_ADMIN --cap-add=SYS_RAWIO --device=/dev/cpu --device=/dev/mem -v /lib/modules:/lib/modules -v /full/path/to/config.json:/etc/xmrig/config.json minerboy/xmrig:latest --cpu-max-threads-hint 50 --threads 2

Here is the same command as a lovely docker compose file ...

version: '3.3'
services:
  xmrig:
      image: 'minerboy/xmrig:latest'
      container_name: xmrig
      devices:
        - /dev/cpu
        - /dev/mem
      volumes:
        - '/lib/modules:/lib/modules'
        - '/root/docker/stacks/xmrig/config.json:/etc/xmrig/config.json'
      command:
        - --cpu-max-threads-hint 50
        - --threads 2
      restart: "no"

Help

DB Tech

https://www.youtube.com/channel/UCVy16RS5eEDh8anP8j94G2A

https://gist.github.com/dnburgess

Anonymous

Search

Docker

Introduction

Images

Useful Wiki

Installation

New All In One Official Method

Engine

Compose From Command Line

Using Ansible

Usage

Run Command In Docker Container

Volumes

Multiple Containers

Networks

Basic Usage

Advanced Usage

Block IP Address Using IPTables

Docker Compose

Restart Policy

Management

Cleaning Pruning

Delete All Stopped Docker Containers

Updating with Docker Compose

Logs Logging

Cleaning Space

Dockge

Portainer

Server

Agent

Updating

Backups

Monitoring

Gotchas

Applications

Backups

Updates

Tracking

Watchtower

Updating

Bitwarden

InfluxDB

Grafana

NGiNX Proxy Manager

NGiNX

Typical LEMP

WordPress

PHP File Uploads Fix

WordPress Clone

WordPress CLI

SSL Behind A Reverse Proxy

Email Server (mailu)

Email Server (docker-mailserver)

Postscreen

Postgrey

SpamAssassin

Fail2Ban

Backups

Autodiscover

Internet Speedtest

Emby Media Server

AWS CLI

Let's Encrypt

VPN

Gluetun

OpenVPN

WireHole

WireGuard

ffmpeg

MediaInfo

MKV Toolnix

MakeMKV

HandBrake

FileBot

Setup

Usage

Automated Downloaderr

FlareSolverr

Jackett