Creating a full Media Server on Ubuntu with Docker

A full configuration guide of a complete media server including programs such as PVR's, Jackett, qBittorrent(With VPN), Reverse Proxy, NextCloud, MySQL, Portainer and more!

Installing Ubuntu(or kubuntu) and installing Docker(Part 1)

This is going to be a pretty long series but I will try to keep it easy to read and understandable. This is not hard. But I am around for questions. Lets get started!

A few things to note before we begin

Here are the steps to install kubuntu19.04:

  1. Download the .iso file from here.
  2. Download Rufus and install it from here.
  3. Run Rufus as an administrator and select the device in the top menu(the USB drive you want to install the OS on)
  4. Boot Selection(Disk or ISO) and select the ISO image you downloaded.rufus.png
  5. Start the process. Then make sure Write in ISO is selected and proceed then itll tell you its deleting everything on it. Press ok.
  6. When it says READY again at the bottom in green the drive is ready to be inserted into the machine your installing it.

Now there might be different things you need to do but for me I had to make sure I selected to boot from the USB Drive during startup so it could begin the installation. It's all self explanatory from there on out and is really easy. When it is done it'll prompt you telling you you need to reboot. Once you reboot you'll be greeted with a login screen. Enter the login information you entered during the installation. That's it! kubuntu is installed. You might need to configure drivers, etc. If you do just google it, ubuntu instructions work for kubuntu.

Now it is time for the docker and docker-compose installation!

  1. First you want to open terminal(can find that by typing terminal into the search bar)
  2. Copy and paste this into the terminal: curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo apt-key add -
  3. Then you will want to copy and paste this into the terminal as well: sudo add-apt-repository "deb [arch=amd64] https://download.docker.com/linux/ubuntu $(lsb_release -cs) stable edge"

  4. Your going to have to run a update now to retrieve information from the newly added repository by copy and pasting this into the terminal:sudo apt-get update && apt-cache policy docker-ce

  5. After you see the output with the docker version, its time to actually install docker with copying this into the terminal: 

    sudo apt-get install -y docker-ce
  6. You want to check the status of docker to make sure it's running with entering this into the terminal:
    sudo systemctl status docker
  7. If that says active(running) your all set! If you don't want to need to use sudo each time you use docker or docker-compose you can add this into the terminal to fix it:
    sudo usermod -aG docker ${USER}
  8. After you have ran that command, logout and log back in to your account. That's to fix you not needing to type sudo before docker and docker-compose.
  9. Now to install docker-compose copy and paste this into your terminal:sudo curl -L "https://github.com/docker/compose/releases/download/1.24.0/docker-compose-$(uname -s)-$(uname -m)" -o /

After you retrieve docker-compose you need to make it executable with:

sudo chmod +x /usr/local/bin/docker-compose && sudo ln -s /usr/local/bin/docker-compose /usr/bin/docker-compose

Once that command has been ran, type docker-compose -v and see if it outputs the version. If so congratulations! If not, there are some other problems that may be of concern. Feel free to post a comment asking about it!

Static IP, SSH, Environment Variables, Mounting drives, Permissions and directories(Part 2)

In the last section we installed kubuntu19 and setup docker as well as docker-compose. In Part 2 we will be doing the following things:

  1. Set up Static IP
  2. Installing Open-SSH server
  3. Setting the environment variables that'll make configuring your docker containers easier.
  4. Mounting External Drives if needed to a location. I use /media/${USER}/ExternalDriveHere.
  5. Creating directories and changing permissions to make everything communicate better

Setting Static IP:

  1. Go to terminal and type
    sudo apt install net-tools -y && ifconfig -a
  2. Make a note of the connection that has a default gateway, like 192.168.0.1,etc. You'll need the DNS Servers, and the gateway.
  3. Open Network Connections on the bottom right.
  4. Go to your active internet connection(The one that has access to your router)
  5. Edit the settings of the connection under IPv4 and set it to manual.
  6. Fill in your DNS Servers, with a comma. You can use 1.1.1.1,1.0.0.1 if you don't want to use your ISP's DNS Servers
  7. Press Add and type in the IP address it shown in ifconfig earlier, netmask will fill itself and use the gateway from ifconfig as well
  8. Press Apply and Ok. Now your local IP address will not change!

Setting up Open-SSH Server(Optional if server is not on your main computer):

  1. Open terminal and type
    sudo apt install openssh-server -y
    sudo service ssh status
  2. If it says active you can download putty on your computer.
  3. Install Putty and then open it. Connect to the IP address you set earlier, Port 22.
  4. Press connect and type in your username and password.
  5. Now you can control your server without being at it.

Setting your Environment Variables for your account:

  1. Open terminal and type
    id
  2. Get the Numbered UserID of your user and the number for docker(mine is 130)
  3. Now you need to edit your environment variable file by typing this in the terminal:
    sudo nano /etc/environment
  4. Go to the end of the file and add a few different lines
    PUID=(The ID of your user usually 1000)
    PGID=(The ID of docker group, was 130 for me)
    TZ="America/New_York"(The TZ code of which you reside)
    USERDIR="/home/USER"#fill in USER with your username
    MYSQL_ROOT_PASSWORD="password you will choose for MySQL"
    Let me show you what a completed file looks like:
    PATH="/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games"
    PUID=1000 PGID=130 TZ="America/New_York" USERDIR="/home/mediaserver" MYSQL_ROOT_PASSWORD="123456789"
  5. Now that it is done, press ctrl+x, press y and hit enter.
  6. Logout and log back in for the changes to take effect!

External Drive Mounting(Optional):

  1. Open terminal and type
    sudo fdisk -l
  2. Find the drive you are looking for mounting that you will put media in. All you usually need is /dev/sdd2 /dev/sde1 etc.
  3. Once you know which drive you need to mount check to see if its mounted anywhere by typing
    sudo umount /dev/PARTITION_HERE
  4. Then you can mount the drive to your location of choice, mine is inside /media/USERNAME/
    sudo mount /dev/PARTITION_HERE /media/USERNAME/NAMEYOUWANT

Creating Docker directory and setting permissions!

  1. Open terminal and type
  2. 3
     
    1
    2
    3
  3. This should make a directory and set permissions to it called docker it will be located inside /home/USERNAME/docker. This is where all of your Docker files will be stored from all your different containers. Your other volumes get stored in /var/lib/docker/volumes.

Okay so now you have completed the prep work before you get started configuring docker containers! I look forward to seeing you all in Part 3.

Have a wonderful day!

Creating Docker-Compose file!(Part 3)

So far we have installed an operating system and did a ton of other stuff but we haven't gotten to any of the actual Media-Server guides. So in this guide I will be giving you a docker compose file that will only need a few tweaks to it and I will let you know what you need to change. I will put # before lines that need to be changed with what to change them with.

There will be no numbering as its one long file. You will however need to open your terminal and type
sudo nano /home/USERNAME/docker/docker-compose.yml

Now you have initiated starting a file called docker-compose.yml inside of the docker folder you created earlier. I strongly suggest getting a good text editor or using Kate on kubuntu and copy and pasting this:

version: "3"
services:
  plexms:
    container_name: plexms
    restart: always
    network_mode: host
    image: plexinc/pms-docker
    volumes:
      - ${USERDIR}/docker/plexms:/config
      - ${USERDIR}/Downloads/plex_tmp:/transcode
      #Where all your media is stored
      - /media/USER:/media
      - ${USERDIR}/docker/shared:/shared
      - ${USERDIR}/docker/letsencryptcerts:/certs:ro
    ports:
      - "32400:32400/tcp"
      - "3005:3005/tcp"
      - "8324:8324/tcp"
      - "32469:32469/tcp"
      - "1900:1900/udp"
      - "32410:32410/udp"
      - "32412:32412/udp"
      - "32413:32413/udp"
      - "32414:32414/udp"
    environment:
      - TZ=${TZ}
      - HOSTNAME="Docker Plex"
      #You get this From Plex
      - PLEX_CLAIM="claim-##########################"
      - PLEX_UID=${PUID}
      - PLEX_GID=${PGID}
      #- ADVERTISE_IP="http://192.168.0.0:32400/"

  portainer:
    image: portainer/portainer
    hostname: portainer
    container_name: portainer
    restart: always
    command: -H unix:///var/run/docker.sock
    ports:
      - "9000:9000"
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock
      - ${USERDIR}/docker/portainer/data:/data
      - ${USERDIR}/docker/shared:/shared
    environment:
      - TZ=${TZ}

  organizr:
    container_name: organizr
    hostname: organizr
    restart: always
    image: organizrtools/organizr-v2
    volumes:
      - ${USERDIR}/docker/organizr/www:/config/www
      - ${USERDIR}/docker/organizr/organizrlog:/config/log
      - ${USERDIR}/docker/shared:/shared
      - ${USERDIR}/docker/organizr/organizrconfig:/config
    ports:
      - "9983:80"
    environment:
      - PUID=${PUID}
      - PGID=${PGID}
      - TZ=${TZ}

#Keeps containers up to date
  watchtower:
    container_name: watchtower
    hostname: watchtower
    restart: always
    image: v2tec/watchtower
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock
    command: --schedule "0 0 4 * * *" --cleanup

#For this make sure to download an .ovpn file from VPN provider and put in openvpn folder inside /home/user/docker/qbittorrent/openvpn, possibility of using a credentials.conf
#Make sure to create categories so sonarr, radarr, lidarr and lazylibrarian can all use them. Set download locations for each category such as Movies category would be /downloads/movies
  qbvpn:
    image: markusmcnugen/qbittorrentvpn
    container_name: qbvpn
    privileged: true
    cap_add:
      - NET_ADMIN
    devices:
      - /dev/net/tun
    environment:
      - PUID=${PUID}
      - PGID=${PGID}
      - TZ=${TZ}
      - VPN_ENABLED=yes
      - LAN_NETWORK=192.168.0.0/24
      - NAME_SERVERS=1.1.1.1,1.0.0.1
      - VPN_USERNAME=
      - VPN_PASSWORD=
    volumes:
      - ${USERDIR}/docker/qbittorrent:/config
      #These are all the locations where categories get set in the UI of qBittorrent at ip:8080 username admin password passwordpassword 
      - /media/USER/HDD:/downloads
      - /media/USER/HDD/shows:/downloads/shows
      - /media/USER/HDD/movies:/downloads/movies
      - /media/USER/HDD/books:/downloads/books
      - /media/USER/HDD/audiobooks:/downloads/audiobooks
      - /media/USER/HDD/music:/downloads/music
      - /media/USER/HDD/blackhole:/downloads/torrents
      - /media/USER/HDD/torrents:/downloads/blackhole
      - ${USERDIR}/docker/shared:/shared
    ports:
      - 8080:8080
      - 8999:8999
      - 8999:8999/udp
    restart: unless-stopped

  radarr:
    image: "linuxserver/radarr"
    hostname: radarr
    container_name: "radarr"
    volumes:
      - ${USERDIR}/docker/radarr:/config
      #External Mount for current drive im using(The torrent downloader plays friendly when theyre the same directories inside each container)
      #Remember this can be changed to ${USERDIR}/docker/downloads/movies:/downloads/movies
      - /media/USER/HDD/movies:/downloads/movies
      - ${USERDIR}/media/movies:/movies
      - ${USERDIR}/docker/shared:/shared
      #External Mount so radarr can find archived movies
      #- /media/USER/HDD2:/media/HDD2
    ports:
      - "7878:7878"
    restart: always
    environment:
      - PUID=${PUID}
      - PGID=${PGID}
      - TZ=${TZ}

  sonarr:
    image: "linuxserver/sonarr"
    hostname: sonarr
    container_name: "sonarr"
    volumes:
      - ${USERDIR}/docker/sonarr:/config
      #External Mount for current drive im using(The torrent downloader plays friendly when theyre the same directories inside each container)
      #Remember this can be changed to ${USERDIR}/docker/downloads/shows:/downloads/shows
      - /media/USER/HDD/shows:/downloads/shows
      - ${USERDIR}/media/media/tvshows:/tv
      - ${USERDIR}/docker/shared:/shared
      #External Mount
      - /media/USER/HDD2:/media/HDD2
    ports:
        - "8989:8989"
    restart: always
    environment:
      - PUID=${PUID}
      - PGID=${PGID}
      - TZ=${TZ}

  lidarr:
    image: "linuxserver/lidarr"
    hostname: lidarr
    container_name: "lidarr"
    volumes:
      - ${USERDIR}/docker/lidarr:/config
      #(The torrent downloader plays friendly when theyre the same directories inside each container)
      #External Mounts for where the torrent downloaders download the music too and the default directory of where the qbvpn is set for default download directory(Used with Jackett Blackhole and other Torrent Links)
      #Remember this can be changed to ${USERDIR}/docker/downloads/music:/downloads/music and ${USERDIR}/docker/downloads/torrents:/downloads/torrents
      - /media/USER/HDD/music:/downloads/music
      - /media/USER/HDD/torrents:/downloads/torrents
      - ${USERDIR}/media/media/music:/music
      - ${USERDIR}/docker/shared:/shared
      #External Mount
      - /media/USER/HDD2:/media/HDD2

    ports:
      - "8686:8686"
    restart: always
    environment:
      - PUID=${PUID}
      - PGID=${PGID}
      - TZ=${TZ}

  lazylibrarian:
    image: linuxserver/lazylibrarian
    container_name: lazylibrarian
    environment:
      - PUID=${PUID}
      - PGID=${PGID}
      - TZ=${TZ}
    volumes:
      - ${USERDIR}/docker/lazylib/config:/config
      #External Mount (The torrent downloader plays friendly when theyre the same directories inside each container)
      #Remember this can be changed to ${USERDIR}/docker/downloads/books:/downloads/books
      #Remember this can be changed to ${USERDIR}/docker/downloads/audiobooks:/downloads/audiobooks
      - /media/USER/HDD/books:/downloads/books
      - /media/USER/HDD/audiobooks:/downloads/audiobooks
      - ${USERDIR}/media/media/books:/books
      - ${USERDIR}/docker/shared:/shared
    ports:
      - 5299:5299
    restart: unless-stopped

  tautulli:
    container_name: tautulli
    hostname: tautulli
    restart: always
    image: linuxserver/tautulli
    volumes:
      - ${USERDIR}/docker/tautulli/config:/config
      - ${USERDIR}/docker/tautulli/logs:/logs:ro
      - ${USERDIR}/docker/shared:/shared
    ports:
      - "8181:8181"
    environment:
      - PUID=${PUID}
      - PGID=${PGID}
      - TZ=${TZ}

  ombi:
    container_name: ombi
    hostname: ombi
    restart: always
    image: linuxserver/ombi
    volumes:
      - ${USERDIR}/docker/ombi:/config
      - ${USERDIR}/docker/shared:/shared
    ports:
      - "3579:3579"
    environment:
      - PUID=${PUID}
      - PGID=${PGID}
      - TZ=${TZ}

  jackett:
    image: "linuxserver/jackett"
    hostname: jackett
    container_name: "jackett"
    volumes:
      - ${USERDIR}/docker/jackett:/config
      #External Mount for Blackhole downloads
      #Remember this can be changed to ${USERDIR}/docker/downloads/torrents:/downloads
      - /media/USER/HDD/torrents:/downloads
      - ${USERDIR}/docker/shared:/shared
    ports:
      - "9117:9117"
    restart: always
    environment:
      - PUID=${PUID}
      - PGID=${PGID}
      - TZ=${TZ}
  
#This will allow you to generate certficates and keep a webserver up for reverse proxies, ill go into more detail about it later
  letsencrypt:
    image: linuxserver/letsencrypt
    container_name: letsencrypt
    cap_add:
      - NET_ADMIN
    environment:
      - PUID=${PUID}
      - PGID=${PGID}
      - TZ=${TZ}
      #Website you made on freenom
      - URL=yourwebsite.tk
      #Subdomains you chose to use like www,www.plex,plex,www.sonarr,sonarr etc.
      - SUBDOMAINS=www,portainer,sonarr,radarr,plex
      - VALIDATION=http
      #An email is important so you get notifications about your certs.
      - EMAIL=admin@yourwebsite.tk
      - DHLEVEL=2048 #optional
      - ONLY_SUBDOMAINS=false #optional
      #Extra domains seperated by commas that need to have an SSL as well
      - EXTRA_DOMAINS=otherdomain.tk
    volumes:
      - ${USERDIR}/docker/letsencryptcerts:/config
    ports:
      - 443:443
      - 80:80
    restart: unless-stopped

  ubooquity:
    image: linuxserver/ubooquity
    container_name: ubooquity
    environment:
      - PUID=${PUID}
      - PGID=${PGID}
      - TZ=${TZ}
      - MAXMEM=1024
    volumes:
      - ${USERDIR}/docker/uboo/config:/config
      #Remember this can be changed to ${USERDIR}/docker/downloads/books:/downloads/books
      - /media/USER/HDD/books:/downloads/books
      - ${USERDIR}/docker/uboo/comics:/comics
      - ${USERDIR}/docker/shared:/shared
    ports:
      - 2202:2202
      - 2203:2203
    restart: unless-stopped

  psitransfer:
    container_name: psitransfer
    image: psitrax/psitransfer
    restart: unless-stopped
    ports:
      - 3000:3000
    environment:
      - PSITRANSFER_ADMIN_PASS=
    volumes:
      #External Mount for the PSI transfer files
      #Remember this can be changed to ${USERDIR}/docker/downloads/psifiles:/data
      - /media/USER/HDD/files:/data
      
  mydb:
    image: mariadb
    command: --transaction-isolation=READ-COMMITTED --binlog-format=ROW
    restart: always
    container_name: mysqldb
    volumes:
      - db:/var/lib/mysql
    ports:
      - 3306:3306
    environment:
      - MYSQL_ROOT_PASSWORD=${MYSQL_ROOT_PASSWORD}
      - MYSQL_PASSWORD=12345
      - MYSQL_DATABASE=nextcloud
      - MYSQL_USER=12345
#Links to mysql so you can use MySQL for the Database backend of Nextcloud!
  nextcloud:
    image: nextcloud
    container_name: nextcloud
    volumes:
      - /home/USER/docker/newcloud/config:/var/www/html/config
      - /home/USER/docker/newcloud/html:/var/www/html
      - /home/USER/docker/newcloud/apps:/var/www/html/custom_apps
      #Remember this can be changed to ${USERDIR}/docker/downloads/cloud:/var/www/html/data
      - /media/USER/HDD/cloud:/var/www/html/data
      #External Mounts for External Storages inside NextCloud
      #- /media/USER/HDD/blackhole:/media/downloads
      #- /media/USER/HDD/movies:/media/movies
      #- /media/USER/HDD/shows:/media/shows
      #- /media/USER/HDD/music:/media/music
      #- /media/USER/HDD/books:/media/books
      #- /media/USER/HDD2:/media/HDD2
      #Uses the same DB volume as the mysql container
      - db:/var/lib/mysql
    environment:
      - MYSQL_DATABASE=nextcloud
      - MYSQL_HOST=192.168.x.x
      - MYSQL_USER=12345
      - MYSQL_PASSWORD=12345
    ports:
      - 8010:80
    links:
      - mydb
    restart: unless-stopped


volumes:
  db:

If you notice the environment variables we made earlier get used a lot here, it saves time. All the directories it needs gets created and put inside /home/USERNAME/docker and everything that doesn't need readily access goes to /var/lib/docker/volumes like docker_db for nextcloud/mysql.

In the next section we will be taking this docker-compose file and using it to build our entire server in one go! Part 4 here we come!

Running your server for the first time!(Part 4)

So we have setup everything we need to get the show on the road!

First thing we need to make sure we are in the docker directory located at /home/USERNAME/docker. You should have created a docker-compose.yml file that is the basic configuration of each of our media server components! What that means is its very easy to get them started. All you have to do is go to the terminal and type:

cd ${USERDIR}/docker
docker-compose up -d

You will see it slowly pulling and extracting all the image files it needs to run the containers, so first run takes the longest. Its a lot to download. But it will then say that they have all been started. I usually like to verify that everything is all good by going to portainer which is located at ip:9000 and looks like this when all is said and doneportainerstats.png

I do not run the nginx with docker, that starts at the beginning of my system. I just use letsencrypt for certificates. But lets encrypt also has a ton of sample proxy configs that will help you with most of these.

So what happens here is all your containers are being configured and creating the necessary files they need to run. Most of the work left is configuring each service which I will not go into... It takes a long time to get it right. Theres so many options. I will go into the setup of organizr and configuring the options inside the letsencrypt container since that is where your proxy will be that will send to subdomains, or if you prefer, folders. So take this time to setup plex. Since that is definitely the main concern here everything else can come after.

Configuring your letsencrypt Reverse Proxy!(Part 5)

This is how you will access all of your services easily without having to remember ports or ip address's. A reverse proxy just passes traffic to the local ip:port on the server side so there's no need to expose more ports. Everything actually stay's local except plex and the Lets Encrypt container.

I have put together some configs you can download here. I will have a more detailed guide tomorrow. I have wrote a lot today I think I need a break :)