Jump to content


Photo

Freenas 11.0-U2 - Running emby via docker in an Ubuntu VM

docker freenas vm

  • Please log in to reply
7 replies to this topic

#1 Hisma OFFLINE  

Hisma

    Advanced Member

  • Members
  • 71 posts
  • Local time: 04:41 PM

Posted 13 August 2017 - 08:45 PM

Note that there's a lot of steps to this, but individually none are particularly difficult... I'm not a linux or bsd expert by any means, in fact I'd consider myself a complete amateur.  I  just pieced this all together from guides I read online, none of this is really my own, I'm just compiling it all in one place.  I'm a newb myself so I wrote this as newb-friendly as possible... not to insult anyone's intelligence, just to reach the largest audience possible.

 

That said, docker for freenas has been getting a lot of attention since corral, and while I never migrated to corral, everyone seemed to rave about docker so I really wanted to give it a shot.  It's now possible on freenas 11.0, thanks to native VM support.  So here i'll explain step-by-step how I got an emby docker container running on ubuntu server VM in freenas.  At the end you'll have emby server running in a docker container on a Ubuntu Server VM, with everything set to auto-start on reboot.

 

Set up Docker for Ubuntu Server

 

Note: I installed this on an Ubuntu Server 16.04 VM directly from the Freenas 11 GUI.

PreReqs:

  • freenas 11.0-U2 (may work on other 11.0 versions but it's confirmed working on this version for me)
  • Ubuntu Server 16.04 ISO x64
  • vnc viewer
  • (optional but highly recommended) - dedicated ssh app - i like secureCRT

First Step - Create the Ubuntu VM

Inside the freenas GUI:

  • Go to VM tab, and create a new VM.  Create VM name, and give it a good amount of resources, since this will be your container manager.  Based on the specs of my server (xeon 1220v3 w/ 32GB RAM), I gave my VM 4 CPUs and 12GB of RAM.  
  • Once VM is created, before you start it, go to devices tab, add CDROM, and point it to Ubuntu Server 16.04 iso image (it's free, just grab it from wherever)
  • Also add a disk, must have zvol to tie to. If no zvol, create a zvol with sufficient size.  Mine was 80GB.  I won't show how to set up a zvol here, but just search for it, it's a simple process

Now you can start the VM and connect initially via vnc viewer to the port specified on the VM details.  It should boot to the ubuntu cdrom and start the installation process.

Install Ubuntu with basic settings.  No GUI, no extra features.  Just keep it bare bones to keep it lightweight as possible.  It will come up with an option to use UEFI boot.  Choose that option as that will allow you to auto-start the VM via the freenas GUI.

UEFI Boot Fix

One annoying thing is that the freenas VM system will not boot to uefi automatically. There's an extra step you need to follow to fix this. https://forums.freen...ing-uefi.54039/

Summarizing steps from that guide, if you have shutdown your VM already and try to reboot it (which was my case), it will not automatically start the VM. Instead it will boot to shell. To eliminate this - Just type exit at the shell prompt, and in the EFI menu system navigate to "Boot Maintenance Manager" and then select "Boot from file" to locate and select your grubx64.efi file.

Once the VM is started, log in as root.

sudo -i

As root, cd to the

/boot/efi/EFI

directory of your VM in order to create the new BOOT directory and copy the existing grubx64.efi to /EFI/BOOT/bootx64.efi.

mkdir /boot/efi/EFI/BOOT
cp /boot/efi/EFI/ubuntu/grubx64.efi /boot/efi/EFI/BOOT
mv /boot/efi/EFI/BOOT/grubx64.efi /boot/efi/EFI/BOOT/bootx64.efi

The end result should look like this, using Ubuntu as an example:

root@ubuntu-vm:/boot/efi# tree -L 3 .
.
└── EFI
├── BOOT
│ └── bootx64.efi
└── ubuntu
├── fbx64.efi
├── grub.cfg
├── grubx64.efi
├── mmx64.efi
└── shimx64.efi
3 directories, 6 files
root@ubuntu-vm:/boot/efi#

The file bootx64.efi is a copy of the grubx64.efi in your VM.

NB: If grubx64.efi gets updated you will need to re-create bootx64.efi

 Restart the VM from the freenas GUI, and see if it boots directly to VM.

Now your ubuntu VM should be up and running, and it will auto-start on reboots.  

As soon as I got Ubuntu up and running, I made sure to enable ssh access.

 

Set Up SSH Access (Optional but Reccomended)

sudo apt-get install openssh-server 

Now you can access the VM via ssh (I like secureCRT) instead of relying on vnc viewer.  

Also, make sure to mount NAS folder to give NAS access to your VM -

  • Install cifs-utils
apt-get install cifs-utils

Create the directory and mount the device using the following commands (I named my mount point "freenas".  Name it whatever you want).

sudo mkdir /mnt/freenas

Now mount the device using cifs 

mount -t cifs //192.168.0.155/nas -o username=anonymous,password=,rw,nounix,iocharset=utf8,file_mode=0644,dir_mode=0755 /mnt/freenas

Change the ip address and folders to match the path to your particular NAS setup.  My nas is setup for anonymous access internally.  You might want to be more secure if you have multiple users, so change those settings as you see fit.

sudo df -Th /mnt/freenas/

Test the directory and make sure your VM now has access to your freenas files.  Once tested, you'll wan to make permanent so it persists on reboot.  

edit /etc/fstab and append -

//192.168.0.155/nas /mnt/freenas cifs username=anonymous,password=,rw,nounix,iocharset=utf8,file_mode=0755,dir_mode=0755

Again, change the path and options as it matches your particular setup.  Test that all this is working by stopping your VM, restarting it, and making sure that your NAS is still accessible via your mount point.

Now you're ready to actually install docker.

Installing Docker

taken from here

https://www.digitalo...on-ubuntu-16-04

The Docker installation package available in the official Ubuntu 16.04 repository may not be the latest version. To get the latest and greatest version, install Docker from the official Docker repository. This section shows you how to do just that.

First, add the GPG key for the official Docker repository to the system:

curl -fsSLhttps://download.docker.com/linux/ubuntu/gpg  | sudo apt-key add -

Add the Docker repository to APT sources:

sudo add-apt-repository "deb [arch=amd64] https://download.docker.com/linux/ubuntu/ $(lsb_release -cs) stable"

Next, update the package database with the Docker packages from the newly added repo:

sudo apt-get update

Make sure you are about to install from the Docker repo instead of the default Ubuntu 16.04 repo:

apt-cache policy docker-ce

You should see output similar to the follow:

Output of apt-cache policy docker-ce

docker-ce:
Installed: (none)
Candidate: 17.03.1~ce-0~ubuntu-xenial
Version table:
17.03.1~ce-0~ubuntu-xenial 500
500 https://download.docker.com/linux/ubuntu xenial/stable amd64 Packages
17.03.0~ce-0~ubuntu-xenial 500
500 https://download.docker.com/linux/ubuntu xenial/stable amd64 Packages

Notice that docker-ce is not installed, but the candidate for installation is from the Docker repository for Ubuntu 16.04. The docker-ce version number might be different.

Finally, install Docker:

sudo apt-get install -y docker-ce

Docker should now be installed, the daemon started, and the process enabled to start on boot. Check that it's running:

sudo systemctl status docker

The output should be similar to the following, showing that the service is active and running:

● docker.service - Docker Application Container Engine
Loaded: loaded (/lib/systemd/system/docker.service; enabled; vendor preset: enabled)
Active: active (running) since Sun 2016-05-01 06:53:52 CDT; 1 weeks 3 days ago
Docs: https://docs.docker.com
Main PID: 749 (docker)

Installing Docker now gives you not just the Docker service (daemon) but also the docker command line utility, or the Docker client. All we need for now.

 

Install and Set Up Portainer (Optional but recommended)

taken from here

https://www.ostechni...-manage-docker/

sudo docker pull portainer/portainer

Let us check whether the Portainer image has been pulled or not.

sudo docker images

Sample output:

portainer/portainer latest ec91653336d4 7 days ago 9.132 MB

As you see in the above output, Portainer image size is less than 10 MB, which is very small so that I won’t consume more RAM and Hdd space.

We have now Portainer in our local Ubuntu system. Let us start the container using command (this command allows portainer to be persistent on restart):

docker run -d -p 9000:9000 --restart always --name portainer -v /opt/portainer-data:/data -v /var/run/docker.sock:/var/run/docker.sock portainer/portainer

Now, Portainer is running! Let us go ahead and access the Portainer UI. To do so, open your wbe browser and point it to – http://localhost:900..._Address:9000/.You will be presented with a screen like below that asks to setup a password for the admin user.

Enter the password twice and hit Validate button to create the admin user password.

 

Finally - Add Emby Docker Image

Taken from here -

https://hub.docker.c...dukrbdutjzkjtz/

We recommend you install directly from the Docker Hub. Before starting the install procedure please verify the following prerequisites are fulfilled:

  • ensure the user running installation command can run docker

Start the installation by issuing the following command from within a terminal (restart always & define port will make persistent on restart):

docker run -it --rm -v /usr/local/bin:/target emby/embyserver instl
Set Up Emby

run the emby-server command

emby-server

It will ask you to add volume locations.

Use the mounted freenas directory to give access to NAS (or whatever folder you set up to mount with your NAS)

/mnt/freenas

That's it for the cli portion of emby set-up.  Just ctrl+D to exit.

Now, go into portainer, check images. You should see emby server. Click the link hyperlink. Look for the expose detail. Use one of the available tcp ports. For me, it was tcp port 8096.

Now just enter the url of the docker VM with the correct port number:

That should go directly to the emby set up screen. You're done!

But one last thing before we finish...

Enable Systemd Emby (Optional - allows emby-server to auto-start with docker VM)

By deafult, containers don't auto-start on reboot.  If you want this to happen, which is extremely convenient, you need to give it an appropriate run command.  There's several ways to do this with docker, but in Ubuntu, systemd seems to be the recommended way to go. This is probably the most complex part of this tutorial, since the contents of your custom unit will vary depending on your personal setup.  Use mine as a general guide and tweak as needed.  

 

On some emby docker tutorials, there are one-line commands that get emby-server to get included in systemd, but I couldn't get any of them to work for me for some reason, so I had to set up my own systemd service to make it work.  

I installed an app called systemd-docker for docker auto starts.  Found it in a suggestion in the emby docker thread and it worked for me.   

apt-get install systemd-docker

Then create the unit emby-docker.service file that goes into /etc/systemd/system/

things to note on the switches used in the "ExecStart" section - 

-v = volume

-e = environment variable

A good tip is to go into Portainer, look at the emby-server container you set up earlier, and look at the details of the container to get the settings of your ENV and volume sections.  Then use the switches as shown in my example to configure your unit to match your personal setup.  Some other tips - 

  • For my set-up, my volume referred to my mount point of my NAS.  Change that to match what yours is.
  • I run my emby container as root user.  I have no problem with this.  If you want to run it as some other user, you need to change the user as needed
  • TZ = timezone.  It's important to have that match your local timezone.  Look up linux timezones and find one that matches yours.  

The rest I think you can keep the same and it shouldn't cause you many issues, but of course depends on your set-up.  This took me a while to figure out and get right, and there may very well be easier ways to do this.

Here's my sample emby-docker.service file.  Make sure to give the file full rw permissions so it can be executed by systemd.

[Unit]
Description=Emby-Server Docker
After=docker.service
Requires=docker.service

[Service]
ExecStart=/usr/bin/systemd-docker run --rm --name %n --net=host -v /home/root/.emby-server:/config -v /mnt/freenas:/mnt/freenas -e APP_UID=0 -e APP_GID=0 -e APP_USER=root -e EDGE=0 -e UMASK=002 -e TZ='America/New_York' emby/embyserver
Restart=always
RestartSec=10s
Type=notify
NotifyAccess=all
TimeoutStartSec=120
TimeoutStopSec=15

[Install]
WantedBy=multi-user.target

Enable it to make it persistent - 

sudo systemctl enable emby-docker

Then start it

sudo systemctl start emby-docker

You should see the container running in portainer in your containers list, and you should be able to access your emby server on the default 8096 port.  This time however, the container will always auto-start with your VM.  

 

 


Edited by Hisma, 13 August 2017 - 08:57 PM.

  • Abobader, PhilWhite and teiva like this

#2 PhilWhite OFFLINE  

PhilWhite

    Advanced Member

  • Members
  • 387 posts
  • Local time: 11:41 AM

Posted 13 August 2017 - 10:32 PM

Nice, but why not just run Emby in an iocage jail, which is also available in Freenas 11.0?  

 

Many fewer steps and there is no need to create a full Linux VM to then run  the docker image.  When 11.1 is released it will have full GUI support for the iocage jails.  For now, you can set up Emby to run in a few steps through the command line, though it's best to use the FreeBSD install for Emby, which requires a few steps to change ffmpeg.

 

Since this is a VM running inside Freenas, don't you have to import your media from Freenas into the Ubuntu VM?  Iocage jails, like the current warden system, have more direct access to your Freenas volume, since both use FreeBSD as their OS.  You can mount your Freenas dataset into the iocage jail.

 

I don't know how interested you, or anyone else might be in playing with it, but to create the jail you call up a shell and enter something like the following, assuming you have an ip address you know that you can assign and that your subnet is 255.255.255.0 (which is /24 in this example); and that your NIC uses em0.  You can alter the ip address and subnet as needed for your purposes.

iocage create tag=jailname ip4_addr="em0|192.168.x.x/24" -r 11.0-RELEASE

Then, start the jail:

iocage start jailname

Then mount the dataset attached to your primary volume (the source as when you add storage to a plugin jail) and choose a destination:

iocage fstab -a jailname /mnt/Yourvolume/Yourdataset /media nullfs rw 0 0

You can decide to make it read only and so use r0, but Emby will then throw lots of errors and you can't write to files from Emby.

 

Then add the emby server. First get into the jail:

iocage console jailname

and then:

pkg install emby-server

It's probably best to follow the instructions for the freeBSD install and alter ffmpeg accordingly, which takes a little while.

 

Then, start emby.

service emby-server start

And it's probably best to have emby restart at boot:

sysrc emby_server_enable="YES"

Then, go to the ip address you entered (192.168.x.x:8096) at port 8096 and enjoy.


Edited by PhilWhite, 13 August 2017 - 10:35 PM.

  • Abobader and Hisma like this

#3 makarai OFFLINE  

makarai

    Advanced Member

  • Members
  • 545 posts
  • Local time: 06:41 PM

Posted 14 August 2017 - 06:24 AM

quick question, whats the difference between the plugin (emby) and the iocage method?



#4 PhilWhite OFFLINE  

PhilWhite

    Advanced Member

  • Members
  • 387 posts
  • Local time: 11:41 AM

Posted 14 August 2017 - 07:05 AM

quick question, whats the difference between the plugin (emby) and the iocage method?

Good question.

 

Right now, the answer is that iocage jails are supposed to be more stable and efficient than the warden jails created when you install a plugin currently.

 

In a few months there will be no difference, as far as anyone using a jail will see, because Freenas 11.1 will migrate all the regular and plugin jails to iocage jails -- so, everyone will have iocage jails (if using the jail system).  At that point, if you want to install a new plugin in a jail, instead of creating the old warden-style jail, it will create an iocage jail.

 

The hope, from what I gather, is that iocage jails will have far fewer headaches than the current warden-jail system.  If you look at the Freenas forum, one of the most active 'help' areas is the plugin section.  Plex, for one, has caused numerous headaches from what I can tell.

 

So, personally, if everything is working for you now, I would wait for that update later this year and you won't have to do anything to get an iocage jail -- provided the migration works well.  (Unless you want to play with it, since it isn't that hard to do.)  The main reservation I have about completely switching now is that at least one of the Freenas updates broke iocage for several users.  The next iocage update will fix the problem, but it will not be released until Freenas 11.1 is available.  I have Emby in both an iocage jail and a plugin jail just to hedge my bets.



#5 Hisma OFFLINE  

Hisma

    Advanced Member

  • Members
  • 71 posts
  • Local time: 04:41 PM

Posted 14 August 2017 - 09:45 AM

Nice, but why not just run Emby in an iocage jail, which is also available in Freenas 11.0?  

 

Many fewer steps and there is no need to create a full Linux VM to then run  the docker image.  When 11.1 is released it will have full GUI support for the iocage jails.  For now, you can set up Emby to run in a few steps through the command line, though it's best to use the FreeBSD install for Emby, which requires a few steps to change ffmpeg.

 

Since this is a VM running inside Freenas, don't you have to import your media from Freenas into the Ubuntu VM?  Iocage jails, like the current warden system, have more direct access to your Freenas volume, since both use FreeBSD as their OS.  You can mount your Freenas dataset into the iocage jail.

 

I don't know how interested you, or anyone else might be in playing with it, but to create the jail you call up a shell and enter something like the following, assuming you have an ip address you know that you can assign and that your subnet is 255.255.255.0 (which is /24 in this example); and that your NIC uses em0.  You can alter the ip address and subnet as needed for your purposes.

iocage create tag=jailname ip4_addr="em0|192.168.x.x/24" -r 11.0-RELEASE

Then, start the jail:

iocage start jailname

Then mount the dataset attached to your primary volume (the source as when you add storage to a plugin jail) and choose a destination:

iocage fstab -a jailname /mnt/Yourvolume/Yourdataset /media nullfs rw 0 0

You can decide to make it read only and so use r0, but Emby will then throw lots of errors and you can't write to files from Emby.

 

Then add the emby server. First get into the jail:

iocage console jailname

and then:

pkg install emby-server

It's probably best to follow the instructions for the freeBSD install and alter ffmpeg accordingly, which takes a little while.

 

Then, start emby.

service emby-server start

And it's probably best to have emby restart at boot:

sysrc emby_server_enable="YES"

Then, go to the ip address you entered (192.168.x.x:8096) at port 8096 and enjoy.

 

 

Looks good Phil, thanks for the tutorial.  The way I see it, the more options we have, the better.  For me, it was mostly an exercise to see what docker was all about, and see if I could actually get it working.  And so far I am happy with the performance, and what sort of possibilities it can have in the future.  

 

But you bring up a good point, native performance vs virtualization overhead.  BSD is BSD, linux is linux.  If we want to use linux-based apps in freenas, we need to implement a VM environment, period.  For me, I have a system that is built for handling VMs quite well, so I'm okay with this.  And the flexibility of using linux apps to me is worthwhile benefit.  And as I mentioned, I had heard nothing but rave reviews when it came to docker, so I had to see it for myself.

 

IOCage is probably the best path for majority of users that want a seamless experience.  But perhaps for those of us who are more adventerous, and want to implement a system that gives you access to countless more possibilities that come with using a linux-based container system, then docker is a good solution.

 

I have them both right now, and when 11.1 comes around, I'll play with both docker & iocage, and see which one is a better fit for my use-case.  


Edited by Hisma, 14 August 2017 - 09:45 AM.

  • PhilWhite likes this

#6 PhilWhite OFFLINE  

PhilWhite

    Advanced Member

  • Members
  • 387 posts
  • Local time: 11:41 AM

Posted 14 August 2017 - 10:41 AM

Sounds like a good plan to me.  My impression is that iocage is not quite ready for prime time in Freenas, so it's probably best to wait; and your excellent post can get folks up and running with an Ubuntu VM and docker straight away.  From what I gather the great advantage of using docker is that there are so many applications that already have docker containers, so it opens a whole world of applications to Freenas.

 

I totally agree -- the more solutions, the better.

 

I basically did the same thing as you in setting up an iocage jail -- I just wanted to see how to do it.

 

 

ETA: I just recently added ubuntu to a laptop to 'force' myself to learn the command line.  So far, I think it's been great.  I'm just starting to learn this stuff myself, but it opens up all sorts of possibilities.  I'm all for learning and trying new things.


Edited by PhilWhite, 14 August 2017 - 11:07 AM.


#7 makarai OFFLINE  

makarai

    Advanced Member

  • Members
  • 545 posts
  • Local time: 06:41 PM

Posted 15 August 2017 - 02:49 AM

Good question.

 

Right now, the answer is that iocage jails are supposed to be more stable and efficient than the warden jails created when you install a plugin currently.

 

In a few months there will be no difference, as far as anyone using a jail will see, because Freenas 11.1 will migrate all the regular and plugin jails to iocage jails -- so, everyone will have iocage jails (if using the jail system).  At that point, if you want to install a new plugin in a jail, instead of creating the old warden-style jail, it will create an iocage jail.

 

The hope, from what I gather, is that iocage jails will have far fewer headaches than the current warden-jail system.  If you look at the Freenas forum, one of the most active 'help' areas is the plugin section.  Plex, for one, has caused numerous headaches from what I can tell.

 

So, personally, if everything is working for you now, I would wait for that update later this year and you won't have to do anything to get an iocage jail -- provided the migration works well.  (Unless you want to play with it, since it isn't that hard to do.)  The main reservation I have about completely switching now is that at least one of the Freenas updates broke iocage for several users.  The next iocage update will fix the problem, but it will not be released until Freenas 11.1 is available.  I have Emby in both an iocage jail and a plugin jail just to hedge my bets.

 

 

have you done any direct performance comparisons? I have a VM in bbyve with emby in docker and the warden jail, and the warden jail outperforms during transcoding. (Just based on the CPU  that is consumed, somehow the docker uses twice as much)



#8 PhilWhite OFFLINE  

PhilWhite

    Advanced Member

  • Members
  • 387 posts
  • Local time: 11:41 AM

Posted 15 August 2017 - 07:04 AM

have you done any direct performance comparisons? I have a VM in bbyve with emby in docker and the warden jail, and the warden jail outperforms during transcoding. (Just based on the CPU  that is consumed, somehow the docker uses twice as much)

Actually, no I hadn't but just did.  I have most files as mkv or mp4, so I had to search around for one that would transcode, but it looked essentially identical in both types of jail.  That makes sense to me, since the means of accessing the files is identical.

 

 

ETA:  Had a few minutes to try an MKV from a recording last night -- running the same file in the iocage jail reached a peak CPU usage of 14.4% while it loaded and then it ran around 0.75-1.2%. In the warden jail peak usage was 15.4% while the file loaded, and it was running around 3.5% for the 5 minutes or so that I watched.  So maybe a slight performance benefit for iocage over the warden plugin jail?


Edited by PhilWhite, 15 August 2017 - 09:00 AM.






Also tagged with one or more of these keywords: docker, freenas, vm

0 user(s) are reading this topic

0 members, 0 guests, 0 anonymous users