Empty guide

[squalet 1]

Hi, I just started using Emby (version 4.6.7.0) and I'm struggling to have the live tv guide working. I added the XMLTV guide from https://raw.githubusercontent.com/dracohe/SMARTIPTV/master/guide.xml but apparently there is an issue with the file system based on the error message I see from the logs.  I am using the official docker image in a Synology server. I've manually checked all the folders from within the docker and I'm perfectly able to read/write in all the folders. I also set the debug mode to check whether it would inform about the folder/file that cannot be created/accessed but it didn't show any extra information apart from what can be seen from the logs attached. Any help would be much appreciated. Thanks !!

2021-11-24 14:04:23.402 Error LiveTvManager: Error refreshing channels for service
        *** Error Report ***
        Version: 4.6.7.0
        Command line: /system/EmbyServer.dll -programdata /config -ffdetect /bin/ffdetect -ffmpeg /bin/ffmpeg -ffprobe /bin/ffprobe -restartexitcode 3
        Operating system: Linux version 4.4.180+ (root@build13) (gcc version 7.5.0 (GCC) ) #42218 SMP Mon Oct 18 19:16:55 CST 2021
        Framework: .NET Core 3.1.21
        OS/Process: x64/x64
        Runtime: system/System.Private.CoreLib.dll
        Processor count: 4
        Data path: /config
        Application path: /system
        System.UnauthorizedAccessException: System.UnauthorizedAccessException: Access to the path is denied.
         ---> System.IO.IOException: Operation not permitted
           --- End of inner exception stack trace ---
           at System.IO.FileSystem.CopyFile(String sourceFullPath, String destFullPath, Boolean overwrite)
           at System.IO.File.Copy(String sourceFileName, String destFileName, Boolean overwrite)
           at Emby.Server.Implementations.IO.ManagedFileSystem.CopyFile(String source, String target, Boolean overwrite)
           at Emby.LiveTV.Listings.XmlTv.XmlTvProvider.UnzipIfNeeded(String compressedFile, String targetPath, Boolean isRemoteSource)
           at Emby.LiveTV.Listings.XmlTv.XmlTvProvider.GetXml(ListingsProviderInfo info, String path, MediaProtocol protocol, CancellationToken cancellationToken)
           at Emby.LiveTV.Listings.XmlTv.XmlTvProvider.GetChannels(ListingsProviderInfo info, CancellationToken cancellationToken)
           at Emby.LiveTV.EmbyTV.GetEpgChannelData(IListingsProvider provider, ListingsProviderInfo info, CancellationToken cancellationToken)
           at Emby.LiveTV.EmbyTV.QueryChannelsAsync(ChannelManagementQuery query, CancellationToken cancellationToken)
           at Emby.LiveTV.EmbyTV.GetChannelsAsync(CancellationToken cancellationToken)
           at Emby.LiveTV.LiveTvManager.RefreshChannelsInternal(ILiveTvService service, IProgress`1 progress, CancellationToken cancellationToken)
           at Emby.LiveTV.LiveTvManager.RefreshChannelsInternal(IProgress`1 progress, CancellationToken cancellationToken)
        Source: System.IO.FileSystem
        TargetSite: Void CopyFile(System.String, System.String, Boolean)
        InnerException: System.IO.IOException: Operation not permitted
        Source: 
        TargetSite: 

 

embyserver.txt

[Carlo 4330]

Hi, That is clearly a permission error based on the error shown.
On Synology installed on the host (no docker) you would need to set permission in control panel for any shares that Emby will use.  You need to give read/write access to the system internal emby user.

Besides the docker permissions you will need to make sure the file system has given permission to the system internal emby user as well on Synology. I'm not sure exactly how your docker image is setup so it's only a guess why this fails short of knowing Emby doesn't have proper access.

But let me ask you this.  On Synology why are you running Emby in docker?  That's just going to make life harder for you with an additional abstraction level of OS involved for storage, networking, CPU, GPU and it will cause things to be harder to get working correctly as well as hamper performance. Docker is nice for small things or when you need to "isolate" things but it's not the best thing to do for dedicated apps where you want the best performance.

Why not install Emby directly on the Synology box?  If you are running DSM 7 you can install it directly from Package Center.  If running DSM 6 you can download the package from our website and manually install that from Package Center.
For DSM 6 you can download Emby here: https://emby.media/synology-server.html
For DSM 7 you can download Emby here or install from package center: https://emby.media/synology7-server.html

[squalet 1]

Hi, many thanks for your quick help.

From my understanding, it doesn't make any sense that permission error. Like I said, within the Emby docker instance, I have full read/write access to all folders and files and everything works except for that. I started different Emby servers from scratch using different 4.6.x versions (with empty config folders that are automatically populated on startup) and I had that issue with all of them. It would definitely be really helpful if the logs in debug mode gave more information regarding the path location that is raising that error.

In my case, I have many things running in dockers and to me, it was easier as I also had a docker instance that checks and downloads the latest docker images available automatically. Also, that way, Emby doesn't use my NAS disks but the external hard drive I have attached allowing them to hibernate. That being said, I just installed Emby from the Synology package center and the issue is gone. Everything seems to be working fine. I will keep using the server from the DSM package center though I will also be checking the dockers to see if this issue is magically gone and I can go back to using them.

Thanks again.

[Luke 37106]

Is /config mapped to a path on the Synology?

[squalet 1]

I only have 2 mappings. 1 for /Config and another for the actual data. Both /config and the data are mapped to different folders of my external drive attached by USB. Both folders are automatically mounted by DSM with 777 permissions so I don't see why it can complain about it. 

[Carlo 4330]

19 hours ago, squalet said:

I have full read/write access to all folders and files and everything works except for that.

Right, you have access but does the "emby" user? Are you checking this on your Synology file system or from inside the docker where Emby is running?
What does the file ownership and access rights look like inside the container?

Depending on why/how this happened you might have to look at other permissions as well for the instance or even docker itself.
This is what I meant earlier by an extra layer being in the way.

You should really never give 777 permissions but instead verify the user needing access or ownership and give rights and/or ownership to that ID.
 

[squalet 1]

Hi, I understand granting 777 permissions is not good practice, but if that way is raising a permission error, I don't see how limiting the access can help in this particular case. I'm not truly familiar with how Synology permissions work to be honest which is why I left it by default, but I will give it a shot when I have something working for sure.

I checked permissions both from the Synology file system (DSM) and from the Emby docker instance running. This is how the config folder permissions looks like within the docker container:

Quote

/config # ls -l
total 24
drwxrwxrwx    1 1024     users         4096 Nov 24 20:59 cache
drwxrwxrwx    1 1024     users            0 Nov 24 14:02 config
drwxrwxrwx    1 1024     users         4096 Nov 24 14:01 data
drwxrwxrwx    1 1024     users         4096 Nov 26 05:08 logs
drwxrwxrwx    1 1024     users         4096 Nov 24 14:17 metadata
drwxrwxrwx    1 1024     users         4096 Nov 24 14:01 plugins
drwxrwxrwx    1 1024     users            0 Nov 24 13:58 root
drwxrwxrwx    1 1024     users         4096 Nov 24 20:59 transcoding-temp

And this is how it looks like if I `ls` the / folder:

Quote

/ # ls -l
total 12
drwxr-xr-x    1 root     root          9152 Nov 22 01:25 bin
drwxrwxrwx    1 1024     users         4096 Nov 24 20:58 config
drwxrwxrwx    1 1024     users         4096 Nov 24 13:56 data
drwxr-xr-x    6 root     root           360 Nov 26 05:08 dev
drwxrwxr-x    1 root     root           268 Nov 26 05:08 etc
drwxr-xr-x    1 nobody   nogroup          0 Dec 31  2018 home
-rwxr-xr-x    1 root     root           389 Mar 21  2019 init
drwxr-xr-x    1 root     root          5898 Nov 16 12:43 lib
drwxr-xr-x    1 root     root           104 Nov 22 01:25 libexec
drwxr-xr-x    1 root     root          1016 Oct 29 14:31 licenses
dr-xr-xr-x  491 root     root             0 Nov 26 05:08 proc
drwx------    1 root     root             0 Dec 31  2018 root
drwxr-xr-x    1 root     root            24 Nov 16 12:43 share
dr-xr-xr-x   12 root     root             0 Nov 26 05:08 sys
drwxr-xr-x    1 root     root         16034 Nov 21 22:20 system
drwxrwxrwt    1 root     root           234 Nov 26 05:12 tmp
drwxr-xr-x    4 1000     1000            80 Nov 25 08:52 transcodes
drwxr-xr-x    1 root     root            24 Nov 22 01:25 usr
drwxr-xr-x    1 root     root            22 Nov 26 05:08 var

In the docker-compose file, this is how I configured the uid and group ids. I made them match with the default Synology uid corresponding to the administrator uid and the default group id.

Quote

emby:
    image: emby/embyserver
    container_name: emby
    environment:
      - PUID=1024
      - PGID=100

 

[Carlo 4330]

That looks like it could be a configuration issue but at this point after you've sort of messed up the container it would be better to remove and reinstall the docker image.

But this time setup the file system correctly.  It doesn't look like you did that previously as you have what looks like a mismatch of permissions between docker and the owners of the files.

Sorry this is going to be long, but I'll try and give you a short overview of some potential things to look for and ways to set things up to save yourself grief.

This is in general and not Emby specific but If you're mounting or binding things outside the docker image, you want to set those up first. Until you get the hang of setting permissions properly logout and back in as that specific user and create the folders as well as any files that will need to be present.  That user should then own the paths and the permission should be correct.  You can check that using the ls -l .  At least you will know the user can access and manipulate the files!

In the case of using "foreign or external devices" or what is actually going to typically be system devices you will want to grant permission to access those locations to the group and user as well or reference multiple ID in the docker setup (ie GPU video cards and possibly USB attachments).

In Synology for example to use a USB3 HDD you would want to make sure it's using either btrfs or ext4 file system and probably not as an ntfs drive as it will mount as a fuse block device. If you want to know why I can follow this up but it's not that important to the rest of this info.

From a command prompt do a "df -h".  You should see something like this (this is a portion of mine)

/dev/md0                           2.3G  1.4G  825M  63% /
devtmpfs                           3.8G     0  3.8G   0% /dev
tmpfs                              3.9G  236K  3.9G   1% /dev/shm
tmpfs                              3.9G   42M  3.8G   2% /run
tmpfs                              3.9G     0  3.9G   0% /sys/fs/cgroup
tmpfs                              3.9G  2.6M  3.9G   1% /tmp
/dev/mapper/cachedev_0              35T   32T  3.9T  89% /volume1
/dev/usb2p1                        3.6T   40G  3.6T   2% /volumeUSB3/usbshare
/dev/usb3p1                        3.6T   89M  3.6T   1% /volumeUSB7/usbshare
/dev/usb5p1                        3.6T  2.9T  798G  79% /volumeUSB5/usbshare
/dev/usb7p2                        3.7T  2.7T  981G  74% /volumeUSB10/usbshare10-2
/dev/usb1p2                        3.7T  3.3T  407G  90% /volumeUSB8/usbshare8-2
/dev/usb6p2                        3.7T  2.4T  1.4T  64% /volumeUSB6/usbshare6-2

You can clearly see the multiple 4 TB drives attached to the Synology.  Right now I've got 12 attached just not showing the whole list.
Now use this command "sudo mount -c" and look through the list and you'll see your USB drive looking similar to

/dev/usb2p1 on /volumeUSB3/usbshare type ext4 (rw,relatime,nodelalloc,synoacl,data=ordered)
/dev/usb3p1 on /volumeUSB7/usbshare type ext4 (rw,relatime,nodelalloc,synoacl,data=ordered)
/dev/usb5p1 on /volumeUSB5/usbshare type ext4 (rw,relatime,nodelalloc,synoacl,data=ordered)
/dev/usb7p2 on /volumeUSB10/usbshare10-2 type fuseblk.ntfs (rw,relatime,user_id=0,group_id=0,default_permissions,allow_other,blksize=4096)
/dev/usb1p2 on /volumeUSB8/usbshare8-2 type fuseblk.ntfs (rw,relatime,user_id=0,group_id=0,default_permissions,allow_other,blksize=4096)
/dev/usb6p2 on /volumeUSB6/usbshare6-2 type fuseblk.ntfs (rw,relatime,user_id=0,group_id=0,default_permissions,allow_other,blksize=4096)

That's another snippet from my list.  Here you can see both ext4 and the ntfs mounted via fuse block. In the case of the fuseblk mounts you see the user, group IDs as well ad overall permissions and if the device can be used by only the one owner or "allow_other" as well.  So for docker mounting use I would normally NOT want to use the usbshare6-2, 8-2 or 10-2 drives if I didn't need to but instead use the others USBs drives 3, 5 & 7 with ext4 formatting.

You can see these have already been mounted to the file system as well.  I tend to like to mount these as /volumeUSB followed by a number. That let's me know what drive bay these are mounted it. The reason I do this (learned almost the hard way) is Synology is goofy and will load these with different dev IDs.  Look at the left column at the first USB as it's device usb2 partition 1 but I know it actually the 3 drive in cabinet bay. I've almost partitioned and formatted the wrong disc because of that. Now I carefully load my bays drive by drive formatting and mounting them.  Now I don't care what order they load in or what device ID they get.

Anyway, you want this mounted already.  So if I were going to use USB3 for docker it should already be mounted to the file system (ie  /volumeUSB3)
As root user if I do an ls -l / I'll see these are all "root root" owned

drwxr-xr-x   1 root root   976 Nov 23 17:15 volume1
drwxr-xr-x   5 root root  4096 Nov 20 14:53 volumeUSB10
drwxr-xr-x   4 root root  4096 Nov 20 14:53 volumeUSB3
drwxr-xr-x   3 root root  4096 Nov 20 14:53 volumeUSB4
drwxr-xr-x   5 root root  4096 Nov 20 14:53 volumeUSB5
drwxr-xr-x   5 root root  4096 Nov 20 14:53 volumeUSB6
drwxr-xr-x   4 root root  4096 Nov 20 14:53 volumeUSB7
drwxr-xr-x   5 root root  4096 Nov 20 14:53 volumeUSB8

Now if I do this:

cd /volumeUSB3
mkdir emby
chown -R emby:emby /volumeUSB3/emby

I get this when doing an ls -l
drwxr-xr-x 2 emby emby 4096 Nov 26 18:12 emby

That folder is now owned by emby.  Just a hint and personal preference but I would do this in real life /volumeUSB3/docker/emby
That way when I see 'docker" I know what ever directories are under this are used by docker. (just a habit of mine to use a parent folder matching the container or app).
Now I would create folders I'd use for the docker volumes such as "mkdir config" followed by
chown -R emby:emby /volumeUSB3/emby/config
which gives me this
drwxr-xr-x 2 emby emby 4096 Nov 26 18:21 config

Now I've got my directories setup where I want them using mounted volumes that have been shared in control panel and set to use the system internal emby users (if present).
Now I want to know the user and group IDs to use for docker that match the ownership of the folders I just set for user "emby".

if I type "id emby" I will get this info back.  On my system since I have the system internal emby user (new to DSM7) these numbers will be high so don't let that scare you as it's just an example but would be what I would use on my system.
uid=240699(emby) gid=240699(emby) groups=240699(emby),937(videodriver)

If on the other hand I were to run the id command against my normal user id of carlo I'd get different IDs back:
uid=1027(carlo) gid=100(users) groups=100(users),65536(Media)

These are the IDs you would then use for setting up the docker container.  You can see these IDs at the bottom of the docker-compose.yml file and you would change the values to match the user on your system.  Other wise you can run into permission issues that will drive you nuts.

Now at least you know things external to docker should be setup correctly. If you then still get permission problems it's very likely going to be inside the container. You then check ownership and read/write permissions of those areas.

Hope that helps

[squalet 1]

Wow, many many thanks for the answer. I'm doing a backup of the disk to format it in ext4 and try what you said. I will update with the results

[Carlo 4330]

Hopefully there was a nugget or two in there that will help.

Keep us updated how you make out!

[squalet 1]

Good news !! Now that I'm using an ext4 external drive, the issue is gone and I'm now capable of using Emby through Docker perfectly fine. The guide problem is now gone. Many thanks for your thorough explanation and help. 

Now that I have it working I wanted to ask something else. Apart from the fact that using it through docker I am adding a few more layers that could affect the performance... Are there any optimisations made to the Synology application that are not in the docker image or is it supposed to be basically the same server?

[Carlo 4330]

Awesome, I wasn't sure if I should have included so much low level info or not but glad there was something there that helped.

I think the best way to answer your new question is simply stating that any application running on the host vs docker or some other container format is built specifically for that OS vs a generic docker Linux assembly that will be abstracted from the host.  So yes there is always going to be a performance and memory penalty for doing that.

With that said, docker on Synology and other NAS boxes is great for some apps because it allows an easy install of apps you normally could not easily install otherwise.  Synology for example uses a stripped down version of Linux with many commands changed to their own version.  That can make running or building some apps very hard to do.  By using docker you essentially get a small foot print Ubuntu (or other Linux) running for that instance which makes it easy to run any of the apps built for that OS.

That's a super high level explanation of course.  But in general for an application such as Emby Media Server, given the choice you are going to better off running a native package designed for it. So in this case that would be one of our packages designed for DSM 6 or DSM 7 depending on what you're currently running.

[squalet 1]

Fantastic, thanks very much for all your help. I really appreciate it