Problems with using rclone with emby

[Krauz 1]

Hello, my newly set up emby media server adds my locally stored files to my library just fine without any kind of additional input by me. I have a few problems though when adding my rclone mount into the mix and I would appreciate it if anyone with a bit more know-how than me could help me sort them out.

Using the official rclone fork with my webdav cloud all of my files are put into a folder called "links". This is regardless of whether it is a tv show, movie or else. So adding this "links" folder to my library emby gets quite confused with correctly sorting all of my media, which is why I thought of using the "Auto Organize" Plugin. My idea was to tell it to move all of the files from the "links" folder to where all of my local files are (in their respective folders of course). At first it seemed to be working correctly, since it created a folder for the TV Series it was working on currently and put the episode into the folder. I later found out that while "moving" the file it seemed to be downloading it from my webdav cloud into the folder it was only supposed to be moving the file to. From what I understand rclone stores some sort of "dummy file" in the mounted folder that can be accessed and played by emby but moving/copying the file to another folder will result in my computer downloading the file from the cloud to my local storage which completely defeats the point of using rclone in the first place.
One more thing to add: Before emby I used plex and to be honest, it worked fine with rclone since I could just point my movies as well as my tvshows library to that "links" folder and plex would just know which file is an episode from a show and which file is a movie and it would add them to the respective libraries automatically without having to move the files on the disk into separate folders. From what I understand emby requires a bit more work than that and I am totally fine with that if it means that I can select an external video player in the android tv app (this is why I made the switch from plex to emby. It's just not an option in plex)

I've been tinkering with my media server for quite a while now and I feel like I am running in circles. All I really want to achieve is to combine my locally stored media with my media that is stored in my cloud so that I can watch it with emby in a kind of all-in-one solution but every time I seem to find a solution to my problems some get fixed but completely new problems arise and I feel like it's just all too complicated. What's most frustrating is that for most of them I wouldn't even know what to google to be completely honest with you, which is why I am now resorting to here in hopes that someone can help me out. I really hate asking on forums because I feel like these questions have been asked 1000 times before but somehow all of my search results on google that I get when searching for solutions to my problems really don't get me anywhere. I really appreciate any kind of help I can get.

Setup: Emby in docker, rclone in docker, host is debian amd64

[Luke 37118]

HI there, can you please provide a specific example? Thanks !

[Krauz 1]

hi, yes of course.

 

So on my webdav cloud there are two files currently for testing purposes. Their names are :

House Of The Dragon S01E01 The Heirs Of The Dragon 2160p HDR 5.1 x265 10bit.mkv

and

House Of The Dragon S01E08 The Lord Of The Tides 2160p HDR 5.1 x265 10bit.mkv

 

They are both in said "links" folder. When adding this folder to my TV Shows library in emby it doesn't group them together into a single "The House of The Dragon" show but instead lists these singular episodes with their file names as episode names under "latest TV shows". Not even listing it under "TV Shows" in the library even after multiple restarts of the media server and rescans of my library. Under "latest TV shows" I can click play and watch them but when I finish watching them they completely disappear from my library nowhere to be found, even after rescanning and restarting again. I am guessing this is just all consequence of emby not being able to know where to list it because the files are just thrown together in a single folder. This is why I thought the "Auto Organizer" would help me but this is also where I came to find out what I described earlier. The files in the rclone mount folder obviously don't occupy nearly as much space on my disk as it says when inspecting it with "du -s --apparent-size" just because of the fact that my internet speed isn't fast enough to download 1.3 GB as fast as the files appear in the folder. Now trying to sort the files with the "Auto Organizer" plugin creates that exact problem I stated earlier where instead of moving/copying the "dummy file" (still for the lack of a better term) it downloads the file from my WebDAV cloud to the folder where it is supposed to copy the "dummy file" to. This is apparent by looking at my router admin panel and seeing the download speed of my media server go up exactly the moment I order the copy command. 

I hope you can help me and be sure to let me know if you need any more information, I'll be glad to help you help me

My emby server is running version 4.7.8.0, rclone is running version 1.59.2 if that is any help to you.

[Luke 37118]

Can you please show us the exact folder structure? Thanks.

[Krauz 1]

Here you go.

 

First one is the root of my disk, second one is in the "realdebrid" folder which is the mount folder for rclone and third one is my two files inside the "files" folder. This last one would be the folder where all of the files that are on my WebDAV cloud would be thrown in without any sorting.

[Luke 37118]

OK so you just have flat files without series folders. Currently that is not fully supported and that is the reason for your problem.

This is something we'd like to support in future updates, we're just not there yet.

[Krauz 1]

Okay, well, that's kind of what I thought would be the problem. As I am not getting any replies on the rclone forums where I also asked about this problem regarding rclone, do you happen to know what I can do? Having to use the "Auto Organizer" plugin because emby needs series folders isn't as much of a problem as the one I am having with rclone.

[Luke 37118]

What is the rclone specific problem?

[Luke 37118]

I'm guessing you're referring to the downloading. Do you have auto organize configured to copy or to move?

[Krauz 1]

Yes, precisely. Currently I have it set to move because I discovered the problem when having it set to copy but both of them cause this behavior.

[Luke 37118]

49 minutes ago, Krauz said:

Yes, precisely. Currently I have it set to move because I discovered the problem when having it set to copy but both of them cause this behavior.

Right so Emby Server is built with the .NET runtime, and that means all of our file operations are using the methods they provide.

We have noticed on some file systems that when we use the api to move a file, that internally it ends up copying. It appears to do a copy followed by a deletion of the original.

[Krauz 1]

After some more testing it appears that even moving the file by mv command causes this behavior. I also tried moving the file via smb on windows, same problem. Maybe I am just looking for a solution to a bug that's not actually a bug but a feature and thus there wouldn't be a solution, maybe this is why it seems so difficult to solve.