Ubuntu & rclone: 18% of Log are Errors (from 77k lines)

[camel_coal 2]

Evening,

I have attached my log file, it contains around 77k lines in total, with at least 13.7k with the word "Error". It appears the majority of errors (68+%) are related to `ffprobe` with around 8% due to Chapter Image extraction.

My setup is Ubuntu installed on bare metal and Emby installed natively (no VMs or containers). I'm looking to understand the nature of the errors and if there is anything I could do to mitigate them. The media is stored on a combination of LVM and an `rclone` mount. The vast majority of errors are related to the path `/mnt/archive` indicating `rclone` mount and appear to have the word "Timeout".

A spot check of a random selection of files for `ffprobe` related errors show the files are accessible and are infact within the Emby library itself. It could be the error occurred and then was successful a second time.

I don't mind it taking a while to build up chapter markers, intro detection over a long period of time as I accept I have a large library. My key concern is are the errors significantly slowing this process down and is there anything I can do to mitigate them. Currently a media scan takes 20 hours - in other forums, posts with a similar setup report scans in the minutes, although I suspect this is for a more mature setup.

My key questions:

* Is there anything I could do to mitigate these errors? I'm an emby newbie and formerly a long time Plex user.

* If response from `rclone` is an issue, is there a way to increase the timeout window for `ffprobe`?

 

Many thanks in advance!

embyserver.txt

[Luke 37232]

Hi, are you hitting your request limits in google drive perhaps?

[camel_coal 2]

Hi Luke, thanks for taking a look. 

Based on your prompt I looked at my rclone.log. It has around 30k entries for the past 72 hours. The overwhelming majority are about vfsCache expiring. 

Only *15 lines* contain errors with only 2 instances regarding the connection being reset. Last GOAWAY message was 6 days ago. These don't seem to correlate with the error messages I've identified in the Emby log. 

For context, I use Dropbox as the cloud provider. 

 

Is there a way to check how much of my library now has say Intro Detection complete or Thumbnails generated? Is there a database query I can run or API call to make?

[Luke 37232]

hi @camel_coalare you still having an issue with this?

[camel_coal 2]

Hi Luke. 

 

Yes I'm still having this issue. However, given recent changes at Dropbox I'm looking to migrate off rclone on the next few months altogether and any underlying cloud service. 

 

It's not having a significant detrimental effect on the server so happy to close this out. 

 

Thanks for looking into this. 

[camel_coal 2]

I managed to resolve this issue, it was due to the VFS File Cache setting. For the `--vfs-cache-mode` I used `full`, however the act of filling the cache was the limiting factor. I have now switched to just caching `writes` until at least the inital scan is done.

 

# VFS File Cache
--cache-dir=/flippy/cache \
--vfs-cache-mode writes \
--vfs-cache-max-size 200G \
--vfs-cache-max-age 72h