Any solition when Rclone had been error and folder become empty

[dinhtienvu 1]

I utilize Rclone in conjunction with Google Drive alongside Emby, and everything is functioning properly.

The designated directory is named "Movies."

However, occasionally, Rclone encounters errors and crashes, causing the "Movies" folder on the server to appear empty.

Consequently, Emby proceeds to remove all thumbnails and associated information.

Upon reestablishing the connection with Rclone, Emby initiates a complete rescan, which can be quite time-consuming due to the extensive number of files on my Google Drive.

Is there any viable solution to retain the previously scanned data?

Edited August 9, 2023 by dinhtienvu

[kikinjo 281]

Post a log from rclone, seems your mount becomes unavailable.

[dinhtienvu 1]

11 minutes ago, kikinjo said:

Post a log from rclone, seems your mount becomes unavailable.

Yes, my mounted drive contains files with Unicode characters causing rclone to crash. As a result, I need to restart rclone.

However, each time I restart Rclone, all of Emby's data is lost, and Emby has to perform a fresh scan from the beginning. This process takes a significant amount of time.

Do you comprehend my situation?"

[kikinjo 281]

You need to reconfigure your locale to utf8, you didn't mention your OS and configuration so its hard to recommend solution.

But you need to use utf8 , maybe this can help https://github.com/rclone/rclone/issues/2125

[dinhtienvu 1]

Thank you, 

I've posted my issue, and Luke is currently working on processing it. Many others are experiencing the same error as me regarding the "Error log PathTooLongException."

Currently, the only solution I've found is to delete the folders with long paths. However, the issue with the long paths is not solely due to their length, but also because they contain Unicode characters in the folder names (though I'm not entirely certain, there are similar cases like mine).

As a result, we'll need to reconfigure rclone and restart the system. This will lead to Emby removing all the thumbnails and data that were previously scanned, which is quite disheartening.

Therefore, I'm actively searching for a solution to retain the old thumbnails and movies that were scanned before.

 

 

 

[Edrock200 53]

I didn't think emby did this if the root folder was missing. If in your scenario when rclone unmounts, but the root is still there, you will want to change that behavior. e.g. If rclone is mounting to remote:Movies to /mnt/media/Movies, change it to mount mount remote: /mnt/media with flags such as --allow-non-empty if /mnt/media has other folders and/or --include=Movies/** if your remote has other folders you don't want to appear.

Alternatively you can make a script to remount. e.g. Linux example, make a file in Movies called "mounted.bin"

[ ! -f /mnt/media/Movies/mounted.bin ] && remount.sh

The above would ideally be in a bash file and set as a cron job to check every minute.

remount.sh would be something like

#!/bin/bash

docker stop emby (assuming emby is in docker container)

fusermount -uz /mnt/media/Movies

sleep 2s

rclone mount remote: .... or systemctl restart name_of_rclone_service

docker restart emby

[Luke 42077]

I think the server's resilience to this will be improved in the upcoming 4.8 release. Thanks.

[dinhtienvu 1]

7 hours ago, Edrock200 said:

I didn't think emby did this if the root folder was missing. If in your scenario when rclone unmounts, but the root is still there, you will want to change that behavior. e.g. If rclone is mounting to remote:Movies to /mnt/media/Movies, change it to mount mount remote: /mnt/media with flags such as --allow-non-empty if /mnt/media has other folders and/or --include=Movies/** if your remote has other folders you don't want to appear.

Alternatively you can make a script to remount. e.g. Linux example, make a file in Movies called "mounted.bin"

[ ! -f /mnt/media/Movies/mounted.bin ] && remount.sh

The above would ideally be in a bash file and set as a cron job to check every minute.

remount.sh would be something like

#!/bin/bash

docker stop emby (assuming emby is in docker container)

fusermount -uz /mnt/media/Movies

sleep 2s

rclone mount remote: .... or systemctl restart name_of_rclone_service

docker restart emby

Thank you greatly for your assistance. I comprehend that you provided me with very detailed instructions, but could you perhaps assist me a bit further?

Your approach involves consistently retaining previously scanned old data, correct? Could you kindly explain the outcomes when I perform this action? I apologize for my limited technical proficiency.

In my situation, the root folder remains present; however, all the subfolders within it are empty due to an rclone error.

 

[Edrock200 53]

9 hours ago, dinhtienvu said:

Thank you greatly for your assistance. I comprehend that you provided me with very detailed instructions, but could you perhaps assist me a bit further?

Your approach involves consistently retaining previously scanned old data, correct? Could you kindly explain the outcomes when I perform this action? I apologize for my limited technical proficiency.

In my situation, the root folder remains present; however, all the subfolders within it are empty due to an rclone error.

 

Theres a lot of assumptions in my initial post:

1. You are using linux

2. You are running in a docker container

3. You have an empty media folder root that you mount rclone into

Assuming all of this is true, and assuming my note about emby not deleting content when root is missing, the first option of mounting differently would prevent media from deleting upon dismount as it would essentially unmount that entire root folder path on crash.

The second script I noted would automate the fix for you. e.g. 

[ ! -f /mnt/media/Movies/mounted.bin ] && remount.sh ### This line basically says if /mnt/media/Movies/mounted.bin doesn't exist then run remount.sh, if it does exist, do nothing.

The below script, which would run if mounted.bin is missing, would stop emby to prevent media deletion, remount the media, then start emby. I've modfied the last line slightly to check for mounted.bin before starting emby back up, so if the remount fails for any reason emby will stay stopped.

Descriptions added after ###

docker stop emby (assuming emby is in docker container)   ### Stop emby

fusermount -uz /mnt/media/Movies  ### force unmount any remnants of the rclone mount

sleep 2s ### sleep for 2 seconds to ensure clean unmount completion

rclone mount remote: .... or systemctl restart name_of_rclone_service ### remount rclone mount

[ -f /mnt/media/Movies/mounted.bin ] && docker restart emby  ### if mounted.bin exists, then restart emby