AgileHumor 123 Posted May 13, 2015 Posted May 13, 2015 I had about a thousand movies and TV shows fully synced. I had a new sync job that was partially synced. Today on Dev Version 3.0.5610.43036 all items seem to be resyncing. Anyone else have this experience or is it unique to my library?
Luke 42079 Posted May 13, 2015 Posted May 13, 2015 only reason that should happen is if it thinks the files have been removed from the cloud account
AgileHumor 123 Posted May 13, 2015 Author Posted May 13, 2015 No changes seem evident. It's not a big deal...it's just CPU which sits idle most days. Maybe something weird happened on my setup...hopefully it's a on off. My JSON file was a couple MB last night...and now it's only 400KB. If the JSON gets corrupt, does it resync? Or can it test the file path too?
Luke 42079 Posted May 13, 2015 Posted May 13, 2015 the json file represents the synced database. if that file is removed or tinkered with then yes that could cause this
AgileHumor 123 Posted May 14, 2015 Author Posted May 14, 2015 I can promise I did not make changed (don't use google drive for anything else). Just an FYI if others report there might be some potential "reset" scenarios on the server (perhaps internet down at time).
AgileHumor 123 Posted June 6, 2015 Author Posted June 6, 2015 (edited) Synced 171 "Kids Movies" - Stayed Synced for a Week. Added another 1000 "Regular Movies" - Now, the Kids movies are reconverting/resyncing. I'm on build 3.0.5625.23705, but I've been trying this for over two month and each time sync decides to resync at some point (i delete all sync jobs in between). Any other reports of resyncs or other undesired sync anomalies? Looking for a pattern... PS - Log available on request...but trying to understand patterns that work and don't. Edited June 6, 2015 by AgileHumor
Luke 42079 Posted June 6, 2015 Posted June 6, 2015 I think I see one change that can be made that might improve this. Will update later. 1
Luke 42079 Posted June 7, 2015 Posted June 7, 2015 I'm sorry, I was incorrect. What I intended to look at looks fine to me. I have cloud synced test content that has been synced for over two months and does not exhibit this problem. We really need to catch it in the act rather than after the fact because at that point it's hard to figure out why. The only thing I would suggest is that if you are tinkering with any of the cloud synced directories within google drive - don't do that. It is all based on unique file identifiers assigned by the cloud sync provider. If for example, you were to manually delete all of the cloud synced content, then manually put it back in, it would not match up with Emby Server because the files would surely have new unique identifiers. That is a scenario that would cause resyncing. If you are moving the files around in google drive, I would think the identifiers would stay the same, but I can't say for certain.
AgileHumor 123 Posted June 7, 2015 Author Posted June 7, 2015 I haven't changed anything in cloud (I don't use Google Drive except for Emby). Good to know the GUID though for the future. I'll to try when it starts next time... PS - I did update the "Kids Movies" synced first to sync new content (i normally run large jobs with that unchecked and update it end)...could that be the issue? PSS - 5th time I've tried to backup my media. I've uploaded everything at least twice by now. I've tried stabilizing things where I'm sure things don't change on cloud side, but there have been some hard resets if a dirty Emby Server shutdown could caused.
Luke 42079 Posted June 7, 2015 Posted June 7, 2015 Thanks. The root cause of this would most likely be that the server downloads the data.json file and somehow gets an empty file back, then proceeds to resync because it thinks the content is missing from destination. Or if you're uninstalling and re-installing Emby Server from scratch this could also happen because the server would have a new server Id. That's something we'll have to figure out and make sure the backup and restore plugin can handle that situation. The server could also get a new Id if you're tinkering with the server's data folder. Since you say you're not tinkering with the cloud content, I think these are the two most likely scenarios.
AgileHumor 123 Posted June 7, 2015 Author Posted June 7, 2015 In regards to the JSON, I do get a lot of these errors that seem to resolve themselves (temp internet glitch I assume): Cloud & Folder Sync failed. 2 hours ago Couldn't find file 0B4i-GJkNuqecNlJ3VmtsTUNyQkk/data.json Cloud & Folder Sync failed. 3 hours ago Couldn't find file 0B4i-GJkNuqecemltQjdMTER1ZU0/Tinker Bell and the Legend of the NeverBeast (2015).mp4 Cloud & Folder Sync failed. 3 hours ago Couldn't find file 0B4i-GJkNuqecemltQjdMTER1ZU0/Tinker Bell and the Legend of the NeverBeast (2015).mp4 Cloud & Folder Sync failed. 3 hours ago Couldn't find file 0B4i-GJkNuqecNlJ3VmtsTUNyQkk/data.json I've not reset my server or program data....so don't think it's the 2nd option. If it did happen on one dev build, it wouldn't explain it happening more than three times.
Luke 42079 Posted June 7, 2015 Posted June 7, 2015 ok. if it can't find the data json file then that would be the problem. then it's going to think the cloud destination is empty and will start resyncing everything. 1
AgileHumor 123 Posted June 8, 2015 Author Posted June 8, 2015 You might want to do an internet check (any other test file can be written/read)...or maybe retry or something before assuming a resync is needed (assuming it's been synced at least once). I understand it's a tough problem... The internet is normally not dead, just congested. I even limit sync to half of upstream bandwidth. Glad at least I understand why
Luke 42079 Posted June 8, 2015 Posted June 8, 2015 that's already there. if the server fails to talk to onedrive we don't resync. it's only if the server asks for a file and then gets a response back saying it doesn't exist, that we resync
hamstercat 63 Posted June 8, 2015 Posted June 8, 2015 In the plugin I catch any 404 and translate them to FileNotFound, could that be a problem? I took that from the Folder Sync plugin, but maybe that's what's happening here.
AgileHumor 123 Posted June 9, 2015 Author Posted June 9, 2015 Never. I do have an AT&T modem that likes to pretend to be the web page when the internet isn't working (much like hotpots) using transparent proxy. i.e going to ford.com will show me the "internet is down" web page of the router.
hamstercat 63 Posted June 9, 2015 Posted June 9, 2015 I'm thinking maybe if the server is unreachable it'll return a 404 also.
Luke 42079 Posted June 9, 2015 Posted June 9, 2015 I'm thinking maybe if the server is unreachable it'll return a 404 also. Yup, that we should distinguish. If unreachable you can throw anything else appropriate aside from a not found exception,
hamstercat 63 Posted June 9, 2015 Posted June 9, 2015 (edited) Good, in that case it's fine already! I was worried maybe I was sending too many not found. Error must be elsewhere then. Edited June 9, 2015 by hamstercat
Luke 42079 Posted June 9, 2015 Posted June 9, 2015 yea that's really why we have to capture in the act. it's hard to diagnose after the fact. maybe you could add more logging to the plugins though? it appears they use their own logger and don't write very much to the server log file.
AgileHumor 123 Posted June 9, 2015 Author Posted June 9, 2015 I'd suggest a temp token file written and then verified (or some API call that is hidden that confirms service is google, and all is well). That way a bad logon or fluke Google Drive API won't cause an entire resync.
hamstercat 63 Posted June 10, 2015 Posted June 10, 2015 I'll add more logging to the plugin then, it definitely can't hurt.
hamstercat 63 Posted June 18, 2015 Posted June 18, 2015 New version of Google Drive plugin is up with more logging. 1
Recommended Posts
Create an account or sign in to comment
You need to be a member in order to leave a comment
Create an account
Sign up for a new account in our community. It's easy!
Register a new accountSign in
Already have an account? Sign in here.
Sign In Now