Folder Sync - Endlessly re-syncing

[HeWhoWas 0]

I'm using Folder Sync to sync to a local folder, and for some reason the jobs seem to go through this state:

1. Queued
2. Converting
3. Ready To Transfer
4. Transfering
5. Synced...
6. Queued

In an endless cycle, similar to what was reported in this thread: http://emby.media/community/index.php?/topic/24076-completed-sync-jobs-re-syncing/. Does anyone have a solution to this? Happy to provide debug logs so long as someone can point me in the direction of enabling/collecting them, in the meantime here's some info that might help:

 

• Folder being synced to is a folder on the same drive. The disk is in a RAID0 and IO is not an issue.
• Ubuntu 14.04 running Docker Emby
• Permissions are open on all folders being written to, and owned by the user running Emby (99)

Thanks all. 

 

[Luke 42077]

this was resolved, at least on windows. once it is synced, you want to make sure the server can still access the synced content

[HeWhoWas 0]

Unfortunately I can confirm that this is still the case on Linux. A sync job with 10 items in it has encoded 3 times in the last 2 days.

 

There doesn't seem to be too much sync logging under the logging dir, is there some way to increase the verbosity?

 

I've confirmed that the sync location is readable by the emby user. It makes sense it would be, as it's the user that creates them.

Edited November 4, 2015 by HeWhoWas

[Luke 42077]

you can enable debug logging under advanced, restart the server, then please see:

 

http://emby.media/community/index.php?/topic/739-how-to-report-a-problem/

 

thanks.

[HeWhoWas 0]

Ok, I've managed to isolate the issue somewhat and can reliably recreate it. My process:

 

1. Go to TV -> Latest on your desktop.
2. Hit "Sync" and set up a folder sync. The settings don't matter, but I use Baseline High. The number of items I have is 239.
3. Allow the sync to start, and wait for an item to progress to the 'Synced' stage.
4. Go to User Sync page in the mobile Android application (note that it will say "Statys: Idle") and hit the refresh icon.
5. Viola, all videos that were in the "Synced" state now revert to the "Queued" state, and the queue progresses from its current point.

I'm not sure what the issue could be, I've tested the same behaviour with smaller queues, up to 10 items. I've enabled debugging mode and dropped the logs and more important information here: https://cryptobin.org/c2d0n790 - I've PM'd you the password as it's got a lot of information in it.

 

Edit - Added some information after some more testing. The android app can be used to trigger a sync in this manner, but I'm sure something else is doing the same in the background.

Edited November 5, 2015 by HeWhoWas

[Luke 42077]

how about the log from when they were synced?

[HeWhoWas 0]

Attached in the paste bin, please see your pm for the decrypt code. Or is there a specific log file for sync jobs?

[Luke 42077]

nothing appears to have changed in the log.

[HeWhoWas 0]

Sent you another PM, I've uploaded the full days logfile to MEGA: https://mega.nz/#!YRhBhbra

 

Decryption key in PM again.

[HeWhoWas 0]

Sent you through another logfile, same issue with dev plugins installed.

[HeWhoWas 0]

This is still a problem with the latest release, and this bug more or less destroys my use case for the platform. I'm not expecting hand-holding support, but some pointers about where to look in the code would be nice.

 

Are there any instructions on how to get a Linux Dev Env up and running?

[Luke 42077]

Sorry, when I said next release, I really meant the upcoming one. The one that just went out this week was just a surgical update on top of the previous release.

 

Some of our linux packages do have dev releases. which one are you using?

[HeWhoWas 0]

I'm using the Docker package at the moment, but what I'd really like to do is get the SLN file to load and build in monodevelop. I started on it last night and found a few things that had to be changed to work:

 

* Post-build tasks modified to use cp instead of xcopy

* Paths to NuGet.exe modified to be nuget.exe

 

But kept running into build issues aound other dependencies. Has there ever been a working development environment under Linux? If it's not going to take too long I'd love to be able to investigate or fix the issue...

[Luke 42077]

yes there is. there's a separate linux build script under /tools, including a readme

[HeWhoWas 0]

Facepalm. Thanks.

[HeWhoWas 0]

FYI, This is still an issue in the most recent version. Not seeing any new logging either, so don't have much more information to provide sorry.

[Luke 42077]

Ok, we're going to have a patch release in a day or two and I've merged in all the sync changes from the dev branch up to master. So please retest once you have that. Thanks.

[HeWhoWas 0]

Problem still exists on the latest version. Caught the logs from it today, beware that it's massive because at this point it's re-encoding about 1TB of media each time this bug is encountered. PM'd you the details.

[HeWhoWas 0]

I've removed the docker config and installed emby natively on Ubuntu itself. Let's see how it goes like this.

 

I've also started pushing some of the sync items up to Google Drive in the hopes that those one's won't spontaneously vanish, so hopefully the issue is mitigated somewhat.

[HeWhoWas 0]

Problem still exists with latest beta version on Ubuntu 14.04.

Edited January 4, 2016 by HeWhoWas

[HeWhoWas 0]

Any news, suggestions or, well, anything? 500 items re-encoding every 30-40 hours, including the stuff being pushed to Google Drive. Once all items are "Re-queued" they become totally unavailable until they're encoded again - making some media available for a grand whooping total of 3 minutes before it's automatically deleted and queued again.

 

Sync for Emby is fundamentally broken on my Linux box.