Sync tasks stick on Nvidia Shield server (to iOS client)

[r_c_ 0]

Hi there.

 

Moving over from Plex I have set up Emby server on my shield (latest 16GB model) and it is all working well.

 

I have all my media on a large external usb drive and have set all the temp folders (cache, transcode, metadata, sync etc) to said drive.

 

When syncing files, larger mkv files especially seem to start converting at a good rate (about 4x ish) and then after around 20% (slightly varies per file) the conversion stage just hangs. The Device/sync settings page simply displays the percentage it is stuck on as does the same part of the iOS app.

 

The ffmpeg log seems to cut off mid line and not be updated any further when this happens. Logs attached.

 

Any help much appreciated. I have tried varying combinations of speed throttling, cpu cores etc to no avail. I have even opened the file in the sync folder on the server and it seems to play up until the point the conversion failed. Plenty of space on the drive and the file plays fine when simply watching via the iOS app or tv emby client (lg).

 

Thanks,

Richard.

 

ffmpeg.txt

 

HardwareDetection.txt

 

embyserver.txt

Edited June 2, 2019 by r_c_

[r_c_ 0]

Just to follow up, after heading out and returning a few hours later the sync now as an error message.

Logs at this stage here.

 

Thanks,

Richard.

 

enbyserver_v002.txt

 

ffmpeg_V002.txt

 

hardwaredetect_v002.txt

[Luke 38493]

Hi, what was the error message?

[r_c_ 0]

Hi Luke.

 

After a fair while (After the conversion hung I went out for a few hours and checked on my phone what the status was when I returned) the error in the download settings area of the iOS app said...

 

The conversion has failed!

 

Or something very similar (might have said download/transcode for example) in orange writing.

 

The first set of logs I uploaded was from when it was in the “stuck state” at around 17.5% and the second was on my return as above.

 

Thanks again,

Rich.

[r_c_ 0]

Some more info.

 

The file attempting to be synced is a high bitrate mkv from a Blu-ray rip (makemkv) and plays fine in both emby and Plex. I also tried a pass through converted version of it as mp4 with the spare audio streams removed. It got a slightly higher percentage through but ultimately had the same problem.

[r_c_ 0]

Just looking through the logs. Doesn't mean a huge amount to me, but this seems to highlight at least some of the offending moments....

 

 

2019-06-02 14:55:00.149 Info TaskManager: IntervalTrigger fired for task: Convert media

2019-06-02 14:55:00.149 Info TaskManager: Queueing task SyncConvertScheduledTask

2019-06-02 14:55:00.149 Info TaskManager: Executing Convert media

2019-06-02 14:55:00.509 Debug XmlSerializer: Deserializing file /storage/emulated/0/Android/data/com.emby.embyserver/files/config/sync.xml

2019-06-02 14:55:00.599 Debug XmlSerializer: Deserializing file /storage/emulated/0/Android/data/com.emby.embyserver/files/config/users/60a7183cc613440eb5795a49e391be4b/config.xml

2019-06-02 14:55:00.628 Info App: Bitrate exceeds DirectPlay limit: media bitrate: 24199089, max bitrate: 320000

2019-06-02 14:55:00.629 Info App: Bitrate exceeds DirectStream limit: media bitrate: 24199089, max bitrate: 1500000

2019-06-02 14:55:00.629 Info App: Profile: Unnamed, Path: /storage/28B6-BA2A/NVIDIA_SHIELD/Emby/media/Test02/MacGyver - S01E01 - HDBluRay.conv.m4v, isEligibleForDirectPlay: False, isEligibleForDirectStream: False

2019-06-02 14:55:00.695 Debug XmlSerializer: Deserializing file /storage/emulated/0/Android/data/com.emby.embyserver/files/config/encoding_diagnostics.xml

2019-06-02 14:55:00.765 Debug MediaEncoder: CodecValidation: FindVideoDecoder - MediaType: h264, Mode: 1

2019-06-02 14:55:00.786 Debug MediaEncoder: CodecValidation: FindVideoDecoder - Checking: 'Nvidia.h264' (Priority: 10)

2019-06-02 14:55:00.802 Debug MediaEncoder: CodecValidation: FindVideoDecoder - Check successful - selecting 'Nvidia.h264'

2019-06-02 14:55:00.803 Debug MediaEncoder: CodecValidation: FindVideoEncoder - Media: h264, UseHardwareCodecs: True, Mode: 1

2019-06-02 14:55:00.807 Debug MediaEncoder: CodecValidation: FindVideoEncoder - Checking: 'Nvidia.h264' (Priority: 10)

2019-06-02 14:55:00.807 Debug MediaEncoder: CodecValidation: Encoder supports input stream

2019-06-02 14:55:00.807 Debug MediaEncoder: CodecValidation: FindVideoEncoder - Check successful - selecting 'Nvidia.h264'

2019-06-02 14:55:00.992 Info MediaEncoder: ProcessRun 'Encoding 6997c9' Execute: /data/data/com.emby.embyserver/files/ffmpeg/ffmpeg -c:v h264_mediacodecndk -mediacodec_name OMX.Nvidia.h264.decode -f mp4 -i file:"/storage/28B6-BA2A/NVIDIA_SHIELD/Emby/media/Test02/MacGyver - S01E01 - HDBluRay.conv.m4v" -map 0:0 -map 0:1 -map -0:s -c:v:0 h264_mediacodecndk -mediacodec_name OMX.Nvidia.h264.encoder -force_key_frames "expr:gte(t,n_forced*5)" -mediacodec_output_size 720x404 -b:v:0 1308000 -maxrate 1308000 -bufsize 2616000 -profile:v:0 8 -level:v:0 4096 -vsync -1 -map_metadata -1 -map_chapters -1 -threads 0 -codec:a:0 copy -metadata:s:a:0 language=eng -disposition:a:0 default -y "/storage/28B6-BA2A/NVIDIA_SHIELD/Emby/sync/14/14/6997c9c9-cb11-4198-b072-d8b5784bf1ec.mkv"

2019-06-02 14:55:01.004 Info MediaEncoder: ProcessRun 'Encoding 6997c9' Started.

2019-06-02 14:55:02.777 Info MediaEncoder: ffmpeg successfully started

2019-06-02 14:56:53.501 Info MediaEncoder: ProcessRun 'Encoding 6997c9' Process exited with code 137

2019-06-02 14:56:53.504 Debug MediaEncoder: Disposing stream resources

2019-06-02 14:56:53.510 Info MediaEncoder: FFMpeg exited with code 137

2019-06-02 14:56:53.592 Error App: Error during sync transcoding

*** Error Report ***

Version: 4.1.1.0

Command line: /data/app/com.emby.embyserver-qu7ibyfi_s5W-HNecAlK9Q==/base.apk

Operating system: Unix 4.9.109.4

64-Bit OS: True

64-Bit Process: True

User Interactive: False

Runtime: file:///mscorlib.dll

Processor count: 4

Program data path: /storage/emulated/0/Android/data/com.emby.embyserver/files

Application directory: /data/user/0/com.emby.embyserver

Mono: 5.14.0 (explicit/62031dcabf4)

Android Version: 8.0.0-REL - SDK: 26 'O'

Patch Level: 3507953_1441.7411 (2018-12-05)

Fingerprint: NVIDIA/darcy/darcy:8.0.0/OPR6.170623.010/3507953_1441.7411:user/release-keys

Model: SHIELD Android TV - NVIDIA/NVIDIA

Hardware: darcy/darcy/darcy/unknown

SupportedAbis: arm64-v8a, armeabi-v7a, armeabi

System.Exception: System.Exception: Encoding failed

  at Emby.Server.MediaEncoding.Encoder.MediaEncoder+<EncodeVideo>d__75.MoveNext () [0x0017c] in <759de1aea5af43339ca4a93f5c524532>:0 

--- End of stack trace from previous location where exception was thrown ---

  at Emby.Server.Sync.SyncJobProcessor+<SyncVideo>d__30.MoveNext () [0x00305] in <101e2a99ff6a4b15a0f9dbb8b8104576>:0 

Source: mscorlib

TargetSite: Void Throw()

Edited June 3, 2019 by r_c_

[Luke 38493]

Can you please attach the conversion log? thanks.

[r_c_ 0]

Hi Luke.

 

My first two posts have the logs attached, or is the conversion log something other than the ffmpeg, hardwareDetecr and embyServer ones I see I the logs section?

 

As another point in the graph, I set up a temporary server on my mac and the file converted without issue when a sync was initiated which would lead one to assume the issue is perhaps with the shield server conversion process?

 

Thanks,

Rich.

[Luke 38493]

As a test, if you turn off hardware transcoding, does that make a difference?

[r_c_ 0]

I turned it off in the transcode (rather than sync) settings and the error persisted unfortunately.

 

I also tried a few combinations of full speed off and on as well as max cores and fewer cores to no avail.

 

Out of the box I my Mac air (having pointed the cache and transcode etc locations to a specific temp folder) as mentioned it converted without issue.

[Luke 38493]

 

 

I turned it off in the transcode (rather than sync) settings and the error persisted unfortunately. 

Can you provide a conversion log from this? thanks.

[r_c_ 0]

Hi Luke.

 

I have just attempted a conversion with the hardware acceleration off again and it went through. 

Turning it back on and the same error happens, perhaps I didn't commit the preferences change last time I checked, apologies there.

 

Set up remote access so I can tweak from here today...

After a bit of a play it seems the error is with the hardware decode rather than the encode.

 

If I keep hardware encoding on and disable hardware decoding then the conversion completes. Change nothing else but enabling the hardware h.264 decoding and we are back to failing again. Various logs attached.

 

It halves the speed of the conversion with the decode off but that is better that it being 1/4 speed with all hardware options off.

 

Hopefully it is something that could be fixed up in a later version or with a server tweak somewhere?

As well, is there somewhere I can set the sync process to not use hardware decoding, but turn it on for the regular streaming/playback?

 

As well, it seems to be stuck in the "ready to transfer" stage, I assume it should jump strait to transferring to my phone when done? I am off the local network at the mo, but my browser and the (iOS) phone app can see, interact and steam for the server fine.

 

 

Many thanks again,

Rich.

embyserver_hardwareH264__decodeOFF_encodeON.txt

embyserver_hardwareH264__decodeON_encodeON.txt

ffmpeglog_hardwareH264__decodeOFF_encodeON.txt

ffmpeglog_hardwareH264__decodeON_encodeON.txt

[Luke 38493]

@@softworkz any thoughts?

 

Looks like our fallback from GPU to CPU only covers the start of the process, but doesn't fallback if there's a failure somewhere in the middle of the conversion. So I guess that's an improvement we should look to add, not for streaming, but certainly for sync conversions.

[r_c_ 0]

Just to flag up the transfer seems to work now, although without the % complete updating. Saying transferring 0% for however long then snaps to downloaded.

 

Thanks,

Rich.

[r_c_ 0]

Actually with hardware decoding off, minus the lack of transfer percentage the syncing is working quite well now.

 

It does seem a bit arbitrary as to when the transfer will start though, I sometimes have to (on the phone) open the download, change nothing and hit save for it to nudge and start.

 

Thanks,

Rich.

[softworkz 3948]

@@r_c_ All your logs are about the same file.

 

Does the problem occur

• with all files?
• with only some files?
• or so far only with that single file shown in the logs?

[softworkz 3948]

@@softworkz any thoughts?

 

Looks like our fallback from GPU to CPU only covers the start of the process, but doesn't fallback if there's a failure somewhere in the middle of the conversion. So I guess that's an improvement we should look to add, not for streaming, but certainly for sync conversions.

 

Thats not as easy as it might seem. At the transcoding start, any failure can be credited to hw acceleration issues with high probability. But once it's running - any failures that may occur are rather unlikely caused by the fact that it's hw accelerated. Typically there are other reasons when that happens, so this will be tricky to determine...

 

What I think we should have, though, is separate hw acceleration settings for sync transcoding and live transcoding (not necessarily duplicated, but we should think about a way...)

[r_c_ 0]

Hi @@softworkz and @@Luke

 

My initial tests had issues on what as far as my testing got seemed to be higher bitrate files.

I tried several episodes from the series all with the same results so for ease of testing used the same file from then on.

 

Separate options for sync and playback transcoding would be great (although solving the issue would be even greater of course!).

The same issue occurs on the original mkv as well as a passthrough converted mp4 version with the spare audio channels left out, although it does get slightly further along with the mp4 version.

 

If I were a guessing man, perhaps it is a file size issue? I have plenty of space on the designated convert and sync folder paths though and about 9 gig of the shield's own storage is free.

 

Thanks,

Rich.

[softworkz 3948]

If this is not just a single-file issue (like a specific one being damaged in some way), then I can only offer trying to reproduce this locally on a Shield.

Would you be able to provide a file for testing?

[r_c_ 0]

Hi @@softworkz. I’ll prepare and message you a link shortly.

 

Many thanks,

Richard.

[Luke 38493]

Thanks !

[Luke 38493]

@@softworkz will review your PM. Thanks.