High GPU memory usage while transcoding

[Sungray 6]

 

This time it worked, these seem to change all the time :D. I'm not on my LAN right now. But no effect on the mem.

 

 

ffmpeg-transcode-6a2081f2-d169-482d-836e-e1da143299f0_1.txt

Edited June 14, 2022 by Sungray

[softworkz 5066]

The last idea that that I have is that maybe they are only encoding a small chunk and each time and then stopping the process again, so you might just see an average in the nvidia tool.

At least there log has new entries every 10s in average. You could check that by using "top", watching their transcoder process and then looking at the PID of the process whether it is constant or changes every 10 seconds.

Another thing you could try is to run their command manually (with sudo):

/usr/lib/plexmediaserver/Plex Transcoder -codec:0 hevc -hwaccel:0 nvdec -hwaccel_fallback_threshold:0 10 -threads:0 1 -hwaccel_output_format:0 cuda -hwaccel_device:0 cuda -codec:1 dca -analyzeduration 20000000 -probesize 20000000 -i "/data/Movies/The Contractor (2022)/The.Contractor.2022.2160p.UHD.BluRay.HDR.DTS-HD.MA.5.1.x265-SPHD.mkv" -analyzeduration 20000000 -probesize 20000000 -i /transcode/Transcode/Sessions/plex-transcode-kxp7vaj0vfisl67oy3tnenwf-483a5004-5727-4f56-971c-ab0110495bb4/temp-0.srt -filter_complex "[0:0]hwupload[0];[0]scale_cuda=w=3064:h=1280:format=p010[1];[1]tonemap_cuda=mobius:nv12[2]" -map "[2]" -metadata:s:0 language=eng -codec:0 h264_nvenc -b:0 14023k -maxrate:0 18698k -bufsize:0 37396k -forced-idr:0 1 -r:0 23.975999999999999 -force_key_frames:0 "expr:gte(t,n_forced*1)" -filter_complex "[0:1] aresample=async=1:ocl='stereo':rematrix_maxval=60.000000dB:osr=48000[3]" -map "[3]" -metadata:s:1 language=eng -codec:1 aac -b:1 136k -f dash -seg_duration 1 -dash_segment_type mp4 -init_seg_name 'init-stream$RepresentationID$.m4s' -media_seg_name 'chunk-stream$RepresentationID$-$Number%05d$.m4s' -window_size 5 -delete_removed false -skip_to_segment 1 -time_delta 0.0625 -manifest_name "http://127.0.0.1:32400/video/:/transcode/session/kxp7vaj0vfisl67oy3tnenwf/483a5004-5727-4f56-971c-ab0110495bb4/manifest?X-Plex-Http-Pipeline=infinite" -avoid_negative_ts disabled -map_metadata -1 -map_chapters -1 dash -map 1:s:0 -metadata:s:0 language=eng -codec:0 ass -f segment -segment_format ass -segment_time 1 -segment_header_filename sub-header -segment_start_number 0 -segment_list "http://127.0.0.1:32400/video/:/transcode/session/kxp7vaj0vfisl67oy3tnenwf/483a5004-5727-4f56-971c-ab0110495bb4/manifest?stream=subtitles&X-Plex-Http-Pipeline=infinite" -segment_list_type csv -segment_list_size 5 -segment_list_separate_stream_times 1 -segment_format_options ignore_readorder=1 -segment_list_unfinished 1 -fflags +flush_packets "sub-chunk-%05d" -start_at_zero -copyts -vsync cfr -init_hw_device cuda=cuda: -filter_hw_device cuda -y

You'll probably need to tweak this cmd a bit to get it working standalone (maybe it doesn't work at all - I don't know)

You could also try the simplified command line I posted above with both Emby and their transcoder.
Your findings will be interesting, but I can't walk you though each detail.

Thanks,
sw

[softworkz 5066]

Nevermind - I found it.

Try this:

Find: -copyts 
Replace: -copyts  -threads:v:0 1

[Sungray 6]

Just now, softworkz said:

Nevermind - I found it.

Try this:

Find: -copyts 
Replace: -copyts  -threads:v:0 1

Yes! Care to explain? Is this a better setting?

[softworkz 5066]

6 minutes ago, Sungray said:

Yes! Care to explain? Is this a better setting?

 

Historically, the threads setting  has been a user-configurable option in Emby.

I want to change that since quite a while for several reasons, the primary one is that I'm saying "this is nothing for the user to decide".

There doesn't even exist a single threads value that is good in all situations - it depends. And in reality, there are multiple many settings:

• For the video decoder
• audio decoder
• filter graph
• video encoder
• audio encoder
• etc.

What we have had throughout the years was nonsense, but there was no final decision about the removal yet.

One point for example is that hw decoders should not be operated with multiple threads. Standalone decoders don't allow this, but the hybrid ones (which can also do sw decoding) allow multiple threads to be configured (default is 0 == auto).

The effect is that much more frames are decoded than which can be processed, and all those decoded frames that are congested are causing the high memory.

Edited June 14, 2022 by softworkz

[Sungray 6]

All I have is this:

As you say in ffmpeg the thread count is configurable for video decoding, video encoding, and audio. What does this setting control? Video encoding?

I suppose the replace changes thread count for video decoding?

Edited June 14, 2022 by Sungray

[softworkz 5066]

2 minutes ago, Sungray said:

As you say in ffmpeg the thread count is configurable for video decoding, video encoding, and audio. What does this setting control? Video encoding?

It's accidental - whatever of the used streams is the first one gets it applied. So it can be video or audio decoder (if stream 0 is audio).

[softworkz 5066]

7 minutes ago, Sungray said:

Video encoding?

It depends on the position in the command line. What comes before the -i INPUT file applies to decoding what comes after is applied to encoding. Filtering has differently named parameters.

Edited June 14, 2022 by softworkz

[Sungray 6]

Just now, softworkz said:

It depends on the position in the command line. What comes before the -i INPUT file applies to decoding what comes after applied to encoding. Filtering has differently named parameters.

Gotcha.

So all in all this looks to me like a good improvement with no ill effect? Will this be added in a later release or should I just keep the substitution for myself?

[softworkz 5066]

Please note that the replacement doesn't survive server restarts, you need to re-apply each time.

22 minutes ago, Sungray said:

Will this be added in a later release

I hope, but I don't know at the moment

Edited June 14, 2022 by softworkz

[Sungray 6]

6 minutes ago, softworkz said:

Please note that the replacement doesn't survive server restarts, you need to re-apply each time.

I hope, but I don't know at the moment

Ah. Well I really hope this will be changed soon then.

Thanks a lot @softworkz.

Edited June 14, 2022 by softworkz

[Luke 42077]

@Sungray are you still having an issue with this?

[Sungray 6]

12 minutes ago, Luke said:

@Sungray are you still having an issue with this?

Fixed on 4.8.0.9 beta.