Scale subtitle stream to target dimension instead of the original dimension

[Waldonnis 148]

Still working on it.  I have it working perfectly now, but think the scale2ref expression is ridiculously complicated and can be simplified.  I'll finish it up tomorrow.

[Waldonnis 148]

Well, I do have it working, but after loading up a Japanese movie that splashed rendered captions all over the screen, it just didn't look good and I had a hard time figuring out a good way to keep positional elements in place while also keeping the sub frame AR.  Long story short, I concluded it's probably just easier to "stretch to fit" the sub frame to the video (which is super-easy).  Yeah, it'll probably squish or stretch captions a bit, but barring drastic AR changes, it would be the same behaviour that Emby seems to do now and at least it would keep text that was intentionally positioned to be directly above some character's head in a scene exactly where it's supposed to be.

 

So, we're back to simple with no real explanation needed other than the sub frame scaling happens after any desired video scaling:

ffmpeg -canvas_size 1920:1080 -i input.mkv -filter_complex "null[v];[0:2][v]scale2ref[sub][ref];[ref][sub]overlay" -c:v libx264 -c:a copy output.mkv

scale2ref by default can mangle the AR of the sub frame if it's different than the video's AR (think of it like a 16x9 movie on a 4x3 television set up to "stretch to fill the screen"; everyone looks taller/thinner).  As mentioned above, though, it's pretty much the best way to go IMO so positional intent is maintained and the distortion is relatively minor in all but the most extreme cases that are too rare to care about (e.g. 4:3 sub frame on a 36:10 video).  I also changed the pad naming to [v] because I was tired of typing (lol) and because I didn't have a reason to explicitly name the output pad for overlay any longer now that I don't have any debugging/info filters in the filtergraph (I never post those bits; they just confuse things, but leave the pad name in place until I'm done).

 

Oh, and it's safe to use scale2ref whenever you overlay subs if you want, even if the video isn't scaled or the sub/video frame sizes are equal.  If the frame sizes match, it's just a 1:1 scale anyway.  It's probably wasteful to do so from a CPU cycles perspective, but it won't cause output issues.

 

Here's a really quick benchmark comparison for a 2160p video being downscaled to 1080p and overlaying subtitles for 30s using the original arrangement (as noted in the OP) vs. the above on my admittedly-busy machine (using a hardware encoder, but software decoding...and outputting to null)...

 

Original:

frame=  720 fps= 20 q=20.0 Lsize=N/A time=00:00:30.03 bitrate=N/A speed=0.823x
video:7620kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown
bench: utime=111.781s stime=5.891s rtime=36.528s
bench: maxrss=779288kB 

New:

frame=  720 fps= 30 q=20.0 Lsize=N/A time=00:00:30.03 bitrate=N/A speed=1.26x
video:7598kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown
bench: utime=74.219s stime=1.391s rtime=23.880s
bench: maxrss=662488kB

Edited July 21, 2018 by Waldonnis

[Luke 38985]

Can you try that when the subtitles are external sub/idx? thanks.

[Waldonnis 148]

Yep, works fine.  Same command line, except for the stream IDs obviously (in this case, the IDX is [1:0] since i specified it as the second input file, and the input of the first filter would be [0:0] for the movie file's video track stream ID).

[Luke 38985]

I'll play with it some more but when i tested this the other day the transcode was successful but i just couldn't see the subtitles.

[Pittiplatsch 2]

Hi,

 

I'd like to know what the status of the issue is? I've a problem with misaligned/scaled subtitles (vobsub) with some of my media and it seems to be very related to the issue described in this thread.

 

i figured out, that the issue only come up when I crop letterboxed media to get the actual or intended aspect ratio in my media files while rendering. I also tested if anamorphic rendering has any influence to it, but the issue seems to be not affected by this, only the cropping.

 

I'm pretty sure this issue come up with a Emby update a while ago, but I cannot reproduce this anymore.

 

Thanks,

Peter

[Luke 38985]

Hi @@Pittiplatsch, this topic is not about fixing any alignment issues, this topic is about finding optimization in our subtitle transcoding command line. Let's look at your issue from scratch. please see how to report a media playback issue. thanks !

[Carlo 4504]

Sounds like this might be related based on the only media he sees this issue on.

[Pittiplatsch 2]

Hi @@Pittiplatsch, this topic is not about fixing any alignment issues, this topic is about finding optimization in our subtitle transcoding command line. Let's look at your issue from scratch. please see how to report a media playback issue. thanks !

 

Hi @@Luke. Sorry for the misunderstanding. The title and some discussion of this thread suggested that this might be related to my issue. Anyway, I created a new post: https://emby.media/community/index.php?/topic/63466-subtitle-alignmentscaling-when-transcoding-with-vobsub-subtitles/

 

Thanks,

Peter

[Luke 38985]

Thanks.

[hud2009 0]

Hello, first of all sorry to ressurect this post... but i have an issue and think you guys can help me.

 

Can i hardcode this subtitles on the video and create outputs with differents resolutions WITH the subtitle "burned"?

 

If yes, can you guys tell me how?

 

Thanks !

[Luke 38985]

Hello, first of all sorry to ressurect this post... but i have an issue and think you guys can help me.

 

Can i hardcode this subtitles on the video and create outputs with differents resolutions WITH the subtitle "burned"?

 

If yes, can you guys tell me how?

 

Thanks !

What format subtitles?

[hud2009 0]

What format subtitles?

Hello,

 

It's dvbsub.

 

With your code, I can burn the subtitle with only one resolution (1080x720), but I need the subtitle to work at 3 other resolutions (728x432 ; 640x460 and 320x160).

 

My code with one resolution below:

 

ffmpeg -i 'udp://xxx.x.xx.xxx:xxxx?interface=xx.xx.xx.xx&fifo_size=5000000&reuse=1' \

         -map 0:0 -map 0:1 -map 0:2 -map 0:3 \

         -filter_complex "null[0:0];[0:3][0:0]scale2ref_{[ref];[ref]overlay" \}

         _{-sn -s 1080x720 -vcodec:0 libx264 -preset medium -b:v 5000k -x264-params keyint=60:scenecut=0 -g 60 \}

         _{-pix_fmt yuv420p -profile:v high422 -level 3 \}

         _{-profile:v baseline -c:a aac -b:a 96k -c:a:2 aac -b:a:2 96k  -c:s:3 dvbsub \}

         _{-f mpegts udp://xxx.x.xx.xx:xxxx?pkt_size=1316}

 

_{PS: It's a live stream channel.}

Edited October 22, 2019 by hud2009

[Luke 38985]

Hello,

 

It's dvbsub.

Ok, our media conversion feature can help you do this.