when would the resolution of embedded subtitles not match the resolution of the video stream?
The most common example would be if someone re-encoded a BRD rip and cropped the letterbox area out leaving just the video in its actual AR rather than the padded 16x9 frame on the disc...and doing this without cropping the sub stream too (nobody crops the sub stream, in my experience...they just copy it). For example, one of my files started out with 16x9 1080p dimensions, but I cropped all of the letterbox bar area out and re-encoded it to just include the 2.40:1 AR of the film itself (so the final dimensions are 1920x810). The PGS sub stream, however, is still 16x9 1920x1080 (1080p). If the subtitle text would ordinarily be rendered in the area that I had cropped out, then not scaling would mean some/all of the caption text would be rendered off screen, so I would need to scale down the subtitles so the entire sub frame fits within the smaller vertical area of the 2.40:1 AR, 810-pixels of video.
One of my other files still contains the letterbox bars and is still 1920x1080 16:9 (same as the subtitle stream), so there's no risk of rendering off-screen and don't have to scale the subtitle frame at all and a simple overlay would work fine.
In the case of the UHDs I've seen so far (maybe all?), the PGS subtitle streams have all had dimensions that were 25% of the video's 2160p frame (so, the subs are 1080p while the video is 2160p, but both are the same AR). If I rendered a 1080p frame on top of a 2160p video, I'd end up with a caption frame that is small compared to the video and "floats" somewhere within the much larger video frame (likely smack dab in the centre). It's entirely possible that ffmpeg could autoscale in these situations, but I've never tried it and wouldn't expect it to do so.
In both situations (cropped input video and UHD), the sub stream's dimensions would differ from the source video's...and scaling would be appropriate for different reasons (for cropped, because we might render offscreen and with UHD because the sub frame would be too small). In the case of my "still 1080p" video example above, no scaling is needed because the dimensions do match between the streams.
I haven't thought about how anamorphic would fit in to this, but I'd expect you could choose to always use scale2ref and if the presentation seems odd during testing, changing the force_original_aspect_ratio option will probably compensate for it. Honestly, I have no idea how picture-based subs are rendered with anamorphic content, so that's more of a guess. I do know that the subs on my anamorphic test file have no SAR/DAR (obviously), so they normally are either stretched to match the DAR-derived dimensions or are rendered as-is without any scaling (no idea which is done on players since I've it's something I've never had to think about). Either way, it's easy to reproduce that behaviour once we figure out what it's supposed to be.
Edited by Waldonnis, 20 July 2018 - 12:22 AM.