Smarter Zoom / Cropping

[oihsdfhdif 1]

At the moment on a TV especially (but applies to all devices), if you select Zoom to crop the picture and remove the black bars instead of zooming in "just enough" until the black bars are gone, it seems to "zoom" in by some constant amount. This is quite frustrating because sometimes the aspect ratio is such that you only need a little crop "little zoom". Instead Emby goes waaay too far zoomed in and ruins the picture.

Suggestions:

1. Set Zoom function to zoom only until black bars are gone (variable based on content)

OR...ideally:

2. Allow users to manually adjust zoom/crop factor

Edited July 20, 2021 by oihsdfhdif

[Smashpen 1]

Oih,

Zooming on video is hardly the same as zooming a photograph.  Video has line height and width of pixels.  The monitor is set to play that pixel dimension.  Monitors can be adjusted to different pixel scan sizes, but it's not continuous.

So the video itself is playing back with it's pixel height and width.  And to make this issue even more complex videos often have black bars in the video itself.  So the signal is actually playing back top and bottom black bars 68 pixels each, for example.

Neither the TV electronics, nor the software knows what part is a black bar and what is a dark alley.  Both are in the video, sometimes.  You could try to grab just pure black, but that can sometimes also fail in dark scenes.  No one bothers doing this.

Zooming a few lines of pixels at a time, as you requested.  Ok, say you only want 10 lines shaved off top and bottom.  The TV still has to show the same number lines of video, so for example a video that is 1260 by 720, you want to now show as 1260 wide by 700.  That means all the horizontal scan information has to be converted taking only 700 lines and changing it to 720 lines to fill the TV screen that is 16:9.  A TV would have to do that on the fly 24 or 30 frames a second in high quality, taking a single frame that is perhaps 720 x 1260 x 10 color depth bit = 9,072,000 bits of information 24 times a second = 217,728,000 bits of information per second that has to be blurred over a 720 pixel height image.  And the end result is most likely a stretched video that has an incorrect aspect ratio.  If you also move in from the side, then the resize has to occur in height and width which doubles the processing task.  So, obviously doing it with software is not going to happen.

A TV Zoom hardware solution is more likely and some TV's have that, but it most always means cutting off part of the picture.  An incremental horizontal only would only rarely be needed to fix incorrectly formatted videos where the Aspect Ratio is wrong.  Aspect ratio being wrong is pretty much 100% pirated videos.  So, you are asking the TV manufactures to offer a feature that essentially only fixes illegal videos.  That's not going to happen.  Some TV's have a height adjustment feature that spreads the height of individual horizontal lines, but that usually was associated with Standard TV's and not pixel based Widescreen TV's of today.

You might have this preference to have a full TV screen rather than cut top and bottom with black bars.  Look for TV models that still do that.  It's been mostly dropped as a feature.  Some older cable boxes had this feature and that has also been dropped.  I don't know what the percentage of users who prefer original aspect ratio being correct versus cutting off part of the image to fill the screen. I never care for cutting off the image, just like when I go see art at a museum, I want to see what the artist created and not an extra floating frame in front of a painting that cuts off parts of the artwork.

How many users who read this question prefer to crop out part of the movie so the TV screen is filled?  

In any case, It is really not practical to do with software and if you tried to do it, the image quality would instantly suffer a lot.

[speechles 1917]

20 minutes ago, Smashpen said:

Oih,

Zooming on video is hardly the same as zooming a photograph. 

Actually it is the same exact thing. You scale the group.

 

20 minutes ago, Smashpen said:

Neither the TV electronics, nor the software knows what part is a black bar and what is a dark alley. 

Yes. The application can easily know. You do that math and can easily tell when you render inside or outside the video frame. It isn't magic.

 

The issue is the "frame buffer" usually is made to consume frames of a certain dimension at a certain scale itself. When you scale up the video player this is easy. Once you reach the screen edges and need to push part of the playing video into the ether is when it will/can leak out of video addressing. That means the video signal will corrupt itself once you move out of the bounds of the screen (Roku). But this does not mean that other apps on other platforms cannot do this if they have an adjustable frame buffer. The Roku does not allow adjustable frame buffering. That is the issue when you zoom larger. When you zoom smaller (shrink) any application on any platform can do this as smaller will always fit into the frame buffer.

When scale you know the exact aspect ratio ahead of time. Can predict exactly how many pixels will be needed to scale up to eliminate. The issue will be the frame buffer likes to align on 8 or 16 or 32 or 64. You have to zoom into those multiples. If you "Stretch" and don't preserve aspect ratio ALL clients can do this even the Roku since no pixels are pushed off screen. It is when you must keep aspect ratio and part of the playing video must be pushed off the screen that problems happen.

20 minutes ago, Smashpen said:

In any case, It is really not practical to do with software and if you tried to do it, the image quality would instantly suffer a lot.

Yeah. It is practical and most modern clients do this. Image quality is affected by zoom. The terms "instantly" and "a lot" are subjective. If you must zoom something 1.18x to get edges all filled in while respecting aspect ratio it will be using larger zoomed in pixels. It is what it is. It doesn't make the image worse. It makes it what it was.

It is easier than you make this out to be...

Edited July 20, 2021 by speechles

[Smashpen 1]

Speech,

Good points.  I am coming from a professional photographer, sound engineer and video post production and movie special effects background.  Any Zoom in the industry drops too much image quality and is never done for production.  If you sit a certain distance from the TV, say around 12 feet and the monitor is, say 55 inch diagonal, then there is a point where the average eyesight has a real hard time to make out any pixelation problems with a zoomed image, but I would probably see issues, since that was my job to check quality on post production special effects.  I am also an audiophile and never can listen to MP3's or music on a phone.  It's a matter of preference.

When I was shooting for corporate, magazines and newspapers the editor did not publish photos that had to be heavily cropped.  They would just discard the shot, as useless and a side effect was I didn't get paid unless I brought in better quality.  I think that really got me started, looking extremely close at image quality.

I don't see this' scale pixel by lines' feature anywhere in TV models and I still think there is so little demand and the problem with causing any video artifacts that could kill a product sales. TV manufactures are also concerned by any way to show a TV image that is not as perfect as possible.  Any bad reviews that the image doesn't look good and the kind of extra cost to make this happen with TV hardware, just rules out TV builders from offering such a feature.

To do it with software,  I have to ask, who is paying for your development time?  Can you find a patron to actually pay for this?  If you can, go with it.  I think when it comes down to asking the cost estimate, up front, the project will not go far.

Edited July 21, 2021 by Smashpen

[speechles 1917]

<Group id="Scaler">
    <Video id="idPlayer" width="1920" height="1080" visible="false" />
</Group>

 

m.scaler = m.top.findNode("Scaler")

m.scaler.scale = [xScale,yScale]

 

You just need to know the amount of scale required to get to the edges if you stretch. If you keep aspect ratio the Roku will choke. The frame buffer isn't bigger than 1920x1080 (3840x2160 4k). It will start to macro the pixels and repeat a blocky pattern once you go "off the edge".

But it is that easy on the Roku. You just wrap a group around the video like any other object and scale it. It isn't magic at all.

 

@Smashpen Who pays for development time of what? I am at Emby. What exactly do you mean? I would write the scaler for the Roku if I could do zoom-with-aspect-ratio. It doesn't work I already wrote the demo for it. It works. It shows the Roku cannot do this. I mean I am curious. Interested. Not trying to be pretentious or lead you into something where I am not sharing as much as I can. I am at Emby. What I produce becomes property and entity of Emby. Whether or not it becomes useful or utilized. There are experiments all the time. We want to do better. It isn't about anything else other than Emby on Roku for me. Razor focus on that and must one up yourself every single day. What you did yesterday is old news. What did you bring me today? This is evolution. It takes time.

 

 

As far as the zoom/scale stuff we could already do stretch-to-fill on the Roku but who would want that? It makes people look fat. Nobody would want it. Abandoned the idea. Also because the Roku cannot scale outside the frame buffer I give up on the idea that the Roku can zoom-with-aspect-ratio too. That is where we are on that front...

The Roku most definitely can make the video player smaller and stash it into a corner on say, the Live TV guide, that is possible. But we felt it was better a full screen player with a mini-guide. Not a full-guide with a mini-player. It is possible to make a mini-player appear anywhere, at any resolution, at any scale on the screen while you browse and etc. All of this is possible.

Edited July 21, 2021 by speechles

[Smashpen 1]

Speech.

Thanks for your quick response.  I see.  So software resizing  to fill the screen with a crop is fairly easy.  I was thinking the new video has to be rendered into the number of pixels on the TV.  I don't think TV's have any of this custom resizing built in, so it has to be done by software in real time. I don't think you can just send a magnify command to the TV and get a certain zoom increase to be rendered by TV hardware.  So you are talking about rendering the zoom change with software bringing it up to 1920 x 1080 and then sending the new version to the TV in real time.  I know you can render a resize like this, but that is not in real time.  Maybe with a dedicated hardware line doubler it could be possible.  I don't know of any product that does this, because it would cause cropping of the original and the hardware processor is part of my estimate of cost idea.  

Regarding the estimate, you can dev a solution with a GUI interface, but it needs QA and revisions.  What would a sprint story points be on something like this.  If a dev suggested that it's a 3 day job, then QA plus revisions, could easily make it much longer and everyone's hours are paid in a payroll for profit company.  Volunteer work is, of course different.  I was referring to that the project manager would look at design effort, dev time, QA, testing revision and Customer service feedback and everyone who touches it, if they are on the payroll get paid.  So that combination is the cost.  The end result is to help how many people who want to watch content cropped?

I think more useful would be to use it to fix aspect ratio problems, but, like I mentioned above, the only problem aspect ratio are amateur or pirate illegal videos.

Product manger make the decision of where to spend team hours.  I just cannot see this pixel zoom work that useful.  I would guess that for 500 users questioned, maybe 1 or 2 might play with it.  Also, as I mentioned, any outside review might easily mention that the image resolution suffers with this change and I would think no product manager would ever want those words in a review, even if the next sentence says the image remains high quality if you don't use this feature.

The blurring is the main problem... So if it's a 16:9 1080 TV.  And there is a 2.39 movie. the correct aspect ratio video height would be 804 (rounded up to an even number).  If you want to fill the screen, that 804 height needs to be rescanned by the TV to 1080 height which is a zoom in of about 25%.  So you would lose also 25% of the left and right part of the movie to make that happen..  You are spreading the height with a resolution of 804 to 1080, so fractions of each line of pixels are blended together. That's sounds awful.  We could never do that in post production for films or TV and no studio could release any product with that in the playback.

If you guys really want to make this, go ahead.  I think it's a bad idea.

[oihsdfhdif 1]

7 hours ago, Smashpen said:

...the only problem aspect ratio are amateur or pirate illegal videos.

This isn't early 2000s anymore....

It's a viewer's preference what the crop should be. The simplest solution is just scale the group as others have suggested, and expose the variable in the settings to the users can play around with their desired crop as needed. Don't need to even bother the devs to calculate the exact cropping factors based on video properties (unless they desire).

 

[Deathsquirrel 741]

9 hours ago, oihsdfhdif said:

This isn't early 2000s anymore....

It's a viewer's preference what the crop should be. The simplest solution is just scale the group as others have suggested, and expose the variable in the settings to the users can play around with their desired crop as needed. Don't need to even bother the devs to calculate the exact cropping factors based on video properties (unless they desire).

 

I was thinking something that started the same.  This isn't the early 2000s any more.... stop zooming shit like you have a 4:30 tube TV