An observation about recent unexpected transcoding

[raven-au 17]

Hi all,

 

I noticed a problem just recently when a 480p SD show (originally from one of my DVDs) was unexpectedly transcoded.

 

First, if I start playback and exit out soon after (say less than a minute) the show gets marked as watched which isn't right (seen on Android TV client v1.2.1 and v1.2.2).

 

After investigating I found that a lot of these shows where missing the "Media Info" section, they had the video path info only.

 

The lack of Video Info" might be the cause of them getting marked as watched incorrectly, I can't tell, it was enough to cause it to transcode.

 

As and aside, it still puzzles me why transcoding gets it wrong when the storage aspect ratio doesn't match the presentation aspect ratio such as is the case with this video. It is stored as 720x480 with a presentation aspect ration of 16:9 (actually Handbrake says presentation dimensions are 853x480, 15.99:1).

 

I get that the video not having the Media Info is a problem but in the recent past I've seen videos that do have the Media Info play at the wrong presentation dimensions.

 

What I'm struggling to understand is, given that ffmpeg is used to do the transcode and there is sufficient info to construct ffmpeg parameters to transcode the video to display it as it is intended, why are they transcoded incorrectly?

 

If you know the player being used won't or can't use (or doesn't know) the presentation aspect ratio to display it properly why doesn't the transcode do this job?

Isn't transcoding meant to accommodate limitations of the player on the playback device, then why not this one?

Edited June 5, 2016 by raven-au

[Luke 42077]

Hi, we can't really answer that without discussing a specific example. I would refresh it from the detail screen to bring back the media info, then we can look at the transcoding logs and go from there. Thanks.

[raven-au 17]

Hi, we can't really answer that without discussing a specific example. I would refresh it from the detail screen to bring back the media info, then we can look at the transcoding logs and go from there. Thanks.

 

Yes, refreshing the shows does get the needed information.

 

So now they don't transcode so I can't provide a transcoding log.

 

I might be able to fish out an occurrence of it but there was no Media info so I'm guessing the trancode result will be pretty much undefined so probably not useful. 

 

What I have described above is an observation, more for information to yourselves, since the transcoding doesn't occur once the server has the Media info.

 

The situation is much the same for the question about the incorrect aspect ration on transcode but it still stands as that behaviour has been seen in earlier versions of the Android TV app.

 

I don't remember seeing this in the tvOS app (which used to trancode pretty much everthing) but if I do see it there or if I see the Android TV app trancode when it shouldn't and I see the incorrect aspect ration problem again I'll send a debug log.

 

I don't see how that will help though as the incorrect aspect ratio on transcode has been see by others for a fairly short while and providing logs didn't result in any meaningful feedback but did result in the addition of an option to stretch the video. That option still resulted in an ugly (still a little squashed) image for me.

 

Maybe re-read what I wrote, the question is why does trancoding not scale videos that have a different aspect ratio to the storage dimensions when the player it is being sent to is known to be unable to handle this (I'm assuming you can get hold of target app information, maybe you can't)?

 

Don't get me wrong, the current Android TV release has resolved many problems I have seen over time and is working quite well for me, however I thought I'd mention these thoughts because it's worth you guys being aware of it, maybe you'll spot something while working on other things because of it, who knows.

 

Ian

[ebr 16169]

Maybe re-read what I wrote, the question is why does trancoding not scale videos that have a different aspect ratio to the storage dimensions when the player it is being sent to is known to be unable to handle this (I'm assuming you can get hold of target app information, maybe you can't)?

 

Because we don't know that.  The item is being transcoded properly.  The problem is with the Google player on the client end.  So that is where the correction needs to be made.

 

As for why your items transcoded in the first place, that is because, if we have no information about the format of the item we have to transcode it to a format we know will play.  So, the whole issue there was the lack of media information we needed to make a proper playback decision.

[raven-au 17]

Because we don't know that.  The item is being transcoded properly.  The problem is with the Google player on the client end.  So that is where the correction needs to be made.

 

So the server doesn't know what client has requested this or is it that the server doesn't know if the client will use a player that's not capable of it, such as whether the client will use the VLC library or other player?

 

Assuming the later that would be something that's needed in the protocol between the server and client.

I still can't see why the protocol between the server and client can't be enhanced but there's no detail been given about how that works so I don't understand the problem.

 

As for why your items transcoded in the first place, that is because, if we have no information about the format of the item we have to transcode it to a format we know will play.  So, the whole issue there was the lack of media information we needed to make a proper playback decision.

 

I get that and I understand how that case results in best effort, essentially undefined behaviour.

 

Ian

[Luke 42077]

the client apps do report everything they support to the server. what is the specific issue you're reporting here?

[raven-au 17]

the client apps do report everything they support to the server. what is the specific issue you're reporting here?

 

I'm asking a question.

 

When the server knows that a client is going to use a player that won't play a title using the correct display dimensions why doesn't it just use the ffmpeg scale video filter.

I know there are other considerations like whether to transcode or not but a 16:9 show squashed to 4:3 isn't really watchable.

I'm assuming the scale filter will behave this way but I haven't tested this case, maybe I'm wrong about how the scale filter works so that does need to be checked.

If the scale video filter doesn't actually do it this way then there may be another filter.

 

Anyway, and @@ebr, please correct me if what I'm saying isn't right.

 

Recently (for a time when libVLC wasn't being used) we saw the Android TV client play titles with an aspect ratio of 4:3 because the storage dimensions were 4:3 but the video header said it was 16:9 and we ended up with a squashed picture.

 

That resulted in an option in the Android TV client to stretch the video in various ways but, for me, didn't quite get a show that looked OK.

 

IIRC (and the point of what I'm saying) the squashed shows were being transcoded so I'm suggesting that it should be possible to identify when the client will have this problem and since the show is already going to be trancoded why not also use the ffmpeg scale video filter to send a result that has the correct display dimensions.

 

As I say I could be wrong about the filter, there may be a different one or there might not be any, that's something I'll need to check if you think this suggestion is useful.

 

Ian

Edited June 6, 2016 by raven-au

[ebr 16169]

I'm asking a question.

 

When the server knows that a client is going to use a player that won't play a title using the correct display dimensions why doesn't it just use the ffmpeg scale video filter.

 

As I tried to say before, we don't know that.  It is only certain videos that exhibit this problem and only when played with the Google player.  Therefore, the proper place to handle this was on the client end with that player.  So, the option to stretch within the interface was added.  Also, this problem would occur whether transcoded or direct streamed so the server may not even be involved.

 

If we can figure out exactly what conditions cause this problem within that player, we might be able to automatically set the zoom in the interface.  Beyond that, we hope that updates to the Google player will solve the issue altogether.

[raven-au 17]

As I tried to say before, we don't know that.  It is only certain videos that exhibit this problem and only when played with the Google player.  Therefore, the proper place to handle this was on the client end with that player.  So, the option to stretch within the interface was added.  Also, this problem would occur whether transcoded or direct streamed so the server may not even be involved.

 

If we can figure out exactly what conditions cause this problem within that player, we might be able to automatically set the zoom in the interface.  Beyond that, we hope that updates to the Google player will solve the issue altogether.

 

Oh well, you get that I guess .....