Streaming quality

[Luke 37060]

I've added a streaming quality setting. It's a numeric scale from 1-5, although for those using api client there is an enum.

 

See api client for usage examples. If you omit the param you'll be left to the mercy of encoder defaults.

 

Scott L / Mark - Try playing with this to see if it helps alleviate the blocky video you've complained about.

 

These are my general guidelines:

 

1-2: Mobile device on 3g or lower

3-4: Mobile device on 4g or wifi hotspot

5: Any device within a user's local network

 

Server horsepower is also a factor, of course.

[Redshirt 1487]

Haven't played with this yet, but after getting my new phone today and playing... Video streaming is working great for me using webm and specifying the max height/width based on the device metrics. I couldn't have hoped for better.

[Redshirt 1487]

and I had to jinx it.

 

I'm seeing some interesting results with my high-res movies.

 

If I watch "Days of Thunder" it stutters quite a bit.

 

General
Unique ID                                : 47235021645236141341344866039511049953 (0x2389225EEE5DF3D5382197F9BE7AE6E1)
Complete name                            : \\GALACTICA\Movies\All Movies\Days of Thunder (1990)\Days of Thunder (1990).mkv
Format                                   : Matroska
Format version                           : Version 2
File size                                : 1.91 GiB
Duration                                 : 1h 47mn
Overall bit rate                         : 2 543 Kbps
Encoded date                             : UTC 2009-05-08 22:25:44
Writing application                      : Lavf54.11.100
Writing library                          : Lavf54.11.100

Video
ID                                       : 1
Format                                   : AVC
Format/Info                              : Advanced Video Codec
Format profile                           : High@L4.1
Format settings, CABAC                   : Yes
Format settings, ReFrames                : 4 frames
Codec ID                                 : V_MPEG4/ISO/AVC
Duration                                 : 1h 47mn
Nominal bit rate                         : 2 133 Kbps
Width                                    : 1 280 pixels
Height                                   : 528 pixels
Display aspect ratio                     : 2.40:1
Frame rate                               : 23.976 fps
Color space                              : YUV
Chroma subsampling                       : 4:2:0
Bit depth                                : 8 bits
Scan type                                : Progressive
Bits/(Pixel*Frame)                       : 0.132
Writing library                          : x264 core 66 r1115bm 11863ac
Encoding settings                        : cabac=1 / ref=4 / deblock=1:-3:0 / analyse=0x3:0x133 / me=umh / subme=7 / psy_rd=0.0:0.0 / mixed_ref=1 / me_range=32 / chroma_me=1 / trellis=2 / 8x8dct=1 / cqm=2 / deadzone=21,11 / chroma_qp_offset=0 / threads=2 / thread_queue=2 / nr=0 / decimate=1 / mbaff=0 / bframes=3 / b_pyramid=1 / b_adapt=2 / b_bias=0 / direct=3 / wpredb=1 / keyint=250 / keyint_min=25 / scenecut=40(pre) / rc=2pass / bitrate=2133 / ratetol=1.0 / qcomp=0.60 / qpmin=10 / qpmax=51 / qpstep=4 / cplxblur=20.0 / qblur=0.5 / ip_ratio=1.40 / pb_ratio=1.30 / aq=1:1.00:0:10.00
Default                                  : Yes
Forced                                   : No
Encoded date                             : UTC 2009-05-08 22:25:44
HANDLER_NAME                             : GPAC ISO Video Handler

Audio
ID                                       : 2
Format                                   : AAC
Format/Info                              : Advanced Audio Codec
Format profile                           : LC
Codec ID                                 : A_AAC
Duration                                 : 1h 47mn
Channel(s)                               : 6 channels
Channel positions                        : Front: L C R, Side: L R, LFE
Sampling rate                            : 48.0 KHz
Compression mode                         : Lossy
Delay relative to video                  : 1mn 5s
Language                                 : English
Default                                  : Yes
Forced                                   : No
Encoded date                             : UTC 2009-05-08 22:30:04
HANDLER_NAME                             : 5.1

 

However if I watch "The Hunger Games" which is the same resolution, and a higher bitrate... It doesn't stutter. The colors are a bit washed though.

 

General
Unique ID                                : 195103647634916756765558689898201025680 (0x92C798815CFC83151C15EF09A4330C90)
Complete name                            : \\GALACTICA\Movies\All Movies\The Hunger Games (2012)\The Hunger Games (2012).mkv
Format                                   : Matroska
Format version                           : Version 2
File size                                : 6.55 GiB
Duration                                 : 2h 24mn
Overall bit rate                         : 6 511 Kbps
Encoded date                             : UTC 2012-08-04 05:45:28
Writing application                      : mkvmerge v5.0.1 ('Es ist Sommer') built on Oct  9 2011 16:57:02
Writing library                          : libebml v1.2.2 + libmatroska v1.3.0

Video
ID                                       : 1
Format                                   : AVC
Format/Info                              : Advanced Video Codec
Format profile                           : High@L4.1
Format settings, CABAC                   : Yes
Format settings, ReFrames                : 5 frames
Muxing mode                              : Header stripping
Codec ID                                 : V_MPEG4/ISO/AVC
Duration                                 : 2h 24mn
Bit rate                                 : 5 004 Kbps
Width                                    : 1 280 pixels
Height                                   : 528 pixels
Display aspect ratio                     : 2.40:1
Frame rate                               : 23.976 fps
Color space                              : YUV
Chroma subsampling                       : 4:2:0
Bit depth                                : 8 bits
Scan type                                : Progressive
Bits/(Pixel*Frame)                       : 0.309
Stream size                              : 4.90 GiB (75%)
Writing library                          : x264 core 125 r2208 d9d2288
Encoding settings                        : cabac=1 / ref=5 / deblock=1:0:0 / analyse=0x3:0x133 / me=umh / subme=7 / psy=1 / psy_rd=1.00:0.00 / mixed_ref=1 / me_range=16 / chroma_me=1 / trellis=1 / 8x8dct=1 / cqm=0 / deadzone=21,11 / fast_pskip=0 / chroma_qp_offset=-2 / threads=12 / lookahead_threads=2 / sliced_threads=0 / nr=0 / decimate=1 / interlaced=0 / bluray_compat=0 / constrained_intra=0 / bframes=3 / b_pyramid=2 / b_adapt=1 / b_bias=0 / direct=1 / weightb=1 / open_gop=0 / weightp=2 / keyint=250 / keyint_min=23 / scenecut=40 / intra_refresh=0 / rc_lookahead=40 / rc=2pass / mbtree=1 / bitrate=5004 / ratetol=1.0 / qcomp=0.60 / qpmin=0 / qpmax=69 / qpstep=4 / cplxblur=20.0 / qblur=0.5 / ip_ratio=1.40 / aq=1:1.00
Language                                 : English
Default                                  : Yes
Forced                                   : No

Audio
ID                                       : 2
Format                                   : DTS
Format/Info                              : Digital Theater Systems
Muxing mode                              : Header stripping
Codec ID                                 : A_DTS
Duration                                 : 2h 24mn
Bit rate mode                            : Constant
Bit rate                                 : 1 510 Kbps
Channel(s)                               : 6 channels
Channel positions                        : Front: L C R, Side: L R, LFE
Sampling rate                            : 48.0 KHz
Bit depth                                : 24 bits
Compression mode                         : Lossy
Stream size                              : 1.52 GiB (23%)
Default                                  : Yes
Forced                                   : No

 

All my standard definition media plays great.

[Tikuf 663]

I have just noticed this with some 1080p and 720p movies I have and I'm viewing through the localhost (with chrome). Not so much a stutter but it just stops for a bit then continues on. Not sure what is happening but ffmpeg is only using 15% processor and the box I'm on is more than capable of handling it . I am going to reconstruct some of the c/l and try to see where the bottle neck is. I can reconstruct from the videohandler.cs unless you have the c/l handy Luke?

[Luke 37060]

When you play using the video handler, check the server log file and you'll see the command line that was used.

[Luke 37060]

Mark, that is strange because they both look very similar. Have you tried fiddling with the new quality param I added?

[Tikuf 663]

When you play using the video handler, check the server log file and you'll see the command line that was used

Excellent gives me a good start. Almost bed time for me.

 

On a quick look its not encoding fast enough @about 15 fps what is weird though it doesn't appear to be using the -thread 0 cpu is only at 15%

now if i pump the thread count to 8 it uses 56% cpu but it still doesnt increase the fps to a fast enough rate (@ about 20 fps)

 

I will look at it more tomorrow.

[Luke 37060]

Actually, not only will there be the command line, but the entire ffmpeg log will be dumped to s separate log file too.

[Tikuf 663]

Hmm there are 2 ffmpeg logs does that mean that it is encoding it twice? They start at the same time the fps starts off at at good rate until the second drops into the mix.

 

 

01:14:24.356, Info, Http Server received request at: http://localhost:8096/mediabrowser/api/video?id=efb7cea19fda7db453409cfdf78cc6da&outputformats=webm&quality=5
01:14:24.356, Info, Http Headers: Connection=keep-alive,Accept=text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8,Accept-Charset=ISO-8859-1,utf-8;q=0.7,*;q=0.3,Accept-Encoding=gzip,deflate,sdch,Accept-Language=en-US,en;q=0.8,Host=localhost:8096,User-Agent=Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.4 (KHTML, like Gecko) Chrome/22.0.1229.79 Safari/537.4
01:14:24.361, Info, Responding with status code 200 for url http://localhost:8096/mediabrowser/api/video?id=efb7cea19fda7db453409cfdf78cc6da&outputformats=webm&quality=5
01:14:24.368, Info, O:\MB3 Source Code\Mb Server Mb Ui\MB3\Server\Server\ProgramData-Server\MediaTools\ffmpeg.exe -i H:\Bluray\THIS_MEANS_WAR\bdmv\stream\00000.M2TS -threads 0 -vcodec libvpx -b:v 1200000 -acodec libvorbis -ar 44100 -ab 320000 -f webm -
01:14:24.911, Info, Http Server received request at: http://localhost:8096/mediabrowser/api/video?id=efb7cea19fda7db453409cfdf78cc6da&outputformats=webm&quality=5
01:14:24.912, Info, Http Headers: Connection=keep-alive,Accept=*/*,Accept-Charset=ISO-8859-1,utf-8;q=0.7,*;q=0.3,Accept-Encoding=identity;q=1, *;q=0,Accept-Language=en-US,en;q=0.8,Host=localhost:8096,Referer=http://localhost:8096/mediabrowser/api/video?id=efb7cea19fda7db453409cfdf78cc6da&outputformats=webm&quality=5,Range=bytes=0-,User-Agent=Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.4 (KHTML, like Gecko) Chrome/22.0.1229.79 Safari/537.4
01:14:24.912, Info, Responding with status code 200 for url http://localhost:8096/mediabrowser/api/video?id=efb7cea19fda7db453409cfdf78cc6da&outputformats=webm&quality=5
01:14:24.912, Info, O:\MB3 Source Code\Mb Server Mb Ui\MB3\Server\Server\ProgramData-Server\MediaTools\ffmpeg.exe -i H:\Bluray\THIS_MEANS_WAR\bdmv\stream\00000.M2TS -threads 0 -vcodec libvpx -b:v 1200000 -acodec libvorbis -ar 44100 -ab 320000 -f webm -

[Luke 37060]

It's normal, but you should see one of those requests get aborted pretty quickly followed by a big long error message.

 

Basically, the client first sends a request for a progressive download.

 

Then while this is going on it sends a byte range request to see if we're capable of sending back certain sections of the file. If we indicate yes, then it will send a third request to get the last few hundred bytes of the file which is where the time code information is stored. This allows seeking. But when we encode with ffmpeg we can't handle byte range requests, so we just start another progressive download from the beginning. The player will see this and abort the second request.

[Redshirt 1487]

I've revisited this trying all the available quality settings. I don't see a difference between 5 or 4. 3 is a little better and 1 lets the video play for 20-30 seconds before it starts to hiccup. My system is also showing very little cpu usage for ffmpeg, so I'm not sure where the bottleneck is.

[Tikuf 663]

Ok i have done a fair bit of testing on this today. Finding a happy balance proved to be a bit difficult. I used and undoctored m2ts stream because I figure it was probably the highest res that will be used as an input.

 

I ended up with some preset files that are contained in this zip https://dl.dropbox.com/u/1349444/libvpx-presets.zip

 

The command lines i used were

 

ffmpeg -i H:\Mediabrowser3\MEDIA\THIS_MEANS_WAR\BDMV\STREAM\00602.m2ts -codec:v libvpx -fpre libvpx-1.ffpreset -vf scale=1920:-1 -threads 0 -codec:a libvorbis -ac 2 -b:a 128k -f webm 1.webm
ffmpeg -i H:\Mediabrowser3\MEDIA\THIS_MEANS_WAR\BDMV\STREAM\00602.m2ts -codec:v libvpx -fpre libvpx-2.ffpreset -vf scale=1280:-1 -threads 0 -codec:a libvorbis -ac 2 -b:a 128k -f webm 2.webm
ffmpeg -i H:\Mediabrowser3\MEDIA\THIS_MEANS_WAR\BDMV\STREAM\00602.m2ts -codec:v libvpx -fpre libvpx-3.ffpreset -vf scale=1280:-1 -threads 0 -codec:a libvorbis -ac 2 -b:a 128k -f webm 3.webm
ffmpeg -i H:\Mediabrowser3\MEDIA\THIS_MEANS_WAR\BDMV\STREAM\00602.m2ts -codec:v libvpx -fpre libvpx-4.ffpreset -vf scale=640:-1 -threads 0 -codec:a libvorbis -ac 2 -b:a 128k -f webm 4.webm
ffmpeg -i H:\Mediabrowser3\MEDIA\THIS_MEANS_WAR\BDMV\STREAM\00602.m2ts -codec:v libvpx -fpre libvpx-5.ffpreset -vf scale=640:-1 -threads 0 -codec:a libvorbis -ac 2 -b:a 128k -f webm 5.webm

 

The bitrates contained in the presets may need a bit of tweaking but I can comfortably stream 5 through a 3g connection from my server. Ymmv.

 

I have downgraded all the audio to 2 ch since I figured if you are using webm you prob don't have 6ch surround the bitrates can be downgraded /upgraded a bit to suit outputs (although downgrading them gave me an extra boost with the conversion frame rate

 

As far as the -vf scale is concerned they represent the highest resolutions that the video is watchable at and the transcoding frame rate is high enough to produce a fps conversion rate of over 24 frames per second.

In order to use these settings some extra logic will have to be produced in the videohandler to use the original width and to only down size the video never up size (the first number in the vf scale is the max that the output should be, the -1 says keep the height relative to the width.

 

For example if the input is only 640 wide and preset 1 is used then the 1920 should be replaced with the 640.

If the input width is 1920 and preset 2 is used then the 1920 should be replaced by the 1280.

Not sure I have explained it properly but I will detail it further if required.

[Luke 37060]

I'll get to this later this week.

 

In the meantime, Mark, are you using the audio channels param? If not then you should to make sure you get back a maximum of two channels. It might help decrease the amount of data coming over the wire.

[Redshirt 1487]

No worries. I've got enough other stuff to do, playback quality isn't holding me back.

 

In regards to what I'm requesting from the server. An example would be

 

http://192.168.0.13:8096/mediabrowser/api/video?id=blah&outputformats=Webm,Mp4,Mkv&audiochannels=2&maxWidth=1920&maxHeight=1080&quality=2

[Luke 37060]

Have you tried setting the max resolution values to 720p? I have really fast hardware, but even when I test locally it struggles at 1080p. Granted, I think a lot can be done with the parameters I'm using to optimize the process, but in general I think 1080p streaming is going to be tough.

[Tikuf 663]

1080p is doable i have tested it on my lan with the preset 1 i think i may be able to squeeze a bit more out of it bitrate wise (so its higher)

[Redshirt 1487]

well after googling my own hardware specs :oops: I've only had the phone 2 days. I'm actually sending 1280x720 as the max res. I'm poling the device metrics, not specifing hard coded resolutions.

[Tikuf 663]

i just got 1080p streaming across the lan while converting it @10000k/bits/s its converting at 24 fps

 

this is the preset i used

 

vcodec=libvpx

g=120
lag-in-frames=16
deadline=realtime
cpu-used=1
vprofile=1
qmax=25
qmin=17
slices=4
b=10M

[Luke 37060]

Can you tell me how many of these parameters are webm specific and how many can be used for any output format?

[Tikuf 663]

Can you tell me how many of these parameters are webm specific and how many can be used for any output format?

 

I will have single line command lines for you in the morning mate (its 2am). I have paired them down a heap and have done some testing (some means buckets of)

 

I have come up with 6 diff bitrates all convert with fps to spare.

 

And will be easy to convert into the existing code.

 

If you are using chrome or firefox some of the results are here

profile 1 4000Kb/s

http://www.nyadev.com/webm/2-bd.webm 2000Kb/s

http://www.nyadev.com/webm/3-bd.webm 1200Lb/s

http://www.nyadev.com/webm/4-bd.webm 800Kb/s

http://www.nyadev.com/webm/5-bd.webm 280Kb/s

http://www.nyadev.com/webm/6-bd.webm 150Kb/s

 

I can get more out of all of them (converting speed wise)

Ds can play all of them via the net with no pause

Red and I max out on number 3 (Im on nextg)

 

 

Edit all the bits in the preset are webm specific except the b (bitrate)

[Luke 37060]

Ok. I've removed ogv as a supported encoding format because I doubt anyone's going to use it and it's just going to cause us more work.

 

So we really only have to worry about webm, asf and ts.

[Tikuf 663]

Yup happy with that .

Did you have a chance to look at those webm files if so how did they look to you?

Any suggestions re bitrate size?

[Redshirt 1487]

Tikuf. Based on Lukes comments in the original post as well as our testing and real-world specs. I think you should aim for the following bitrate maximums for each Quality level...

 

Q 1 & 2 = 3G, Q 3 & 4 = 4G

 

Q 1: .15M/ps

Q 2: .25M/ps

Q 3: .35M/ps

Q 4: .45M/ps

Q 5: 1.9M/ps

[Tikuf 663]

Would we be better served having 6 settings rather than 5.

 

2 for 3g

2 for 4g

2 for lan (wifi and wired).

 

I would really like to see at least 2 hq settings.

[Luke 37060]

We can always do that but that's a separate issue. Adding another hq setting isn't going to help mark stream over 3g with his internet connection.