Q on format and quality for video files and tools to get there

[rliepins 9]

Hi all

 

I am rehashing the age old topic of what media type to use and how to do it (quality and tool).  I've looked over past posts and have found most people use MP4 or MKV  using H264.  But I've slowly seen more and more people using H265.  I'd love to know the difference and why one would go that route.  

 

I have a bunch of DVD and BluRay rips and have converted some to MKV, some to MP4 and some to simply AVI.  Qualtiy is as wide as the number of tools out there.  Vidcoder works great for batches.  Handbrake has more fine tuned control.  I've used Video Redo for special case editing.  I'm unsure what to use to get an H265 format, however.

 

I'm at the point where I want to standardize my library and rework the file structure.  Disk space isn't an issue, but I'd like to use a common format (and tool for that matter).  File size is not an issue as I have plenty of disk space so I want to make sure the quality is as high as can be.  If I start going H265, am I shoehorning myself?  I use Emby for my media, but often take the files with me on long trips on the ipad.  I still use WMC with Emby as the movie server.  The extender has had issues in the past with some file formats, but I do need to rip to files rather than use DVD folder structure.

 

So what is the current recommended format for converting DVDs and BluRays?  What tools do you use and and why do you choose that?  What settingds do you use to get there?  Again, I know this is a rehash, but I'd like to see what people are doing these days.

 

Thanks again.  Emby rocks.

 

 

[ryandavidg 30]

I'm sure you'll get plenty of different opinions posted here when it comes to tools and setups, but the difference when it comes to H265 over H264 is it is a much more efficient method of encoding, so you get more quality for the same size / or smaller size for the same quality. The drawbacks are not all devices support it (or can handle it as it requires more processing power) and encoding also takes longer. But then you gain in needing less storage space and streaming bandwidth. I was going to say it's the future, because it is becoming more widespread, but there have also been rumblings about various companies not being keen on the royalty payments/patents and developing their own alternatives, but I haven't kept up to date with all those plans and notice that Google Chromecast Ultra supports H265 so it does seem to be gaining ground. Personally, I'm using it more and more and no longer rip to H264.

 

If you want the technical details on H265 and how it works, pm me and I'll link you some in-depth articles, but they are obviously not needed in order to use it.

 

In terms of software, most encoding applications support H265 from what I've seen, including Handbrake which you've used. Take a look at Staxrip. If you're going to start encoding in H265 check out some of the discussions on the forums for these applications for a starter setup, then when you're more experienced you can finetune according to what you want. It can get a bit technical if you want the best possible efficiency/results, but one can always throw a little bit more bitrate at it as you mentioned you aren't storage constrained.

 

edit: Forgot to emphasise, check your devices and see if they support H265. Test a couple of H265 files on them if they are a bit dated or underpowered.This may decide the direction you should go in right there.

Edited November 19, 2017 by ryandavidg

[Marc_G 95]

This is a topic I'm interested in, and revisit about twice a year now.

 

For me, I'm sticking with H264 for the foreseeable future. Here's why:

 

My primary TV watching device is a 1080p TV, not a newer 4k. I've seen 4k TVs, but at my seating distance I have no need for it. Until my TV dies, I'm 1080p. When my last-of-its-breed plasma dies, it will probably be replaced by a 4k because that's pretty much all that's out there for quality TVs these days, but that doesn't mean that I want to start shoving larger bitstreams at it. From 12 feet, a 60" 1080p TV is already pushing the limits of what the retina can resolve.

 

So, given that I've got no need for higher res video than 1080p, is the efficiency difference of H265 worth converting from H264? For me, no, because I have plenty of space and most of my media is stored short term (rent, rip, watch, delete). The media I own outright and keep, I don't particularly care to spend any effort to shrink.  If I were to watch it outside my local network I'd be transcoding down anyway, so H265 probably wouldn't help there.

 

Every device I have is fine with H264 and I'm in a "it just works" mode. Not looking to push the envelope. The compatibility issue will eventually go away I expect, but I'm not going to fool with it.

 

I'll look at this again next summer, and Xmastime in a year from now, but I doubt I will plan any changes anytime soon.

 

Good luck!

 

Marc

[Gilgamesh_48 1240]

@@ryandavidg is quite correct in all that he says. However there is an additional factor that should be considered and that is that storage has become quite cheap and very reliable. You do not need an expensive NAS for your drives as a regular computer with a USB hub or two or even more can support a virtual ton of drives. My server, that also runs EMBY, has 12 external USB drives totaling 50TB and using StableBit's DrivePool (Windows 10) to pool them all into one big drive for holding my media. That setup has been running for over 4 years with only two drive failures and zero data loss from the failures and one of the "failures" was due to me dropping a drive when rebuilding the rack that holds my server.

 

Having said that I keep almost all my media in MP4 format and I use HandBreak using their "Fast 1080p" preset to produce the files. I know there are "better" settings but my old eyes do not see the difference so I figure "why bother."

 

I had some old AVIs but I have converted all of those to MP4s so now all I have is MP4s and a few MKVs that I have not converted. I see no reason to use anything other than H264 for anything in the future. Storage is cheap and all my playback devices support that format.

 

I also should add that I keep all my audio in stereo because I find that anything "above" Dolby 2.1 is wasted on my old ears. (They are as old as my eyes)

 

The bottom line is find out what the limits are of your perceptions and do not bother with video or audio beyond what you can hear or see. There are many resolution snobs that insist they hear or see differences that they really don't so do not fall into the trap that many do of thinking they really need "better" because "better" is subjective and be sure you really need "better" because there really is no need for "better" differences that you can neither see or hear.

 

Just my two cents. Make your own choices.

[ryandavidg 30]

Had another thought on your statement that storage is not an issue but you want the best quality possible. To get away from the DVD folder structure remember that you don't have to reencode at all, just use something like MakeMkv which will make exact copies of the video streams on the disc but in an MKV container. Most people don't use this approach as they want to save a bit of space rather than store full BluRays, but as Gilgamesh_48 said storage is getting cheaper all the time so you could always make 'lossless' rips and reencode at a later date if you decide to. I know of only one person who has built a library this way, but it's another alternative to consider if you truly want the best possible quality or don't want to commit to an encoding approach right now.

[rliepins 9]

I've thought about MKV and have used that on occasion.  However, my extender has issues at times and MP4 is better for it.  Thats probably a function of my existing codecs on the system.  Is there an easy way to convert MKV to MP4 without reencoding?

 

BTW, that you all for the responses.  I think this is an ever evolving topic to consider and its helpful to newbs and vetrans alike.

[ryandavidg 30]

It's easy to convert files between MKV and MP4, it's called 'remuxing' as you don't reencode the video/audio inside the container. Look online for some batch scripts using FFMpeg, or use something like AviDemux (leave video and audio as "copy", just change the file type you're saving as). Very quick due to the lack of reencoding.

 

Might be worth just checking this comparison if you run into any issues though, as MKV/Matroska is a more recent "open" standard and supports just about any type of video/audio/subtitles, whereas MP4 has more restrictions. Those tables for audio and subtitles towards the bottom are particularly relevant - at least MP4 supports DTS these days, but not PCM or Dolby TrueHD.

Edited November 19, 2017 by ryandavidg

[Waldonnis 148]

Lots of good advice above.  Great posts to all 

 

The bottom line is find out what the limits are of your perceptions and do not bother with video or audio beyond what you can hear or see. There are many resolution snobs that insist they hear or see differences that they really don't so do not fall into the trap that many do of thinking they really need "better" because "better" is subjective and be sure you really need "better" because there really is no need for "better" differences that you can neither see or hear.

 

Just my two cents. Make your own choices.

 

This is definitely the least-mentioned but most important advice anyone can give when it comes to encoding settings and quality.  For example, if you don't hear a difference or have the equipment to support things like Dolby Atmos or 7.1 sound, you'll save a ton of storage by just downmixing and re-encoding the sound tracks.  Same goes for video, really.  I know some people who are fine with ~2Mbit 1080p encodes, so all of my crowing about higher-bitrate sources doesn't (and shouldn't) matter to them.  It's all about what looks/sounds good to you in your viewing environment.  Of course, if you're planning on improving your home theatre setup, then take that into consideration as well.

 

Also, if you're always transcoding everything, then it may make more sense to just encode to something your playback devices will support if you don't want to tax your server with transcoding tasks.  For example, if you went HEVC for everything, but none of you devices actually support it, then you spend a lot of duplicate effort transcoding it every time rather than just sticking with something more compatible (usually h.264 and AAC or AC3 audio).  I know a few people that love keeping direct copies from their BDs, but they never really get to watch the files without having to transcode them anyway (usually due to bitrate or audio codec limitations of their playback equipment).  As was already said, storage is cheap, so it's really up to what you can live with quality-wise, what your playback devices support, and how capable your system is if you choose to rely on on-the-fly transcoding.

 

One other note is that manual hardware encoding is something worth bringing up, especially if you choose to go the HEVC route (if supported by your hardware).  I'm personally not satisfied by the limitations of the hardware implementations so far (won't expand on that here as it's more technical than is needed here), but I can't deny that they're significantly faster than software encoding and do a decent job.  Handbrake only supports QSV (Intel's), but Staxrip can do NVENC (nVidia's) encoding as well.  You won't get the same compression levels or as low of a bitrate as you would with software encoding, but the encoding time difference (~1hr or less vs. 12hrs+) may be worthwhile if you can't dedicate that kind of computer time for encoding...again, storage is cheap.

 

As for quality with HEVC or h.264, it's all relative and mostly subjective.  I can get good quality and/or compression out of either codec with each "beating" the other in certain circumstances. Don't get sucked in by the oft-repeated "same quality at half the bitrate/filesize" crap the HEVC fans will spout - it's not always true, and it's better to evaluate the merits on your own with your eyeballs and equipment.

 

Matroska (mkv) vs. mp4/m4v is mostly personal taste these days, although if you have an Apple device in the mix, stick with mp4 since they still refuse to support mkv files.  Both are only containers, each having their merits and shortcomings - I think there's still a comparison chart floating around on Wikipedia somewhere if you're curious.  I prefer Matroska, but mostly because I can edit metadata and track info much more easily, and it supports a few codecs that I use frequently that mp4 can't deal with.  There are tools to manipulate or edit each container type (mkvtoolnix and mp4box), and ffmpeg is a solid "all around" tool as well.

[JeremyFr79 228]

I think the most important thing to consider is this, what is the end device you're watching it on and what meets your needs best.  Every situation is different.  For me I have nearly 100TB of storage on my server, however I'm slowly switching to h.265 as well as 4k material.  My main end device is an 80" 4k display and a full Dolby Atmos/DTS-X setup so I strive for the best picture and sound that I can in my files.  Now if I was watching on a 40" 1080p with a soundbar it'd be a whole other matter.  h.265 if you have the hardware to play it back, or the server powerful enough to transcode it is worth it for the savings in storage space.  Luckily I have both so h.265 isn't an issue for me and if encoded right yields amazing picture quality with even better file sizes.  Audio is the one thing you can't compress so as someone else mentioned if you don't need Atmos or even TrueHD/DTS-HD then only use AC-3/DTS or even MP3/AAC if you're listening on a stereo only setup.  This will save you quite a bit in file size.  Atmos/DTS-X Soundtracks are gigabytes in size themselves not including the video.

 

In the end everything will move to 265 it's inevitable just like everything moved to 264 before it from MPEG2 and the likes.  So why not make the move if you can now instead of wishing you had done so a couple years down the road.

[mawazi 1]

The march of progress never stops. After HEVC & VP9 comes HEVC's successor & AV1. I imagine the progress is slowing down, they can't keep doubling the efficiency.

 

I now encode to VP9 and Opus in a Webm container. My main device for viewing is a Chromecast Ultra, but also an Android phone, Chrome browser, and audio through a Google Home. Why those codecs and container? They're very high quality and efficient, they're open and freely licensed, and they match well with the technology ecosystem that I use. HEVC's and VP9's efficiency gets more and more meaningful as the resolution of the videos being encoded goes up.

 

My re-encoding of Blu-ray H264 video and lossless audio to VP9 and Opus (with, of course, my particular settings) usually results in a webm file 50% or so the original file's size. Any loss of fidelity in the video is not apparent to me, especially when watching in real-world conditions (which is probably how you'll be watching, I imagine).

 

If file size is truly not an issue for you, then I'd say just use MakeMKV for your Blu-rays, which only re-muxes the original streams into a mkv file without re-encoding.

[Waldonnis 148]

In the end everything will move to 265 it's inevitable just like everything moved to 264 before it from MPEG2 and the likes.  So why not make the move if you can now instead of wishing you had done so a couple years down the road.

 

I disagree with this.  h.264 will still be in widespread use for a LONG time, especially for 720p and 1080p.  Heck, MPEG2 is still in widespread use in many regions, so I wouldn't expect h.264 to disappear any time soon either.  HEVC is the standard codec for UHD, but outside of that, it's not as widely used as you might think, mostly because it's just not cheap to do so (processing and licensing costs) and isn't nearly as widely supported quite yet.  "Less than legal" circles seem to have fallen in love with HEVC, but having inspected some of those encodes, very low bitrates are the norm and HEVC's efficiency often (but not always) yields a more watchable video at those bitrates than h.264 would.  Those folks are also (overly) DNR-happy, but mostly because it helps their quest for smaller files.  At higher bitrates, though, any "clear advantage" that HEVC has gets less clear.  HEVC is also much more demanding and time-consuming to encode, but that's another matter entirely (maybe "a couple years down the road" is the better option since we'll probably have more cores to throw at it by then, lol).

 

Also, if you need fine detail or grain retention at 1080p or lower, then you likely just erased any file size or bitrate edge HEVC had (and in some cases, the HEVC file will be larger than the h.264 version at similar PSNR/SSIM levels).  Every codec sucks at encoding noise/grain, as it's not something that can be predicted or compressed away - HEVC is no different in that regard.  Sometimes there's just no substitute for more bits regardless of what codec is used.  Anyone who's had to use the grain tune with x265 will attest to how quickly the bitrates/file sizes will balloon...

 

Long story short: these codecs are different, with no clear winner for every case.  At non-4k/8k resolutions, use the codec that all of your equipment supports at bitrates that are pleasing to your eyes; HEVC is the obvious choice for 4k/8k, but VP9 or maybe even h.264 may also be an option if supported (yes, h.264 does support 4k, but most hardware decoders don't).  If storage is a problem, target bitrates for predictable file sizes and use two-pass encoding or just buy more storage.  Also, don't be afraid to experiment with encoder settings since some sources are just problematic and require a bit more encoder tweaking to yield "watchable" results (read: less macroblocking/halos/etc).  Just be sure to use shorter clips for testing so you don't have to run 5hr-24hr encodes just to test a setting change.  If you were encoding for a disc, life would be so much simpler - having a pre-determined codec and hard bitrate cap makes encoding life easier... 

 

Encoding is a very deep subject and you're sure to get LOTS of opinions, but it's never as simple as "X is better and the future"...believe me, I wish it was.  It's also constantly changing, with modern encoders still being actively developed and new codecs/encoders on the horizon.  AV1 is the most prominent of these, but AOM still has some things to finalise before AV1 can be truly usable...and industry acceptance is still a big unknown.

 

Side note for mawazi: Interesting choice of VP9, but it makes sense given your environment.  I'd be interested to hear what settings you use with vpxenc in PMs if you feel like sharing.  I just don't have that much experience with VP9 and would like to see what others are doing with it outside of YouTube.

Edited November 20, 2017 by Waldonnis

[Deathsquirrel 745]

Just to add to @@Waldonnis post above, all those complications combined with disc space being cheap are why I just stick with the encoding from the disc as often as possible.  If the results play back without problem on my devices I don't reencode.  There absolutely is not a single universal encoder setting that produces pleasing results from all inputs so your option is to either fiddle around with re-encoding for practically every file or just keep what they already put out from the factory.

 

I used to reencode but it was just a massive time-suck and I'm too old for that

[Guest asrequested]

Just build a beast of a server to handle anything you throw at it

[Waldonnis 148]

Just build a beast of a server to handle anything you throw at it

 

Try using aomenc on a video on your new system.  The encoding speed will probably give you an excuse to start shopping for more hardware 

[Guest asrequested]

Try using aomenc on a video on your new system. The encoding speed will probably give you an excuse to start shopping for more hardware

Don't get me started lol

[mawazi 1]

Try using aomenc on a video on your new system.  The encoding speed will probably give you an excuse to start shopping for more hardware 

 

I've been meaning to give av1 a try, with the full realization that it is early days. Not sure if there's a way to play it other than to decode it to a lossless file first.

[Waldonnis 148]

I've been meaning to give av1 a try, with the full realization that it is early days. Not sure if there's a way to play it other than to decode it to a lossless file first.

 

You pretty much have to decode it with aomdec and pipe that to a player right now.  Last I heard, the codec spec hadn't been finalised, so nobody wants to add it to lav or players until it is.  Supposedly, they're in "crunch time" for finishing up, but I have no idea what the timeline is on that.  aomenc is a bit rough still, but getting better and I expect more optimisation to occur once the codec is finalised.  We'll also likely see a couple more encoders for AV1 pop up, although most will probably either be professional (non-free) or research projects.