Wanting to build NAS that can transcode. What hardware?

[xnappo 1593]

Yep - definitely building your own is best.  I used to know how to make those shared links work, but guess I don't anymore.

 

Try this? http://secure.newegg.com/WishList/PublicWishDetail.aspx?WishListNumber=28259986

 

Otherwise...:

Edited July 25, 2015 by xnappo

[famulor 1]

Yep - definitely building your own is best.  I used to know how to make those shared links work, but guess I don't anymore.

 

Try this? http://secure.newegg.com/WishList/PublicWishDetail.aspx?WishListNumber=28259986

 

Otherwise...:

Looks good   Would it be easier if i ran the client software on it aswell for my own needs or should i just build another machine for that? I guess the transcoding option is a "on/off" kinda thing so i cant say like my client machine transcodes by itself but the chromecasts needs to be transcoded for them? 

[xnappo 1593]

I have the server in my home office (also has a nice video card and 42" 4K TV for games ) - for the actually living room / home theater I would recommend NUCs, FireTV or Nexus Player.

 

xnappo

[JeremyFr79 228]

You could also do something like this and have a unit that would absolutely TROUNCE the system shown above and save a few bucks.  I bought my file server from these guys same setup but lower CPU's as I have a separate transcoding server.

 

And you will never worry about room for adding drives

 

http://www.ebay.com/itm/SUPERMICRO-846E1-R900B-X8DTE-F-2x-X5650-SIX-CORE-XEON-CPUS-16GB-MEM-24x-TRAYS-/201217587925?pt=LH_DefaultDomain_0&hash=item2ed980bad5

[legallink 187]

Looks good Would it be easier if i ran the client software on it aswell for my own needs or should i just build another machine for that? I guess the transcoding option is a "on/off" kinda thing so i cant say like my client machine transcodes by itself but the chromecasts needs to be transcoded for them?

I have done both on running the client software on the actual machine and running a separate box for TV interface. I have found that a roku wins in the WAF over say: a windows install with a harmony one remote and whatever user interface. Windows, despite what most say, is not as good a remote interface as those TV specific devices like a roku or a fire TV etc. there is lag. There is programming. Unless you spend several hundred on a good rf remote your experience will be less than great. Additionally, depending on the box, whether windows or Linux, you'll have maintenance and therefore box downtime (updates, drivers, etc.). A fire TV or a roku has none of those issues really.

 

That being said, the prettiest interfaces are available on a nuc or on the media server as Xbmc and/or emby theater. And if the remote use is good enough for you, then you should be fine.

[famulor 1]

You could also do something like this and have a unit that would absolutely TROUNCE the system shown above and save a few bucks.  I bought my file server from these guys same setup but lower CPU's as I have a separate transcoding server.

 

And you will never worry about room for adding drives

 

http://www.ebay.com/itm/SUPERMICRO-846E1-R900B-X8DTE-F-2x-X5650-SIX-CORE-XEON-CPUS-16GB-MEM-24x-TRAYS-/201217587925?pt=LH_DefaultDomain_0&hash=item2ed980bad5

I live in Denmark so the tax and customs etc would be crazy on that thing but it looks really solid! and you mention seperate transcoding server. is that something i should look in to or wont it matter for my needs? 

 

I have the server in my home office (also has a nice video card and 42" 4K TV for games ) - for the actually living room / home theater I would recommend NUCs, FireTV or Nexus Player.

 

xnappo

I'll look for a NUC then. Always had a soft spot for them and never owned one

 

I have done both on running the client software on the actual machine and running a separate box for TV interface. I have found that a roku wins in the WAF over say: a windows install with a harmony one remote and whatever user interface. Windows, despite what most say, is not as good a remote interface as those TV specific devices like a roku or a fire TV etc. there is lag. There is programming. Unless you spend several hundred on a good rf remote your experience will be less than great. Additionally, depending on the box, whether windows or Linux, you'll have maintenance and therefore box downtime (updates, drivers, etc.). A fire TV or a roku has none of those issues really.

 

That being said, the prettiest interfaces are available on a nuc or on the media server as Xbmc and/or emby theater. And if the remote use is good enough for you, then you should be fine.

Sounds like a NUC is the right one for me then about the transcoding service: i guess i cant decide what devices my media server transcodes to? so it only transcodes for the chromecast and not the NUC?

 

 

btw im getting a 60mbit upload. Is that enough for 4 concurrent 1080p streams?

 

 

You guys are really really helpful! 

[JeremyFr79 228]

I live in Denmark so the tax and customs etc would be crazy on that thing but it looks really solid! and you mention seperate transcoding server. is that something i should look in to or wont it matter for my needs? 

 

I'll look for a NUC then. Always had a soft spot for them and never owned one

 

Sounds like a NUC is the right one for me then about the transcoding service: i guess i cant decide what devices my media server transcodes to? so it only transcodes for the chromecast and not the NUC?

 

 

btw im getting a 60mbit upload. Is that enough for 4 concurrent 1080p streams?

 

 

You guys are really really helpful! 

You don't necessarily need a dedicated transcoding server.  I have several servers in my rack and use it for a home lab among other things so having a dedicated Emby server just made sense to me.  But it's far from required. 

[legallink 187]

You don't necessarily need a dedicated transcoding server. I have several servers in my rack and use it for a home lab among other things so having a dedicated Emby server just made sense to me. But it's far from required.

 

Agreed. If you are getting a nuc, just use the nuc. No need for multiple transcoding capable machines.

 

However a nuc won't handle the storage issue unless it is external.

Edited July 29, 2015 by legallink

[famulor 1]

You don't necessarily need a dedicated transcoding server.  I have several servers in my rack and use it for a home lab among other things so having a dedicated Emby server just made sense to me.  But it's far from required. 

Alrighty. Will build a media server then. About the transcoding is it a on/off sorta thing or can i choose which clients should be transcoded to and which shouldnt? 

 

What hardware would i need for 4 concurrent 1080p steams at max?

 

Agreed. If you are getting a nuc, just use the nuc. No need for multiple transcoding capable machines.

 

However a nuc won't handle the storage issue unless it is external.

I'll get some external storage

[Deathsquirrel 741]

Alrighty. Will build a media server then. About the transcoding is it a on/off sorta thing or can i choose which clients should be transcoded to and which shouldnt? 

 

 

No, you can control what client devices you use and what formats you store on your server though.  If you buy hardware that can play the formats you store then there is no transcoding.  If you buy hardware that can't play the videos you make then there is.  Simple as that.

 

I rip my blu-rays to MP4 or MKV with a standard video and audio profile that works well with emby theater and emby classic.  Most any other client device will require transcoding of audio and sometimes video.  That's on me.  I made the files and I picked the devices.  The server has nothing to do with it.

[xnappo 1593]

Just a note that anything that runs Emby for Kodi (which is a lot of stuff) will play without transcoding.  The Emby for Android client is making great strides there too.

[famulor 1]

No, you can control what client devices you use and what formats you store on your server though.  If you buy hardware that can play the formats you store then there is no transcoding.  If you buy hardware that can't play the videos you make then there is.  Simple as that.

 

I rip my blu-rays to MP4 or MKV with a standard video and audio profile that works well with emby theater and emby classic.  Most any other client device will require transcoding of audio and sometimes video.  That's on me.  I made the files and I picked the devices.  The server has nothing to do with it.

Awesome. Seems like a good solution for me then what hardware would you recommend for my needs? (and as low power comsumption as possible )

 

Just a note that anything that runs Emby for Kodi (which is a lot of stuff) will play without transcoding.  The Emby for Android client is making great strides there too.

Im not totally sure what youre saying here. Do you mean that if something runs emby for kodi (which im thinking about doing) then its gonna be a bad experience? about the emby for android it will stream to chromecast from a mobile device (iOS or Android) so thats gonna be a bad experience as well?

[xnappo 1593]

No, I am saying anything that runs Kodi will play everything without transcoding.   Aka good experience. 

[famulor 1]

Ah cool. so what was the final thoughts of the hardware needed for this project?

[famulor 1]

Just a quick thought. Would it work with having all of the content on ex amazon cloud server?

[xnappo 1593]

I know people who do that..

[Kent 91]

I'm looking at putting together a new media server + NAS as well, and exploring the idea of FreeNAS + Emby plugin on the below hardware. I'm still discovering if this will allow 4-5 transcodes but I think it should.

 

I started a thread in the FreeBSD forum to see if the experts there can give some advice on the build.  Feedback from you guys is welcome as well!

 

FYI - Below build positions me for 15 total drives (using 3 FlexCages) but i'll start with one FlexCage and three drives for now.

 

• SUPERMICRO MBD-X10SLH-F-O uATX Server Motherboard LGA 1150 Intel C226 DDR3 1600 - $90
• Intel Xeon E3-1246 v3 Haswell 3.5GHz 8MB L3 Cache LGA 1150 84W Server Processor BX80646E31246V3 - $290
• Kingston 32GB (4 x 8GB) 240-Pin DDR3 SDRAM ECC Unbuffered DDR3 1600 Server Memory w/TS Model KVR16E11K4/32 - $240
• ICY DOCK FlexCage MB975SP-B 3.5" & 5.25" Black SATA Tray-less 5 x 3.5" HDD in 3 x 5.25" Bay SATA Cage - $90
• 3 X Seagate NAS 4TB ST4000VN000 Internal Hard Drives - $390
• Sharkoon T9 Value white Edition ATX (Case) - $100
• Antec EDGE 650W 80-PLUS Gold Power - $98
• + a 32GB thumb drive to run FreeNAS off of - $12

 

 

Total: $1,310 + tax

[famulor 1]

I know people who do that..

And its working fine? Do you have any more info on that?

[xnappo 1593]

Sorry - no I don't.  Hopefully someone who does that will chime in.

[legallink 187]

I'm looking at putting together a new media server + NAS as well, and exploring the idea of FreeNAS + Emby plugin on the below hardware. I'm still discovering if this will allow 4-5 transcodes but I think it should.

 

I started a thread in the FreeBSD forum to see if the experts there can give some advice on the build.  Feedback from you guys is welcome as well!

 

FYI - Below build positions me for 15 total drives (using 3 FlexCages) but i'll start with one FlexCage and three drives for now.

 

 

• SUPERMICRO MBD-X10SLH-F-O uATX Server Motherboard LGA 1150 Intel C226 DDR3 1600 - $90
• Intel Xeon E3-1246 v3 Haswell 3.5GHz 8MB L3 Cache LGA 1150 84W Server Processor BX80646E31246V3 - $290
• Kingston 32GB (4 x 8GB) 240-Pin DDR3 SDRAM ECC Unbuffered DDR3 1600 Server Memory w/TS Model KVR16E11K4/32 - $240
• ICY DOCK FlexCage MB975SP-B 3.5" & 5.25" Black SATA Tray-less 5 x 3.5" HDD in 3 x 5.25" Bay SATA Cage - $90
• 3 X Seagate NAS 4TB ST4000VN000 Internal Hard Drives - $390
• Sharkoon T9 Value white Edition ATX (Case) - $100
• Antec EDGE 650W 80-PLUS Gold Power - $98
• + a 32GB thumb drive to run FreeNAS off of - $12

 

 

Total: $1,310 + tax

That setup should do 4-5 transcodes. I have an i5 2500k, and I can get 3-4 transcodes out of it without a hiccup for the clients.

[famulor 1]

What if i were to build a media server that still needs to stream to 5-6 clients concurrent but the clients do the transcoding by themself? would it be cheaper or should i still go for the option to transcode to be on the safe side?

[NomadCF 15]

Here's my newest quick Setup ,I've got 3 cheap i5-3570K "work servers (windows)" , 2 storage servers (Linux) and my main server (Linux) in the house. The Setup works like this.

 

• All connections are over Gigabit and only use DNS for connections (more on that later)​
• The 3 "work servers" main job is just to pre "sync/transcode" their own type of content (one syncs only Anime, one Movies, TV shows) and save the converted to it's respective save location on one of the storage servers.
  • 1 nic for saving media streams
  • 1 for reading media/network traffic
  • Each "work server" also doubles as "steam host" for gaming.
  • Each runs under the global name of mediacenter.nomadcf.com & gamingstation.nomadcf.com along with an individual name of  mediacenter(1|2|3).nomadcf.com & gamingstation(1|2|3).nomadcf.com
  • Each client connects to one either though haproxy if remote OR by DNS name given to it by dnsmasq based on load/connections.
    • Each "work server" reports back to the main Linux server on what and who is connected to it. And removes that server ip from the global name pools. This eliminates two user connection to the same server for media or game streaming.  
• ​The 2 "storage" servers each are
  • Debian 9 boxes using raidz3
  • Have a usable 11T (6x MD04ACA400 hard drives) using RaidZ3,
  • Almost a mirrored pair as far as data goes. That is accomplished via a few rync scripts using  (rsync demons NOT over ssh).
  • Each server is a called global name of storage.nomadcf.com and then there respective name storage(1|2).nomadcf.com 
  • each runs Emby
    • only shows access to pre-transcoded content. 
    • Never to the RAW files (aka non-transcoded)
    • for lan devices Direct play only
  • All Lite streaming devices are configured to connect to these (Roku,fire sticks, etc)
  • The nics are broken down like this
    • 1 nic dedicated to talking with just the "workstations" for writing content to the servers.
    • 1 nic for talking to the lan
    • 1 nic for taking for streaming to the wan or lan (if there are no wan connections and the other lan nic has over 2 connections already)
  • All connections are done via UNC with DNS only ! no mapped drive for media content. (this avoids a mess of issues including croto virus). aka \\storage.nomadcf.com\Movies this also allows for easier direct play access.
  • All Media save locations (write able) shares are also hidden, and only allowed access from X ipaddress AND server name (where read only is open to any one).
•  The main Linux server is also my firewall/bridge & gateway.
  • One nic to the modem (in bridge mode) and the other 2 to the network.
  • For emby I use haproxy,DNS and dhcpd on the server.​​
    • Haproxy is used to make sure to connect each user connects to one of the storage servers that is up and has the least connections.
      • It load balances my remote clients across my storage severs
      • It handles a roll over in the event a server goes down.
      • it compresses and non compressed web traffic (html,js,etc)
      • It breaks up my connections by virtual host naming.
        • remote users connect VIA a url like http:\\streaming.nomadcf.com OR http:\\mediacenter.nomadcf.com
        • both run on same ip and port (80) and haproxy sends all the streaming.nomdcf.com to the storage severs EMBY or and all the mediacenter.nomadcf.com to the "workstations" for direct access to the raw content. Plus it separates my other port 80 website traffic to the web server (Again all on the same port & ip)
    • All server devices are given static DHCP address from a pool based on the devices name (pools contain 16 address each and are given a 1 month lease time)

 

It seems way more complicated when I see it written out, but really it's was extremely easy to setup (about 1 day and a few more to flush out some issues/scripts). And overall the cost was minimal minus the drives. Everything else was hobbled together from old machines and parts from over the years.

 

At the same time you should slim this all down to just 2 server, 1 that only does Direct play access and the other to pre-transcode all your content. In which case really BOTH servers hardware could virtual be anything. As the the work is being done pre them seeing/getting anything. And there is "little" while before any view-able content is available to any one. That is until server one has anything trans-coded and ready for server 2 to see. 

[NomadCF 15]

Here's my newest quick Setup ,I've got 3 cheap i5-3570K "work servers (windows)" , 2 storage servers (Linux) and my main server (Linux) in the house. The Setup works like this.

 

• All connections are over Gigabit and only use DNS for connections (more on that later)​
• The 3 "work servers" main job is just to pre "sync/transcode" their own type of content (one syncs only Anime, one Movies, TV shows) and save the converted to it's respective save location on one of the storage servers.
  • 1 nic for saving media streams
  • 1 for reading media/network traffic
  • Each "work server" also doubles as "steam host" for gaming.
  • Each runs under the global name of mediacenter.nomadcf.com & gamingstation.nomadcf.com along with an individual name of  mediacenter(1|2|3).nomadcf.com & gamingstation(1|2|3).nomadcf.com
  • Each client connects to one either though haproxy if remote OR by DNS name given to it by dnsmasq based on load/connections.
    • Each "work server" reports back to the main Linux server on what and who is connected to it. And removes that server ip from the global name pools. This eliminates two user connection to the same server for media or game streaming.  
• ​The 2 "storage" servers each are
  • Debian 9 boxes using raidz3
  • Have a usable 11T (6x MD04ACA400 hard drives) using RaidZ3,
  • Almost a mirrored pair as far as data goes. That is accomplished via a few rync scripts using  (rsync demons NOT over ssh).
  • Each server is a called global name of storage.nomadcf.com and then there respective name storage(1|2).nomadcf.com 
  • each runs Emby
    • only shows access to pre-transcoded content. 
    • Never to the RAW files (aka non-transcoded)
    • for lan devices Direct play only
  • All Lite streaming devices are configured to connect to these (Roku,fire sticks, etc)
  • The nics are broken down like this
    • 1 nic dedicated to talking with just the "workstations" for writing content to the servers.
    • 1 nic for talking to the lan
    • 1 nic for taking for streaming to the wan or lan (if there are no wan connections and the other lan nic has over 2 connections already)
  • All connections are done via UNC with DNS only ! no mapped drive for media content. (this avoids a mess of issues including croto virus). aka \\storage.nomadcf.com\Movies this also allows for easier direct play access.
  • All Media save locations (write able) shares are also hidden, and only allowed access from X ipaddress AND server name (where read only is open to any one).
•  The main Linux server is also my firewall/bridge & gateway.
  • One nic to the modem (in bridge mode) and the other 2 to the network.
  • For emby I use haproxy,DNS and dhcpd on the server.​​
    • Haproxy is used to make sure to connect each user connects to one of the storage servers that is up and has the least connections.
      • It load balances my remote clients across my storage severs
      • It handles a roll over in the event a server goes down.
      • it compresses and non compressed web traffic (html,js,etc)
      • It breaks up my connections by virtual host naming.
        • remote users connect VIA a url like http:\\streaming.nomadcf.com OR http:\\mediacenter.nomadcf.com
        • both run on same ip and port (80) and haproxy sends all the streaming.nomdcf.com to the storage severs EMBY or and all the mediacenter.nomadcf.com to the "workstations" for direct access to the raw content. Plus it separates my other port 80 website traffic to the web server (Again all on the same port & ip)
    • All server devices are given static DHCP address from a pool based on the devices name (pools contain 16 address each and are given a 1 month lease time)

 

It seems way more complicated when I see it written out, but really it's was extremely easy to setup (about 1 day and a few more to flush out some issues/scripts). And overall the cost was minimal minus the drives. Everything else was hobbled together from old machines and parts from over the years.

 

At the same time you should slim this all down to just 2 server, 1 that only does Direct play access and the other to pre-transcode all your content. In which case really BOTH servers hardware could virtual be anything. As the the work is being done pre them seeing/getting anything. And there is "little" while before any view-able content is available to any one. That is until server one has anything trans-coded and ready for server 2 to see. 

[famulor 1]

So whats the benefit of doing a "2 server setup" like you just wrote? compared to just build 1 server that can handle the transcoding etc?

Edited September 13, 2015 by famulor

[NomadCF 15]

So whats the benefit of doing a "2 server setup" like you just wrote? compared to just build 1 server that can handle the transcoding etc?

 

Smaller cheaper (any old machines laying around) hardware can be used and a better client experience.

• one server can be use for used all the heavy lifting
  • transcoding
  • chapter image extraction
  • image/nfo downloading
  • backups (data not just emby configs)
• One for dealing just with clients with out having to deal with any of the heavy stuff. No heavy stuff means more resources for clients. 

 

At this point emby can't restrict users to only direct play or direct stream content. So you could easily overwhelm your system with users requesting media that all needs transcoding along with the system it self trying to transcode something. Take a simple i5-3570K with it's 4 cores. 1 core is needed for the system and emby's  mediacenter.exe and then that will leave you with 3 cores for transcoding. You could take it down to 1 core PER trancoding but if your trancode say a full Bluray mkv down to a say something a roku can handle. Using only one core can lead to buffering issue. Now image 5 users all needing transcoding.  

 

Now on the flip side I've used a dual core 1.8 Ghz can direct *stream* (not play) to over 7 users with out any users seeing any buffering issues. Also having your content *all* transcoded makes it way easier to have ready for transport. You can just copy a few of the videos to X device to play in the car,plan,etc and your ready to go with having to worry about having to wait for things to transcode & sync. This happens to me from time to time as we'll have some impromptu weekend or other trips to places with little or not internet.