Setting up an epic emby server room.

[rbjtech 4265]

Until disk I/O catches up - a good compromise is 2.5G or 5G networking - which is a lot more tolerant of cabling (works fine on Cat5e) - and the switches/NICS are substantially cheaper too.

Unless you have a significant disk I/O system - you will never get anywhere close to max 10Gbit throughput.  350Mbytes per second is 'normal' so yes it's 3x faster than 1Gig, but it's not 10x faster ...  

 

 

[chef 3745]

There is an end in sight! My lighting isn't exactly straight, but I think it is close enough.

Yeah, I can lift 4x8 sheets of dry wall over my head, and install a ceiling on my own, but my entire body hates me for it.

Mental note: I can not do dry wall for a living, or I might actually die.

I also updated the AP's in the house.

 

I decided to spend of the of the money on a security gateway from unifi for the network, and one of those unifi network keys that in the Ubiquity dashboard.

Edited April 30, 2021 by chef

[chef 3745]

On 4/23/2021 at 6:05 AM, Hxemby001 said:

What did you think about the fiber optic USB 3 cables? Any experience with those?

I've decided to put those drives closer to the computer. I'd love to get them out of the room because of the fan noise. The fiber usb was a little pricy.

I did run extra copper inside the walls just in case it was use full.

I may have gone overboard, but I put an extra six runs in the wall.

 

I had an extra 1000 feet of cat6, so I figured, why not.

Edited April 30, 2021 by chef

[Guest]

43 minutes ago, chef said:

Yeah, I can lift 4x8 sheets of dry wall over my head, and install a ceiling on my own, but my entire body hates me for it.

Mental note: I can not do dry wall for a living, or I might actually die.

LOL.. yeah usually needs a pair of helping hands to get it secured and flat..

Looking good so far... might wind up with a cozy little room.. with a touch of bad-ass...LOL... Waiting to see whats coming next in the theatre/cinema sound and processing department..

[sooty234 266]

On 4/30/2021 at 4:28 AM, rbjtech said:

Until disk I/O catches up - a good compromise is 2.5G or 5G networking - which is a lot more tolerant of cabling (works fine on Cat5e) - and the switches/NICS are substantially cheaper too.

Unless you have a significant disk I/O system - you will never get anywhere close to max 10Gbit throughput.  350Mbytes per second is 'normal' so yes it's 3x faster than 1Gig, but it's not 10x faster ...  

 

 

That's a limited way of thinking. Do you only have one stream of data in your network at a time? I can and do write to multiple arrays simultaneously. So that can be around 700 - 800MB/s. And while that's happening, I'm downloading, streaming and have a few other network tasks going on. 10G = 1250MB/s, 5G = 625MB/s and 2.5G = 312.5MB/s. With NVMe drives being common, and having read/write speeds upward of 3000MB/s, anything less than 10G when building a new network is ill advised. Always build for future expansion.

[rbjtech 4265]

3 hours ago, sooty234 said:

That's a limited way of thinking. Do you only have one stream of data in your network at a time? I can and do write to multiple arrays simultaneously. So that can be around 700 - 800MB/s. And while that's happening, I'm downloading, streaming and have a few other network tasks going on. 10G = 1250MB/s, 5G = 625MB/s and 2.5G = 312.5MB/s. With NVMe drives being common, and having read/write speeds upward of 3000MB/s, anything less than 10G when building a new network is ill advised. Always build for future expansion.

No - my view is a realistic/economic way of thinking.

Unless you have a specific need for 10Gig, then currently, putting it in for 'future expansion' is poor judgement as it's 10x the price of 1 Gig networks and a home user simply cannot use it and is highly unlikely to need it.

On the SAME interface - 1gig will give you simultaneous capacity for :-

10 x 80Mbit 4k remux streams

50 x 16Mbit 1080p streams

100 x 8Mbit 1080p streams

Price of a 16 Port 10Gig switch (not including all the expensive SFP's, 10Gig Nics) - £500-600

Price of a 16 port 1Gig switch (no need for expensive SFP's, NIC's will be standard or included) - £50-60

 

So in summary, 10Gig = 10 x the Performance (which you cannot use or do not need) and 10 x the Price

 

This is where 2.5Gig and 5Gig come in, still expensive vs 1Gig, but nowhere near the cost of 10Gig.

 

btw - I build DC's for a living, 1Gig, 10Gig, 40Gig or 100Gig is chosen as needed - you build for the demand - if a 1gig network meets that immediate demand with capacity, then that's what we build.  Granted in the DC, most things are 10Gig as standard now - but this is in the DC, not the home.

 

 

 

 

 

[chef 3745]

I've seen a lot of home labs on the internet lately, and they all seem to run 10 gig lines between two separate redundant drive arrays for backups.

I think for my home setup I'll have to use my cat 6 for one gig at the moment. I do intent on upgrading a full rack of ubiquity server stuff though.

Edited May 1, 2021 by chef

[Carlo 4330]

Then there is the other side of the router or WAN side. If you "think" you need 10Gig internally then you plan on moving lots of data around internally.

If this is mainly for Emby use then you have to ask where all these clients are going to be located.  If these are remote clients and you have at best a 1Gig connection to the Internet then that will always be your limiting factor and the speed of your internal network is mostly pointless.

Where higher speed networking is quite useful is between different file servers or NAS boxes when you need to copy/move data around often.

Keep in mind the majority of things you likely use won't support more than 1Gig like your Rokus, Android, FireCubes, XBox, PS4 and likely not your laptop/notebooks either. Low end NAS boxes from the likes of WD and Synology don't support more than 1Gig. Even a decent Synology box like the 920+ only has 2 1Gig connections.

If your storage disks used for Emby are directly attached and not network shares then the speed of your network is even less important.

So use matters a lot.  Like @rbjtechmentioned you can overbuild and waste money on bandwidth that may never come close to getting used in all but the most extreme cases.

[chef 3745]

Had anyone build a vpn between two houses?

I was wondering about these ubiquity security gateways. Apparently, they can be used  to create a vpn connection.

[Carlo 4330]

I've been doing this for years but I've been using ASUS routers which can do OpenVPN or MS VPN. I currently have 4 VPN tunnels running 24/7. I then have another used for general remote access.

But I'm no help on the ubiquity side of things.

[rbjtech 4265]

11 hours ago, chef said:

Had anyone build a vpn between two houses?

I was wondering about these ubiquity security gateways. Apparently, they can be used  to create a vpn connection.

They will just use a flavour of OpenVPN - I'd stick to that as opposed to proprietary protocols as we know that is tried,  tested and trusted.

Have you thought about a segmented network design at all - ie the use of VLAN's - keeping media traffic off your main network for example ?  If your switches support 802.1Q (I seem to recall you have netgear switches?) which should be compatible.   

This is another important aspect of network 'design' that gets overlooked and of course adds a very important layer of security as well.

[sooty234 266]

On 5/1/2021 at 12:27 PM, rbjtech said:

No - my view is a realistic/economic way of thinking.

Are you sure you know what those words mean? 10G is very realistic and isn't at all expensive. I've got a whole bunch of it. I've even got a dual 10G NIC in my pfsense box. And I've even got spare 10G parts. I was thinking about upgrading, just for the hell of it.

@chef ebay is your friend. If you choose to use sfp+ instead of RJ45, the NICs are much cheaper (not that 10G RJ45 are expensive). Here's a lot of 8 for $199. I mean, what a deal!

Even Linus says it's cheap (and this was in 2019)

On 5/1/2021 at 12:27 PM, rbjtech said:

btw - I build DC's for a living

meh

Edited May 6, 2021 by sooty234

[Carlo 4330]

Ok so assuming you already have wiring in place using cat 6/RJ45 then you will not be looking for sfp+ or other types of adapters and likely won't want to be purchasing used/refurb or other equipment off ebay.

Besides the NICs you still need other equipment like a smart switch as well. This is likely where you'll have most of your money invested in the upgrade.

But it still comes down to NEED.  If all of your devices are just common PCs used for house hold needs or video playback devices then it won't need anything above 1Gbit.  Heck most devices might be WIFI only.  That would leave you only the "core network" as 10Gbit.  Again, what is the need once things are setup?

How fast do you need to move files from a NAS/File Server to an Emby Server? Are you going to get any speedup from the increase in bandwidth?  In most homes, no, not at all.

The fact is, if you're not moving large amount of files around like full daily backups then you have little need for more than 1Gbit adapters in a typical Emby setup. Hell you might be better off moving to direct attached storage then a seperate network box anyway which really makes this even less worth while an upgrade.

Is 10Gbit nice to have?  SURE
Is it needed?  Not by most people and actually will do very little for the average person.

[chef 3745]

So close to the end! 

I'm thinking a 4k UHD TV at 75". However, I'm not going to get a smart TV, because I have a Fire Cube.

[rbjtech 4265]

6 hours ago, sooty234 said:

Are you sure you know what those words mean? 10G is very realistic and isn't at all expensive. I've got a whole bunch of it. I've even got a dual 10G NIC in my pfsense box. And I've even got spare 10G parts. I was thinking about upgrading, just for the hell of it.

@chef ebay is your friend. If you choose to use sfp+ instead of RJ45, the NICs are much cheaper (not that 10G RJ45 are expensive). Here's a lot of 8 for $199. I mean, what a deal!

Even Linus says it's cheap (and this was in 2019)

 

tbh, I can't really take you seriously - especially with comments like use 'ebay' and 'Linus' says this - you do realise he buys NOTHING right and is given it all by sponsors - with headlines purely to get more views from people thinking he is an IT guru - he is not ..  

[Guest]

First I agree on the TV.. I hate smart TV's but I go for running windows on them.. more control over every aspect of the device.. Fire Cube seems like a nice piece of equipment.. but I would probably wind up. getting a PC to run on it.. just the way I am.. LOL  I do see processing and killing the system.. over time the wear... better to replace on something like the cube versus a PC...

----------------------------------------------------------------------------------------

As far as the ongoing discussion.. I probably already stated my stance.. but I support the 10G idea.. However.. since this project was finished by the time the discussion started .. I would say it is sort of after the fact... 10G is coming.. and I put this in a spoiler tag... and you have to go in two views.. one versus the other in difference now... and tearing it all out and doing it again later... In this case it would have been the combo of the two.. in cost..

AFTER THE FACT..  

eBay is great for NEW-IN-BOX products usually... I go with what I know though. Stay away from PayPal..
 

Spoiler

 

Picture just two people in the household.. since we are doing 'HOME' usage... as a hose utilitarian perspective how much traffic is going to be flowing through switches and routers total for the home by itself.. ( This is with a projection of the next 20-30 years, not just short-sighted now.. and this comes with the idea of looking at what you needed 20 years ago ).. All possible home automation, protection, safety, appliances, lighting... including security camera streams on the property and recording/upload to security company... Next you have Entertainment usage.. hopefully from PURE sources, not transcoded.. for two phones.. three tablets.. however many PC's.. at least three/possibility of 5 or 6- TV's/Media boxes for processing... then we can have a possibility of a few more media devices which access data for Music by itself... and we may be serving them to the outside world as well when we are away from the home... Updating the systems as well.. and allowing the necessary connectivity...and protections of course.. ( which may mean scanning over the network )... You may even run a website from your home for your own personal use.. much like a Facebook profile.. or even a place for your things, even a portfolio.. ( see the following ).. There may be a home based business... and you may even work from home.. which leads me to.. -->

Now two people.. who use their systems as a tool but not for storage.. so all data is stored on EHD's... as a designer, photographer, artist, ... you are talking about RAW data files that are used and transferred, in some cases, over the network to said repository.. and then is used by a program over that network... and goes into processing.. especially 3D Printing and 3D Design... and effects... you will not get the same processing speed which available from working off the hard drive at 6GB/s or 12 GB/s.. instead its 1GB/s if your lucky.. and you have it all to use...As a media developer, DJ, Artist, or even a producer ( music ).. the same and even more so ( Not to mention CPU requirements.. ) of someone who is editing and creating videos... Next we have the code developer.. who may need to access the data they are working on... this can be thousand of tiny little files but accessing and working on them may come in one request.. and it may need to be compiled with is up and down stream... AND some of their files being created may come for or include such RAW data ( LARGE files ) as well.. These things may also be on a server for the office.. or even to be accessed from the place of business...

The who thing makes it a lot easier to work with... Even with a simple setup like I have right now... I get entirely frustrated with the prolonged processing and strain placed on my system by a 1 GB/s bottleneck...when things could be done and the system cooling down a long time beforehand... I strayed away from being able to do a lot of those things but in digital media creation, which has become more and more mainstrean.. it takes more.. and we really don't have the extra hour to wait in our lives.. Some people do only consume.. but we wind up creating things on a system.. where we used to do them another way in the old days so the demand for them to happen quickly and efficiently is definitely there less we just spend our lives tormented by it. OR wasted as I see it.

Properly editing home videos, photographs... setting up the Digital DVD Rack, or Record Shelf... Our basic needs WAAY exceed what was originally thought to be needed in the 60's when they did it for the first time and it keeps expanding... even if yes there are methods to do SO much with it.. sitting still there stands to reason its supply and demand...

 

 

[chef 3745]

I think my NAS situation limits me a bit for 10gig. Because I currently have five of these raid drive enclosurers, which are connected to the main emby machine with usb3.

Do you think the usb3 would bottle neck the data transfer at 10gig?

 

I can't wait for the rack to show up, so I can finally start building a new service.

 

[Guest]

No in fact.. that was what I was looking at for a solution for my own needs.. via USB3.. which gets around the network issue..

Having five of them would present me with a n issue though... LOL

EDIT: I am the guy though that buys the $490,000 McLaren SLR Benz.. and upgrades it to 900+ HP... custom carbon fiber aero and hood... then repaints with custome rims and racing tires.. MORE... I say.. LOL..

Edited May 6, 2021 by Guest

[chef 3745]

32 minutes ago, Hxemby001 said:

First I agree on the TV.. I hate smart TV's but I go for running windows on them.. more control over every aspect of the device.. Fire Cube seems like a nice piece of equipment.. but I would probably wind up. getting a PC to run on it.. just the way I am.. LOL  I do see processing and killing the system.. over time the wear... better to replace on something like the cube versus a PC...

----------------------------------------------------------------------------------------

As far as the ongoing discussion.. I probably already stated my stance.. but I support the 10G idea.. However.. since this project was finished by the time the discussion started .. I would say it is sort of after the fact... 10G is coming.. and I put this in a spoiler tag... and you have to go in two views.. one versus the other in difference now... and tearing it all out and doing it again later... In this case it would have been the combo of the two.. in cost..

AFTER THE FACT..  

eBay is great for NEW-IN-BOX products usually... I go with what I know though. Stay away from PayPal..
 

  Reveal hidden contents

 

Picture just two people in the household.. since we are doing 'HOME' usage... as a hose utilitarian perspective how much traffic is going to be flowing through switches and routers total for the home by itself.. ( This is with a projection of the next 20-30 years, not just short-sighted now.. and this comes with the idea of looking at what you needed 20 years ago ).. All possible home automation, protection, safety, appliances, lighting... including security camera streams on the property and recording/upload to security company... Next you have Entertainment usage.. hopefully from PURE sources, not transcoded.. for two phones.. three tablets.. however many PC's.. at least three/possibility of 5 or 6- TV's/Media boxes for processing... then we can have a possibility of a few more media devices which access data for Music by itself... and we may be serving them to the outside world as well when we are away from the home... Updating the systems as well.. and allowing the necessary connectivity...and protections of course.. ( which may mean scanning over the network )... You may even run a website from your home for your own personal use.. much like a Facebook profile.. or even a place for your things, even a portfolio.. ( see the following ).. There may be a home based business... and you may even work from home.. which leads me to.. -->

Now two people.. who use their systems as a tool but not for storage.. so all data is stored on EHD's... as a designer, photographer, artist, ... you are talking about RAW data files that are used and transferred, in some cases, over the network to said repository.. and then is used by a program over that network... and goes into processing.. especially 3D Printing and 3D Design... and effects... you will not get the same processing speed which available from working off the hard drive at 6GB/s or 12 GB/s.. instead its 1GB/s if your lucky.. and you have it all to use...As a media developer, DJ, Artist, or even a producer ( music ).. the same and even more so ( Not to mention CPU requirements.. ) of someone who is editing and creating videos... Next we have the code developer.. who may need to access the data they are working on... this can be thousand of tiny little files but accessing and working on them may come in one request.. and it may need to be compiled with is up and down stream... AND some of their files being created may come for or include such RAW data ( LARGE files ) as well.. These things may also be on a server for the office.. or even to be accessed from the place of business...

The who thing makes it a lot easier to work with... Even with a simple setup like I have right now... I get entirely frustrated with the prolonged processing and strain placed on my system by a 1 GB/s bottleneck...when things could be done and the system cooling down a long time beforehand... I strayed away from being able to do a lot of those things but in digital media creation, which has become more and more mainstrean.. it takes more.. and we really don't have the extra hour to wait in our lives.. Some people do only consume.. but we wind up creating things on a system.. where we used to do them another way in the old days so the demand for them to happen quickly and efficiently is definitely there less we just spend our lives tormented by it. OR wasted as I see it.

Properly editing home videos, photographs... setting up the Digital DVD Rack, or Record Shelf... Our basic needs WAAY exceed what was originally thought to be needed in the 60's when they did it for the first time and it keeps expanding... even if yes there are methods to do SO much with it.. sitting still there stands to reason its supply and demand...

 

 

One good thing is that in the future I built the network lines to have enough room to pull them back through all the holes in the joists. This means that at some point I could pull new fiber though the holes .

[Guest]

Always good to think and plan ahead..  ... especially for repairs.. I have seen some really stupid/painful/costly things done in my time..

Edited May 6, 2021 by Guest

[rbjtech 4265]

@chef - you have prepped for 10Gig by using Cat 6a cable, so unless any of your runs are over 100m - you are good to go when and if you wish to upgrade.  

[Guest]

Was watching this Burke BunchTV - YouTube a few minutes ago.. Pretty awesome theater.. he has some videos about building it on down the page.. some good stuff in there.. The only thing he is missing is EMBY.. and some Media Sonic Boxes..LOL.. Maybe a DELL Micro Form Factor Windows 10 PC to run it some ...

I remember the first home cinema I was in.. out at a lake property actually way back on 2003 sometime... nice setups.. quite a bit different then versus now..  

Edited May 11, 2021 by Guest

[chef 3745]

So this is cool! 

I'm running the Unifi Controller on a Raspberry Pi 4. It has a small screen that will load a kiosk view of the dashboard in chrome browser. It sits inside the network box

I have a new USG showing up, and two new Unifi switches. 

And.. I am finally ready for paint!

Edited May 14, 2021 by chef

[Guest]

Looking great as always... People in the house should be excited about what's to come

I have to look some of this up though.. Unified Security Gateway for example..  unfamiliar with Ubiquity items..

[BAlGaInTl 279]

That's so clean.

I'm impressed.  It makes me want to fish some cables through my old house... maybe not.

I'm running my UniFi controller in a docker, but I would love to move it off my main server.  Do you have any links to the stuff you used for your RPi?  That looks really cool.

Edited May 14, 2021 by BAlGaInTl