Jump to content


Photo

My Emby Server setup

emby server setup

  • Please log in to reply
13 replies to this topic

#1 plexman OFFLINE  

plexman

    Advanced Member

  • Members
  • 340 posts
  • Local time: 07:05 AM
  • LocationSpain

Posted 21 July 2016 - 06:01 AM

Hello everybody!

 

Since a long time ago I wanted to share my server setup in this awesome forum, sorry for my english in advance :P . In my university residence some years ago everybody was constantly asking for external HDD's to share movies and tv shows, so one day I wondered what could I do to make this sharing simpler. After using Plex for several years I switched to Emby cause Plex added a 100 users limit share and we were +-120 users. I rented a SoYouStart Opteron Server with 6Tb of storage in order to make Emby accesible outside the residence's LAN. Everything was perfect but we were (all the users) paying 45€ a month for the server, in addition I hadn't control of the hardware and if I wanted to change something or add more disk space was impossible. So with the installation of 300/300 fiber in my house, I decided to setup a home server and migrate all the stuff from the cloud server to my home server. That way we'll only have to pay for the electricity and the initial hardware cost. So, went to amazon and started looking for a nice budget setup for my server, the key of the purchase was the energy saving.

 

The home server specs are the following:

- Intel i5-6600 Processor

- Asus B150M-A Motherboard

- 2x4Tb HGST Desktar NAS HDD's (installed in Software RAID 0), for storing the media files

- Kingston SSDNow V300 120Gb for the OS and Emby metadata, etc.

- 16 Gb DDR4 HyperX Fury Kingston RAM

- Corsair Series CX430 PSU

 

The system is running under Ubuntu 16.04 after trying to do it under Debian 8.5, but the system due to the recent processor and the Linux Kernel in Debian 8.5 (I suppose) wasn't very stable with Emby. In order to do the migration transparently for the users I downloaded all my library (1300 SD Movies, 500 1080p 5Mbps Movies and 130 SD TV Shows) from the cloud server to my 8Tb /home partition in RAID 0, after this I installed and configured Emby and letting it scan the library and download all the metadata (notice that I didn't back up the library from the cloud server, just the users and their play status). With this completed, I configured my TP-Link wr1043nd router with OpenWRT installed to forward the necessary ports to the server, and setted up a dynamic DNS with NOIP as my ISP IP is dynamic. Everything almost ready, users backed up in the cloud server and restored in the home server I switch my Domain to point to my home ddns instead of the cloud server and watch the users loging in. Everything was fine and nobody noticed the change (correction, somebody had dns issues but after erasing history and cookies in their browser the problem was solved).

 

The average number of users connected at the same time is 5-10, with 20-30 in peak hours (siesta time 3-4pm, after dinner 11pm, before luch 1-2pm, you know spanish regime  :D ), almost everybody uses the server through the Web interface (some tablets or chromecast but it's 5-10% of the users) so what I do to deal with transcoding, or better said, how do I do to AVOID transcoding without losing quality. It's very simple, the solution was to convert all the media to h264/AAC/mp4 with a daily script which executes ffmpeg at 2am, looks for .avi or .mkv files and converts them to h264/AAC/mp4. The script is veeeery veeery simple (i'm not a scripting or programming boss) and this is the conversions it does:

 

(quality/video_codec/audio_codec/file_extension) 

 

Input --> Process --> Output

------------------------------------

 

- Movies and episodes in sd/mpeg/mp3-AC3/avi:

 

sd/mpeg/mp3/avi    -->    x264 CRF 20, audio_bitrate aac 128kbps, web_optimized    -->    sd/h264/aac/mp4

 

-Movies in sd/h264/DTS-AC3/mkv

 

sd/h264/DTS-AC3/mkv    -->    video stream copy, audio_bitrate aac 128kbps, web_optimized    -->    sd/h264/aac/mp4

 

-Movies in hd/h264/DTS-AC3/mkv with video bitrate around 5mbps

 

hd/h264/DTS-AC3/mkv    -->    video stream copy, audio_bitrate aac 192kbps, web_optimized    -->    hd/h264/aac/mp4

 

For movies with bitrate over 5mbps I use another script to do a 2-pass conversion cause it takes time for each file. In addition the scripts removes all subtitles and conserves only one audio track in spanish.

 

The question now is, why I do this? Well mainly because all my users don't care about DTS, surround or having a 20mbps full hd bluray. The only thing they want is that the movie plays well, fast, and good quality if possible, so this is a way of optimizing resources and disk space. The SSD gives an amazing speed charging the menus and all the web frontend I really recommend it, the server is now running perfectly and growing little by little. Oh! important, I have one directory for SD movies and another for HD movies, and for TV shows the structure is like this --> /home/shows/show_name/season/episodexx.avi

 

Hope you like it! and if you have questions I'll be pleased to answer them. (converting script attached and modified for understanding, the script is named wololo :D , if you played Age of Empires you should know why ;) ).

Attached Files


Edited by plexman, 21 July 2016 - 11:36 AM.


#2 dcook OFFLINE  

dcook

    Advanced Member

  • Members
  • 866 posts
  • Local time: 01:05 AM

Posted 21 July 2016 - 08:50 AM

Your taking a big risk with RAID 0, you will lose all 8TB of your content very quickly and easily with a single failure.

I hope you have backups of everything.



#3 MSattler OFFLINE  

MSattler

    Advanced Member

  • Alpha Testers
  • 1449 posts
  • Local time: 01:05 AM

Posted 21 July 2016 - 10:29 AM

What he said.... Raid 0 is just asking for trouble.

 

Good job on the conversions.  I have myself wondered about adding enough storage to where local users can use the full quality rip and remote users use a lower quality rip to forego transcoding.  But with GPU Transcoding working well, I have not gone that route yet.


  • plexman likes this

#4 legallink OFFLINE  

legallink

    Advanced Member

  • Alpha Testers
  • 1276 posts
  • Local time: 01:05 AM

Posted 21 July 2016 - 10:48 AM

Please don't discuss the notion of potentially downloading shows illegally and then sharing them on the forums.  You're just going to bring problems to everyone here.


Edited by legallink, 21 July 2016 - 10:54 AM.

  • MSattler likes this

#5 plexman OFFLINE  

plexman

    Advanced Member

  • Members
  • 340 posts
  • Local time: 07:05 AM
  • LocationSpain

Posted 21 July 2016 - 11:41 AM

Please don't discuss the notion of potentially downloading shows illegally and then sharing them on the forums.  You're just going to bring problems to everyone here.

 

Removed the lines refering to that, thanks!

 

What he said.... Raid 0 is just asking for trouble.

 

Good job on the conversions.  I have myself wondered about adding enough storage to where local users can use the full quality rip and remote users use a lower quality rip to forego transcoding.  But with GPU Transcoding working well, I have not gone that route yet.

 

Yes I know the risks of having RAID 0, but until I can afford a new HDD for RAID 5 I'll have to take them because I need that disk space. I thought about using them separately also, not in RAID.


Edited by plexman, 21 July 2016 - 11:49 AM.


#6 dcook OFFLINE  

dcook

    Advanced Member

  • Members
  • 866 posts
  • Local time: 01:05 AM

Posted 21 July 2016 - 03:00 PM

You would not want to do RAID 5 either which such large disks, high percent chance of bigger failure.

RAID6 would be ok, or you could do a RAID10

 

 

Removed the lines refering to that, thanks!

 

 

Yes I know the risks of having RAID 0, but until I can afford a new HDD for RAID 5 I'll have to take them because I need that disk space. I thought about using them separately also, not in RAID.


Edited by dcook, 21 July 2016 - 03:29 PM.


#7 MSattler OFFLINE  

MSattler

    Advanced Member

  • Alpha Testers
  • 1449 posts
  • Local time: 01:05 AM

Posted 21 July 2016 - 03:21 PM

You would not want to do RAID 5 either which such large disks, high percent change of bigger failure.

RAID6 would be ok, or you could do a RAID10

 

Or go the unRaid route.  Still have parity protection, including dual parity you get with Raid 6.  But you can mix disk sizes.



#8 JeremyFr79 OFFLINE  

JeremyFr79

    Advanced Member

  • Members
  • 925 posts
  • Local time: 10:05 PM
  • LocationSeattle, WA

Posted 21 July 2016 - 04:35 PM

Honestly you can use whatever RAID makes you happy as long as you understand the risks, accept those risks and if the need is there plan and account for those risks.  No matter what form/level of RAID you use, if you are relying on it as your soul form of data protection then you'll learn at one point or another how wrong you are and I say that in relation to ALL forms of RAID.  RAID however can give you performance enhancements etc and you should choose your form of RAID based on the type of performance data you're working with etc that you need.  In the end though if you have data you truly care about and can't live without then you need to have a bulletproof backup plan in place and that plan MUST include a form of offsite backup for optimum protection of your data.


  • porkbone likes this

#9 MSattler OFFLINE  

MSattler

    Advanced Member

  • Alpha Testers
  • 1449 posts
  • Local time: 01:05 AM

Posted 22 July 2016 - 09:22 AM

Honestly you can use whatever RAID makes you happy as long as you understand the risks, accept those risks and if the need is there plan and account for those risks.  No matter what form/level of RAID you use, if you are relying on it as your soul form of data protection then you'll learn at one point or another how wrong you are and I say that in relation to ALL forms of RAID.  RAID however can give you performance enhancements etc and you should choose your form of RAID based on the type of performance data you're working with etc that you need.  In the end though if you have data you truly care about and can't live without then you need to have a bulletproof backup plan in place and that plan MUST include a form of offsite backup for optimum protection of your data.

 

While very true, your backup is the original disc's that you own.  So in a true worst case scenarios, you simply re-rip your content.  I think if we are talking about critical data like business data, databases for a company, maybe family pictures for an individual, yes, do not rely on it as your sole backup.  But really everyone here should have the physical DVD/BluyRay discs on hand, so it should not be a huge deal to recover from.



#10 legallink OFFLINE  

legallink

    Advanced Member

  • Alpha Testers
  • 1276 posts
  • Local time: 01:05 AM

Posted 22 July 2016 - 11:47 AM

Eh, not a huge deal, and yet a huge deal.  I've got 8TB drives, and losing one of those and all the data on it, to rerip them would take a long time (probably a couple of months doing it intermittently while working).  So, I highly recommend an offsite backup.  I use Backblaze, simply because I tried crashplan, and it took way to long to transfer the files (they are both relatively equal in cost).



#11 JeremyFr79 OFFLINE  

JeremyFr79

    Advanced Member

  • Members
  • 925 posts
  • Local time: 10:05 PM
  • LocationSeattle, WA

Posted 22 July 2016 - 03:37 PM

While very true, your backup is the original disc's that you own.  So in a true worst case scenarios, you simply re-rip your content.  I think if we are talking about critical data like business data, databases for a company, maybe family pictures for an individual, yes, do not rely on it as your sole backup.  But really everyone here should have the physical DVD/BluyRay discs on hand, so it should not be a huge deal to recover from.

 

 

Eh, not a huge deal, and yet a huge deal.  I've got 8TB drives, and losing one of those and all the data on it, to rerip them would take a long time (probably a couple of months doing it intermittently while working).  So, I highly recommend an offsite backup.  I use Backblaze, simply because I tried crashplan, and it took way to long to transfer the files (they are both relatively equal in cost).

My point was simply that you in all reality can use whatever RAID you personally want to use as long as you're comfortable with the risk's of said RAID and that people yelling don't use this, or don't do that or use this, is really very pointless as each and everyone's requirements and risk assessment is different.  RAID is best chosen based on performance needs and cost.  After that if you want to rely on any redundancy said RAID will offer then so be it, but redundancy should never be a reason why you choose one form of RAID over another.


Edited by JeremyFr79, 01 August 2016 - 12:58 PM.


#12 colejack OFFLINE  

colejack

    Advanced Member

  • Members
  • 81 posts
  • Local time: 01:05 AM

Posted 28 July 2016 - 11:17 PM

My point was simply that you in all reality can use whatever RAID you personally want to use as long as your comfortable with the risk's of said RAID and that people yelling don't use this, or don't do that or use this, is really very pointless as each and everyone's requirements and risk assessment is different.  RAID is best chosen based on performance needs and cost.  After that if you want to rely on any redundancy said RAID will offer then so be it, but redundancy should never be a reason why you choose one form of RAID over another.

 

 

+1 Listen to this guy. He knows what he's talking about. *RAID is for uptime not backup.

 

*RAID is also good for performance plus uptime in certain cases, ie,  RAID10


Edited by colejack, 28 July 2016 - 11:19 PM.


#13 justdrooit OFFLINE  

justdrooit

    Advanced Member

  • Members
  • 52 posts
  • Local time: 12:05 AM

Posted 13 September 2016 - 03:35 PM

My Emby server used to be a system of spare parts. It's now the most powerful system in my possession. It's just myself and my girlfriend that use it, but it has mostly worked quite well. It also pulls double duty as the system that controls my front and back home cameras for security.

No, of course I don't think this is the end-all setup that's better than everyone else's, but it's working pretty well for me.

Here are the specs:

AMD FX8320E 8-core processor
ASROCK 970M Pro3 mainboard
Nvidia NV295 Quadro graphics (simple graphics card - what I had on hand)
16GB DDR3 RAM
Windows Server 2008r2 x64
1 120GB OCZ Agility3 SSD (Boot)
1 275GB Crucial SSD (work drive)
2 Netgear ReadyNAS NV+ v2
   1 with 4 2TB HDD for 6TB storage X-RAID2 (similar to RAID5)

   1 with 4 3TB HDD for 9TB storage X-RAID2 (similar to RAID5)
1 Drobo with 8TB storage - Unused at the moment but will eventually serve as a form of backup.
100/100 FIOS internet

x265 has given my end-of-life ReadyNASs an extended life but the entire setup is not perfect because my girlfirend's old Android tablet isn't quite powerful enough to run those files natively while most of my things can. So, as a result, the server gets pushed a bit in transcoding if she's watching something in x265 and I'm viewing my local TV on my smartphone at work.

My hope is that things continue to progress in hardware transcoding and we eventually have a clear/easy path to simply put a low-mid range, new generation video card (RS 460 or GTX 950/1060) in and it'll run more easily. I know Nvidia has not made that easy as they don't like precompiled versions of the transcoder being distributed, but I'm hoping maybe AMD takes advantage of that.
 



#14 Luke OFFLINE  

Luke

    System Architect

  • Administrators
  • 157148 posts
  • Local time: 01:05 AM

Posted 08 August 2017 - 03:28 PM

@plexman

@dcook

@JeremyFr79

@justdrooit

@colejack

@legallink

 

We're please to announce Emby for NetGear ReadyNAS is now available for testing:

 

https://emby.media/c...etgear-readynas

 

Please try it out and report your experience. Thanks and enjoy.


  • plexman likes this





Also tagged with one or more of these keywords: emby, server, setup

0 user(s) are reading this topic

0 members, 0 guests, 0 anonymous users