igeoorge 26 Posted September 16, 2021 Share Posted September 16, 2021 I'm having emby issues. This problem only occurs at night, when I have many friends watching simultaneously. So I'm thinking about splitting the load of hits. Currently I have only one machine receiving access. After the change, it will look like this: Machine A with nginx server in front doing the balancing. Machine B and C with Emby. Machine C will be a clone of machine B. Has anyone used nginx for this? Is it possible to do that? Link to comment Share on other sites More sharing options...
Luke 37095 Posted September 16, 2021 Share Posted September 16, 2021 @pir8radio have you tried this? 1 Link to comment Share on other sites More sharing options...
jaycedk 383 Posted September 16, 2021 Share Posted September 16, 2021 (edited) Will that not be a problem ? 2 emby server = 2 databases, right ? Since Emby dos not support 2 server access to 1 database. Then user watch status would not be in sync. Edited September 16, 2021 by jaycedk 1 Link to comment Share on other sites More sharing options...
jaycedk 383 Posted September 16, 2021 Share Posted September 16, 2021 Quote Machine C will be a clone of machine B. What if user x has watched something on Machine B, and Machine C is cloned over that. Again watch status issues. 1 Link to comment Share on other sites More sharing options...
igeoorge 26 Posted September 16, 2021 Author Share Posted September 16, 2021 At the moment I have a database that makes the current Emby sync. This same database should be able to synchronize the two Emby Link to comment Share on other sites More sharing options...
igeoorge 26 Posted September 16, 2021 Author Share Posted September 16, 2021 Thanks @Jaycedk for the explanations. So for it to work, I must do it like this: Machine A with nginx server in front doing the balancing. Machines B and C with Emby. Machine D making a clone of machine C in real time Would that work? Today I have problems with Emby HTTP crashes. Do you have any tips on what I can do to make the emby lighter or more balanced? Link to comment Share on other sites More sharing options...
jaycedk 383 Posted September 16, 2021 Share Posted September 16, 2021 Well anyways not sure, just listing pitfalls 1 Link to comment Share on other sites More sharing options...
BillOatman 501 Posted September 16, 2021 Share Posted September 16, 2021 Or maybe go simple and have all Emby servers accessible and assign users to servers. Not load balanced of course but should be able to get you past your problem without jumping through all kinds of hoops. 1 1 Link to comment Share on other sites More sharing options...
igeoorge 26 Posted September 16, 2021 Author Share Posted September 16, 2021 The idea of load balancing came after we realized that emby only crashes when it is heavily used. Unfortunately splitting each user by server is not an option 1 Link to comment Share on other sites More sharing options...
jaycedk 383 Posted September 16, 2021 Share Posted September 16, 2021 Hmm ok so how many users are we talking about, and simultaneously connections ? Never had this issue, only serving to family. I think emby is designed as an house hold media server. Not Network streaming service. Link to comment Share on other sites More sharing options...
Carlo 4330 Posted September 16, 2021 Share Posted September 16, 2021 15 minutes ago, igeoorge said: At the moment I have a database that makes the current Emby sync. This same database should be able to synchronize the two Emby How exactly do you do this? You can't access an Emby database while it's in use so you would have to stop Emby in order to sync. The only safe way I can think of to do this would be using a custom notification plugin or websocket notification service that keeps the watched status updated in real time. Remember once you have users using 2 different servers this isn't a simple database copy as you have watched status entries in two database that need to be synced. Concerning nginx if you setup load balancing and use the IP Hash method this might work ok, but you would need to test it. "The server to which a request is sent is determined from the client IP address. In this case, either the first three octets of the IPv4 address or the whole IPv6 address are used to calculate the hash value. The method guarantees that requests from the same address get to the same server unless it is not available. If one of the servers needs to be temporarily removed from the load‑balancing rotation, it can be marked with the down parameter in order to preserve the current hashing of client IP addresses. Requests that were to be processed by this server are automatically sent to the next server in the group:" https://docs.nginx.com/nginx/admin-guide/load-balancer/http-load-balancer/ This could be the best method to try as it would keep users on the same server so watched status would be less of an issue. You would need to test this to see if a user gets the same server day after day or only for a few hours, but it might be a starting point. 1 Link to comment Share on other sites More sharing options...
igeoorge 26 Posted September 16, 2021 Author Share Posted September 16, 2021 The database I am referring to is the user database. I have another server that activates or deactivates users, without the need to access each user manually. Sorry, I believe I've been expressing myself wrong. Yes, I don't have access to the Emby database. Perhaps if this could be made available one day, it would be very useful. I will follow your directions, thank you very much! @cayars Link to comment Share on other sites More sharing options...
Carlo 4330 Posted September 16, 2021 Share Posted September 16, 2021 I don't use nginx myself but have used load balancing in commercial ventures. That to me looked like the safest option but you need to do some testing of course. You might find a better way but that should be a good starting point to try as it could work straight away for you. 1 Link to comment Share on other sites More sharing options...
pir8radio 1292 Posted September 19, 2021 Share Posted September 19, 2021 (edited) If you can keep the two databases in sync it will work.. Im weak when it comes to db's but my understanding was you cant sync sqlite without taking one or both out of production during the sync.. and you need them to be in constant sync not once a day. IMO On the nginx side you can use load balancing but you need to use the "ip-hash" function on the load balance config, otherwise the load balancer will split connections across the other servers.. for example if one person is streaming a movie, it gets started buffered, then on its next request for the next chunk it would try from another server which would fail because that server isnt transcoding the movie. IP-hash will keep user1 on server A then user2 connects he gets put on ServerB and stays there.. How many "friends" do you have connecting? and what issues are you seeing... I have done testing in the past inviting everyone on this forum to hit my server with a stream... I only had 35 participants but all were transcoding HD video and the server had room to go, just didn't have enough testers... see the results in this thread: This was all done on one bare-metal server, with one GPU RTX4000. If you have way more than 35 then ok maybe you need load balancing.. But if you are not transcoding at all the bottle neck is somewhere else. I've noticed the hard drives are usually the first to be the bottleneck (if you have a good GPU) transcode drive needs to be a good SSD, array where the video is stored needs to have good throughput. add those streams up and see where your bottle neck is. For example ignore transcoding.. if you have 35 movies being played at 10Mbps each you are talking about 350Mbps which can be a bottle neck for a single drive, like a transcode drive, especially if its reading and writing at the same time. So, again just curious how many friends and what were your issues? Edited September 19, 2021 by pir8radio 1 1 Link to comment Share on other sites More sharing options...
Carlo 4330 Posted September 19, 2021 Share Posted September 19, 2021 Nice post @pir8radio. Lots of good info there. 1 Link to comment Share on other sites More sharing options...
igeoorge 26 Posted September 19, 2021 Author Share Posted September 19, 2021 Hi friend, thanks for your feedback. I'm not currently transcoding. All are in direct reproduction. At the moment the machine where the emby is has the following specification: 12 cpus 120gb of ram Raid with 48tb We don't have GPU The use of this machine is still very small. Cpu is no more than 30% usage. The maximum memory that the emby came to use was 10GB. But the average daily ram usage is 4GB. Every day at 6:00 am the emby server is automatically restarted. We do this in an attempt to lighten the emby and avoid problems. We have an average of 250 simultaneous reproductions. We are studying all possible scenarios and I think the cheapest and simplified scenario would be to just add another server and direct new users to it. The problem I've been facing every night is with emby HTTP crashes. The embyserver.exe service is still active, but it cannot be accessed. When the emby http server goes down, all functions on the machine the emby is on are still active and working fine. We believe that this problem in emby is due to the amount of reproductions, as the problem only occurs at night, when there are more simultaneous reproductions. I'm in Brazil and my English is bad. I'm using google translator. Sorry about that. Thank you so much for your help. Link to comment Share on other sites More sharing options...
Carlo 4330 Posted September 19, 2021 Share Posted September 19, 2021 Are you saying you have 250 simultaneous average streams going or just 250 users connected and browsing your libraries along with streaming? Link to comment Share on other sites More sharing options...
igeoorge 26 Posted September 19, 2021 Author Share Posted September 19, 2021 (edited) The average is 250 simultaneous. With a minimum of 15 and a maximum of 390 simultaneous. Which is really weird; We've had moments with 300 simultaneous people and no problem with Emby. We've had moments with 200 simultaneous people where the emby had problems. Sorry, I'll try to express myself better. Edited September 19, 2021 by igeoorge Link to comment Share on other sites More sharing options...
pir8radio 1292 Posted September 20, 2021 Share Posted September 20, 2021 On 9/19/2021 at 1:50 PM, igeoorge said: The average is 250 simultaneous. With a minimum of 15 and a maximum of 390 simultaneous. Which is really weird; We've had moments with 300 simultaneous people and no problem with Emby. We've had moments with 200 simultaneous people where the emby had problems. Sorry, I'll try to express myself better. Ok so public service vs many "friends" connecting at night like the first post says lol.. makes troubleshooting a little different.. you use nginx today? or cloudflare? I would enable caching on nginx, your issue is probably the web server emby uses, taking on a lot of hits.... if you can offload some of those hits to nginx cache it would probably help you a lot. keep your users from hitting the emby server. what kind of raid (0,1, 5, 7? SSD or spinning disk? any SSD raid Caching?) what connection to raid (fiber ch, ethernet, 1G, 10G)? What kind of ethernet connection? multiple 1G, 10G, copper or fiber? Does your ethernet adapter have features enabled like offloading, caching etc? have you tried disabling emby logging? what kind of drive is your database stored on, raid? single drive? single SSD? Link to comment Share on other sites More sharing options...
igeoorge 26 Posted September 21, 2021 Author Share Posted September 21, 2021 Hey friend, Today I don't use nginx. Today I use cloudflare. I don't cache on cloudflare, totally disabled. Could I download these accesses in cloudflare, instead of nginx? The emby today is in a datacenter, VPS on Raid. 1G connection, maximum use of 700M We do the normal server cache, it's 120gb of cache I hope I have answered all your questions. Thanks for helping. Link to comment Share on other sites More sharing options...
igeoorge 26 Posted September 21, 2021 Author Share Posted September 21, 2021 Do you think if I put an nginx server in front of the emby I will stop having problems? Can I put the nginx server and continue using cloudflare? Link to comment Share on other sites More sharing options...
pir8radio 1292 Posted September 21, 2021 Share Posted September 21, 2021 13 minutes ago, igeoorge said: Do you think if I put an nginx server in front of the emby I will stop having problems? Can I put the nginx server and continue using cloudflare? yes i run like this INTERNET---------Cloudflare---------Nginx---Emby Link to comment Share on other sites More sharing options...
Carlo 4330 Posted September 21, 2021 Share Posted September 21, 2021 I thought I asked you about Cloudflare caching previously in PM? I would certainly turn this on in standard mode and set up a page rule to cache images. About 5 minutes work. That alone will save a lot of hits to the server when people are browsing your libs as Cloudflare will cache the images. It will also make your pages load a lot faster as well. Link to comment Share on other sites More sharing options...
igeoorge 26 Posted September 21, 2021 Author Share Posted September 21, 2021 The cache for cloudflare was active. We turned it off because cloudflare started wanting to cache the videos. What happens is that when the cache is active, videos do not play. What did you do to keep the cache active and videos to continue playing? Link to comment Share on other sites More sharing options...
igeoorge 26 Posted September 21, 2021 Author Share Posted September 21, 2021 this is my cloudflare configuration Link to comment Share on other sites More sharing options...
Recommended Posts
Create an account or sign in to comment
You need to be a member in order to leave a comment
Create an account
Sign up for a new account in our community. It's easy!
Register a new accountSign in
Already have an account? Sign in here.
Sign In Now