Emby Server stops working

[omerome 4]

I have a weird problem and I am not even sure where to start.

 

At least once a day, my Emby-Server stops working and I manually have to restart the jail (not the plugin) to get it to work again. It is the only app on my FreeNAS server that does this. I am also running Plex, Resillio, SabNZBD, and Sonarr with 16GB ram.

 

Has anyone experienced something like this before?

 

I am currently on Version: FreeNAS-11.0-U1 but this has been happening since I was on 9.10 and I hoped that a version upgrade would solve this problem.

 

server-63635328000.txt

Edited July 12, 2017 by omerome

[Luke 42077]

Hi, can you try uninstalling the cover art plugin and seeing if the issue persists?

[omerome 4]

Sure. I just uninstalled it.

 

May I ask, is there an issue with cover art?

[Luke 42077]

Occasionally for some systems it can be a little too much. We are switching the image processing library from image magick to skia which lightens the load a bit, but that switch hasn't made it's way to the bsd release yet.

[omerome 4]

It has happened again.

 

Emby Server (I am currently away from my pc to physically restart the service) is not working once again and I have removed CoverArt.

[Luke 42077]

Can you provide a new server log? thanks !

[omerome 4]

Here is my log.

 

Thanks!

server-63635414401.txt

[Luke 42077]

@@omerome can you go into the emby server dashboard -> dlna, disable the server's dlna features and then see if that makes a difference? thanks !

[omerome 4]

I disabled all of the DLNA features and I'll let you know the results (most likely, tomorrow is when it would happen).

Edited July 13, 2017 by omerome

[Luke 42077]

Thanks for the feedback.

[omerome 4]

Thanks for the feedback.

Hmm. The server stopped working again.

 

I am noticing that my logs are growing in size each time, too. The last log is over 124MB so I can't upload it though, but hopefully, another recent one can be useful.

server-63635546043.txt

server-63635609275.txt

Edited July 14, 2017 by omerome

[omerome 4]

Here is the latest log.

 

It seems that at 3AM that's the time when the logs are being written. Maybe then a process or service is possibly hanging up Emby Server?

 

EDIT: It might be the Server Configuration Backup plugin. It's the only one that I think it being run during that time. I will remove it for now and see what happens.

server-63635609275.txt

Edited July 15, 2017 by omerome

[mrvirtual 9]

Sorry if this is not appropriate but Me Too!

I too am having this issue but, I can get it to fail by too much clicking about in emby web interface, by starting movies, stopping and starting something else. It also seems to fail after an hour or two.

 

I can restart it by ssh'ing into the server with putty and using su to service emby-server restart.

- you need to enable SSH on the FreeBSD jail.

 

If I do an emby-server status it returns that the service is running, so it's not crashing rather something within the service, by apperance somenting web interface related is failing.

If I perform the restart while attempting to load the web interface, it is not timing out, it then times out and fails.

so when emby is becoming unresponsive, the interface is accepting connections but not serving up the full content.

 

Hoping this can help narrow down something.

 

 

 

server.log

[omerome 4]

Sorry if this is not appropriate but Me Too!

I too am having this issue but, I can get it to fail by too much clicking about in emby web interface, by starting movies, stopping and starting something else. It also seems to fail after an hour or two.

 

I can restart it by ssh'ing into the server with putty and using su to service emby-server restart.

- you need to enable SSH on the FreeBSD jail.

 

If I do an emby-server status it returns that the service is running, so it's not crashing rather something within the service, by apperance somenting web interface related is failing.

If I perform the restart while attempting to load the web interface, it is not timing out, it then times out and fails.

so when emby is becoming unresponsive, the interface is accepting connections but not serving up the full content.

 

Hoping this can help narrow down something.

No, that's perfectly acceptable because your problem is very similar to what I am going through. I don't get any "error", but the web interface is not responsive or the server doesn't connect when I try load Emby. I just get a spinning icon, like it's loading and only after I manually restart the server, does it go back to normal.

 

I have tried removing plugins, checked my router settings/rules, but it just keeps happening. At first I thought it was something wrong with my OS, but Emby is the only plugin/jail that gives me problems so there is something weird that is happening and I can't figure it out yet.

[mrvirtual 9]

Occasionally for some systems it can be a little too much. We are switching the image processing library from image magick to skia which lightens the load a bit, but that switch hasn't made it's way to the bsd release yet.

 

 

@@omerome, I see you are running at least 4 other Jails, I am not sure how FREENAS handles memory allocation, from the guest/Jail point of view.

 

Just thinking about what @@Luke has said previously about emby server being stressed, could your freenas be a little stressed running ~5Jails )Guest OS's) and the FREENAS services, being that FREENAS requires at least 8GB of memory?

 

I removed my emby backup plugin and it has so far been more stable, although again not functioning earlier today.

 

It seems that when I perform quick operations, this can stress the Jail and the web interface fails form that point until I manually restart the service. I do not recall having this issue on Ubuntu running emby, I lost that functionality with a FREENAS upgrade . but again I am only running one Jail and was one Virtual Box VM in the VB Jail Plugin.

 

Just did a quick search and found this https://wiki.freebsd.org/Jails

 

check some of the FREENAS performance reports/graphs and see what the memory and CPU are doing when you think this is occurring.

 

Have you enabled SSH to the Jail? I find this makes restarting the service much easier.

[Luke 42077]

Thanks @@mrvirtual !

[omerome 4]

 

 

@@omerome, I see you are running at least 4 other Jails, I am not sure how FREENAS handles memory allocation, from the guest/Jail point of view.

 

Just thinking about what @@Luke has said previously about emby server being stressed, could your freenas be a little stressed running ~5Jails )Guest OS's) and the FREENAS services, being that FREENAS requires at least 8GB of memory?

 

I removed my emby backup plugin and it has so far been more stable, although again not functioning earlier today.

 

It seems that when I perform quick operations, this can stress the Jail and the web interface fails form that point until I manually restart the service. I do not recall having this issue on Ubuntu running emby, I lost that functionality with a FREENAS upgrade . but again I am only running one Jail and was one Virtual Box VM in the VB Jail Plugin.

 

Just did a quick search and found this https://wiki.freebsd.org/Jails

 

check some of the FREENAS performance reports/graphs and see what the memory and CPU are doing when you think this is occurring.

 

Have you enabled SSH to the Jail? I find this makes restarting the service much easier.

That would be disappointing if the server was being stressed from the beginning since I thought my components were sufficient enough for it not to be. When I checked the CPU usage in the reporting tab in FreeNAS, it doesn't seem like my server is being taxed at all and most of the time, it looks like the CPU is being idle. I actually thought went overkill with the parts for a while lol.

 

And yeah, I am using putty to remote into the particular jail. Thanks.

 

Here is what I am running:

 

ASRock E3C226D2I Mini ITX Server Motherboard

Intel Xeon E3 1230 Processor

16GB (2x8GB) Crucial Memory

6x3TB WD Red HDs

Edited September 2, 2017 by omerome

[mrvirtual 9]

That's a well spec'd host, the CPU is more than worthy as is the memory for running Freenas, freenas utilises all the resources it can and  8GB is the minimum recommended, the ZFS pools hit the memory quite a bit as I understand it.

 

I only run a HP microserver 16GB and Celeron CPU, 2 core. the server only had 4GB I thing to start and Freenas was OK but sluggish.

 

the thing is from what I can tell from Jails is that the entire system resources are granted to the Jail, so there is no memory management in regard to how much memory and CPU the Jail and other resources consume, if it was VMware for example we could place resource limits on what the VM (Jail) can utilise.

googling around I have come up with things that the good folks out there have developed however, adding this to Freenas may have undesirable effects upon the NAS system.

 

Think of it this way you have:

1 Socket, 4 cores and 8 threads;

16 GB Memory.

So your CPU can handle quite a bit of processing throughput, it's a Xeon so it's designed for enterprise processes and virtualization this is great. You in theory should be able to have 1 - 6 or even 1 to 10 vCPU over commitment. that's 1 physical core to say 6, as virtualization does not execute idle threads. just on cores alone 24 VM's

 

Each Jail then also requires memory, your physical OS/Host in this case Freenas needs 8GB minimum to perform. leaving you say 8GB to play with. however we can't tell freenas to limit itself to 8GB.

Freenas takes all it can to provide you with the best NAS service it can along with the ability to do other stuff like Jails (plugins etc.)

FreeBSD jails present the full amount of system resources to each Jail, so each jail sees 16GB.

If processes are running on the jails, and taking more memory and more memory the remaining 8GB is soon going to run out, in fact from what I can see in Freenas it takes all the available memory and only leaves a bit free.

My Freenas shows 1.6GB free, this is if I'm running a JAIL or not

 

I would expect 2 - 3 Jails would be a comfortable limit for your system as it is memory constrained IMHO.

 

Can you try shutting down the other Jails and seeing how this goes?

 

This command will show you the system memory on FreeBSD 

FreeBSD memory command sysctl hw.physmem

 

Hope this helps

[MRobi 161]

Here is the latest log.

 

It seems that at 3AM that's the time when the logs are being written. Maybe then a process or service is possibly hanging up Emby Server?

 

EDIT: It might be the Server Configuration Backup plugin. It's the only one that I think it being run during that time. I will remove it for now and see what happens.

I had a similar issue a while back and it turned out it was the scheduled task for the trakt plugin. I deleted the scheduled task and it fixed everything right up. Running the task manually has never caused an issue aside from it showing I'm at 40 million % complete lol

[omerome 4]

That's a well spec'd host, the CPU is more than worthy as is the memory for running Freenas, freenas utilises all the resources it can and  8GB is the minimum recommended, the ZFS pools hit the memory quite a bit as I understand it.

 

I only run a HP microserver 16GB and Celeron CPU, 2 core. the server only had 4GB I thing to start and Freenas was OK but sluggish.

 

the thing is from what I can tell from Jails is that the entire system resources are granted to the Jail, so there is no memory management in regard to how much memory and CPU the Jail and other resources consume, if it was VMware for example we could place resource limits on what the VM (Jail) can utilise.

googling around I have come up with things that the good folks out there have developed however, adding this to Freenas may have undesirable effects upon the NAS system.

 

Think of it this way you have:

1 Socket, 4 cores and 8 threads;

16 GB Memory.

So your CPU can handle quite a bit of processing throughput, it's a Xeon so it's designed for enterprise processes and virtualization this is great. You in theory should be able to have 1 - 6 or even 1 to 10 vCPU over commitment. that's 1 physical core to say 6, as virtualization does not execute idle threads. just on cores alone 24 VM's

 

Each Jail then also requires memory, your physical OS/Host in this case Freenas needs 8GB minimum to perform. leaving you say 8GB to play with. however we can't tell freenas to limit itself to 8GB.

Freenas takes all it can to provide you with the best NAS service it can along with the ability to do other stuff like Jails (plugins etc.)

FreeBSD jails present the full amount of system resources to each Jail, so each jail sees 16GB.

If processes are running on the jails, and taking more memory and more memory the remaining 8GB is soon going to run out, in fact from what I can see in Freenas it takes all the available memory and only leaves a bit free.

My Freenas shows 1.6GB free, this is if I'm running a JAIL or not

 

I would expect 2 - 3 Jails would be a comfortable limit for your system as it is memory constrained IMHO.

 

Can you try shutting down the other Jails and seeing how this goes?

 

This command will show you the system memory on FreeBSD 

FreeBSD memory command sysctl hw.physmem

 

Hope this helps

Thanks for the info!

 

Just as an update: I deleted a jail that wasn't used anymore and rebuilt the Emby jail and now everything looks to be running great.

 

I am still not 100% sure what was the main case, but maybe it was a misconfiguration somehow or a corrupted installation. I did notice that since I upgraded to FreeNAS v11, the newer jail template needed to be downloaded as well and maybe that had something to do with it since I was still using 9.10 templates with my other jails.

 

Thanks to everyone for their help.

[mrvirtual 9]

Hi @@Luke,

 

I am having this issue once again on BSD, Freenas Plugin.

 

I recently upgraded to Freenas11 and rebuilt my jails from scratch, restored my config. but Emby service stops accepting connections, you can connect to HTTP but it does not load, only times ojut when the service is stopped for the restart. pre and post server logs attached.

Thanks in advance.

 

PreRestart.log

PostRestart-Server.log

[omerome 4]

Hi @@Luke,

 

I am having this issue once again on BSD, Freenas Plugin.

 

I recently upgraded to Freenas11 and rebuilt my jails from scratch, restored my config. but Emby service stops accepting connections, you can connect to HTTP but it does not load, only times ojut when the service is stopped for the restart. pre and post server logs attached.

Thanks in advance.

The problem came back for me as well. I made no updates or changes to my server in weeks so I can safely rule out a version issue.

 

Here is my log.

server-63639820800.txt

[Luke 42077]

3.2.30 will be a little lighter on it's feet so once that is available for BSD I'd be curious to know the results. Thanks.

[mrvirtual 9]

Just tried to update to 3.2.30, but not yet relaease. I'll keep trying and let you know.

[mrvirtual 9]

Hi, any ETA on the emby 3.2.30 update for Freebsd?

 

thanks