Emby server does not start anymore after failed update

[DrZombi 4]

Hi Guys,

Sorry to bother you but I am more of a Windows than a FreeBSD user so I'm totally lost when it comes to troubleshooting my emby plugin.

I kept having notifications in FreeNAS about an emby server update so I just went to plugins -> Update -> Update Jail as well. I got a failed to update message and now the emby server does not start anymore. I confirmed with service emby-server status that the service is not running and I cannot start it with service emby-server start. It just says Starting Emby server and then nothing.

I saw in another post that it may be worth adding the mlock parameter to the jail.conf so that's what I did but with no more luck.

 

Can you please guys help me get my emby server up again ?

Thanks a lot in advance.

 

P.S: This may give you some information. Do not hesitate to ask for more.

root@emby-jail:~ # pkg info emby-server
emby-server-4.7.4.0_1
Name           : emby-server
Version        : 4.7.4.0_1
Installed on   : Wed Jun 29 10:46:53 2022 CEST
Origin         : multimedia/emby-server
Architecture   : FreeBSD:12:amd64
Prefix         : /usr/local
Categories     : multimedia
Licenses       : UNKNOWN
Maintainer     : driesm@FreeBSD.org
WWW            : https://emby.media/
Comment        : Media server to organize, play & stream audio/video to variousdevices
Options        :
        LAME           : on
Shared Libs required:
        libzvbi.so.0
        libx265.so.192
        libx264.so.163
        libwebpmux.so.3
        libwebpdemux.so.2
        libwebp.so.7
        libvpx.so.7
        libvorbisenc.so.2
        libvorbis.so.0
        libva.so.2
        libva-drm.so.2
        libunwind.so.8
        libunwind-x86_64.so.8
        libtheoraenc.so.1
        libtheoradec.so.1
        libpng16.so.16
        libopus.so.0
        libmp3lame.so.0
        libmfx.so.1
        libjpeg.so.8
        libinotify.so.0
        libgnutls.so.30
        libfribidi.so.0
        libfreetype.so.6
        libfontconfig.so.1
        libdrm.so.2
        libdav1d.so.6
        libchromaprint.so.1
        libass.so.9
        libaribb24.so.0
        libOpenCL.so.1
Shared Libs provided:
        libmscordbi.so
        libmscordaccore.so
        libhostpolicy.so
        libhostfxr.so
        libdbgshim.so
        libcoreclrtraceptprovider.so
        libcoreclr.so
        libclrjit.so
        libSystem.Security.Cryptography.Native.OpenSsl.so
        libSystem.Net.Security.Native.so
        libSystem.Native.so
        libSystem.IO.Compression.Native.so
        libSystem.Globalization.Native.so
        libSkiaSharp.so.80.2
Annotations    :
        FreeBSD_version: 1203000
       libclrjit.so
        libSystem.Security.Cryptography.Native.OpenSsl.so
        libSystem.Net.Security.Native.so
        libSystem.Native.so
        libSystem.IO.Compression.Native.so
        libSystem.Globalization.Native.so
        libSkiaSharp.so.80.2
Annotations    :
        FreeBSD_version: 1203000
        repo_type      : binary

 

[Duffyx 25]

Did you restart the jail so that the "mlock" parameter becomes active?
You can try running the EmbyServer binary manually by going to the relevant directory in the jail (/usr/local/lib/emby-server/system/) and then, "./EmbyServer".
This will print more debug info.

Edited June 29, 2022 by Duffyx

[DrZombi 4]

Hi Duffyx,

Thanks a lot for your answer. Yes I did restart the jail and even the whole TrueNAS.

Strangely, when I run the EmbyServer manually with your command line, it launches fine and the web interface is accessible. Although it presents me with the launching wizard and 0 libraries, like a brand new installation. I really fear that my DB has been wiped with all my hours of configuration and tweaking gone is the wild.

I also tried to restart the Emby Plugin after that to see if it changed anything but no, the service still does not start by himself. The only way to launch Emby Server is now to run your command line.

Does that give you any clue ? Do you think there can be any way to get my metadata back or is it gone forever ??

[Duffyx 25]

Thats normal, as you didn't specify the datadirectory it should use to start up, can you copy paste the start-up script that's installed? (/usr/local/etc/rc.d/emby-server)
Does it look like this: emby-server.in « files « emby-server « multimedia - ports - FreeBSD ports tree
You can add the command line options, and see if it also runs by keep adding them, eg "./EmbyServer -os freebsd -programdata /var/db/emby-server"

Try adding "ip6 = inherit;" to the jail config too

Edited June 29, 2022 by Duffyx

[DrZombi 4]

The script looks like this :

emby-server
root@emby-jail:/usr/local/etc/rc.d # cd emby-server
emby-server: Not a directory.
root@emby-jail:/usr/local/etc/rc.d # cat emby-server
#!/bin/sh

# $FreeBSD$
#
# PROVIDE: emby-server
# REQUIRE: LOGIN
# KEYWORD: shutdown
#
# Add the following lines to /etc/rc.conf.local or /etc/rc.conf
# to enable this service:
#
# emby_server_enable: Set to yes to enable the emby-server service.
#                       Default: no
# emby_server_user: The user account used to run the emby-server daemon.
#                       This is optional, however do not specifically set this to an
#                       empty string as this will cause the daemon to run as root.
#                       Default: emby
# emby_server_group: The group account used to run the emby-server daemon.
#                       This is optional, however do not specifically set this to an
#                       empty string as this will cause the daemon to run with group wheel.
#                       Default: emby
# emby_server_data_dir: Directory where emby-server configuration
#                       data is stored.
#                       Default: /var/db/emby-server
# emby_server_ffmpeg: Path of the ffmpeg binary.
#                       Default: %%PEFIX%%/lib/emby-server/bin/ffmpeg
# emby_server_ffprobe: Path of the ffprobe binary.
#                       Default: %%PEFIX%%/lib/emby-server/bin/ffprobe

. /etc/rc.subr
name=emby_server
rcvar=${name}_enable
load_rc_config ${name}

[ -z "${emby_server_user}" ] && emby_server_user=emby
[ -z "${emby_server_group}" ] && emby_server_group=emby
[ -z "${emby_server_data_dir}" ] && emby_server_data_dir=/var/db/emby-server
[ -z "${emby_server_ffmpeg}" ] && emby_server_ffmpeg=/usr/local/bin/ffmpeg
[ -z "${emby_server_ffprobe}" ] && emby_server_ffprobe=/usr/local/bin/ffprobe

: ${emby_server_enable:="no"}
: ${emby_server_user:="emby"}
: ${emby_server_group:="emby"}
: ${emby_server_data_dir:="/var/db/emby-server"}
: ${emby_server_ffmpeg:="/usr/local/lib/emby-server/bin/ffmpeg"}
: ${emby_server_ffprobe:="/usr/local/lib/emby-server/bin/ffprobe"}

export LD_LIBRARY_PATH=/usr/local/lib

pidfile="/var/run/emby-server.pid"
procname="/usr/local/bin/mono"
command="/usr/sbin/daemon"
command_args="-f -p ${pidfile} ${procname} /usr/local/lib/emby-server/system/EmbyServer.exe \
        -os freebsd \
        -ffmpeg ${emby_server_ffmpeg} \
        -ffprobe ${emby_server_ffprobe} \
        -programdata ${emby_server_data_dir} \
        -updatepackage emby-server-freebsd_{version}_amd64.txz"

start_precmd=emby_server_precmd
emby_server_precmd()
{
        [ -f ${pidfile} ] || install -g ${emby_server_group} -o ${emby_server_user} -- /dev/null ${pidfile}

        [ -d ${emby_server_data_dir} ] || install -d -g ${emby_server_group} -o${emby_server_user} -- ${emby_server_data_dir}

}

stop_postcmd=emby_server_postcmd
emby_server_postcmd()
{
        rm -f ${pidfile}
}

run_rc_command "$1"

root@emby-jail:/usr/local/etc/rc.d #

 

This looks like the one you provided at first sight.

Also, when running the application with your extended command line, it finds my db and I can get my conf back (thanks a looooooot).

This still does not explain why the service cannot start but that's a big improvement !

Any idea of what could be borken ?

 

[Duffyx 25]

The start-script is still using the mono runtime, but you have installed a dotNET version, I think? Weird.
I only maintain the official FreeBSD port, which works fine. I'm not sure where the plugin in FreeNAS gets all the sources from, like the startscript etc.

In the jail: pkg delete -a, pkg install emby-server, run the service.

Although when you do this, it wont be able to show in the services maybe in the webUI in FreeNAS.

Edited June 29, 2022 by Duffyx

[DrZombi 4]

pkg delete -a, pkg install emby-server, run the service

I assume that you are sure it won't delete all my database and configuration ?

[Luke 37028]

Hi, are you actually still on FreeNAS and not TrueNAS? How did you initially install Emby Server?

[DrZombi 4]

Hi Luke,

No that was just bad language habits ! I'm running TrueNAS-12.0-U8.1.

I cannot remember for sure how I deployed Emby initially. But since I'm not very at ease with FreeBSD, I think that was through the plugin manager in TrueNAS. There is an Emby plugin in the community section. When deployed it worked perfectly and I even upgraded it a few times. But this time the upgrade failed and now if I restart the plugin/jail, Emby won't restart automatically, the only way to start it is through the command line provided above.

[DrZombi 4]

Hi Guys,

I'm adding something new. In fact the solution is not really stable as after some time (a few hours I'd say), the Emby server goes down and I need to relaunch it manually.

I would be glad to follow @Duffyx and go with pkg delete -a, pkg install emby-server to renew the server with a fresh install (that's what I understand from reading those lines), but I'm still worried about my database. I REALLY do not want to loose all those hours of tweaking and personnalization on movies metadata.

Can some please confirm that it is safe to uninstall/resintall the packages and that it will preserve the database ?

Is there any way to "backup" my database or at least the metadata for all medias, just to be safer than safe ?

 

Thanks a lot in advance !

[wareb 3]

HI!

I have the same problem.

I'm running TrueNAS-13.0-RELEASE

Emby not auto-start after update to 4.7.4.0_1

I'm install second plugin emby2 but clear install does't help!

And i REALLY do not want to loose all those hours of tweaking and personnalization on movies metadata TOO.

Edited June 30, 2022 by wareb

[GunbleR 1]

Hi. Same thing. According to freshports, at 4.7.4.0_1 they moved from mono to .Net.

I tried fresh jail (13.1) with [pkg install embyserver], it wont try to start:" service start emby-server" produce no pid

/var/db/emby-server is not populated. 

Atm rolled back to daily snapshot, maybe will try to troubleshoot later

Edited June 30, 2022 by GunbleR

[DrZombi 4]

Hey !! Thanks a lot @GunbleR !!!

I did not know that the Emby plugin takes an automatic snapshot before upgrading and did not even think about it ! I feel so dumb right now... I should really take a deep tour of my TrueNAS because knowing things like this can totally save your day !!

I did the same as you in the end, just rolled back to the snapshot taken before the upgrade, rebooted the TrueNAS just for the sake of my psychorigidity and everything went back up perfectly ! I will care about upgrading Emby later, for now I'm just happy that I found everything back !

Thanks a lot !

[Luke 37028]

Make sure to enable mlock on the jail.

[DrZombi 4]

Hi Luke,

Yes I saw that on another thread after I had the problem. I enabled it but it changed nothing. Maybe the harm was already done ? Do you think it could be good if I have it enabled before launching the upgrade ?

[Luke 37028]

It shouldn't matter because there have been plenty of others who enabled it after already trying to launch. 

[DrZombi 4]

So there must be another problem...

[Luke 37028]

Is any emby server log file generated?

[DrZombi 4]

I cannot tell what's inside, now that I have rolled back but I remember that I went to the log directory and yes, there were some files generated.

[wareb 3]

Hope the problem will fixed

I'm rollback to 4.7.1.0

[Luke 37028]

2 hours ago, DrZombi said:

I cannot tell what's inside, now that I have rolled back but I remember that I went to the log directory and yes, there were some files generated.

Can you upgrade again and then attach the latest log file? Thanks.

[dhenzler 9]

Be sure that the allow_mlock is checked... see below

[DrZombi 4]

Hi all !

I took some guts and time (and a snapshot ) to re-test an update and I can confirm that this time it went on smoothly.

So the solution is confirmed to be that allow_mlock must be configured on your jail BEFORE the upgrade. Afterwards it will be too late to activate it and re-upgrade.

 

Thanks a lot guys !

[dctheee 2]

On 8/25/2022 at 2:13 AM, DrZombi said:

Hi all !

I took some guts and time (and a snapshot ) to re-test an update and I can confirm that this time it went on smoothly.

So the solution is confirmed to be that allow_mlock must be configured on your jail BEFORE the upgrade. Afterwards it will be too late to activate it and re-upgrade.

Wanted to log in and thank you for posting this solution! This solved an issue I had with updating Emby!

[wareb 3]

Hi, im enabled "allow_mlock" and then update from 4.7.1.0 to 4.7.6.0, confirms that all works!