losing server connection almost daily - SSD upgrade causing?

[whyteknt 3]

I recently upgraded my NVME boot drive to a larger 2tb drive.    On the Boot drive I have Emby Server running.

I think something about that drive cloning process maybe is causing me to lose connection about every day.   All my clients can no longer find the server within about 24 hrs (not always at the same time of day though)?   I've looked through the server log files, and I do see an ERROR message:  System.Net.Http.HttpRequestException: System.Net.Http.HttpRequestException: No connection could be made because the target machine actively refused it

when I reboot the emby server, everything connects and works fine again - until the next day.

I did also move my library of TV recording over manually to the new boot drive, previously was on an external drive.   and I changed the default library location of the live TVs to the new boot as well.

something in my process must have been too much of shortcut and I've broken something?

before I do a complete re-install of emby server in windows (which perhaps is what is needed), hoping someone might have some suggestions?   I always worry about losing all my series settings if I do a re-install of the server.....

 

(expanded error message)

System.Net.Http.HttpRequestException: System.Net.Http.HttpRequestException: No connection could be made because the target machine actively refused it. (xxx.xxx.x.xxx:80) [I masked]
     ---> System.Net.Sockets.SocketException (10061): No connection could be made because the target machine actively refused it.
       at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.ThrowException(SocketError error, CancellationToken cancellationToken)
       at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.System.Threading.Tasks.Sources.IValueTaskSource.GetResult(Int16 token)
       at System.Net.Sockets.Socket.<ConnectAsync>g__WaitForConnectWithCancellation|277_0(AwaitableSocketAsyncEventArgs saea, ValueTask connectTask, CancellationToken cancellationToken)
       at System.Net.Http.HttpConnectionPool.ConnectToTcpHostAsync(String host, Int32 port, HttpRequestMessage initialRequest, Boolean async, CancellationToken cancellationToken)
       --- End of inner exception stack trace ---

[Luke 42078]

Hi there, please attach the complete emby server log file. Thanks.

[whyteknt 3]

thanks Luke - I've attached the whole server log file now.

 

same thing happened again today.  something is causing a disconnect, and my clients are not re-establishing.

embyserver (before reboot).txt

[Luke 42078]

Are you saying the server process crashed? if so, is there anything in windows event viewer about it?

[whyteknt 3]

No, I don't think that the emby server is crashing  on my windows computer (I don't think).  But when I go to try and view emby through any of my clients (Android phone, laptop, firetv, LGsmart tv), Emby on each of those is disconnected from the server, and I can't reconnect from the client.  If I reboot the emby server on my windows computer, then all the clients reconnect; until the following day when they are disconnected again.

I had not thought to look in Event Viewer to see if there were problems showing up there. It appears there are.  The one that pops up daily that could be  related to my problem is this:

 

The server {7966B4D8-4FDC-4126-A10B-39A3209AD251} did not register with DCOM within the required timeout.

 

Does that offer any clue?   it shows as an error at least once a day since the date I swapped/cloned my boot drive to a larger NVME, so it appears to match the frequency of when I'm losing connection.  

 

this is the event detail (I'm always paranoid about revealing license keys and such, so I've masked anything like that in the message below)

<Event xmlns="http://schemas.microsoft.com/win/2004/08/events/event">

- <System>

  <Provider Name="Microsoft-Windows-DistributedCOM" Guid="{xxxxxxxx-xxxA-xxxx-xxxx-xxx3A001407E}" EventSourceName="DCOM" />

  <EventID Qualifiers="0">10010</EventID>

  <Version>0</Version>

  <Level>2</Level>

  <Task>0</Task>

  <Opcode>0</Opcode>

  <Keywords>0x8080000000000000</Keywords>

  <TimeCreated SystemTime="2023-08-13T18:33:11.4644487Z" />

  <EventRecordID>33970034</EventRecordID>

  <Correlation ActivityID="{xxxxxx7e-cxx2-0000-9x-7xx74562cad901}" />

  <Execution ProcessID="1252" ThreadID="17592" />

  <Channel>System</Channel>

  <Computer>BRIX-BOX</Computer>

  <Security UserID="S-1-5-21-1xxxxx07511-62xxxxx8-710751788-1001" />

  </System>

- <EventData>

  <Data Name="param1">{7966B4D8-4FDC-4126-A10B-39A3209AD251}</Data>

  </EventData>

  </Event>

[Neminem 1519]

For that error try this fix.
Its might help fixing your windows error.

Event ID 10010, Server did not register with DCOM within timeout (thewindowsclub.com)

[whyteknt 3]

Thank you Jaycedk!   right after I posted that last response, I found this same information looking up that error as well.

I made that change last night and as of right now, I have NOT had the disconnect occur - so I'm hopeful that may have fixed things.

 

appreciate the help and advice.    I suspect my problem was/is more 'windows' than 'emby' related; emby was just the only thing that was giving me a clue there was an issue at all.

 

[Luke 42078]

Please let us know if that makes a difference. Thanks.

[visproduction 315]

WT, It would be nice if you made jaycedk response as the solution.  That gives him extra points.

[whyteknt 3]

well, I spoke a bit too soon.  I think it's on the right track, but the problem repeated just an hour ago.   I'll continue to troubleshoot.  I do think it's a windows issue, not emby though, so jaycedk was on the right path.

 

[Luke 42078]

14 hours ago, whyteknt said:

well, I spoke a bit too soon.  I think it's on the right track, but the problem repeated just an hour ago.   I'll continue to troubleshoot.  I do think it's a windows issue, not emby though, so jaycedk was on the right path.

 

Let us know what you find. Thanks !

[Luke 42078]

Also when this happens are you able to use the web interface directly on the server machine?

[whyteknt 3]

so I'm still having this issue occur nearly every other day; not sure when during the day but sometime between when I watched the last night to the next night of recorded TV. 

 

When this happens, in addition to disconnecting all the clients, it stops my scheduled recordings from being recorded, so it's made the DVR function unreliable.   

 

Luke, in response to your question; "YES", I am able to use the web interface directly on the SERVER machine when this happens.  And strangely enough, the act of just 'browsing library' on the active server machine, seems to have the same impact as rebooting the server, and all my clients once again regain access to the server.

I can't see that there is any Windows Error occurring that ties to this event.  There is however a Windows 'warning' for 'surprise removal' that occurs regularly (and may synch up to the same time the clients drop connection):

USB device draining system power when system is idle.             
             
USB Device: VID: 0x1BCF PID: 0x28C4 REV: 0x704             
Removal action failed: QueryRemovalInitiated

 

I previously used to record my Live TV content on an external USB hard drive, but moved to an internal SSD some months ago, right before this issue started (and I did not have this issue at all before I moved to the SSD).   It it possible that Emby still thinks I'm trying to record to an external USB device, and when it's not finding it, that's causing a problem?  I appear to have all my library settings and locations updated in the Emby settings to the new SSD location, but maybe I'm missing something??

I'm including the most recent emby server file, and also a 'hardware detection file' that perhaps is related to this issue?

 

Any advice would be appreciated.

 

the previous fix attempt on the Event ID 10010 does not appear to be helpful for this issue.   I can't tell that that event was the cause at this point (appears to not be).

hardware_detection-63834167785.txt embyserver.txt

[Luke 42078]

What exactly is the problem at this point? Just a dvr problem? Because I do see some cases in your log of your HDHR rejecting requests. You could try rebooting that device and see if that resolves it.

[whyteknt 3]

I've rebooted the server device many times over the past few months, that does not resolve the issue. 

I'm about to the point where I think I may need to uninstall Emby and just do a fresh install and see if that fixes it.  Just hoping I can figure out a way to not loose all my series scheduled recordings.  I'm not sure I want to use the plugin backup/restore option since that might must bring my 'issue' along with that restored backup.

so sounds like you can see that HDHR requests were rejected, but can't see the cause from the logs.  was hoping they would have shown more detail.   Anything else I could provide that would assist?

Edited October 30, 2023 by whyteknt

[Luke 42078]

Not reboot the server device, I mean the actual HDHR. Please try that.

But also do you have more than one HDHR? 

[whyteknt 3]

just one, a HD homerun quad that I've been using for some time.  I'll reboot that and see if it helps, but I seem to recall trying that before (looking for any firmware update, and then rebooting). 

I just went and checked again for FW updates for the HDhomerun, and there IS a new firmware version dated October 20, so I'll update that and reboot and then see if the problem continues.  

 

I also did somewhat recently 'remove' 2 old devices; i had two older dual HD homeruns installed as 'sources', and had just left them on  EMBY as recognized devices for a year or two, even though they were no longer physically hooked up.  I just moved to the single Quatro unit only.  I do think I removed those 2 dual HD homeruns from Emby  (deleted them as a Live 'TV source') a few months ago - and maybe that is related to my current issue?

 

I'll respond back after I've had some time with the new firmware for the HD Homerun quattro running, to let you know if that resolved the problem or not.

 

Thank you Luke

[whyteknt 3]

so it's been only 3 hrs, and problem occurred again during that window; the clients lost connection at some point during that time (same issue as before), and scheduled recordings would not occur if any had been scheduled to start.

I'm attaching updated log files in case it helps.   including the Hardware detection log file because that appears to have a time stamp during that 3 hr window.

only thing in the logs that looks suspicious to me is this comment:

System.Net.Http.HttpRequestException: System.Net.Http.HttpRequestException: No connection could be made because the target machine actively refused it. (xxx.xxx.x.xxx:80) [ anonymized by me]

 

Thanks in advance Luke.

embyserver (2).txt hardware_detection-63834279040 v2.txt

Edited October 31, 2023 by whyteknt

[visproduction 315]

I am unsure what the exact setup is for your USB drive, but maybe this will help.  If you use a USB drive and the hardware that is connected to it is restarted, the USB drive often is not recognized again unless there is some special reboot process.

For example: USB drives connected to computers or routers are reloaded and available during the reboot process of the hardware.  But USB connected to TV's will not do this and is not recognized for several TV brands I have tested including Samsung and Sony.  In order for these TV's to recognize the USB drive automatically, you need to have the USB drive receiving DC power.  Of course, that only applies to USB remote to SSD or hard drives in a housing that has a power supply.  If you don't have this setup, turning off the TV and back on again doesn't see the USB drive and you have to load it again from the menu.  I think these TV's don't wait for the drives to reinitiate and just shows them as not connected and they become unavailable.

Are you perhaps running into this issue?

Edited November 2, 2023 by visproduction

[whyteknt 3]

Thanks Vis.   Appreciate the thought.

Problem is- I don't use an external USB drive for my setup any longer.  I used to, back a few months ago, but I moved my library data to an internal SSD (8tb SSD).    

Perhaps because my libraries 'used' to be on an external drive, EMBY is still looking for it there(?); but I don't know why it would.  I've changed all my library locations in Emby to the new drive location.  nothing I can find in my setup refers back to the old external USB drive location.

My internal SSD drive with the libraries is in a NUC11 (intel NUC mini pc box), and even though I have NO external USB drives present, maybe some other device (Keyboard?, mouse? Camera?) that is connected to USB is somehow causing a power drain and windows is trying to shut something down.   I'm still getting a Windows 'warning' (not an error, just a warning) that 'USB device draining system power when system is idle' (surprise removal [Event ID 196])

 

I've tried to go into the 'power management' of all the USB hubs and drives to disable anything that would allow windows to turn anything off, but doesn't seem to help.

Next:  I'm going to uninstall my HD homerun device, re-install.  If that doesn't work, I may just uninstall Emby and re-install again, hoping to clean up whatever is causing this.  was hoping to avoid this so I didn't lose my Series recording setups.

I've used EMBY for many years with very few issues.  This has to be related to my changing the drive locations for my library, or the fact that I swapped out my boot drive to a new NVME drive as well.  Something about one of those changes must be causing some conflict or confusion in the Emby setup.    Either that, or a NUC11 driver/Bios issue a suppose....

appreciate you taking the time to post a suggestion.  I'll post any results after I continue to try fixes.....

[whyteknt 3]

Nothing I've tried to change has fixed this from occurring.   My DVR function still stops working almost daily, and the only way to fix is to open emby on the Server computer, or restart the emby server.    When the DVR stop working, all my clients also lose access until I restart.

Luke,  can you see any other reason why my recordings are not happening from the logs (I have re-started/rebooted my HDHR and even updated to new Firmware that was available 10/20 - none of which seem to help)?     If so, please let me know what to try.   You said from looking at the log, that my HDHR was rejecting requests to record.   Any other causes I should be considering?   I will look on the SiliconDust site to see if there are any clues there, in case it is HDHR related.

If no other suggestions, I'll plan to uninstall Emby this week, and re-install.    wish I could save my series schedules,  but oh well.  probably good to clean it up anyway.   

 

[whyteknt 3]

So I did a complete uninstall of Emby server (removed all config, plugins) and then reinstalled.  It took a while to get most setting back to where they were, and it'll take longer to get some of my Series schedules back in, since I think I have to wait for them to show up in the guide data.

 

I had hoped this would resolve my issue with my DVR stopping, and all my clients losing access (clients include: LG TV, Firestick, Emby Theater (PC), Android phone) - but it DID NOT.   I lost connection about 30 hrs after the re-install (overnight last night).   

Access and DVR functions get restored if I either 'browse library', 'config library' or restart emby server - on the server PC; but only short term.  Eventually it fails again.  Something is getting 'turned off' and impacting the DVR function, I just can't figure out what.

I've attached the most recent Server log and hardware log.   I see multiple hardware logs listed, do I need to include them all (there are 4 or 5 a day created)?   I can't tell that the 'anonymizer' feature on the log downloads is working.  When I open those files, I see IPs and usernames that think should be blinded out.  So I've taken an additional manual steps to find and replace those identifiers in my attached logs.   I do not have confidence that I'm using the anonymizer toggle correctly in the log download process, or it just isn't working to blind things.

 

The only other step I can think to try for my DVR issue, is perhaps if I leave the 'Browse Library' window on my server PC open 24/7, perhaps that will prevent my DVR features from stopping working.   that's my next test.  I don't know why that would be necessary, but if it works, I might be able to live with it.

Also, I saw in another post where someone was told to turn on the 'Enable DLNA server' switch in the DLNA setup.  I know I had that set to 'off' after I reinstalled the ember server, I may have had it OFF before as well.  Any chance that toggle could have been the culprit?   Turning it on now.

 

Appreciate the help and advice to get my system back to being a working/reliable DVR.

 

other notes:  Server is on a NUC11,  running Windows 11.    Router is a TP AXE5400.   I've made no recent changes to the router.  Windows updates regularly.   only Errors or Warnings I see in the Windows event viewer are for DistributedCOM [event ID 10010] and USBhub3 'surprise removal' (Event ID 196]; both of which are just warnings, not errors.

embyserver 11-10-23.txt hardware_detection-63835210802 11-10-23.txt

[visproduction 315]

Related:
I have fixed the problem but it required down grading windows from version (1903) to (1809) and then carrying out a Windows Reset.  I think my system messed up the drivers or something, not really sure how or when it happened but the Reset has fixed all my problems.
From: https://community.intel.com/t5/Media-Intel-oneAPI-Video/MFXInit-returns-MFX-ERR-UNSUPPORTED-since-Windows-10-1903-update/m-p/1131355
 

[whyteknt 3]

Thanks Vis

looks like that was a win10 issue; I'm on Win 11 (for my Emby server), so not sure it would apply to me (?)   i'll scan through my log to see if I see similar debug messages related to what you're pointing out.

appreciate the input!

[Luke 42078]

Hi, has this helped? Are you still having an issue with this?