New Emby Server - reverse proxy, NordVPN and slow scan questions

[cochize1 55]

I wanted to ask about my new setup and the minor problems I’ve encountered so far. Last week, I transferred from Emby Server on my Synology NAS to Emby Windows Server on my NUC. I noticed a massive increase in the server's responsiveness, and I think there is much less transcoding going on.

At that point, I decided to upgrade the user experience even more, and I enabled the Skip Intro function and 10-second thumbnail intervals for the first time. This, however, caused an issue with slow library scans. I spent most of today turning various functions on and off to see what might be causing this (if anything, and to determine if this is expected behavior), but my knowledge of reading logs is limited.

The second problem is that when using NordVPN (even with split tunneling enabled for Emby), my logs are flooded with errors saying the server has no outbound connection. Is there anything I can do in my setup to bypass this?

My setup is as follows:

·       Emby Server on Windows 11

·       All media are on mounted drives on my Synology NAS on my local network (\192.168.0.10\Media)

·       External access for my users is through a reverse proxy DDNS domain pointing to my NAS that routes the traffic to my local NUC with Emby on the local network (192.168.0.20). Initially, I tried to use Nginx or Caddy directly on the Windows server, but since the NAS has a built-in RP with auto-updated Let’s Encrypt certificate, I decided to stick with that.

·       The NUC8i7HVK has 16GB RAM, an i7-8809G CPU, a Radeon RX Vega GH GPU, and an Intel 630 UHD. The Synology NAS is a DS220+ with two HDD drives.

My questions are:

1.      Do my logs (one with VPN on and the other off) indicate why the library scan (7TB of series episodes) now takes almost 40 minutes every time after enabling Intros and thumbnails? Will it get faster after several full scans?

2.      Is there a way I can use NordVPN daily to browse the internet on the same Windows machine as the Emby Server?

3.      Is my configuration ok? (DDNS domain and reverse proxy on NAS -> Emby Server on Windows -> back to NAS to grab media on mounted drives.)

4.      Should I use the CPU or GPU as my first choice for transcoding?

Any tips appreciated.

 Shout out to the people that know thing or two.

@rbjtech

@Luke

@Happy2Play

@jaycedk

embyserver.txt embyserver (1).txt

[Happy2Play 9782]

4 minutes ago, cochize1 said:

At that point, I decided to upgrade the user experience even more, and I enabled the Skip Intro function and 10-second thumbnail intervals for the first time. This, however, caused an issue with slow library scans. I spent most of today turning various functions on and off to see what might be causing this (if anything, and to determine if this is expected behavior), but my knowledge of reading logs is limited.

Yes it is expected as it is creating interval/bif files for every item.  I would put a time limit on the task and run during off peak time.  As this is a large task initially but once done it is as needed when adding media. And you enabled intro markers so this will extent time even more.  But is dependent on when you have the tasks run.  So you may not want them to run during library scans until all item have makers/fingerprints and bifs.

Not sure on your network issue but failed provider queries will slow things down also.

11 minutes ago, cochize1 said:

4.      Should I use the CPU or GPU as my first choice for transcoding?

You will want to use GPU whenever possible.

[cochize1 55]

I am aware from the other posts to set both as a scheduled tasks, I already run the first scan that run for 24h and created all bif files (stored with media). But now (no matter if I set these options to Never or as a scheduled task) library scan without even adding new media takes about 40 minutes (compared to 30 seconds prior ). And as you can see from my logs there are no logs for image extraction or fingerprints for intros, just a bunch of queries to tvdb or tmdb for old series that already have all metadata in nfo files. Hence my question, is that normal?

[Happy2Play 9782]

From what those logs are showing I would say yes it appears normal.  But neither log actually shows library scan unless you just did one library.

[cochize1 55]

If I run full library scan (for all 6 I have) would it show anything else? TV Show library is the largest and this is where I see a notifiable difference in time it needs to scan

[Happy2Play 9782]

All I can say is with the logs provided you have Markers and Interval images being created along with every episode quering TMDB.  So it appears normal to me.  So until your server settled down I think this are going to be slower but only going off the logs provided.

[cochize1 55]

Thanks, I guess I will have to see if this will settle down in a couple of days after several scans (although I was sure that number of bif files equal the number of episodes so I figured all were populated already). And about the second log: do you have any idea is there any option or workaround and using NordVPN and Emby on the same machine?

[Happy2Play 9782]

3 minutes ago, cochize1 said:

And about the second log: do you have any idea is there any option or workaround and using NordVPN and Emby on the same machine?

Sorry I don't as I have not used a VPN in general.  May have to wait for some else to comment.

[rbjtech 5284]

What are you trying to achieve with the VPN ?

Is this to just VPN your non emby connections from the Windows machine to the internet ?

If you turn off the VPN - can emby get to the services ok ?

If split tunneling - then have you tried the opposite way - ie tunnel all browser connections to the VPN and let emby use the default connection ? 

[cochize1 55]

Hi @rbjtech

At first I thought that's brilliant solution to reverse split tunneling and enable on for all browsers. Everything seemed working snappier untill I looked into the logsembyserver (1).txtembyserver.txt, there are errors with connection again. Attached please find logs with VPN on and off for some short time.

As for your questions: when my Emby Server was on Synology and I was accessing it through emby.mydomain.com (reverse proxy on Synology pointing to localhost: 8096) I was behind NordVPN app 24/7 (with some programs excluded by split tunneling). With VPN turned off Emby can access all services.

I guess one more thing I could try is web browser extension instead off standalone Nord app.

Got any other ideas on how to connect these two? Maybe my set up (reverse proxy on NAS pointing to Windows Emby Server has something to do with it)?

[Q-Droid 989]

Some services might block connections coming from VPN exit points.

Reverse proxies handle inbound connections, not outbound unless you've done something out of the ordinary. Are you rebooting after making the changes to Nord VPN settings? 

[rbjtech 5284]

As per comments from Q-Droid and why I suggested you VPN the browser connections.

Metadata Providers don't like VPN connections because they are a method to (potentially) abuse the connection - ie blocking a VPN for abuse is pointless - so they block them all for known public VPN ranges.

If your split tunnel is only being applied on the browser processes - then I'm unsure why emby would be attempting to use them tbh without getting into routing tables etc.

The browser extensions route is a good idea - certainly worth trying. 

Edited May 22, 2024 by rbjtech

[cochize1 55]

@Q-DroidI believe after turning off/on Nord app I was also restarting the Emby server, yes (or maybe just rotating the log files sometimes).

So, with NordVPN app disabled completely in Windows and NordVPN enabled only as chrome extension all is good, no errors in the logs as expected. Not sure why Nord's app is ignoring split tunneling settings (I'd rather us the app and not extension as I could control the traffic from other apps as well).

As for the other things, it still concerns me that after enabling Skip Intro and Thumbnail extraction now the simple library scan (without adding any new media) takes around 40 min compared to 30 seconds prior to that. I already did a full scan at the beginning so I'd figure all bif files should already beembyserver.txt downloaded. Now library is set to only check for these as a scheduled task but it still makes tones of connections to tvdb and tmdb with every scan. Is it normal? Should it 'calm' abd fasten the process after some time when Emby will think all the files are in place? Is there anything standing out in the logs (maybe debugging would show something more)?

[rbjtech 5284]

You just need to let it finish.  Run both a scan media library and scan metadata folder, then Detect Intro's.

Assuming you are not changing settings - It should not be scanning again once it has the info locally. (*)

Intro detection, depending on cpu and I/O etc can take DAYS.

With ~30K episodes and 2K movies - my scan now takes about 40-50 seconds...

(*) the Intro detection does not appear to be faultless in this regard - I have noticed it re-scanning some Seasons for some reason - haven't had time to look into it tbh, but it's only doing for a tiny subset.  

Edited May 22, 2024 by rbjtech

[cochize1 55]

So, after some tests I might have found a solution for the Nord problem, not sure if it is a fix or just temporary. Long story (of trial and error) short, changing Custom DNS within NordVPN desktop app to Google's 8.8.8.8 or Cloudflare's 1.1.1.1 seems to be working (with split tunneling disabled) No more connection errors in Emby logs so far after some library scans and the VPN turned on systemwide. Thank you guys for all the help, just one final question and we could close this topic as solved: 

Is my setup odd or should I leave it as is (DDNS domain and reverse proxy on NAS -> Emby Server on Windows -> back to NAS to grab media on mounted drives) or it would be more efficient to use reverse proxy directly on the same Windows machine aa Emby Server?

 

embyserver.txt

[rbjtech 5284]

2 hours ago, cochize1 said:

So, after some tests I might have found a solution for the Nord problem, not sure if it is a fix or just temporary. Long story (of trial and error) short, changing Custom DNS within NordVPN desktop app to Google's 8.8.8.8 or Cloudflare's 1.1.1.1 seems to be working (with split tunneling disabled) No more connection errors in Emby logs so far after some library scans and the VPN turned on systemwide. Thank you guys for all the help, just one final question and we could close this topic as solved:

Just be cautious that by using open DNS servers you are removing the privacy of using NordVPN's DNS servers.    Therefore while the traffic will of course be encrypted/tunnelled, the REQUEST for that information is stored/used by Google/CF..

2 hours ago, cochize1 said:

 

Is my setup odd or should I leave it as is (DDNS domain and reverse proxy on NAS -> Emby Server on Windows -> back to NAS to grab media on mounted drives) or it would be more efficient to use reverse proxy directly on the same Windows machine aa Emby Server?

 Doesn't really make any difference - If the RP is working on the NAS, then I'd be inclined to leave it alone as that is likely to be the more stable platform anyway (less reboots etc).

[cochize1 55]

Since I need to change DNS for this to work do you have any 'safe' DNS suggestions I could use? AdGuard maybe?

And one more question since I have you here (I already posted that in other topic, but it's 3 months old so I might not get any answer). In my logs you can notice 'ImageProcessor' errors. Luke suggested in that other topic that it is due to

Quote

network share requires a username and password. You'll need to configure this with the folders you added in Emby library setup.

I also have my network share that require username and password (my Emby server is on Windows and my media are on Synology NAS). I have mapped the drives in Windows Explorer with the password and username and Emby sees it and plays all files with no problem. But when I try to change actor's profile picture by using magnifying glass it goes blank (I have to first delete the picture and then use the magnifying glass for this to work). How can I solve this to get rid of this errors?