Improve Performance

[FrostByte 5051]

I was told to post mine in the following section

Tutorials and Guides - Emby Community

Then cayars will add something to the kb linking to the tutorial.  I guess the consensus was to not pin a bunch of these here but use the kb w/links and keep all the tutorials together.

I'm not sure there is a format, but you can see I've created a few for Synology lately.  Using a lot of screen shots helps (worth a thousand words I guess).  Though that may not be so easy now that you're already done.  

I haven't done mine yet, but this is on my list of things to do.  I'm sure cayars can review and I'll be glad to test it out.

Edited March 11, 2022 by FrostByte

[Carlo 4330]

I thought about adding it to the tutorial section but don't think it's a great idea being there.  It should stay here in the Synology section IMHO and be used as a reference only for those looking for the utmost performance.

We know it works today and I've tested it about as good as one can test this looking for different ways it would break other Synology apps and vice versa but we never know what the next upgrade version holds.

We're essentially operating our NAS units outside the scope/intent of the company making it. A mod like this could cause issues getting support or cause other similar issues so it's probably best  not "advertising it" so we don't get into a situation "Emby said it's ok" or "Emby recommended it" type thing if someone had a problem.  This was just me personally sharing some info and doing a bunch of tests on my own time to make sure it was pretty safe.

I might have other similar "speed/functionality hacks" (more extreme) but have been thinking about posting instructions elsewhere (ie home blog/site) vs here as it's better to keep this about Emby and Synology working together nicely.

Carlo

Edited July 16, 2022 by cayars

[Tremas 106]

Hi guys. I'm finally getting ready to do the NVME upgrade and the instructions are pretty clear as far as setting up the drives (very grateful, @cayars). However, I could use some help with the best way to prepare my existing artwork and settings.

Before I moved all of my media from my old PC to my current NAS, I ensured all of my metadata was saved to local NFOs and that I had all my artwork saved with media. I figured that was the best way to ensure I recovered if something went wrong. I did a fresh emby install on my new NAS (DS1520+) and have been running relatively smooth for the past year. However, my artwork in each library is still "saved into media folders" or the "nightmare scenario" as @Dusan78 described it.

The good news is that this should only be posters, backdrops and other artwork, no .bifs. I plan to enable video thumbnails once I get the NVMEs going, but I've never had them previously.

I am ready to uncheck "Save artwork into media folders" to speed things up for each library, but am unsure what to do after that. Do I manually move all of the artwork into the metadata folder and then run a scan? Do I have to ensure there are no image files left in the media folders, or can I keep a backup copy there? Should I do this before I copy the metadata folder to the NVME drive, or afterwords to let it do the rebuilding on the faster HD? For reference, I have about 1000 movies, 2000 tv episodes, and 2500 songs.

Any suggestions from those of you who have already done this type of change would be much appreciated.

[Carlo 4330]

I store artwork/graphics/bif/nfo all with the media. So those objects won't be in the meta-data folders sitting in NVRMe but that's ok because Emby keeps this information in it's cache which will be on NVMe.  Once it's in the cache it won't pull it from the media folders.

So don't change anything to "help" the new use style.  Just follow the directions and you'll be all set.  Then after a few days of use go back and think about changing the location where things are saved.  However, I bet you won't think about it then.

You'll be fine,
Carlo

[Tremas 106]

Thanks. Just for my understanding though, what happens if i uncheck "Save artwork into media folders" and then enable thumbnail generation. Does that end up with a hybrid situation where existing links to art are still pointing to the media folders and any new downloaded art and new .bif files get sent to the metadata folder?

[kaj 242]

@cayars now that DSM 7.1 seems to be rolling out, are there any problems with updating that you know about? Thanks

[DonMacaroni 24]

Works fine here, with 7.1

[Carlo 4330]

On 4/8/2022 at 8:06 PM, kaj said:

@cayars now that DSM 7.1 seems to be rolling out, are there any problems with updating that you know about? Thanks

There is only one gotcha that I know of but it's not 7.1 related so I'll just path this on.  One of the file system mounts is rather small and is used for installations.   If it fills up, you can get crash, can't login. If logged in, can't sudo, etc.  Basically no working room, no working NAS.

Here's how to run check.  Don't blink or you'll miss it.

The first mount for /dev/md0 right now is very nice at only 23% full for the root partition.  If you're over 80% get that lower before doing a major upgrade as it needs the space to work with.  I found out the hard way when testing compatibility with using NVM3 as drives vs caches and had over a half dozen crashes and wipe out trying to reproduce.  Finally figured out what was causing it, and of course had nothing to do with the NVMe drives.

Anything under 75% is probably fine but if you try updating to 7.1 with 87% full / (root) it doesn't work.  Well didn't for me anyway.  Could also be that the 7.1 was fine but the new beta packages that also updated put me over the top.  As you can see it's not very big to begin with.

So that's something that could happen to anyone at any time on any version if you root drive is pretty much full.

Oh lesson learned is not to use disk benchmark programs like DD, FIO or IOZONE having it write to the root, especially since they leave the test file and don't clean up. So an 8Gb test file there isn't a good idea.

So moving past "user fault" I can't think of any significant issues or reasons not to update.

Carlo

[Tremas 106]

Happy to report that I performed a manual update to 7.1 this weekend. Everything went fine with the upgrade, so I proceeded to use the instructions here to create a single 500Gb NVMe drive for emby caching. Running with no problems so far.

Things are pretty snappy, but the whole process has really exposed a bottleneck with one older HDD in my volume (all of my drives aside from the NVMe are spinning platters). A replacement is on the way which should resolve that issue. I have a second 500Gb NVMe, but I am going to monitor the performance for a few weeks before deciding what to use it for. If emby remains happy with 500Gb after I start generating .bifs, I will set it up as a Synology read cache. If nothing else, that should give me a boost with Synology Photos, which is pretty much the only other media/database app I run outside of emby. Thanks again for all of the great information in this forum.

[Carlo 4330]

The only tweak I'd do for use of 2 NVMe is to use a quality stick for everything but the transcode drive.  Then purchase basically the cheapest NVMe stick you can find that's big enough for your need which likely is 128GB or 256GB.

What that does is take the "wear and tear" off your more expensive drive while putting the W&T on your "throw away" cheap drive.  If planning on buying 2 sticks (if you don't have any yet) then purchase one small "throw away" stick for transcoding and one larger that you won't outgrow like 1TB or 2TB.  That IMHO is better than two 500GB drives.

Here's a "cheap" 256GB NVMe that has great speed for the money in 2280 M.2 format. This $32 (include Amazon Prime shipping) Kingston has 2,100MB/s Read & 1,100MB/s Write speed which is great for the cost. Try and make sure/verify the sticks you purchase have 1,000 or better MB/s write speeds.
https://www.amazon.com/Kingston-250G-2280-Internal-SNVS/dp/B09DVP94WC/ref=psdc_1292116011_t3_B07P22T3VD?th=1

[Tremas 106]

Thanks @cayars. I bought my 2 500 GB NVMe drives during a NewEgg sale back before this topic started. I was planning to use them as standard Synology Cache, but then after the discussion started here I just sat on them. There should be no problem using one NVMe as a "forced" emby cache drive and the other as a Synology cache, right?

[hooray4me 52]

Is there any reason NOT to move the whole package over to the new NVMe drive?

Stop Emby...
ssh and as root:
 

sudo mkdir /volume2/@appstore
sudo mv "/volume1/@appstore/EmbyServer" "/volume2/@appstore"
sudo rm "/var/packages/EmbyServer/target"
sudo ln -s "/volume2/@appstore/EmbyServer" "/var/packages/EmbyServer/target"

Start Emby

[Carlo 4330]

Yep there is reason.  It's best to follow the directions as given in the PDF as it's just moving key folders to the NVMe but still allows Synology to run the app from disk.  This will help to make sure it's still "conventional" to Synology and won't cause any problem with updates or installs (our upgrades or theirs).

You're not moving folders such as cache or meta-data but copying to the NVMe where they will get used.  This allows you to keep the data at the old location and with a simple script updated as well.  This way in the event of an NVMe failure you just change where the folders are pointing right from the Emby Server gui and you're back running on disk again.

Moving the EXE and DLLs as well as config files to NVMe doesn't help you performance wise so there is no need to try to uninstall Emby and re-install to NVMe.  That likely a lot more trouble than it's worth plus you give up the easy way to revert to disk if the NVMe ever has an issue.

[hooray4me 52]

got it, was thinking about the database location with it. Thanks!

[Sparker 31]

In my DS720+ Synology NAS I have setup a 500GB (WD Blue) NVMe to be the target of where apps are installed, so Emby Server is installed and run from the NVMe.

I have it setup so I don't need to transcode media content, so I am not even thinking about transcode caching, but I could point it to a directory on the NVMe as well (Haven't done that yet). I will have to give the guide a read through and see if it's for me. I guess I wouldn't want the HDDs spinning up at all if Emby isn't being used by clients (aside from the task instances).

I assumed the metadata (which I don't place in media folders) would be in the same place the app is installed, which in my case is on volume2 (NVMe). If I opt to place metadata next to media content, then it is going to get saved to HDD volume1 pool.

I have a spare 970 Evo 250GB Plus that isn't initialized in the other slot at the moment.

[Carlo 4330]

Hi, just curious, what method did you use to setup an NVMe as a pool/volume?

[newb 14]

On 1/22/2022 at 7:58 PM, cayars said:

Thus far I've tested with 2 1 TB NVME sticks setup as read/write cache.
Above with an addition of Samsung SATA SSD attached via USB.
Then removing the 2 "cache" sticks from Synology control, formatting one manually as a new volume.

For simplicity this I'm sure is the go to. Only question is 1 or two NVMEs.  1 could hold meta-data, cache and the other for DVR recording and transcoding.  That could allow buying 1 nice nvme to hold good data and make loading and general navigation quick.  The 2nd could be a cheap stick maybe 256MB used as a beater for transcodes and maybe  DVR.

This weekend I'm trying:
Above with an addition of Samsung SATA SSD attached via eSATA. I've got 3 news ways to try this (adapters, cables, expander).

I may also see if the NVME slot can be used any other ways as I've got a few other toys like some of these I can try.

some variations 

Except it would be more like this

Hi @cayars and everyone! I currently have a DS216+ii which does not have any M.2 slots, so my only options are using the USB 3.0 or eSATA ports on the NAS. Were you successful in connecting an M.2 NVMe via either of those two ports and do you know which port is the better one for this purpose? I have not purchased any enclosure or connector yet.

The NVMe drive that I have is a 500GB Samsung 970 EVO that I ripped out of a Macbook Pro before trading it in (I had previously upgraded the Macbook Pro with this drive).

I also have a spare 2.5” SSD somewhere that I could also use for Emby purposes, though by reading through this thread it looks like the NVMe drive is the better option?

Apologies if I am posting in the wrong thread - I will happily move my post elsewhere if needed.

Edit: I am looking at an enclosure such as this one https://www.orico.shop/en/nvme-m2-ssd-housing.html. Even though the enclosure supports USB 3.1 gen 2, the NAS I have only supports USB 3.0, which is not the end of the world, I just won’t achieve 10Gbps speeds, rather just 5Gbps, right? Still wondering if the better option is to go with eSATA instead.

Edited April 19, 2022 by newb

[bjjones 21]

On 4/12/2022 at 12:42 PM, cayars said:

Kingston has 2,100MB/s Read & 1,100MB/s Write speed which is great for the cost. Try and make sure/verify the sticks you purchase have 1,000 or better MB/s write speeds.

Carlo, got my EmbyFast drive set up yesterday on a Samsung 980 plain 500GB NVMe (shoutout to @Dusan78 for his help). Samsung 980 appears was what you were using except 1TB when you made the instructions pdf so I followed your lead. Now I'm thinking about using the syno branded SNV3400 instead of the 980 just for the durability improvement but it's overall slower and the write speed is well below 1000 per above.

Do you think there's any arguable reason to make the switch to the SNV3400 in this use case? I got a (I think) good deal on the 980 500GB @$48 vs probably triple that to move to the synology stick

[Bert 49]

I have a dumb question but since I am OP, meh. Did this make a considerable improvement for you?  Like very noticeable? I think I did it right but was expecting a night and day difference. I can hardly tell I did anything. Maybe I expected too much? 

[Tremas 106]

56 minutes ago, Bert said:

I have a dumb question but since I am OP, meh. Did this make a considerable improvement for you?  Like very noticeable? I think I did it right but was expecting a night and day difference. I can hardly tell I did anything. Maybe I expected too much? 

Hi @Bert. I can say that mine is a considerable improvement, but I had to tweak a couple of things. The process certainly made it clear what other bottlenecks were going on, but before I addressed them I had the same reaction as you. 

The most obvious one was increasing the "Database cache size" in emby. I had done this on my old server, but I forgot about it when I moved to Synology. The value was previously something like 96MB. Once I increased it to 512MB things became super snappy. Emby was restricting the amount of data in the cache at the previous value, so the system was still having to spin up the slow HDDs all the time. You might start there if you haven't already changed the default.

The other, less drastic issue was that I had a single drive in my volume that was half the performance of my other drives. It's a long story as to why this drive got mixed in with my others, but after running the benchmark utility on all of my HDDs this one stuck out like a sore thumb. I replaced it with the same model as my other drives, but the volume just finished rebuilding. I will need to use it for awhile to see if makes much of a difference.

I hope that helps.

[usnscpo 24]

Just confirming that the DSM 7.1 update went smoothly.  No damage or changes were done to my "FastEmby" SSD drive.  Emby Server 4.7 went smooth as well.  Was also really happy to see the database maintenance features.  Well done!

[iJoMun 11]

On 3/5/2022 at 4:36 AM, cayars said:

I'd keep an eye on the size of /dev/md0 in the root directory using this command:
df - h

If your use column shows 75% or higher I would find out why and remove the files before doing the NVMe stuff.

@cayarsSo I'm finally ready to install the SK hynix P31 1TB into my NAS. The last couple of months were quite busy for us so I couldn't really focus on the upgrade. Anyway, things are a little more settled now, so maybe I should give it a go now. In preparation, as you suggested, I ran the df -h command line and found the /dev/md0 folder to be at 77%. Here's a screengrab. Should I proceed or should I clean it up? How do I know what to discard and how to do it? Thanks for your help, buddy!

[Carlo 4330]

Hi, I would try to get that down lower first.

It can be hard to search and find the space hogs when you search from root as you'll be searching the whole file system and more than just this mount.

A little trick I do is leave it as is but also do another mount. 
sudo-i
mkdir /mnt/md0
mount /mnt/md0 /dev/mdo

now you can cd to /mnt/md0 and search it much easier.

[iJoMun 11]

Hi @cayars Thanks for the guide. I tried the command lines and didn't succeed. Not sure what I'm doing wrong. If you have time sometime in the near future, perhaps we could do a remote session. Or maybe problem is simple enough to address in this thread. I'm not in a rush, so whenever you're available. Time is a precious commodity for me nowadays. (My mother suffered a stroke back in early June. She's doing better. We've brought her here to live with us so we can help with her recovery. That takes up pretty much all of my available time nowadays.) But I'd still like to set up the cache. I've noticed a significant degradation in playback in recent months--freezing and/or stuttering, especially when we rewind and then try to play again, something that never used to happen previously. Not sure if it has anything to do with the memory upgrade. I doubt it. Just wondering. Please DM me when you can. Thank you and best regards.

[Carlo 4330]

@iJoMunSorry to hear that.  The last couple years before my dad passed was like that for me, so I sympathize with your situation! It's nice to know your mom's in good hands with family.

Send me a PM when you think you'll have some free time and I'll help you get this all done as quick and efficiently as we can. Let's trade a couple PMs and review what needs to be done so we have the full scope and then we'll knock it out.

Carlo