Improve Performance

[usnscpo 24]

Yep, its really just the syntax of linux that sometimes give me pause... understanding most of the variables/switches, etc.  Give me an old dos machine and I'm golden.  I usually google that stuff so I can see and understand all the possibilities of a command/request.

[Bert 49]

I'm down to help although I'm very green. The last programing instruction I had was on an Apple IIe. 

[Cadkey 47]

I look forward to a guide to test this specific configuration of the two nvme on the DSM 7.1 version. Faster image display in Emby would be a huge improvement.

[Cadkey 47]

*Deleted*

Edited February 23, 2022 by Cadkey

[Cadkey 47]

On 1/22/2022 at 4:28 AM, Dusan78 said:

My DS920+ is now unbelievably fast.  At first there was not much improvement until I realised that all my artwork and .bif files were inside the media folders.  That seemed like a cleaner and more manageable option at the time as I could easily have access to artwork.  Once I replaced the artwork, the old files inside media folders were cleaned up.  However, video preview .bif files were a pain.  It would have taken a month to recreate them all on NAS.  So I took a look at the library.db file and made a program that copies the .bif files into the same folder where the corresponding artwork sits.  With exception of few files with special character, it all got copied and works.  It's amazing how much of an improvement this was.  I want to say it should really be mandatory running everything except the media from a decent NVME.  

Moving transcoding-temp to nvme was also a huge improvement and changing position is instant when watching a transcoded video.  I almost want to sign that lego song "Everything is Awesome"!... lol

 

Hello,
Can you indicate which guide you followed to make these modifications, or explain step by step please.
I'm very interested in the procedure, I also have a DS920+ with 2x 512GB NVME
Thanks very much

Edited February 23, 2022 by Cadkey

[DJX 18]

Are you using the 2x NVME slots on the ds920 with a command line change to make it a volume or did you use up one of the 4 slots?

[Carlo 4330]

Let's not discuss the methods to do this yet on the forum until the process is vetted.
My first attempt at upgrading to DSM 7.1 was met with disaster (but not saying it's 7.1 related). My NAS right now is mostly unusable. Most software is now crashing, trying to go SUDO from SSH is a no go and met with errors. Can't copy anything to USB drives, no rsync, etc

I immediately pulled the network connection as a precaution, grabbed a small 8 port managed switch and old laptop to only allow only those two devices on a private network. I was able to access the shares I care about over the network so I grabbed an 8 bay USB3 enclosure and some SATA drives to build a little 50 TB storage box.  I was checking the drives and reformatting them on my normal work computer when part 2 of my mini disaster struck at 4am as I was tired. I fixed a pot of coffee sat back down, hit the space key to get a nice blue screen. I was trying to rush things along catching up in the forums and got sidetracked and thought I was in the VM attached to the storage bay but wasn't.  I formatted the 3 drives in my notebook.

I of course had a full backup from Sunday with incrementals the next couple days. Wanna guess where my notebook was backup up to?  Yep, the Synology 920+ which just added fuel to the fire. I had a Windows 11 thumb drive handy and a Samsung 950 SSD handy so I stuck it in a USB3 adapter inserted the thumb drive, rebooted with a BIOS change and installed Win 11. I didn't want to touch the NVMe drives until after getting sleep.

So making a long story a bit shorter, I got the 8 bay enclosure hooked up to the old notebook and it's moving files off the NAS but with only 1 GB NIC it's seems painfully slow. I finally today unformatted the NVMes and have the notebook running same old, same old again.

At present I've got 23 TB give or take to move off the NAS before I touch it. I had installed by way of thumb drives a few virus and malware package that I've used to scan files moved off the NAS thinking possible could be related but so far it's clean.  I had just ran a full security sweep including virus and malware checks 2 days prior on the whole network which was clean but I just wanted to be sure.

I've had a few goofy things going on with the NAS likely from installing so many different things on it and experimenting with it.  So I'm surely not saying it's 7.1 related but can't say right now it's not either.  I could certainly have had the same problem with a NAS restart and no update.

 Once I've got the files off I care about I'll reset the NAS clean, get DSM 7.01 installed as well as Emby and a few typical programs.  I'll then start testing from this clean install.
I certainly wouldn't want this happening to anyone else who would likely think it's related.  I'm sure it was just a "perfect storm" of events but non-the-less I want to test this properly before releasing any kind of instructions.

[bjjones 20]

@cayarsWas that using the original 7.1 beta that went up or the replacement -42550 one? Sounds like an unmitigated disaster to recover then sort out, ugh.

Edited February 25, 2022 by bjjones

[usnscpo 24]

@cayarsouch!  You have my sympathies bud.

[Carlo 4330]

2 hours ago, bjjones said:

@cayarsWas that using the original 7.1 beta that went up or the replacement -42550 one? Sounds like an unmitigated disaster to recover then sort out, ugh.

Good question as it's hard to tell from the names but they are quite different in size.
The one I installed was 324MB on disc and the version I just downloaded while making this post is 360MB in size.
So it's obviously different.

I have all the media on the NAS backup up offline, but only one copy.  I technically could have just reset the NAS and then started to restore the media but I'd prefer not to touch the backup sets if I don't have to.  It's a pain to wait while I move the media off the NAS but all files spot checked seem fine and play perfectly so it's better than risking having only one copy.

I already got 2 strikes this inning and didn't want to chance being called out. LOL

1 hour ago, usnscpo said:

@cayarsouch!  You have my sympathies bud.

No harm, no foul except for 2 days of lost sleep.  Of course the timing couldn't have been better as I have over 60 SAS drives still waiting to be setup in their new tiered storage. I've been so busy I've not done much with any of it except make sure all components are functional since before Christmas. I was just starting to making time for setting it all up and this happened which will no doubt set that back another week. LOL Had it been up I'd likely have moved Emby over to the new system using the 920+ as only a test server. The 920+ ran "Carlo" style was really past it's limits using common sense but I like to push things to see what can be done.  The 920+ was running with 3 14TB and 1 18TB drive internal on it.  I have an 8 bay USB3 (eSATA won't work) enclosure filled with 16 TB drives connected to the front USB3 and then 2 WD USB3 18 TB drives connected to a USB3 hub on the back USB3 port. That around 225 TB of raw storage.

Fortunately after the incident I was able to disconnect the 10 external drives quite easily and scan/check them with no issues leaving only the 4 internal drives to get data off of.

So you can see how it makes a pretty good test bed for trying to push things to see if something breaks.  But if it does break you can't jump to conclusions especially when you've tested so many different software apps and attempted 3rd party software installs meant for normal Linux and not Synology.  I was planning on resetting the box after this round of testing but figured if it worked in my environment it would likely work for anyone.  Well that was the thought anyway but I tempted fate one to many times. LOL

I feel mostly bad for you guys having to wait longer after the shipping delays to get in more SSD testing, but in another couple days it will be behind us. 
Once I get this tested with the NVMe used as normal storage for Emby and make it though the update to 7.1 I'll post info then try the couple other performance experiments I've wanted to try (have everything for it sitting here) which are quite radical but could be game changers. My expectation is the NAS is too locked down driver and software wise to work, but I got to try it anyway or I'll always wonder if it could have been.

[Cadkey 47]

11 hours ago, cayars said:

Once I get this tested with the NVMe used as normal storage for Emby and make it though the update to 7.1 I'll post info

Thanks for reporting.
I look forward to getting this info.
Not that I will pass on the Beta 7.1 version but to do the manipulations concerning the nvme on DSM 7.0 which will I hope be valid for the official DSM 7.1.
I have 2x nvme 512gb in my 920+
I think putting emby's thumbnail and transcode folders on the nvme will be faster than on an SSD. I am wrong?

Edited February 26, 2022 by Cadkey

[Carlo 4330]

What I found for NVMe as a drive was putting as much as possible on them was best.
That would include cache, metadata, transcode, convert, folders, DVR, etc

How much you can fit and leave room for transcoding will depend on the size of your libraries and especially if you use thumbnails/bif files (not stored with media).
With 2 NVMe you could use both for this purpose or if just one is big enough use only that and use one the normal way as a read cache managed by Synology.

My testing however showed direct use of the NVMe was far more beneficial than having two 1 TB NVMe used as read/write cache which did kind of surprise me.
It surprised me because the hit rates I was seeing with in the 90% range and I had it set to defer disk writes.  The defer writes I thought would have almost eliminated any transcoding going to actual disk.

But real world testing of the system especially with Live TV having a few recordings in progress plus watching back and using trick play showed quite a difference. One of those "oh sh*t" moments where you don't even need to benchmark the difference because it's immediate apparent from screen loading to number of transcodes and recordings it handles.

[Cadkey 47]

Ok I have 2xNVME 512Go and I have thumbnails/bif files not stored with media.

Also I think the solution of mounting the NVMEs in volume that you are testing will be the right configuration for me. One question please, moving the thumbnails on the NVME volume forces Emby to rescan the libraries? Or is it enough to stop the Emby server, make the changes, restart and everything is ok? Thank you

[Happy2Play 8281]

16 hours ago, Cadkey said:

Ok I have 2xNVME 512Go and I have thumbnails/bif files not stored with media.

Also I think the solution of mounting the NVMEs in volume that you are testing will be the right configuration for me. One question please, moving the thumbnails on the NVME volume forces Emby to rescan the libraries? Or is it enough to stop the Emby server, make the changes, restart and everything is ok? Thank you

Emby will not move anything that is already imported/exists.  Changing Emby metadata folder will require both original location for all already import items and new location will only be for newly imported items as path is already recorded in the database.

Edited February 27, 2022 by Happy2Play

[Cadkey 47]

There is a way to modify the paths in the dbase without having to rescan everything or redo the personal modifications.

[Carlo 4330]

I'll have the proper way to move each directory be it cache, metadata or anything else covered so you'll know how to do it without guessing or trial and error.

On 2/27/2022 at 1:18 AM, Cadkey said:

Ok I have 2xNVME 512Go and I have thumbnails/bif files not stored with media.

Also I think the solution of mounting the NVMEs in volume that you are testing will be the right configuration for me. One question please, moving the thumbnails on the NVME volume forces Emby to rescan the libraries? Or is it enough to stop the Emby server, make the changes, restart and everything is ok? Thank you

If not I'd suggest logging in with SSH and checking the sizes you'll be working with.
Find out the size of both the meta-data and cache folders are and then find out how much of that is .bif files.

It's possible because of bif files your meta-data and cache won't fit on a a 512GB disk.
But we can handle that if needed a couple ways

The best method would be to move each bif file to the location where the media is stored. That could be done using the database and a sql script to create a batch or bash file to move them with the media, then change the option in the libraries so they always save with media.  Alternately we don't copy the bif files to the new meta-data folder and instead symlink every one.  Downside to this is any new bifs created will be on the NVMe so you'd have to run a script once it a while to move the bifs off NVMe and create sysmlinks for them.

The first approach is much better for numerous reasons.

On another note I've forgotten how slow 1Gb networks are but it's painful watching the transfer.  I'm actually using both nics on the Synology as well. I'm up to letter W for movies and nearly have all completed TV series done with only active shows to go. That's all that's left.

[Cadkey 47]

8 hours ago, cayars said:

The best method would be to move each bif file to the location where the media is stored

OK.  but it will bother me.  I do not want.  My medias .mkv are on a cloud.  I prefer that the .bif remain on the Nas. For speed I would like to put them on the NVME if it works. My goal would be to have the most fluid Emby interface possible.

I don't know where the .bif etc. are stored.  So I don't know the size of this datas.

Edited March 1, 2022 by Cadkey

[Cadkey 47]

@cayars I'm interested to know where are all the files that it would be useful to move to an NVME volume in order to know the size and have an NVME of sufficient size.
The goal is to have a fast interface for displaying thumbnails and backgrounds, and why not transcoding too if it speeds it up.

[Dusan78 18]

1Tb Nvmes are the best value purchases now days and I can't see anyone needing more for the cache, metadata and transcoding.  No need for gen 4.  Nice Gen 3 samsung will be perfect. 

I am using my 500Gb Evo970 and I'm only using about 100Gb of it.  I also run a file sync which updates and keeps the mirror of the folders on the NAS (hard disks) so in case anything happens with the m.2 volume I can just change the settings on Emby and be back and running until I fix the m.2 volume (should that need arise after update).

Since the new dsm 7.1 will affect the m.2 cache it might affect the Macgyvered solution we have right now but, we'll see.  

All I know, after seeing the light and running the emby metadata, cache and transcoding on nvme there is no going back.  I really hope the new update makes accomplishing this easier for everyone as not everyone is comfortable tinkering with ssh and linux.

 

[Cadkey 47]

1 hour ago, Dusan78 said:

1Tb Nvmes are the best value purchases now days and I can't see anyone needing more for the cache, metadata and transcoding.  No need for gen 4.  Nice Gen 3 samsung will be perfect. 

I am using my 500Gb Evo970 and I'm only using about 100Gb of it.  I also run a file sync which updates and keeps the mirror of the folders on the NAS (hard disks) so in case anything happens with the m.2 volume I can just change the settings on Emby and be back and running until I fix the m.2 volume (should that need arise after update).

I have a 47To movies and series library.
My two NVME 512Go are sufficient according to you?

[Cadkey 47]

1 hour ago, Dusan78 said:

Macgyvered solution

It makes a Frenchman laugh 

[Carlo 4330]

5 hours ago, Cadkey said:

@cayars I'm interested to know where are all the files that it would be useful to move to an NVME volume in order to know the size and have an NVME of sufficient size.
The goal is to have a fast interface for displaying thumbnails and backgrounds, and why not transcoding too if it speeds it up.

Have a look in /volume1/@appdata/EmbyServer/metadata and /volume1/@appdata/EmbyServer/cache

after navigating the metadata run this command to get the total size of content

du -s -h

navigate to the cache folder and run the same du -s -h command

I just had a look at a backup of my system and the bif sizes are far smaller than I was thinking they were.  I had forgotten that softworkz did a fantastic optimization job on these and reduced the sizes a lot.

Bifs on my system are about 400 GB so I'd feel pretty comfortable thinking every key folder could easily fit on a single TB NVMe.
If you do want to find the specific size of bif files try this command from the metadata folder

find -type f -iname *.bif -print0 | du -ch --files0-from=- | grep total$

Add the size of metadata and cache folder will form the base of storage needed.
Then we need some dynamic space for converting and transcoding which only you can determine looking at current use.

I myself would want to do my actual DVR recording to the NVMe as well then a script would move it back to regular HDD storage after it's done recording.  This I'd consider optional but would help to push the envelope on performance enhancements.

I don't think you need to worry about trying to RAID 2 NVMe disks for redundancy as we can create a script that runs at least once a day or more to sync the contents of metadata & cache to the old location in case of NVMe failure.  This would allow using cheap NVMe disks as well to save $ vs seeking out premium high priced ones for max durability.

IE for same cost a no name 1 TB NVMe will likely be better for our use then a 256 GB Premium NVMe since it should be big enough to function for DVR and still remain about half empty which helps to extend the life of NVMe sticks but still give you plenty of margin on use.

5 hours ago, Dusan78 said:

1Tb Nvmes are the best value purchases now days and I can't see anyone needing more for the cache, metadata and transcoding.  No need for gen 4.  Nice Gen 3 samsung will be perfect. 

I am using my 500Gb Evo970 and I'm only using about 100Gb of it.  I also run a file sync which updates and keeps the mirror of the folders on the NAS (hard disks) so in case anything happens with the m.2 volume I can just change the settings on Emby and be back and running until I fix the m.2 volume (should that need arise after update).

Since the new dsm 7.1 will affect the m.2 cache it might affect the Macgyvered solution we have right now but, we'll see.  

All I know, after seeing the light and running the emby metadata, cache and transcoding on nvme there is no going back.  I really hope the new update makes accomplishing this easier for everyone as not everyone is comfortable tinkering with ssh and linux.

 

So are you doing this already yourself?

Yep exactly.  Being able to copy/sync the contents back to HDD is ideal.

Do you use the coverart plugin?  If so have you noticed how much faster screen updates are using NVMe.  Crazy fast.

[Dusan78 18]

7 hours ago, Cadkey said:

I have a 47To movies and series library.
My two NVME 512Go are sufficient according to you?

Well 47tb doesn't actually tell us how much content you have.  If it's 47tb of 4K 20-50gb/movie files than it isn't a lot.  But, if its all 1080P x265 compression 2gb/movie than it's a lot.  Metadata, artwork and cover art as well as bifs are mostly same size so it more depends on the number of movies and episodes you have rather than actual TB size of your library.  Does that make sense?

So I really don't see anyone exceeding 1tb. 

[Dusan78 18]

2 hours ago, cayars said:

Have a look in /volume1/@appdata/EmbyServer/metadata and /volume1/@appdata/EmbyServer/cache

after navigating the metadata run this command to get the total size of content

du -s -h

navigate to the cache folder and run the same du -s -h command

I just had a look at a backup of my system and the bif sizes are far smaller than I was thinking they were.  I had forgotten that softworkz did a fantastic optimization job on these and reduced the sizes a lot.

Bifs on my system are about 400 GB so I'd feel pretty comfortable thinking every key folder could easily fit on a single TB NVMe.
If you do want to find the specific size of bif files try this command from the metadata folder

find -type f -iname *.bif -print0 | du -ch --files0-from=- | grep total$

Add the size of metadata and cache folder will form the base of storage needed.
Then we need some dynamic space for converting and transcoding which only you can determine looking at current use.

I myself would want to do my actual DVR recording to the NVMe as well then a script would move it back to regular HDD storage after it's done recording.  This I'd consider optional but would help to push the envelope on performance enhancements.

I don't think you need to worry about trying to RAID 2 NVMe disks for redundancy as we can create a script that runs at least once a day or more to sync the contents of metadata & cache to the old location in case of NVMe failure.  This would allow using cheap NVMe disks as well to save $ vs seeking out premium high priced ones for max durability.

IE for same cost a no name 1 TB NVMe will likely be better for our use then a 256 GB Premium NVMe since it should be big enough to function for DVR and still remain about half empty which helps to extend the life of NVMe sticks but still give you plenty of margin on use.

So are you doing this already yourself?

Yep exactly.  Being able to copy/sync the contents back to HDD is ideal.

Do you use the coverart plugin?  If so have you noticed how much faster screen updates are using NVMe.  Crazy fast.

Yep, it's been running great since we exchanged messages back in January and you were testing breaking the raid with nvmes.  Which I now don't think there is any need for as just doing an incremental backup takes likes a minute on daily basis and saves me from buying another m.2 nvme drive.

 

[Cadkey 47]

6 hours ago, Dusan78 said:

Well 47tb doesn't actually tell us how much content you have.  If it's 47tb of 4K 20-50gb/movie files than it isn't a lot.  But, if its all 1080P x265 compression 2gb/movie than it's a lot.  Metadata, artwork and cover art as well as bifs are mostly same size so it more depends on the number of movies and episodes you have rather than actual TB size of your library.  Does that make sense?

So I really don't see anyone exceeding 1tb. 

Yes, it's true.. Overall 1000 films and 16000 series/episodes. 17000 files.