SSD attached by USB3 Performance Benefits

[Carlo 4330]

I have mentioned a few times in the forums I was going to do this to get some basic performance numbers to see how fast the SSD would be via USB3 compared to the RAID volume.
Then also try and use the SSD for Transcoding and possible DVR as well to see if it helps or not especially if you already have a read/write cache setup.

I have two different types of 2.5" to USB3 adapters so I can test those as well to see if either is better. At present I have the 1 TB Samsung drive hooked up to my 920+ NAS.

I formatted the drive in Synology as type ext4 but it's didn't let me choose the sector size.  I'll pull it later and connect it up to a Windows computer to see what size it used as well as try changing it to see if that makes any difference for the small TS files written during transcoding.

The built in disk benchmark only tests individual drives and not the RAID volume.  Worse it doesn't do the tests on external drives.

The 920+ only has to 1 Gb Ethernet ports which I have bonded.  The SSD seems to be faster then the RAID drive as doing a copy from HDD to SSD didn't seem to hit a max.  I can probably do a copy from HDD to SSD while using two computers copying movies to the SSD to see if the two Ethernet ports gets maxed out before the HDD & Ethernet copies max the SSD out.

Do any of you guys know of any type of performance test I could get to run on the Synology for a couple of generic disk benchmarks?
If not any creative ideas?

[Carlo 4330]

I was able to find a few useful tools that give us what I think we need.  At least to start with.

iostat is a real time monitor that shows lots of really good information such as:
avgqu-sz--The average queue length of the requests that were issued to the device.
await--The average time (in milliseconds) for I/O requests issued to the device to be served.
r_await--The average time (in milliseconds) for read requests issued to the device to be served.
w_await--The average time (in milliseconds) for write requests issued to the device to be served.
%util--Percentage of elapsed time during which I/O requests were issued to the device.

These will be perfect for watching while doing a recording, then 2, then 3.  Then adding a playback, another playback etc...
Will be interesting to see how this changes.

Some links that show the use of these tools
https://www.techrepublic.com/article/how-to-use-the-linux-iostat-command-to-check-on-your-storage-subsystem/
https://linuxhint.com/benchmark_hard_disks_linux/
https://askubuntu.com/questions/87035/how-to-check-hard-disk-performance

 

[usnscpo 24]

Sub.

Anxiously awaiting your results!

[Carlo 4330]

I've got some numbers using USB3 attached SSD with 2 different controllers.
I ordered another USB Type C 10Mb housing/controller that's back ordered from the Holidays but should be here this week. I want to see to if this makes any difference at all with the same SSD installed in it attached via USB3.

I've got 6 HGST Ultrastar SAS 12Gbps drives that I ordered that should be delivered tomorrow by Fedex that are going to be added to a ZFS/Ceph Storage pool 4+2 that should give me 60TB or so of free space. The reason I mention this is that I want to copy/move what I have on my Synology to this new storage space which will allow me to try a few other things as well on my Synology box without worry.

What I will do is remove both nvme sticks I presently have installed and bench mark this.  That will be interesting to see how the USB3 attached storage compares!

Then I'm going to add one nvme stick in the Synology but not configure it in Synology GUI. Instead I'll try to use the command line to set it up as a 5th disk. I previously did this in DSM 6 but haven't tried it in DSM 7 yet. This would be an extremely fast drive to use for transcoding and maybe the best option overall and we'll find out.

The above should have the core info we've been looking for.  Then maybe I can try adding the 2nd stick in and RAID the 2 sticks together but doubt this will have much impact overall. I'll find out.  What I think a 2nd stick would likely be best is using the GUI and making it a read only cache for general use.

Having the whole system backed up will give me a lot of freedom to get aggressive without worry.

I've been looking at performance numbers while Emby is recording/watching on the Synology and believe the 2 key metrics to watch/benchmark are the IOPs and wait state related items including queue.

The IO wait states themselves I believe tell the tail.  A wait state is when the CPU is waiting for an IO operation to complete.  The higher the number the worse things are.  Also the number of IO operation in Queue comes into play.  Normally you want this to average 0.5.

So between recording these metric while also testing operational use of Emby (FF/RW) of TV while recording this should give some good info.

[gillmacca01 149]

Looking forward to the results. Thinking about adding external SSD to my QNAP for exact same thing

 

 

 

[Carlo 4330]

I don't want you guys to think I've forgotten about this as I haven't.  It's just turning out exactly how I expected and I'll show you what I mean.

First, trying to use built in Synology monitoring tools is hit or miss depending on system load.  At a certain CPU load the monitoring doesn't update making it less than useful.  I've since tried to make sure nothing was going on in the system to avoid this issue.

What I used for this test was an Apricorn EZ Upgrade 3.0 casing with a Samsung SSD 850 Pro 1TB Drive. https://www.samsung.com/us/computing/memory-storage/solid-state-drives/ssd-850-pro-2-5-sata-iii-1tb-mz-7ke1t0bw/

This particular drive will not be a bottleneck as it's a lot faster then what a USB3 connector can handle.
Sequential Read Speed: Up to 550MB/s
Sequential Write Speed: Up to 520 MB/s
Random Read Speed: Up to 100K IOPS
Random Write Speed: Up to 90K IOPS

This was attached to the front USB3 port on the Synology 920+.  I've set Emby to use this drive for DVR recording as well as transcoding. So this drive will handle both of those ops taking the storage array and 2 nvme cache chips out of the picture.

Here I have 8 channels being recorded with no playback.

DSM Monitoring tool set to real-time utilization of just this USB drive shows:

If I look at NetData info for this drive I'm pulling from the Synology 90+ I see these two items which are the most helpful of the stats.

These two stats are very respectable.

So now I open Firefox and start playing back the MLB recording which plays via Direct Stream so it's being remuxed. I open another tab and start playing back Sicario which also Direct Streams. Using another computer I start playing back the young and the restless. Then another:

DSM Resource monitor still shows:

Netdata info is still respectable:

Busy time went up a bit as well as the backlog but still plenty left.  You can see a few spikes in Disk Busy Time when new sessions were started but it quickly recovered.

At this point this were straight recording and playing. On the Shield TV if I FF it was pretty responsive if the channel was an IPTV channel already in H.264 but when it was a channel from an HDHomeRun in mpeg2 format the FF with a single click or two was usually OK but if I did any substantial jumps (ie trying to skip half time) it seemed to freeze.

The above recording is only 3.2 mbps so it's pretty light for mpeg2.

What this tells me is that there is a difference between the codecs and how it affects FF. I think this plays into user issues of not being able to FF during sporting events like NFL games as these will most likely be mpeg2 format on cable and most certainly from OTA tuners.

But what this set of tests do show is a single 6 gbps SATA SSD plugged in using a USB3 port at 5 gbps does work and doesn't seem to be a bottleneck.  Actually when used through USB3 the throughput limit is probably more like 3.5 gbps.

I'm going to try testing a slightly different way by recording some mpeg2 channels and then playing back those recordings to see if the issue is the same from disk or tuner.

 

[usnscpo 24]

Looking good cayars!  This is all where I hope to harvest some efficiency on my 920+.  Looking forward to your test of the USB Type C 10Mb housing/controller, and if you're successful in using one of the NVME's as a 5th disc for transcoding and recording-- this is where I hope your successful and show the greatest performance boost!  Carry on you madman! ;-)

[Carlo 4330]

Oh sorry, didn't mention it but a USB 3.2 Gen 2 (10 Gbps) housing won't help on the 920+ as it only has USB 3.2 Gen 1 (5 Gbps) ports. It the old USB3 port with the new updated name.

But if you were planning on ordering a case for an SSD and specifically wanted USB then a USB 3.2 Gen 2 or even a USB 3.2 (20 Gbps) case makes more sense to future proof yourself as both will still work with lower bandwidth port versions.

What I didn't try as I didn't have an adapter handy is the eSATA port on the NAS which should be 6 Gbps and is SATA so no overhead.  A quick Amazon check would have this $9 cable doing the job for a 2.5" SSD.

If you're not worried about a fancy case this would work for $9 https://www.amazon.com/HDMIHOME-Power-eSATAp-ESATA-combo/dp/B00GM7CMFG

Just had a thought.  Give me a few minutes and I'll check a different way the USB3 overhead.  Just happen to think I've got a 2.5" to 3.5" SSD adapter so I can test USB vs SATA on my PowerEdge for real difference. BRB

 

[Carlo 4330]

The difference between USB3 (5Gbps) and eSata/SATA (6Gbps) is very noticeable. I test a few different ways but I think this shows it best.

In normal software testing it was much closer.  For this I turned off all system caching and did a 16GB file transfer first writing the reading. Kind of worst case that way but true bandwidth.

[usnscpo 24]

Wow, that is pretty substantial.  So, without nvme or disk cache, you've shown the fastest way to cache throughput is achieved via the eSATA port.  Awesome work, bud.  As you mentioned, its going to be interesting to see if you can get a nvme configured as a 5th drive for transcoding and caching (this is what I'm most excited to see).  Should you be successful, I think this is the route I'd most likely follow, if not then I'll grab an eSATA, SSD, and enclosure to help my transcoding and caching.

Really appreciate your research on this.

[Carlo 4330]

I will do this testing for you later this week.  I'm planning to swipe the nvme sticks and do some nmve juggling between different machines and my notebook to end up with a 4 high speed nvme sticks to RAID 5 for a really fast boot and special device to use for the .5 PB ZFS storage pool I'm building.

Once I flush and turn off the cache I  get give this a day or two of trying a couple alternate used with them before I pull them.  I try and set one nmve up as a normal device/disk that can be used for transcoding.  Then I'll try with both transcoding and DVR.  Once I've got some numbers I can try and add the other stick back as a read only cache. No idea if Synology will let me do that or not if it sees two nmve installed.

[usnscpo 24]

No hurry, bud.  Grateful for your efforts!

[Carlo 4330]

I'll likely be starting this later tonight or tomorrow as the server's 4 stick NVME controller should be delivered tomorrow.
So I've got a little extra motivation to get this done before I can switch around the NVME sticks.

[bjjones 19]

On 1/7/2022 at 12:54 AM, cayars said:

What I didn't try as I didn't have an adapter handy is the eSATA port on the NAS which should be 6 Gbps and is SATA so no overhead.  A quick Amazon check would have this $9 cable doing the job for a 2.5" SSD.

If you're not worried about a fancy case this would work for $9 https://www.amazon.com/HDMIHOME-Power-eSATAp-ESATA-combo/dp/B00GM7CMFG

Just had a thought.  Give me a few minutes and I'll check a different way the USB3 overhead.  Just happen to think I've got a 2.5" to 3.5" SSD adapter so I can test USB vs SATA on my PowerEdge for real difference. BRB

 

@cayars Did you get the cable above to work on your 920+ esata port with the SSD? It appears that cable needs the esata port to be powered (eSATAp) to in turn power the drive and I'm not sure whether the Synology ones are the powered version eSATA ports ? I can't find anything on the synology site one way or the other, just that it doesn't handle ones with port multipliers.

[Carlo 4330]

The only thing I've ever tested eSATA wise on the Synology 920+ is my 8 bay storage chassis which isn't recognized but that's not surprising as Synology intentionally doesn't support eSATA connections that would require a multiplier. They reserve the use of a multiplier for only their own use with expansion boxes like the 517.

You need to be careful using eSATAp cables as the pinouts are different and so is the shape. I don't think Synology uses any eSATAp ports but only eSATA. I've never seen an eSATAp port on any Synology.

Here's the difference between the two:

Here's a closeup of eSATAp

The parts inside the yellow section I added show the USB 5V connections. Pretty much any 2.5" drive should work with eSATAp as it will always have 5V available. Most 3.5" HDDs use both 5V and 12V which aren't guaranteed to be present. Almost all laptops using eSATAp have only 5V while some desktop will add both 5V and 12V. This is typically referred to as eSATAdp (dual power).  For the 3.5" drives you can also use external 12V power as well.

In that case you can use a cable like this which allows injecting 12V in.

It's funny that you brought up eSATA testing as I had ordered a couple adapters to try but they got lost in delivery.  I've reordered them and they should be here on Friday.

I reordered these:
 and 

I've got an interesting experiment I'm going to try in a couple week on my 920+ by making use of one of the PCI slots for a different use then the USB connection.

Hopefully this weekend we'll have some new information on speed difference from eSATA to USB3.

[usnscpo 24]

/Thumbsup!

[DJX 18]

Sorry to dig this up but with black friday looming I am interested. I want to move transcoding temp to a external drive rather than it doing on my HDDs. My main use case is chromecast and android phones.

So its best I use the eSATA port with a compatible drive and housing right? Is the 850 Pro still your recommendation?