Choppy playback during file transfer

[rigo8582 16]

Whenever there is files transferring using something like using "Auto-Organize" my playback is very choppy or doesn't play at all. The playback isn't being transcoded and I'm 100% sure its a disk bandwidth issue. Because when the file transfer is done playback resumes with no issues. Does anyone know if there is a method of automatically throttling the a file transfer during playback?

 

Thanks,

[Happy2Play 9783]

50 minutes ago, rigo8582 said:

Whenever there is files transferring using something like using "Auto-Organize" my playback is very choppy or doesn't play at all. The playback isn't being transcoded and I'm 100% sure its a disk bandwidth issue. Because when the file transfer is done playback resumes with no issues. Does anyone know if there is a method of automatically throttling the a file transfer during playback?

 

Thanks,

@chefis there anything that can be done here besides not running the task during prime hours?

[chef 3810]

7 hours ago, Happy2Play said:

@chefis there anything that can be done here besides not running the task during prime hours?

We could look for active sessions before attempting a file transfer. 

I haven't experienced this, but then again  my server is monster.

I can see how transferring file could cause issues. 

I can add a option to wait for clear sessions before attempting file transfer.

I dont know when that will be though... All I want to do is program computers, but people keep wanting to eat food... Go figure.

I try my best to add this ASAP! 

[rbjtech 5284]

You don't mention where the source and destination file are - but if you are attempting any form of network (NAS) transfer - then it will max the ethernet link between them (assuming you are running 1gig)

If it's on the same machine - then you may be able to improve things by moving the 'source' to a different disk and ideally a different sata controller.

Ultimately though, as has been alluded to above - if you have big I/O operations on your disks that you are also using to playback from - then you need to do it out of hours as you are at risk from interrupting playback.

If you really want to get smart - and run Drive Pooling software such as DrivePool - then you can opt to put all 'new' files on SSD (but still 'mapped' to your normal emby library) and then it will transparently move them out of hours ..  Do depending where your bottleneck is - this 'may' help.

Edited December 19, 2022 by rbjtech

[rigo8582 16]

8 minutes ago, rbjtech said:

You don't mention where the source and destination file are - but if you are attempting any form of network (NAS) transfer - then it will max the ethernet link between them (assuming you are running 1gig)

If it's on the same machine - then you may be able to improve things by moving the 'source' to a different disk and ideally a different sata controller.

Ultimately though, as has been alluded to above - if you have big I/O operations on your disks that you are also using to playback from - then you need to do it out of hours as you are at risk from interrupting playback.

Its being transferred from different disks and controllers. The source is an SSD and the destination is a Raid 5 array. On the surface I wouldn't have expected a file being written would be enough but it affects all drives if a file is being copied to the drive where the playback is coming from.

Was curious if anyone had any ideas or work arounds to mitigate it.

[rbjtech 5284]

3 minutes ago, rigo8582 said:

Its being transferred from different disks and controllers. The source is an SSD and the destination is a Raid 5 array. On the surface I wouldn't have expected a file being written would be enough but it affects all drives if a file is being copied to the drive where the playback is coming from.

Was curious if anyone had any ideas or work arounds to mitigate it.

Your issue is likely the RAID5 array - write speed is likely to be 'slow' because of parity.  Your SSD will be maxing it's capabilities without doubt.

My personal view is RAID5 is unsuitable for media storage.    I don't see the need for RAID at all on a typical media mass storage - the storage is basically static and just builds as you add new media.  If you have a known good backup or an 'offline' copy - then if you ever lost a disk - then just restore it from the backup.  It would be substantially quicker than attempting to do it via a RAID parity rebuild.

To add, you don't need large read speed for media playback (an advantage of RAID) - even 4K remux playback is only reading at 8-15 Mbytes/sec - a normal HDD will hit 180-200 Mbytes/sec+ before hitting it's limit.  

If you have a very dynamic environment - then of course RAID will provide redundancy until your next backup, but maybe do what I do and use RAID1 (miroring) on your dynamic, non media files and adopt a different redundancy strategy for the mass media storage. (I effectively use RAID 1 still, but the mirror is offline and thus acts as my backup as well).

 

[rigo8582 16]

51 minutes ago, rbjtech said:

Your issue is likely the RAID5 array - write speed is likely to be 'slow' because of parity.  Your SSD will be maxing it's capabilities without doubt.

My personal view is RAID5 is unsuitable for media storage.    I don't see the need for RAID at all on a typical media mass storage - the storage is basically static and just builds as you add new media.  If you have a known good backup or an 'offline' copy - then if you ever lost a disk - then just restore it from the backup.  It would be substantially quicker than attempting to do it via a RAID parity rebuild.

To add, you don't need large read speed for media playback (an advantage of RAID) - even 4K remux playback is only reading at 8-15 Mbytes/sec - a normal HDD will hit 180-200 Mbytes/sec+ before hitting it's limit.  

If you have a very dynamic environment - then of course RAID will provide redundancy until your next backup, but maybe do what I do and use RAID1 (miroring) on your dynamic, non media files and adopt a different redundancy strategy for the mass media storage. (I effectively use RAID 1 still, but the mirror is offline and thus acts as my backup as well).

 

I know I get all of that.

I also get the same results on a single hard drive. My lower bitrate and resolution media is usually not impacted or affected much less. Its my 4k media is impacted the most. That is the case if its on my RAID array or single drive.

My raid 5 has an ssd used as write cache to improve write performance. So when it does a write it doesn't take all that long.

Some of it I know I can mitigate by trying schedule it at a time that media is not being consumed. But wanting to know if there were any other solutions.