Plex stuttering while moving files from one HDD to another


mgutt

Recommended Posts

I'm moving some movie files from disk2 to disk3 with the following command:

mv -v "/mnt/disk2/Movie/IJ/"* /mnt/disk3/Movie/IJ

At the same moment I'm watching a movie through Plex from disk2 and its stuttering. iotop returns the following:

1194649012_2020-08-1000_28_37.png.2d9ee6d67d20c13a931d4fc6aa4a3e36.png

 

 

If I pause the movie the IO load of "mv" falls to around 60%:

506029184_2020-08-1000_34_44.png.fbc5e0012c6140b967d3fa46ecf76a9d.png

 

If I resume the movie multiple SHFS processes kick in and produce a massive load so everything raises up (up to 99.99%, see first screenshot):

1408220146_2020-08-1000_39_00.png.ea124a907e05ca9aff4af318afdd2ff7.png

 

Why are there so many of them producing so much IO load? I mean its only a movie with a bitrate of around 40 Mbit/s (5 MB/s) and it should be an easy task for a 7200 rpm HDD to have one file transfer and one movie stream at the same time.

 

Server specs

Supermicro A2SDi-8C+-HLN4F

Intel Atom C3758 (4675 cpubenchmark points)

32 GB RAM

10x Western Digital Ultrastar DC HC520 12TB

XFS formatted

 

Sorry I can not provide the full diagnostics as altough unraid states, that they are anonymized, they contain file names that are private. So please ask for specific log files and I will censor them manually, before posting them.

Edited by mgutt
Link to comment
  • 7 months later...
On 3/12/2021 at 6:18 AM, ogi said:

Hi @mguttI'm curious if you got some resolution here... I've noticed my video streams have to pause/buffer while another file is transferring to the array.  I'm hoping to come up with some settings to prioritize plex streams.

 

I retested this scenario as my server specs changed as follows:

Gigabyte C246N-WU2

Xeon E-2146G

64 GB RAM

10x Western Digital Ultrastar DC HC520 12TB

XFS formatted

 

Now this happens only if I transcode and if I don't use a parity disk:

2124867006_2021-03-1313_48_52.png.aff8df1d9c945ecc63125d6e144f8927.png

 

This seems logical as a parity disk limits writes to a target disk to 80 MB/s which limits reads from source disk to 80 MB/s, too. But I don't know why it happens only while transcoding.

 

Are you using a parity disk? Which CPU?

 

 

Another solution to solve this, could be to bypass the Unraid SHFS overhead by changing Plex container paths from /mnt/user/movies to /mnt/diskX/movies. If your movies are on multiple disks, you need to add multiple paths. I do not use this solution as this would cause a full rescan in Plex if I move my movies to more or less disks.

 

Link to comment

I have a 2x  E5-2680 v2 CPUs; I use dual parity drives inside a Supermicro CSE-846 chassis.  My situation I don't think is Plex specific; and unfortunately I couldn't replicate last night while watching a high bitrate video (video is 82615 kb/s), but I did just upgrade from unraid 6.8.3 to 6.9.1.

 

When streaming very high bitrate video (on VLC for examples) periodically the video stream will outright stop for ~10 seconds... and then resume.  I have assumed this was due to some other write operation to the array; but I cannot be certain.  I should point out things did get better over NFS vs. SMB; but I suspect that's just because the faster file transfer allowed for faster buffering (getting NFS mounts on macOS took me a while to sort out, I should probably post a guide about that!)

 

I'll try and monitor things in iotop and the "System Stats" page on the webui and see if I can get a better idea of what else is happening when this occurs.

 

As your issue involved Plex transcoding, and mine seems to have to do with just general file read operations, I'll create a new post if I'm able to reproduce more consistently.

 

Thanks for chiming in!

Link to comment

Are your disks encrypted?

 

Please enable disk shares and start playing a movie directly from this disk share while running the disturbing write operation. Example:

dd if=/dev/random of=/mnt/diskX/newshare/random.file bs=128k iflag=count_bytes count=128GB

 

 

Link to comment

It's running now; I'm getting occasional hangups in the stream; here is a screenshot when during one of those freezes... for the most part its playing well.

duringFreeze.thumb.png.de38696cda527c172a194a44aacf3e70.png

 

The `dd` command finished

 

root@Tower:~# dd if=/dev/random of=/mnt/disk18/movies/random.file bs=128k iflag=count_bytes count=128GB
976562+1 records in
976562+1 records out
128000000000 bytes (128 GB, 119 GiB) copied, 2241.48 s, 57.1 MB/s
root@Tower:~#

 

Given the movie (mostly) played w/o interruption, not sure what to make of it.

Edited by ogi
moreinformation
Link to comment

Thanks for chiming in @mgutt  It's been running for ~10 minutes w/o issue; have to go back to monitoring the kids.... will edit this post later after the dd command returns.

 

EDIT: kids now want to watch a movie too; so this will be a real stress test haha.

 

EDIT2:  dd command finished

 

root@Tower:~# dd if=/dev/random of=/mnt/user/movies/random.file bs=128k iflag=count_bytes count=128GB
976562+1 records in
976562+1 records out
128000000000 bytes (128 GB, 119 GiB) copied, 1954.55 s, 65.5 MB/s

 

Everything seems to be working remarkably well ...two movies playing (one on Plex via transcoding, BTW thanks for your in-memory decoding guide)... I guess whatever I have had going on before, it's no longer an issue? ...maybe I should test some CPU intensive tasks too...but I can do that later.

 

EDIT 3: Just wanted to say thanks again for your debugging steps.

Edited by ogi
Link to comment
3 minutes ago, mgutt said:

Only to be sure: The random.file was written to the disk where the movie of the kids was located, too?

 

 

Sorry for the confusion; random.file was written to the same disk that a very high bitrate movie was streaming from (disk 18).  I'm not sure which disk random.file was written to when I aimed it at /mnt/user/movies/random.file (haven't looked for it yet).  

 

The kids movie, I'm fairly certain is on another disk in the array.  All operations went smoothly so now I am having trouble even replicating the issue I began with (where the movie would freeze for 5-10 seconds, and there would be no data coming in via the network during that time).  

Link to comment
1 hour ago, ogi said:

I'm not sure which disk random.file was written to when I aimed it at /mnt/user/movies/random.file (haven't looked for it yet).  

I missed to say that. You need to check through the Unraid disk overview which disk is writing and start a movie from that disk.

Link to comment
7 hours ago, mgutt said:

I missed to say that. You need to check through the Unraid disk overview which disk is writing and start a movie from that disk.

Ahh that makes more sense, I see what you're getting at.  Will test this when I get some time later today.

Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.