Unraid stalls (freezes) during the files copy (write)

[Omen]

Good day fellas. It's my second week of uneven fight with unraid. It's been a journey with so many solved questions and it is truly a shame the last one seems unbeatable.

 

In short, when I do a file transfer of many small files (music, 10-20mb files) to a share (no cache), all other shares have really low read speeds.

 

I tracked it down from Plex stuttering on video playback when I copy music files into another share which is on the same drive. Plex is set up with RAM transcode and cpu cores isolation, when I play a video file (4.5GB) and copy something to a share it stalls every 1-2 minute. It also doesn't depend on VM, same happens when I copy files to a share with Krusader and watching a video from a share via network. Moreover, write speeds are really low: 30-40Mb/s. I have same Seagate Exos X16 as passthrough unassigned device and it's working on speeds up to 200Mb/s. The issue is that it seems like I can't use array to write stuff to it and reading not so demanding video from it at the same time. Cache is not an option, since there are many posts on the same stall when mover is doing it's job, which doesn't work either, I don't want to depend on it.

 

The last meaningful idea I've tried is changed cache m.2 ssd from BTRFS to xfs and it changed nothing.

 

Any ideas how unraid array can be useful with simultaneous reads and writes? Are there are any setup options left to address or it is it's limits? Thanks!

 

More quick details:

Unraid: 6.8.3

All system/vm files are on EVO 970 ssd xfs cache drive.

CPU: Intel i7-6800K

Unraid has 3 cores for it, other 3 for VM.

32 GB of RAM

CPU use during the stall is normal, but sometimes one core spikes 100%

Drives: Seagate Exos X16 16TB SATA III 

No other docker containers except Krusader or VMs are running during reproduce.

omen-pc-diagnostics-20200320-0244.zip

Edited March 20, 2020 by Omen

[Omen]

Couple search findings with the same issue:

https://www.reddit.com/r/unRAID/comments/6vryw7/read_performance_during_writes/

https://www.reddit.com/r/unRAID/comments/4dq6et/horrible_write_performance/

https://www.reddit.com/r/unRAID/comments/e9nqxy/all_io_operations_freeze_when_reading_and_writing/

https://www.reddit.com/r/unRAID/comments/bxj5eo/array_unresponsive_while_performing_write/

https://forums.unraid.net/bug-reports/prereleases/670-rc2-reading-all-disks-when-writing-to-a-single-one-r378

 

The only new advise there is to buy SAS controller instead of direct SATA motherboard connection, which is appalling.

Edited March 20, 2020 by Omen

[RossEm]

hey, did you fix it? im running the same issue?

 

[Omen]

23 minutes ago, RossEm said:

hey, did you fix it? im running the same issue?

 

Nope, I've just pinned it down yesterday after overcoming pretty standard hassle with GPU passthrough and buying separate USB controller for USB passthrough. Win 10 VM works smooth like butter, but now it is unraid gut issue.

 

The only thing I see specific with similar cases, is that it happens when copy performs a lot of file overwrites. When I was doing this 5GB file copy it was less noticeable, but small stutters still were in place.

Edited March 20, 2020 by Omen

[Omen]

4 hours ago, RossEm said:

hey, did you fix it? im running the same issue?

 

Ok, I've narrowed down the issue to specifically many small files overwrite.

 

SOLUTION (so far): avoid overwrites and enable turbo write mode (reconstruct write) per: https://forums.unraid.net/topic/50397-turbo-write/

 

Deleting files and writing from scratch pumped up write speed to 140-180 MB/s as expected and reading at the same time from same drive with no noticeable lags. 

Edited March 20, 2020 by Omen

[enkrypt3d]

Had this issue and it turned out to be dedup was enabled. I figured I had enough horse power to do it but I guess not

 

8x 14TB SATA 6Gbps drives in RAIDZ1

4x 2TB NVME M.2 for main array

AMD EPYC 7532 32 Core / 64 threads

128GB DDR4 3200Mhz RAM

3090 GPU Passthrough

 

I wish Unraid supported GRAID!