Hi I just setup unRaid a couple of weeks ago and I'm still in the process of getting all my disks added to the array and copying over data from my previous NAS.  I've been running into an issue with write speed for disks in the array.  After a reboot things seem normal, I see between 50-120MB/s write speeds, as time goes on and especially as I've been doing file transfers from my old NAS to unRaid, the write speeds decline eventually ending up in the KB/s range.  I've had this happen as I mentioned during file copies (using Krusader for that, and copying direct to the array /mnt/user0 to bypass cache), but I've also had this behavior while running a preclear (using the plugin).  I remember when the system was first setup and I precleared my very first disk that the zero phase took about 16hours on an 8TB disk, write speed around 120MB/s.  The last disk I precleared the zero phase took about 40hours and write speed started out in the 120MB/s area but declined down into the KB/s area over time.  

 

If I reboot everything goes back to normal and starts the decline in speed all over again.

 

I'm running unRaid 6.8.3.  Diagnostics file is attached

 

Thanks in advance for help.

diagnostics-20201222-0851.zip

Edited December 22, 20205 yr by justme123

The running VM appears to be causing a very high system load, try turning it off (or leaving it off) to see if it makes a difference.

@JorgeB  thanks for the suggestion.  So I had a file transfer that has been running for a couple of hours, the speed had slowed to 350KB/s, when I turned off the VM the speed momentarily went back to 50-60MB/s for a few seconds and then went back down to about 500KB/s.

 

After that I went into the main VM manager and disabled VMs all together just to see if that made any diff, but no change.  However now when I try to re-enable VMs I get an error that the libvert service failed to start. 

Seeing this in the syslog - 

emhttpd: shcmd (996): /usr/local/sbin/mount_image '/mnt/user/system/libvirt/libvirt.img' /etc/libvirt 1

root: /mnt/user/system/libvirt/libvirt.img is in-use, cannot mount

 

The VM that was running is my main system running blue iris so I do need to have that running.  I have an unassigned device that is being passed thru to it as it is always recording.

Edited December 22, 20205 yr by justme123

13 minutes ago, justme123 said:

root: /mnt/user/system/libvirt/libvirt.img is in-use, cannot mount

Stop/start array should fix it, if not reboot.

3 hours ago, justme123 said:

the zero phase took about 16hours on an 8TB disk, write speed around 120MB/s.  The last disk I precleared the zero phase took about 40hours and write speed started out in the 120MB/s area but declined down into the KB/s area over time.  

Pls check does any slow/abnormal disk in array, this problem quite common. You may need docker diskspeeed apps to identify it.

Yes, that's a strong possibility, also note that disks 2 and 3 are SMR, and you can be hitting the SMR wall.

Ok I will perform some speed tests.  I can say though that I've seen the same issues on all of the disks where speed is good at first and then it declines over time.  Would SMR drives show this type of behavior?  Right now I've only been copying video files that are 10+GB in size.

 

thanks

Edited December 22, 20205 yr by justme123

Did some more testing, this time I rebooted the system and left the vm manager completely disabled.  Over time ended up with the same result file transfers slowed to under 300KB/s.  

 

This is a screen shot from grafana.  I was copying a bunch of large movies and it was fast for a long time then suddenly dumped down to 5MB/s and trailed off to the speed it is now while I'm posting this under 300KB/s.

 

Not sure what to try next, I guess I could uninstall plugins, or maybe upgrading to 6.9.x?  

Try writing to disk1 exclusively, since it's the only non SMR data drive.

Tried writing to disk1 and still ran into problems.

 

I hope I don't jinx myself but I've since upgraded to 6.9-RC2 and so far haven't been able to reproduce the issue.  I did make one other change and that was disabling folder caching, but I had done that pre upgrade and it hadn't made a change in behavior.

 

Hopefully the issue is resolved with the upgrade!

 

Thanks for the help