Hello,

I am having an issue with an unRAID pro installation. When copying large files (10GB+) to an SMB Share the transfer moved along quickly (consistently 100MB/s) until the transfer speed drops dramatically. When this happens the entire server becomes extremely slow until the transfer is stopped. Any dockers or VMs are unusable even the unRAID web interface barely or sometimes doesn't load. Thank you for any help! The system specs are below:

Model: Dell R710

M/B: Dell Inc. - 00NH4P

CPU: Intel® Xeon® CPU X5675 @ 3.07GHz

HVM: Enabled

IOMMU: Enabled

Cache: 192 kB

Memory: 64 GB (max. installable capacity 288 GB)

Network: bond0: fault-tolerance (active-backup), mtu 1500 
 eth0: 1000 Mb/s, full duplex, mtu 1500 
 eth1: 1000 Mb/s, full duplex, mtu 1500 
 eth2: 1000 Mb/s, full duplex, mtu 1500 
 eth3: 1000 Mb/s, full duplex, mtu 1500

Kernel: Linux 4.9.19-unRAID x86_64

OpenSSL: 1.0.2k

Uptime: 64 days, 18:33:34

Hi - I'd recommend getting a Tools->Diagnostics during the problem, or after it has occurred and posting it here.

Hi,

I took 3 diagnostics. The "Before Transfer" is exactly as it sounds it is before I started the transfer and the server is running normally. The "During Transfer" is as soon as I saw the file transfer speed drop down and the third "After Cancelling" is after I cancelled the transfer. The after Cancelling was when the server performance is at its worst.

deepskyserver-diagnostics-20171104-1519 (1)_DURING TRANFER.zip

deepskyserver-diagnostics-20171104-1519_BEFORE TRANSFER.zip

deepskyserver-diagnostics-20171104-1519 (2)_AFTER CANCELLING.zip

I briefly looked at your diagnostic files and saw a whole lot of interesting things going on...Looks like you have about 64GB of ram with a 64GB swap file with all but 775MB used.  Really don't need a swap file with that much ram installed and definitely not that large.  Your appdata folder should be cache-only and if you don't have a cache drive I strongly suggest getting one.  As you read/write files you are filling up your ram between flushes as that is what Linux does and then you swap out to disk adding to the disk I/O issue.  Suggest installing the Docker called 'Netdata' from CA and have it open in a web browser the next time you do one of those large SMB transfers and post a screenshot.  Then disable your swap file and try it again.

12 minutes ago, unevent said:

I briefly looked at your diagnostic files and saw a whole lot of interesting things going on...Looks like you have about 64GB of ram with a 64GB swap file with all but 775MB used.  Really don't need a swap file with that much ram installed and definitely not that large.  Your appdata folder should be cache-only and if you don't have a cache drive I strongly suggest getting one.  As you read/write files you are filling up your ram between flushes as that is what Linux does and then you swap out to disk adding to the disk I/O issue.  Suggest installing the Docker called 'Netdata' from CA and have it open in a web browser the next time you do one of those large SMB transfers and post a screenshot.  Then disable your swap file and try it again.

I have a 1TB SSD for a cache drive. Below is the configuration for the appdata folder. I am not sure how to make it "cache only."I installed netdata. Which section of it should I screenshot? There is a ton of information in it.

 

Set 'use cache disk' to only

Got it, the appdata is now set to cache only. How do I disable the swap file? I have looked everywhere.

5 hours ago, Bryce said:

Got it, the appdata is now set to cache only. How do I disable the swap file? I have looked everywhere.

Not sure where @unevent saw it.   It's not showing in the memory.txt file

You want to go back and set cache disk to Prefer, and the run mover.  That will bring the appdata files back from the array.  You can then leave it as Prefer or switch to Only.

Edited November 5, 20178 yr by tdallen

9 hours ago, Squid said:

Not sure where @unevent saw it.   It's not showing in the memory.txt file

 

If you cross your eyes, it's there...

 

I'm also curious on what you mean by slow transfer speeds and what your settings are for the share you were copying a file to.

 

When you write data to the array in a default disk configuration, we expect write speeds to average out between 30-60 MB/s.  Some can go faster under ideal conditions, but that's atypical.  This is due to the nature of a dedicated parity disk.  There are things you can do to improve that performance.  One is to utilize a cache disk and enable any shares you have to use it.  Shares that leverage a cache disk will benefit from full write performance to the cache and then the data will get moved to the array automatically at night when you're ideally not using the system.  NOTE:  Just having a cache disk installed is not enough.  You have to specifically toggle the setting to use the cache disk on any of the shares you have.

 

The reason your write speed may burst to 100MB/s even without a cache disk is due to the sheer amount of memory on your system.  Writes will buffer to RAM before hitting the disk, so it's very possible at the beginning of your transfer, you're just bursting and then when the RAM fills up, the bottleneck of your storage may be kicking in.

 

I would recommend double-checking the share settings for the share you're manually copying data into, make sure that it is cache enabled, and also make sure you have a minimum free space value set for the share.  That value should be equal to the largest sized single file you will ever transfer into that share.

There is also "reconstruct write" mode (a.k.a., turbo write) which speeds up writes considerably, but requires all drives to be spinning for any write operation. For a large copy, though, might be worth turning it on temporarily.