January 17, 20233 yr So I built my first Unraid server this weekend, and I'm currently trying to transfer all my media files to it. I'm really struggling with poor transfer speeds, so I hope someone here may be able to point out something I'm missing/doing wrong. To start, here are my specs: Intel i7-8700 16GB RAM Gigabyte Z390 Aorus Pro 1x 1TB Samsung 980 Pro NVME (Cache Pool for backups) 1x 500GB Samsung 970 Plus NVME (Cache Pool for appdata) 1x Intel DC S3710 800GB SSD (Cache Pool for media - CURRENTLY DISABLED) 1x Intel DC S3700 800GB SSD (Unassigned Disk for incomplete downloads) Dell HBA330 1x WD White Label 20TB (Unassigned Disk containing my media) 3x WD White Label 18TB (assigned to media share) 1x WD White Label 12TB (assigned to media share) Intel X540-T2 What I'm trying to achieve I am trying to copy approx 17TB of data to my Unraid media share from a 20TB mounted Unassigned Device disk. The issue I've encountered I copied my TV folder (approx 8.2TB) to my media share in 'user0'. At first, the transfer speed was as expected; hovering in the 160-190MB/s range. This looked fine to me, so I left it and went to bed. I checked back around 9 hours later, and the transfer had only reached around 78% complete. It had also slowed down, fluctuating between 40-70MB/s. 16 hours later, the transfer speeds had fallen to a consistent 20-25MB/s and the transfer had only reached 84% complete. A further 15 hours later (at the time of writing this), the transfer is 90% complete and the transfer speeds now hover between 15-18MB/s. What I've tried Setting the CPU governor to performance Changing the share write mode to "reconstruct write" Using Krusader to copy a file which was currently Pertinent Info I have not setup a parity drive yet. My prior research suggested that this would be best to do after my initial data dump. I have not setup a cache drive for my media share yet, for the same reason as above. My media share is set to "high-water", but so far it is only filling one drive. My split level is "top two directories". An example of the structure in the share is: "sharename>media/tv/tv show" My HDD's are all CMR drives. They are shucked WD Elements drives. The 20TB drive (containing the media backup) is almost full, with only 550GB left of free space. I'm aware that full drives will slow down, but at this point I'm getting read speeds of less than 10% of the drives peak performance. Something strange I've noticed Out of curiosity, I noted the file that was currently being copied to my array at around 16MB/s. I opened up Krusader and did a couple of test copies. I copied that exact same file and simultaneously pasted it to the share mount. This successfully copied at a transfer rate ranging between 60-75MB/s. The resulting file ended up being stored on disk1, which is where everything else has gone so far. I then pasted the same file directly to disk1, thinking that this would go faster. This crawled along at 16-19MB/s. What am I missing? I know not to expect rapid transfer speeds with Unraid, but surely this isn't normal? For comparison, I copied all 17TB of my media to that single HDD in just under 26 hours (in a virtualised setup!). This new setup hasn't even done 8TB after 40 hours.
January 17, 20233 yr Author Also, why would I see these read and write speeds on the dashboard but not have this reflect in my file transfers? It's like there's a cache bottleneck somewhere that I'm missing.
January 17, 20233 yr Author 15 minutes ago, JorgeB said: What program are you using to copy the data? The built-in Unraid File Manager. My apologies, I lost my original post and missed this info out when I rewrote it.
January 17, 20233 yr Community Expert Try using Midnight Commander or Windows explorer (it will still make a local transfer) to see if there's any difference.
January 17, 20233 yr Author 22 minutes ago, JorgeB said: Try using Midnight Commander or Windows explorer (it will still make a local transfer) to see if there's any difference. Ok. I'll give it a try. Just for clarity, is that mounting the share and UD drive via SMB in Windows? Also, if I cancel the existing copy, will that keep everything it has transferred so far?
January 18, 20233 yr Community Expert 12 hours ago, capfin said: Just for clarity, is that mounting the share and UD drive via SMB in Windows? Yep. 12 hours ago, capfin said: Also, if I cancel the existing copy, will that keep everything it has transferred so far? It should keep anything already transferred.
January 19, 20233 yr Author On 1/18/2023 at 8:10 AM, JorgeB said: Yep. It should keep anything already transferred. So I tried with MC, and in summary, large transfers of data through FUSE can replicate this slow down. Disk share to disk share HDD transfers run at full speed with no issues. I transferred my remaining ~9TB of data in timeframe one would expect from a fully performant drive. Another thing I noticed is that NVME to NVME transfers cap out at 1.1GB/s, even using disk mounts. Considering I'm running NVME's capable of R/W speeds over 3,000+, this seems poor. This will max a 10gbe NIC, so it's not the end of the world, but it would be nice to know that my drives are performing to the best of their capabilities internally. Edited January 20, 20233 yr by capfin
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.