Faster way to get data onto Unraid array?

[RobAinscough]

I have about 40TB worth of data on two external WD USB 3.2 Gen1 drives (22TB each) - NTFS format - rated at 5 Gb/s (625 MB/s) RPM listed as 7200.  I realize that I'm not going to get 625 MB/s from a 7200 RPM SATA drive, it'll be more like 200-300 MB/s.  USB 3.2 cable.

 

I started moving data from these drives via one of the PC's connected to my LAN (10G network thru-out, tested and verified).  Performance was pretty bad, it would start out ok at 200 MB/s but would soon drop to about 50 MB/s and if many small files it would 1 MB/s.  I was getting estimate time to complete around 8-16 days ... yikes.

 

After some research I discovered I could use Unassigned Drive package for Unraid and plug my external drives directly into USB 3.2 port on my UnRaid box ... thinking this would be faster than going across my LAN from another PC.  Installed UD, mounted external drive, selected folder on it and selected destination on my share and hit Copy.  Sadly it mimicked exactly same performance as I experience via using my PC going thru my LAN ... starts out at 200 MB/s and then soon drops to 50 MB/s.

 

Puzzled, USB 3.2 is rated at 2500 MB/s ... so why am I at 50 MB/s?

 

Are there faster alternatives for external storage devices?

 

Cheers, Rob.

 

EDIT: Motherboard is Z790 with USB 3.2 Gen 1 ports also

 

Edited September 16, 2024 by RobAinscough

[Hoopster]

9 minutes ago, RobAinscough said:

so why am I at 50 MB/s?

 

Is your array parity protected? If you have parity enabled on your array, 40-70 MB/s is not unexpected.  When transferring large amounts of data to an Unraid array, many have parity disabled for that process so you can get the full transfer speed of each individual disk and run a parity check after data transfer.

Edited September 16, 2024 by Hoopster

[RobAinscough]

Yes, parity is enabled (2 X 20 TB drives set for parity).

 

Hmmm … but when I added parity that took 2 days to complete.  So if I disable parity, do the transfers, then enable parity again, isn’t that going to be another 2-3 days or more for parity to catchup?

 

I’m a little surprised parity is cutting performance by that much?

[JorgeB]

You can disable parity for the initial transfer or enable turbo write.

[Hoopster]

26 minutes ago, RobAinscough said:

Hmmm … but when I added parity that took 2 days to complete.  So if I disable parity, do the transfers, then enable parity again, isn’t that going to be another 2-3 days or more for parity to catchup?

 

I’m a little surprised parity is cutting performance by that much?

You pay the "parity penalty" at some point either during the data transfer or after.  As mentioned by JorgeB, another option is to enable Turbo Write.  The only penalty" for turbo write is that all data disks are constantly spun up but this not a big deal during a large data transfer.

 

A good description of turbo write and the parity write process can be found at this link

 

[RobAinscough]

Interesting, thanks for the suggestions.  Now, how do I stop my transfer so I can test this out?  I can’t seem to find any place where I can cancel my copy?  I can see the copy status if I click on the Unassigned Drive in Main but can’t find any UI area to stop/cancel the copy?

[JorgeB]

What app are you using to make the copy?

[RobAinscough]

From USB PC via LAN to Unraid, I've tried standard Win11 Copy and Paste, cmd line xcopy, and a 3rd party tool called TeraCopy.

 

Direct USB on the Unraid server I use the following:

 

 

And 

 

 

[JorgeB]

5 hours ago, RobAinscough said:

I've tried standard Win11 Copy and Paste

You can cancel that, not sure about DFM since I've use it very little.

[RobAinscough]

Tried without parity (basically unassign my 2 parity drives), still extremely slow with transfers.

 

Found this thread with someone having similar problem:

Looks like Unraid is just SLOW!  I do feel there is something just not right about this ... the xfers start fast 200+ MB/s but over time crawl to 1-5 MB/s ... and these are single large MP4 or AVI or MKV video files (not millions of small files).  Honestly folks, I think there is something wrong with Unraid in this particular instance.

 

I'm hoping the Unraid folks can come up with an alternate solution or something in the future, lucky I only have about 47TB to put on Unraid ... anyway, hopefully it'll complete sometime next month.

 

 

Edited September 19, 2024 by RobAinscough

[JorgeB]

4 hours ago, RobAinscough said:

Tried without parity (basically unassign my 2 parity drives), still extremely slow with transfers.

That's not normal, start by posting the diagnostics during a transfer.

[RobAinscough]

Great link, thanks.  I’ll do that for the next file transfer job and report back.

 

I’ve confirmed it’s not a network issue by transferring the same file between two PCs (Win11) and it was very fast.  But as soon as I copy the same file to Unraid it fluctuates from 1MB/s to best case 50MB/s but oddly starts at 200MB/s then quickly drops (this is just one single large file about 27GB).  Given that I had the same problem direct from USB mount on the Unraid box … starts out fast at 200MB/s then soon drops to 1-5MB/s.

 

Anyway, will go thru the diagnostics process and see.

 

Cheers, Rob.

[JonathanM]

Have you run the diskspeed benchmarks?

 

[RobAinscough]

I will give that package a try and report back.  I do have 12 drives - 2 largest ones set to parity.

[RobAinscough]

Here are the DiskSpeed test results.  I don't see anything unusual except some warning about a "gap"?

 

 

Drives: