Copying existing NTFS DrivePool to Unraid

[Dan M]

I'm new to Unraid, so I may be missing a few steps and using some wrong terminology, but here goes...

 

I set up an Unraid server with a 14TB Parity Disk, an 8TB Storage Disk (first in the array), and just because I had it lying around, a 512MB SSD. All connected to SATA 6GB. All pretty simple, but "readying" the Parity Disk was going to take 27 hours.  Totally blank/new drives.  27 hours.  OK, whatever.   While it was building the parity disk, I did see that the 8TB share was available and I could connect to from a Windows 11 machine and write to it.   Write performance to the share on a gigabit LAN was a little slower than usual, but acceptable.  Time to build the parity went way up (it was up 9 days at one point) when I copied data, but I was just testing the waters.  After I was done playing around it coalesced back down to ~27 hours.

 

So, anyways, 27 hours later.  My real goal is to copy about 60TB from my backup StableBit DrivePool array (think Windows Storage Spaces, but DrivePool uses standard NTFS drives and you are able to access files from the drives) spanning 10 6TB and 8TB drives to Unraid.  Since I don't have another 60TBs laying around, I knew I had to go through a process where I would copy one drive from the DrivePool to Unraid, then add the DrivePool drive to the Unraid array, rinse and repeat 10 times.   So I hooked up half of the drives to the SATA ports on my motherboard, all were recognized as Unassigned Devices.   I mounted the first drive and used Midnight Commander (MC) to copy the data from /mnt/disks/{drivename} to /mnt/user/{sharename}.  It was SLLLLLLOOOOWWW.   I expected to take about 10 hours (it about about 5TB), but the MC ETA said 24 hours.   I let it run, figuring because it was copying some smaller files first, the ETA would be off.  When it got to the meat of the drive, with quite a 1 to 5GB files, the ETA would drop.   I went to bed and checked 8 hours later, the ETA was still about 24 hours (it fluctuated down a little, but not much).  I figured I'd let it finish while I did some work, ran some errands, etc.   Sure enough 24 hours it completed.   The average speed was about 40MB/s according to MC.  OK, well, I figured, it's a day - I could live with that if I had to, I'm not in a hurry and I don't have to babysit it while it copies.

Then I went to add the drive that was copied to Unraid to the Unraid Array and move on to the next one in the array.  No, Unraid wanted to "clean" the drive before I could format it and add to the array.   How long was cleaning supposed to take?  According to the Unraid Main page, about 25 hours.   OK, I figured, something wasn't right.  2 full days to copy 5TB of data and move on to the next drive?  All SATA connected, no LAN, no USB.   No.  I was about to give up on Unraid, as much as I like what it has to offer, it would take 20+ days to copy the data. Just no.

So I started reading.   It looked like the main thing causing the slowness was the Parity.  I tried disabling/stopping the Parity drive.  Speed from the next drive in the pool to the array (just a few test files) was almost double speed.   But Unraid still wouldn't let me add the old NTFS formatted drive to the array without cleaning it first.  I tried fdisk and wiping the drive, no go.  So I read some more.  Apparently removing the parity drive (adding it back to Unassigned Devices) might help.  When I was done copying, the parity drive would have to be rebuilt, but it should result in faster adding.   Sure enough, removing the parity drive allowed me to format the old pool drive and I was able to add it to the array in just minutes.  I'm copying now (again using MC) from the old drive (/mnt/disks/{drivename}) to the array (/mnt/user/{sharename}).  The average speed is 132MB/s and the ETA was about 9 hours.  When everything is copied, I'll put the 14TB drive back in as parity and let it rebuild - I sure hope it doesn't say 20+ days!!!!

 

So, anyways, yeah, I've got things copying pretty close to the theortical limit of SATA and adding the pool drive after now only takes a few mintues.  (Oh yeah, figuring out how turn on "Destructive Mode" and whatever else needs to be done, is not exactly intuitive.)  But, does anyone know what I could have (or could do) differently?   I don't mind *some* overhead, but 48+ hours for a process that theortically only takes about 9 hours seems like overkill.