unRaid Noob - How to get Parity Working (in my scenario)

[Jason Stevenson]

I'm an old windows guy, with my first unRaid System.

So to begin with I had a 40TB total storage as Windows Server (Plex) consisting of many SATA/USB disks with some NTFS, and some exFat.  My goal was to build the unRaid server, without losing the 40TB of content I had accumulated over the past 8 years, and gaining some fault tolerance, as my drivers are starting to age.

Current Storage Disks:
6 - 4TB WD Reds (Internal SATA)
1 - 8TB WD Red (Internal SATA)
1 - 8TB WD My Book External USB 3.0 drive
1 - 250GB SSD
1 - 120GB SSD

I decided to buy 3 - 5TB External USB 3.0 drives (WD Easystore), and rebuild my system using unRaid.  I had planned to start by building the unRaid array without parity or cache and copy the data from my 2 - 8TB disks into the new 15TB array (made from the 3 5TB externals).

I setup a Krusader docker, and spent a day copying ~14TB of data into the new 15TB array from the the 2 8TB drives copied into the new array.  This worked great.

Next I setup my other containers for my Plex System: Plex, NZBGet, QBittorrent, Radarr, Sonarr, Jacket, and Ombi.  These worked pretty well, but were sluggish running from the array.

I wiped the 2 SSDs and created a RAID0 cache pool, then let mover move all the containers/appdata/etc to the cache.  This fixed all of the performance issues, and everything was working great.  Now I was ready to beginning pulling data from the rest of the old disks 1 disk at a time growing the array, but I wanted to add parity first.

So I formatted the internal 8TB WD Red drive, and added it to the array as a parity disk.  I restarted the array, and it began to perform the sync operation.  Everything was great for about 10 min, afterwhich 1 or more CPUs showed 100%, and the dockers all froze, and only the main unRaid web UI was responsive. The system would not shut down, or reboot.  I had to hard boot the system.

Next I tried again, this time with VMs disabled (I had none), and docker disabled.  Again the same result after the same 34.5GB of syncing, it locked up again.  This time a different CPU, but same result.  I had to hard boot the system.

I found that 2 of my 6 USB 3.0 ports on the motherboard (MSI Z170A PC MATE (MS-7971), Version 2.0) were gen 2 USB3, and the other 4 were gen 1.  I thought perhaps because 1 of the new 5TB externals were on the gen2, and the others on the gen1 ports, perhaps that was causing an issue.  So I rearranged so all array drives were on the gen1 USB3 ports, but again the same freeze trying to sync parity at 34.5GB.

So now I'm at a loss now sure what to do next, how to even proceed, now that see how awesome docker containers are, I'd love to stick with unRaid.  But I really wanted parity, which was originally my primary reason for migrating.

So if anyone has any insight I'd greatly appreciate it.

-- Jake

jakeflix-diagnostics-20210519-2128.zip

[trurl]

2 hours ago, Jason Stevenson said:

15TB array (made from the 3 5TB externals).

USB is not recommended for array or pool.

[Jason Stevenson]

Not recommended?  Or not possible.  I have no other options until I get the array fully expanded with the old disks, then I can remove the 3 USB disks.  However I'm not sure if I can do that without a working Parity Disk to rebuild after removing each disk...

So are you saying I'm not going to get it to work? Or I just shouldn't do it?

[itimpi]

Technically USB disks are allowed but USB disks are not recommended as they do not tend to provide a reliable connection.   UnRaid is not hot-swap aware and cannot handle a USB disk used as a array drive dis-connecting and then re-connecting while in use.   If this happens then UnRaid will end up misbehaving.

 

You might be better of not using the USB drives as array drives, but as temporary drives mounted using the Unassigned Devices plugin (which can handle USB drives reliably) and then copying data off them into the array of SATA drives.    Since under UnRaid each array drive is a self-contained filing system they can be read individually even when removed from the array.    Longer term these USB drives can e used for backup or data transfer purposes.

[Jason Stevenson]

I understand, however I've been using a USB 8TB drive with my Windows Server for 2 years without an issue, I guess I assumed that Linux would be at least as stable as Windows.  I guess I should have just mounted them and copied the data to them on my Windows Server, before wiping disks and trying unRaid with my internal disks.

I have also now read that my Marvel chipset SATA card, could be causing issues.  I wonder if just having it in the server, is causing problems.  So I've ordered one of the recommended controllers to replace my current one.  3 of my unassigned devices, that are currently attached to that controller, show up as not mountable currently, and their FS is unknown.

I'll try swapping this in and attempt parity sync one more time, if it doesn't work, then I'll just copy all the data from the array back off, and then format the 3 USBs disks and mount as unassigned disks, then copy the data back to them, to start over with the 8TB disk.

Thanks for your help.
 

[trurl]

13 hours ago, Jason Stevenson said:

using a USB 8TB drive with my Windows Server

Was this just a single disk, with no parity or any relationship to other disks (striping)?

 

Including parity disk in the array is where you get into trouble with USB connections. When a disk becomes disconnected, it will be out-of-sync with the array and require rebuilding. This may be a frequent occurrence with USB. If there is no parity, then nothing is in sync anyway so nothing to rebuild (or can be rebuilt).

 

Also, a single pool disk could be USB.

 

But...

 

USB not recommended for a pool or array disk with VM or docker contents because disconnects will cause these to no longer function, maybe even force Unraid to recreate them from scratch on another disk.

 

As mentioned, better to use USB as temporary storage to get data transferred to permanent storage connected by SATA/SAS.