*** New Unraid user - so maybe my questions & observations might not be technically correct. ***

Bought a license - though I have not installed it yet as I am not sure if I will continue with this hardware (MB+CPU) combo. Still running the trial version for testing...

Built an array with a bunch of old disks lying around. The thing that attracted me - you can throw any disk in as long as it's not bigger than your parity disk(s).

Had a Supermicro X10SLM-F with 16GB ECC installed, gathering dust - scrounged in my junk and found an i3-4160 that would work in this MB. Had a spare Adaptec 71605. So maxed the gig out with 22 drives (6 SATA on MB + 16 SATA on the 71605), as follows - 2 x 8TB as parity, 8 x 8TB + 2 x 5TB + 4 x 4TB + 2 x 2TB + 2 x 1.5TB + 1 x 1TB - for a 21 device array with a 1TB SSD as a cache drive. What I like about this MB is that it has a USB3 port on the MB (not in the back as most MB's) that I can plug in the Unraid Boot USB. Also two 1GB ports for bonding.

For chassis I used something I had never used before - a MININGEEK 24+4 HDD open frame from Amazon - put in 8 PF-12's Noctua fans and it's more silent than my other servers. Love this chassis and have decided all my future server builds will be using this. Might even migrate some of my existing storage servers to this.

Observations

1) Most of my disks had xfs filesystem without a partition - so had to move the files to a spare disk - create a partition and move the files back - a temporary pain and lost about 2% space per drive.

2) Parity build took about 40 hours.

3) Played around with multiple scenarios - removed disks, replaced disks, etc. etc.

4) Lastly stopped the array - removed and destroyed (reformatted) both parity disks and brought up the array. Surprise - array came up as valid. Attached the parity disks and hit check parity button. Bunch of errors as expected - with a rebuild estimate in excess of a week!

5) Scrapped the old config - created a new config and brought up the array - parity started rebuilding with an expected time of 24 to 36 hours as expected (kept varying, but stabilized to 36 hours after about 12 hours).

Surprises

1) Array coming up as valid without parity disks - no warnings on missing parity disks (when I brought the array up after destroying the parity disks)

2) Estimated parity rebuid time of over one week, once new parity disks installed.

Questions

1) Accessing and copying files directly onto disks via cli (putty) bypasses parity?

2) Copying and moving files to user shares via cli keeps parity intact? - I think so, because files go to cache drive and then move to array.

29 minutes ago, dv-geek said:

Bought a license - though I have not installed it yet as I am not sure if I will continue with this hardware (MB+CPU) combo. Still running the trial version for testing...

What hardware you use for server is irrelevant. License is tied to GUID of USB boot flash drive. It is "portable." I have changed MB, CPU, RAM, etc. five times over the 14 years I have used Unraid and continue to use the same license as I keep the same array/pool drives. Of course, changes and additions can be made but then those are in the config for the next hardware change.

35 minutes ago, dv-geek said:

2) Estimated parity rebuid time of over one week, once new parity disks installed.

I have 12TB parity drive (single parity) and 8TB array drives and a parity check takes about 24 hours.

Unraid can reference file locations directly via diskX (/mnt/diskX) or user share (/mnt/user/sharename. Might as well learn this now that you are new to Unraid; NEVER, EVER mix disks and shares when dealing with files in Unraid. This can result in data loss. Details of why this is the case are all over in these forums. Always deal disk to disk or share to share.

38 minutes ago, dv-geek said:

1) Accessing and copying files directly onto disks via cli (putty) bypasses parity?

Copying files via CLI to /mnt/diskX does not bypass parity if it is present. If your goal is to populate the array disks with data at a faster speed (bypass the parity "penalty"), do not have parity disks assigned during the initial large data transfer. Parity can then be enabled and built after the initial data is transferred to the array. Using cache disk/mover is usually not an effective strategy during initial data transfer as the cache will fill up fast and can't be moved to array fast enough to be available for more data caching. Copying directly to disk will avoid the FUSE system (shares) overhead and will be slightly faster but it does not bypass parity.

44 minutes ago, dv-geek said:

Copying and moving files to user shares via cli keeps parity intact?

Yes, of course

Edited January 18Jan 18 by Hoopster

Assuming all drives are functioning correctly, the 40 hour parity rebuild is due to the small capacity drives. A spinning hard drive's access time is at it's fasted when accessing its first sector, and its slowest accessing the last. The difference is approximately 50% slower. The parity rebuild can only run as fast as the slowest drive in the array, at the sector which is being read at at hat time.

So let's use an example that is a bit simpler than your server - 3 drives, a 8TB, 4TB and 2TB. I'll assume that all have identical access times (which in real life, won't be true, as the smaller drives typically are older technology and are slower at both first and last sector). Let's say 200MB/s to start, 100MB/s at last sector.

• At starting the parity check, speeds are at 200MB/s.

• At 1TB done, the 2TB drive is at 150MB/s. Just before 2TB, at 100MB/s. (An 8TB drive would be capable of 175 and 150MB/s respectively)

• Once past 2TB, the speed will jump back up to 150MB/s, as the 2TB drive is now idle. It will work its way back down to 100MB/s at the 4TB mark.

• Now just 8TB drives are being red, so speeds again jump to 150MB/s, ramping down to 100MB/s until the parity is completed.

As your server now is just a Proof Of Concept test, these old drives are fine to do your testing with. Ideally, a smaller number of larger capacity drives are a better choice, as you benefit from faster access speeds, lower power consumption and less things to go wrong or fail.

My main server has 18TB and 16TB drives. My parity checks, run quarterly, average 185MB/s and run for just under 27 hours. The image is a screenshot from the DiskSpeed docker, available on Community Applications. It is a great benchmark/diagnostic tool to see the performance of your drives.

Oh... And welcome to Unraid.

Many many thanks @ConnerVT and @Hoopster for speeding up my education on Unraid!

54 minutes ago, dv-geek said:

speeding up my education on Unraid!

By the way, there is a very useful plugin called parity check tuning that allows a parity check (like @ConnerVT , I do mine quarterly) to be broken up into several increments over a period of days. A parity check on my main server takes approximately 24 hours with a 12TB parity drive and 8TB array drives. I have it set to run in 9-hour increments from midnight to 9am on the second Thursday of Jan, Apr, Jul and Oct. It finishes before the third increment is scheduled to end. Of course, with larger parity drive(s), it could take more increments but you decide how long each increment will last and a parity check will run in those increments until finished.

Another useful tip is to enable signatures in your forum profile if you do not see them. A lot of forum users put the hardware details of their servers in their signature. This gives you a good idea of the variety of hardware configurations that can be used with Unraid. There is no "recommended" hardware as Unraid runs on just about anything (other than ARM).

Edited January 18Jan 18 by Hoopster