To preface, I will say that I know there are a lot of factors that affect the length of parity checks, such as drive and connection speed, bottlenecking through the PCI bus, etc.  However, neglecting all of that (say we are comparing one unRAID server with itself):

 

Does storing more data on the server mean that parity checks will take longer?

 

I think I know the answer already, but I want to confirm it.  I believe the answer is 'no', since it takes the same amount of time to XOR a bunch of zero's as it does to XOR a bunch of one's.  However, perhaps the parity check is smart enough to recognize large blocks of zero's (unused sectors of disks) and skip them...seems reasonable, and I just don't know the answer.

 

I currently have about 3.7 TB of usable capacity on my unRAID server, and it is about 80% full at the moment.  I just ran a parity check last night, and it took less than eight hours (I estimate between six hours and eight hours).  If I were to clear off all my data and sell my server as-is, would I be able to make that claim that it 'completes a parity check in 6-8 hours', or does that rate depend on the amount of data stored on the server?

 

Thanks for taking the time to consider what is probably a stupid question....

It's physical speed on the disk and bus that's the issue.

It's already been proven that unRAID is not cpu bound, but I/O bound.

Thus there is spare CPU to handle the bit flipping 0's or 1's. I do not think it matters.

 

Also a parity check is different than a parity generate (write).

 

With a check,  everything is a read.

With a generate, there is a write on the parity drive instead of the read.

To preface, I will say that I know there are a lot of factors that affect the length of parity checks, such as drive and connection speed, bottlenecking through the PCI bus, etc.  However, neglecting all of that (say we are comparing one unRAID server with itself):

 

Does storing more data on the server mean that parity checks will take longer?

 

I think I know the answer already, but I want to confirm it.  I believe the answer is 'no', since it takes the same amount of time to XOR a bunch of zero's as it does to XOR a bunch of one's.  However, perhaps the parity check is smart enough to recognize large blocks of zero's (unused sectors of disks) and skip them...seems reasonable, and I just don't know the answer.

 

I currently have about 3.7 TB of usable capacity on my unRAID server, and it is about 80% full at the moment.  I just ran a parity check last night, and it took less than eight hours (I estimate between six hours and eight hours).  If I were to clear off all my data and sell my server as-is, would I be able to make that claim that it 'completes a parity check in 6-8 hours', or does that rate depend on the amount of data stored on the server?

 

Thanks for taking the time to consider what is probably a stupid question....

Parity is not calculated based on the files on the disks, but on every bit in every possible sector on every disk.  It will take the same time if the disks are empty or full of files. 

 

The CPU itself is barely used when doing a parity calculation... Instead, the bottleneck is the bandwidth on the bus to the disks.  My PCI bus is limited to 133 MB/s... divide that up by my 11 disks and I'm lucky to get an overall 11 to 12 MB/s parity check speed when they are all being accessed.  Later in the process, when only a few of my larger disks are involved, I can get up to 40MB/s calc speed.  (but in both cases, taken together, the involved disks are pushing pretty close to the 133 MB/s bit rate through to the CPU)

 

You should do much better with a PCIe bus.  Speeds have been reported as high as 70-90MB/s.

 

Joe L.