Hey unRAIDers, I just have a couple q's about the specific implementation of parity in unRAID. I searched through the forums and wiki and found some good info, but it wasn't quite specific enough for me .

 

 

1) Situation 1: copying a file over the network to a disk in the unRAID array. WHEN exactly, is parity calculated? For example, is this done byte by byte, file by file? Do we calculate parity, write to disk, and then calculate parity again to check, or what? If I were to yank the network cable between my computers in the middle of transferring a large file, what would happen to the temporary file and what state would that leave the parity in? My guess is that every write operation to a disk in the unRAID array does not get committed to disk until parity is calculated and updated. Can someone confirm or deny this?

 

2) When using telnet to access the unraid server directly, cp'ing and mv'ing files from disk to disk, I'm assuming parity is preserved. Are there any command line operations that would circumvent parity? The reason I'm asking is, I do a lot of work on the command line, I'm very comfortable writing scripts. I just don't want to do anything that would accidentally write something to a disk in the array without updating parity. I'm guessing that this is not possible, and that this works similar to other linux software raids.

 

3) When performing a full disk parity check, it is still possible to write to the array. I'm not clear how this works. Specifically, wouldn't any writes to the array invalidate the results of the parity check? Here are two situations I thought of

 

a) parity check is running. Say it is 25% done. Say a bit exists at the 75% mark that is out of sync with parity, so we haven't yet reached it in our check, and we trust parity over our data disk. If we write to this bit, we force a recalc of parity for that bit, so we should be covered, correct?

 

b) parity check is running. Say it is 75% done. Say we write to a bit at the 25% mark on the disk and it turns out that this write operation introduces a bad bit. What would happen in this case? It seems to me in this case, the parity check for the entire disk would eventually complete, giving the illusion of complete coverage.

 

Anyway, loving unraid. It is super awesome.

 

Thanks! K

1. Parity is calculated on the fly while writes are being performed. If you were to disconnect, then the file and parity would be in the state you left it. If you use a utility like Teracopy, then you can resume the transfer. If not, then you'll have to delete the file and begin again.

 

2. Every write operation, whether it be a copy or move, will update parity as it's written.

 

3. I don't think you can. I tried it the other day, and in Windows it gives me an error that the file is in use by another program. I couldn't even change the name of a file.

 

 

1) My guess is that every write operation to a disk in the unRAID array does not get committed to disk until parity is calculated and updated. Can someone confirm or deny this?

 

Yes, parity is updated each time a block of data is written to a disk. They always stay in sync with each other.

 

2) When using telnet to access the unraid server directly, cp'ing and mv'ing files from disk to disk, I'm assuming parity is preserved. Are there any command line operations that would circumvent parity? The reason I'm asking is, I do a lot of work on the command line, I'm very comfortable writing scripts. I just don't want to do anything that would accidentally write something to a disk in the array without updating parity. I'm guessing that this is not possible, and that this works similar to other linux software raids.

 

I believe you could do things to write to the disks without updating parity. However, it should be easy to avoid doing so. It seems to me that each parity protected disk is created as a device and you need to do any disk operations on those devices. If you do disk operations on the SATA ports directly then you could break the parity.

 

3) When performing a full disk parity check, it is still possible to write to the array. I'm not clear how this works. Specifically, wouldn't any writes to the array invalidate the results of the parity check? Here are two situations I thought of

 

a) parity check is running. Say it is 25% done. Say a bit exists at the 75% mark that is out of sync with parity, so we haven't yet reached it in our check, and we trust parity over our data disk. If we write to this bit, we force a recalc of parity for that bit, so we should be covered, correct?

 

b) parity check is running. Say it is 75% done. Say we write to a bit at the 25% mark on the disk and it turns out that this write operation introduces a bad bit. What would happen in this case? It seems to me in this case, the parity check for the entire disk would eventually complete, giving the illusion of complete coverage.

 

You can read and write to the array while a parity check is being performed. It doesn't invalidate the parity check.

 

If a parity bit is wrong then it will still be wrong after you write to a disk. unRAID reads the old parity and assumes it's correct when it calculates the new parity. This is to avoid reading every disk to perform a write but does introduce this issue where a wrong parity  bit will continue to be wrong until a parity check fixes it.

 

a) not really sure what you are asking. Do you mean writing to a data disk that has a bad bit?

b)Yes, the parity will be wrong and you will not know. However, your system should never just introduce bad bits.

 

Peter

Hey unRAIDers, I just have a couple q's about the specific implementation of parity in unRAID. I searched through the forums and wiki and found some good info, but it wasn't quite specific enough for me .

 

 

1) Situation 1: copying a file over the network to a disk in the unRAID array. WHEN exactly, is parity calculated? For example, is this done byte by byte, file by file? Do we calculate parity, write to disk, and then calculate parity again to check, or what? If I were to yank the network cable between my computers in the middle of transferring a large file, what would happen to the temporary file and what state would that leave the parity in? My guess is that every write operation to a disk in the unRAID array does not get committed to disk until parity is calculated and updated. Can someone confirm or deny this?

Parity is calculated simultaneously (and written to the parity disk) with the writes to any of the protected disks in the array.  You guessed wrong.

2) When using telnet to access the unraid server directly, cp'ing and mv'ing files from disk to disk, I'm assuming parity is preserved.

Correct.

Are there any command line operations that would circumvent parity? The reason I'm asking is, I do a lot of work on the command line, I'm very comfortable writing scripts. I just don't want to do anything that would accidentally write something to a disk in the array without updating parity. I'm guessing that this is not possible, and that this works similar to other linux software raids.

It works exactly like other software raids.  You can use command line commands to read and write to the disks mounted as /mnt/disk1 disk2 etc or to the shared file system mounted at /mnt/user.

 

Like EVERY other raid system it is possible to construct a command to write to the raw disk devices and bypass the unRAID "md" driver that maintains parity.  (The raw devices are the /dev/sdX devices) Writinf to a raw device will corrupt parity and your data.  Those will not be anything you will "accidentally" do.

3) When performing a full disk parity check, it is still possible to write to the array.

Absolutely.  Nothing precludes either reading or writing while parity is being initially calculated or subsequently checked.

I'm not clear how this works. Specifically, wouldn't any writes to the array invalidate the results of the parity check?

No they would not.  Both parity calculations and checks are done at the same low level in the "md" driver.    If you are limited by an older motherboard and a PCI bus you might notice that writes to the arrya are a bit slower while calculating parity at the same time if you have a large number of disks.  I have one of the original unRAID servers lime-tech sold and it is PCI based, with mostly IDE drives, and it still works perfectly fine for reading and writing even with 12 disks installed.