Will these components work?

[Rajahal]

You guys are fantastic, thanks so much for the detailed explanations and responses.

 

bjp999: I will certainly use your method for swapping out the parity drive, it is a simple and elegant solution -  just what I need for that little bit of piece of mind knowing that my data is never at risk.  Hopefully I don't have a drive failure during the parity rebuilding and I won't need to use your second set of instructions, but it's good to know how to handle the situation if it arises.  I think your posts would be a great addition to the wiki.

 

prostuff1: I didn't know there's a 'move now' function for the cache drive, that certainly makes things easier.  Even nicer would be a 'move now then power down' function, so that one could just set the server to move everything from the cache drive to the data array when powering down the server for the night.

 

Good to know that a slower cache drive will still be of benefit.  I think I will use one of my old laptop drives, especially since I have no need for external back up drives anymore.

 

Is the preclear script just to make sure that a drive is in good working order before trusting it with any data/parity information?  If so, I'm not terribly concerned with this, since if a drive were to fail I could just replace it and re-construct the lost data/parity info.  That's part of the point of unRAID, as I understand it.  If there's more to it than this, please advise.

 

Good to know my write speeds are normal.  If I were to have transferred all my data from my desktop to my unRAID server without first assigning a parity drive, then the data would all be unprotected until the transfer was complete and a parity drive were assigned, correct?  So fast and unsecure vs. slow and secure...I opt for the second option, I'm patient, and I want my data safe.

 

I had another thought last night, more of a conceptual thing, I doubt I would actually need to implement it.  After I make my 1 TB drive my parity drive (using bjp999's instructions) and my server is more or less complete, I will be essentially wasting 360 GB of storage capacity (since my parity will be 1 TB and my largest data drive is 640 GB).  I've known this from the start, and I figured its worth it to allow me to add 750 GB or 1 TB drives to the data array in the future.  However, I also remember reading that excess space on the parity drive (360 GB in my case) can be used for unprotected data storage.  I don't want to do that, since I want all my data to be protected.  However, could that excess space on the parity drive function as a cache drive?  I'm thinking that my 1 TB drive would have 2 partitions - 640 GB of parity, then 360 GB of cache.  Is this possible?  Even if it is possible, is this something I would want to do?  I imagine it would place a lot of strain on the drive (especially during the mover's move command, since the data would have to be read off the cache drive and simultaneously parity info would have to be written to the same drive).

 

Another conceptual question: Rebuilding the parity and parity checks are different processes, right?  I know that the data array is unprotected while the parity is initially being built, but is the same true during a parity check?

 

Only a bit over a TB of data left to transfer and two more drives to physically move (1 TB and 640 GB) and my unRAID server will finally be complete.  I expect it will be done by the end of the weekend.  9 months in the planning, about a week in the execution.  By the way, regarding that 500 GB drive that fails in the server but works in the desktop, I think I'm just going to leave it in the desktop.  It clearly wants to be there, and I'm not one to deny it it's nature.  I'll just use it as a torrent seeding drive so that if it does fail someday, I won't lose anything important.

[SSD]

You guys are fantastic, thanks so much for the detailed explanations and responses.

 

bjp999: I will certainly use your method for swapping out the parity drive, it is a simple and elegant solution -  just what I need for that little bit of piece of mind knowing that my data is never at risk.  Hopefully I don't have a drive failure during the parity rebuilding and I won't need to use your second set of instructions, but it's good to know how to handle the situation if it arises.

 

prostuff1: I didn't know there's a 'move now' function for the cache drive, that certainly makes things easier.  Even nicer would be a 'move now then power down' function, so that one could just set the server to move everything from the cache drive to the data array when powering down the server for the night.

 

The mover functionality is just a script.  If you install the "powerdown" package, you may be able to add the powerdown command to the end and get it to power off as you would like.

 

Good to know that a slower cache drive will still be of benefit.  I think I will use one of my old laptop drives, especially since I have no need for external back up drives anymore.

 

Is the preclear script just to make sure that a drive is in good working order before trusting it with any data/parity information?  If so, I'm not terribly concerned with this, since if a drive were to fail I could just replace it and re-construct the lost data/parity info.  That's part of the point of unRAID, as I understand it.  If there's more to it than this, please advise.

 

When you add a new drive to the array, unRAID will clear that disk (write binary zeros to the entiire disk) before adding it to the array.  (If you think about it for a second, you'll realize that you can add a cleared disk to the array without affecting parity.)  This is time consuming especially with a big disk.  The preclear script will fill a disk with zeros and uses a super secret codeword to tell unRAID it is already cleared.  It takes just as long (maybe longer with the extra tests it does) but its not in the array so not affecting the array.  When unRAID adds a disk that is precleared, it adds it instantly with no lengthy clearing process.  The testing that the preclear script does is valuable in shaking it down, even if you don't need the disk cleared.  There are other ways to test out a disk, this is just a convenient one.

 

Drives are more prone to fail in their first few hours of life than they are after a good burn in test.  Although you are right that unRAID helps protect you, some of us are anal enough to want to spare ourselves and our array of the ordeal of dealing with a failed drive.  The thought of replacing a failed disk with another failed disk - not too pleasant.  If nothing else, those parity builds and parity checks are time consuming!  But this is your decision.

 

Good to know my write speeds are normal.  If I were to have transferred all my data from my desktop to my unRAID server without first assigning a parity drive, then the data would all be unprotected until the transfer was complete and a parity drive were assigned, correct?  So fast and unsecure vs. slow and secure...I opt for the second option, I'm patient, and I want my data safe.

 

I had another thought last night.  After I make my 1 TB drive my parity drive (using bjp999's instructions) and my server is more or less complete, I will be essentially wasting 360 GB of storage capacity (since my parity will be 1 TB and my largest data drive is 640 GB).  I've known this from the start, and I figured its worth it to allow me to add 750 GB or 1 TB drives to the data array in the future.  However, I also remember reading that excess space on the parity drive (360 GB in my case) can be used for unprotected data storage.  I don't want to do that, since I want all my data to be protected.  However, could that excess space on the parity drive function as a cache drive?  I'm thinking that my 1 TB drive would have 2 partitions - 640 GB of parity, then 360 GB of cache.  Is this possible?  Even if it is possible, is this something I would want to do?  Would it place too much strain on the parity drive (especially during the move command)?

 

Whoa partner.  Not sure what you read, but there is no way to use the unused portion of parity for anything.  Sorry.  1T seems to be the sweet spot right now for new drives.  For that reason, the 1T parity may be a good thing. But if you never plan to go over 640G, you could replace it with a 640G drive and not lose a single byte of usable space on your array.

 

Only a bit over a TB of data left to transfer and two more drives to physically move (1 TB and 640 GB) and my unRAID server will finally be complete.  I expect it will be done by the end of the weekend.  9 months in the planning, about a week in the execution.  By the way, regarding that 500 GB drive that fails in the server but works in the desktop, I think I'm just going to leave it in the desktop.  It clearly wants to be there.  I'll just use it as a torrent seeding drive so that if it does fail someday, I won't lose anything important.

 

Usually a drive that works in one machine but not in another is due to a cabling problem of some kind.  Sometimes its just gremlins.

 

Good luck.

[Rajahal]

You guys are fantastic, thanks so much for the detailed explanations and responses.

 

bjp999: I will certainly use your method for swapping out the parity drive, it is a simple and elegant solution -  just what I need for that little bit of piece of mind knowing that my data is never at risk.  Hopefully I don't have a drive failure during the parity rebuilding and I won't need to use your second set of instructions, but it's good to know how to handle the situation if it arises.

 

prostuff1: I didn't know there's a 'move now' function for the cache drive, that certainly makes things easier.  Even nicer would be a 'move now then power down' function, so that one could just set the server to move everything from the cache drive to the data array when powering down the server for the night.

 

The mover functionality is just a script.  If you install the "powerdown" package, you may be able to add the powerdown command to the end and get it to power off as you would like.

 

Thanks, that would be nice.  I'm not sure I'm comfortable enough with command line and editing scripts right now to want to tackle that, but maybe sometime in the future.  I certainly wouldn't mind learning more about it.

 

The purpose of the preclear script is crystal clear now, thank you.

 

Whoa partner.  Not sure what you read, but there is no way to use the unused portion of parity for anything.  Sorry.

 

Oh really?  No problem, like I said I wouldn't want to do it anyway.  I wonder where I read that...or if I'm just misremembering something else.

 

1T seems to be the sweet spot right now for new drives.  For that reason, the 1T parity may be a good thing. But if you never plan to go over 640G, you could replace it with a 640G drive and not lose a single byte of usable space on your array.

 

Yeah, I understand that.  I'll keep the 1 TB as the parity drive, I'm sure I'll want to add another large disk to the data array within a few months at least.  640 GB is kind of a weird drive size anyway, I just bought it because at one point in the past it was the best GB/$ value.  I doubt I'll ever buy another drive under 1 TB, unless its just a really good deal.

 

Usually a drive that works in one machine but not in another is due to a cabling problem of some kind.  Sometimes its just gremlins.

 

I think it has to be gremlins in this case.  As I mentioned several posts up (on page 3), I've tried every combination of cabling and motherboard/pci card slots that I can think of.  We determined that the problem has to be with the power source, since it happens with no data cables connected.  But I'm done messing with it.  If I really need that extra 500 GB of space in my server, then I'll RMA the drive.  On a somewhat related note, a different 500 GB drive that failed months ago (clicking) and that has been sitting in a drawer since then is now functioning seemingly perfectly in the unRAID server.  I put it in on a whim figuring that if it does fail, well, whatever, data will still be safe.  So far I haven't had any problems with it.  If it does eventually fail, then I guess I'll just RMA it as well.  I guess the biggest downside of this 'cross that bridge when I come to it' kind of mentality is that if I do in fact have to RMA a drive, my entire unRAID server will have to remain offline (hence my data will be inaccessible) until the new drive arrives in the mail. 

[SSD]

I guess the biggest downside of this 'cross that bridge when I come to it' kind of mentality is that if I do in fact have to RMA a drive, my entire unRAID server will have to remain offline (hence my data will be inaccessible) until the new drive arrives in the mail. 

 

Not true.  The array will run and simulate a single failed or missing drive.  I don't really recommend doing this for long periods of time, but this is standard functionality.

[Rajahal]

Well this just keeps getting better and better, I love unRAID, what a great investment.

 

What do you mean by 'simulate'?  Does it mean that the data from the failed drive will be inaccessible, but that the rest of the data on the working drives will still be present?  Or is it that the parity drive will kind of step in and temporarily hold/serve the data until a new drive is installed?  So the data will all be there, but it will be unprotected?

[SSD]

Well this just keeps getting better and better, I love unRAID, what a great investment.

 

What do you mean by 'simulate'?  Does it mean that the data from the failed drive will be inaccessible, but that the rest of the data on the working drives will still be present?  Or is it that the parity drive will kind of step in and temporarily hold/serve the data until a new drive is installed?  So the data will all be there, but it will be unprotected?

 

The latter.

 

It will appear that the disk is in the array.  When data is accessed from the missing disk, parity and all of the data disks will work together to provide the requested data.  Obviously access to the data will be slower than normal, but it will return the data accurately.  But you are not protected. If another drive were to fail, you would lose both drives worth of data.

[Rajahal]

Unless that second drive to fail were the parity drive, then only the original data disk's data would be lost, since the parity of the remaining disks could be recalculated.  I'm starting to understand how parity works after a lot of reading.  At first it seemed like voodoo magic, but it's really pretty straight forward.

 

Regarding unMenu, I'm having a bit of trouble getting it working.  I downloaded the .zip files and followed the instructions in this thread: link, but to no avail.  Since this is a brand new unRAID build, I figured I could skip the part about upgrading from old versions of unMenu.  So basically, I just did this:

There are three zip files attached.  You will want them all.  Unpack them all in the same directory where you want to put all the unmenu files. I suggest a folder name something like /boot/unmenu.

 

I'm not sure about this step:

To start up this newest version of unmenu, either "cd" to the directory with the unmenu files and then type:

uu

 

or, type the full path to the directory with the files (as an example, if your folder is /boot/unmenu)

/boot/unmenu/uu

[RobJ]

I'm not sure what you are asking, but one point to remember:  in Linux, the /boot folder on your server is the root of the flash drive, so don't create a /boot on the flash (or it will be /boot/boot/).

[prostuff1]

There is a lot of stuff on the wiki page and I think even a note about installing unmenu.  You can go to the wiki an take a look at the Add-on's and also the Third Party Boot Flash Architecture as this structure is what we (the community) are trying to move towards.  It allows the knowledgeable members to assist in diagnosing problems.

[Rajahal]

Well I did create a folder named 'boot' in the flash drive's root directory, so that's at least one problem.  I fixed that, so the files are now in a folder named 'unmenu' directly in the flash drive's root directory.  It still isn't working, as in going to http://tower:8080 doesn't lead anywhere.  That should be the address to get it working, correct?  Or is there some other port I should try?

 

I'll check the wiki some more and see if I can figure anything else out.

 

What I meant by being 'not sure about this step' is that I don't know how to do that step.  Do I just go to start>run and type in /boot/unmenu/uu?  Or tower/flash/boot/unmenu/uu or something like that?

[prostuff1]

Well I did create a folder named 'boot' in the flash drive's root directory, so that's at least one problem.  I fixed that, so the files are now in a folder named 'unmenu' directly in the flash drive's root directory.  It still isn't working, as in going to http://tower:8080 doesn't lead anywhere.  That should be the address to get it working, correct?  Or is there some other port I should try?

 

I'll check the wiki some more and see if I can figure anything else out.

 

What I meant by being 'not sure about this step' is that I don't know how to do that step.  Do I just go to start>run and type in /boot/unmenu/uu?  Or tower/flash/boot/unmenu/uu or something like that?

 

You need to start up a telnet session (from a command prompt) and then type:

/boot/unmenu/uu

 

That will start up unmenu.

[Rajahal]

OK, thanks, I can try that tonight.  I'm very inexperienced with anything command line related.  Do I need a specific telnet client, like PuTTY, or is the functionality built into Windows?  How do I start up the telnet session and log into the unRAID server?

 

I'm happy to report that my unRAID server is complete, for all intents and purposes.  I still have a few bells and whistles, like unMenu, to work out, but all my hard drives are installed and all my data is transferred and safe.  The stats are in my sig.

 

One final (I hope) hardware question:  As I was transferring data and moving hard drives one by one, I would just hook up each new hard drive into the unRAID server into whatever SATA port was available, so I filled up the motherboard first, then went on to the Promise card.  Now that it is complete, I'm thinking of switching around some of the connections to hopefully optimize access speed.  I figure that I should put all the largest drives, the 1 TB parity drive, the 640 GB data drive, and 2 of the 3 500 GB data drives on the motherboard slots, and then put the remainder of the drives (500 GB, 320 GB, 250 GB) on the Promise card.  I remember reading that all drives on the Promise card are bottlenecked through the slower PCI bus, so figured I might gain some minimal performance increase by connecting all my largest, most often accessed drives directly to the motherboard.  Is this reasonable, and is it worth the effort?  I understand that I would probably only notice a difference if multiple hard drives were being accessed at once.

 

Also, if I do go through with this SATA slot/cable reorganization, once I'm done and boot up the server again, I can just press 'Restore' and unRAID will figure out where all the hard drives have moved to, correct?  I remember doing something like this when I was building the server, but I wanted to make sure since I've now read about how dangerous that 'restore' button can be. 

 

I have some further questions about how to best use my unRAID server for media streaming and for torrents, but I figure I'll start a new thread about them since I don't consider them to be hardware-related, and because this thread is already pretty muddled with my random thoughts and questions.