WeeboTech

I can compile it if need be.

Sometimes there is a bios option to re-initialize the video bios after returning from sleep. You can check your bios and toggle the option to see if this helps.

It makes you wonder.. According to these articles anything 12TB and over, i.e. a 16 Drive unRAID environment with 1TB drives, is doomed too.

There's a command called IONICE which might help in controlling priority of the DD's. http://linux.die.net/man/1/ionice In my rsyncmv scripts I do /usr/bin/nice -19 /usr/bin/ionice -c3 rsync -avP --bwlimit=${BWLIMIT:=6400} --remove-sent-files "$@" ${DIR} This has the effect of lowering the priorty of the rsync to a level where it does not affect my rtorrent or any streaming of the system.

Duah.. You're right, I did not think it through all the way.

Does it pay to have a FORMAT option in the pre-clear script?

Perhaps submit it to the background via batch. The process can write it's pid in /var/run and log file in /var/log. The browser interface section can refresh on the log and if need be use the pid file to send a kill to the process and it's children.

NIce Lunatic, I did not even think about doing the cardboard blocking thing. Did you do the taping as suggested? How are your hard drive temps? Were the removables worth it in the end? (I think so!) It's a spiffy case with the removables. Thumbs up for the great pics!

It's not the discs so much the issue as it is the case to house them. The other issue I see is the Power Supply. I got the Corsair 650 thinking this could handle 17 discs. How much of a PS would you need for a 24 or 32 disc system? In theory it sounds great being able to go to 32 discs, but in practice I see this being very difficult to achieve. A 2nd unRAID server seems to make much more sense to me. Thoughts? There are so many external drive cases out there with many options. Ramping a machine up to 32 drives is not that difficult if you get past the single case thought. There are port multipliers or multi-lane connectors whereby SATA paths are combined into a single cable. For example: http://www.newegg.com/Product/Product.aspx?Item=N82E16816133007 Other options http://www.caloptic.com/cgi-bin/quikstore.cgi?product=sBOX-X&detail=yes http://www.datoptic.com/cgi-bin/web.cgi?category=RACKMOUNT_CHASSIS http://www.computervideogear.com/sata/sata-enclosure-sr-hd-pro.htm I've seen as large as 15 Drive tower external sata units. FWIW, It's possible, I know it's expensive and may not be worth it for some, I'm not saying it's always practical.

This a factor of the controllers and quality of drives present. Reads are done from multiple drives/sectors to rebuild the failed sector. Sort of like a stiripe, so performance shouldn't be that much of an issue. There are guys on the AVS forum who have huge arrays using raid5/raid6. I'm sure unRAID will be able to serve an ISO or two, it's the writes that will suffer. The issue of practicality boils down to parity generation time and wiring all the drives. I think removing limits is positive step. It's up to the end user to determine the risk. The addition of a second Q-parity is a huge plus. The possibility of splitting the arrays with multiple individual parity disks would be a welcomed addition. I would certainly use it for external units such as a 5 drive external case using a parity drive within that casing.

It's not the discs so much the issue as it is the case to house them. Supermicro has a case to house 24 Disks. Then there are external cases via Multilane or port multiplier connections. 24 Discs is easily feasible, more then that would take some effort. I was planning on using the norco case, but I've decided to use my CM Stacker with 4 5in3 modules and remove the top USB/Power unit (and rig something into the back). If I put in eSATA controllers each port is capable of 5 disks.. you can see how far the possibility could grow. Is it practical, probably not, I think when the disks start expanding past a case, you run the risk of cables issues, thus mutlitple drive failure issues.

A very positive feature addition! Thanks for the hard work!

What's needed is a graceful way to remove a drive from the array. I would prefer to remove a drive by zeroing out it's parity slot without writing to the associated drive. There's gotta be a way to read the parity drive, pretend slot(drive) number X is 0 and re-write the parity sector. In theory this could work, in practice we would have to find out (and be careful about it). I think it's one of the reasons to do a parity check before any drive replacement operations take place.

This is a very interesting concept. I remember reading articles whereby an LBA error reported via SMART could be translated into numbers usable by a filesystem debugger for locating the bad file. In fact the article also mentioned using the badblocks program to force writes over the bad blocks, thus causing reallocation. My friend and I were discussing this yesterday. He mentioned whenever he sees a drive starting to go bad, he takes it offline and runs shred on it, forcing writes to the whole drive, thus reallocating bad sectors. Not the best approach, but It's saved a few drives of mine at times.

Interesting topic, thanks for bringing it up. That was a very interesting article. It does contain some merit and food for thought. I've yet to encounter the situation. I found it interesting on how a date is chosen as a point in time. In reality, it is size vs MTBF and BER on devices. In some sense it makes me think unRAID should have multiple smaller arrays. It should do this by dividing the larger array into smaller chunks protected by individual parity drives. It was interesting in how the author states RAID6 is not much more protection then RAID5. A comment by a reader makes allot of sense This is how I've handled things in the past. What I would like to add is, the chance of multiple drive issues is increased greatly when you purchase multiple drives at the same time. I've seen this issue over and over again, not just with RAID5. In our web server farm, if too many drives were bought from one vendor at the same time. We would see them fail with a pattern. My suggestion, which worked out well, was to choose multiple vendors and to spread the purchase over time. In some sense, it's interesting to see the warnings of building large arrays with unRAID. Points to consider in the unRAID community, Multiple drive purchases at one time... (as explained above) Using old drives to gain every last ounce of use out of it. This is one fo the great things about unRAID. I can use a hodge podge of old drives. Yet, chances of multiple drive failures are greatly increased when trying to get every last ounce of life out of it. Periodic parity checks - I'm a proponent of this procedure also. In fact I think it should be added as a standard unRAID feature allowing scheduling just like the mover script. There are raid cards that do this automatically. A new term to me, called "BIT ROT" makes sense here. Sectors go bad, periodic scans and checks will help you become aware of potential issues allowing you to be proactive, not reactive. SMART monitoring - I'm a proponent of this also, again, I think it should be added as a standard unRAID feature. Cold or Warm Spare - This is a budget concern, but the faster you replace a disk, the less likely you will see a multiple drive failure. This does not save you from bit rot. LOL.. Periodic parity checks are the only way around that one! Some questions to ponder... >> With the relatively slow speed of unRAID parity creation or disk recovery. Is it really that slow.. You have to consider how many spindles there are involved, then bus speed. Parity check is reasonable if you have drives arranged on efficient buses. Parity create is slower because there is a write involved. >> That combined with the fact that during these periods the disks are thrashing their asses off increasing the chance of failure. and in this case, If the array is idle from use and just creating parity, are they really thrashing, or just reading sequentially. To me thrash is significant inefficient movement, Yet during a parity check/create, access is efficient as long as nothing else is causing movement in the machine. >> e.g. "As we now know, a single disk failure means a second disk failure is much more likely." Do we REALLY know that? I agree. I've always questioned this way of thinking. It's not a given as so many people will repeat... Yet the chance of impending failure is greatly increased for drives purchased at the same time and within the same batch of production. I've seen this countless times over the years. One bad batch can wreak havoc. >> this is one reason I do not support the idea of replacing smaller drives in unRAID by replacing them with larger ones and letting them rebuild. I prefer to put the new, larger drive in the system, copy the data, then stop the array and remove the old drive. What is the difference here? Do you feel the rebuild is susceptible to bit rot and you will not be able to reconstruct the new drive reliably? If the drive has been removed and replaced, you still have the old data on the old drive. If before replacement, a parity check is executed, then chances of reading every drive is higher. if a drive is swapped with a larger drive, and the build fails, you have the chance of putting the old drive back and potentially accessing the second failed drive virtually. Just some thoughts. So what does adding a new drive to the system buy you vs swap replacement? A new drive requires parity calculation to be execuited at siome point. removal of the old drive requries parity calculation. (of course these can be combined into one step)

AS a cache drive it will do fine. And they are designed for many spinups over a lifetime.

You might be better off selling that and putting the money towards a larger drive. It may slow down your monthly parity checks. (You do those right?)

I did not know those adapters even existed. Thanks for sharing!

if you use the calculator on Corsair's website they will probably say this power supply is enough. The website says specifically - >> Planned Expansion (on the off-chance that I might need more than 9 drives): Not sure you need all those parts. You could go with 5in3s in a good case and not have any outside wiring. So if you were to double check your power supply and needs, then consider the 5in3's and a new case, you could fit it all in one place.

My Larger Rig. This has been a long time coming.. Although I have the mini-itx version, I finally finished building a larger cost effective version Parts List Abit AB9 PRO 2.6ghz Core 2 Duo - Used on eBay - a Celeron 440 2Ghz is quite sufficient. 8gb OCZ PC6400 RAM - 4G is sufficient for 99% of the people. Centurion 590 - Great cheap case with allot of fan placement. Seasonic 500W Power Supply (it was laying around idle) Extra 120MM fans ~ $15 (box of 4 coolermasters). Kingston 2GB Secure Digital (SD) Flash Card w/MobileLite Reader Model FCR-ML+SD/2GB - http://www.newegg.com/Product/Product.aspx?Item=N82E16820134548 iStarUSA T-5-SA 1x5.25" Bay Trayless Anti-vibration SATA Mobile Rack http://www.newegg.com/Product/Product.aspx?Item=N82E16816215046 Additional items. Gigabute I-RAM PCI Ram disk http://www.gigabyte.com.tw/Products/Storage/Products_Overview.aspx?ProductID=2180 This is for a future application to allow me to use the server as an NFS server for other machines. IDE to CF adapter + 4GB CF card (to backup the battery backed ramdisk.. just in case). Cheap PCIe 4x video card from eBay.. which will end up getting dropped for a 1X. Masscool PCIe 1x 2 SATA port controller (for the ramdisks). http://www.newegg.com/Product/Product.aspx?Item=N82E16815280008 I'll probably upgrade to a 4X PCIe 4x 4 port adapter to expand to 12 or maybe 15 drives in the future.. For right now it seems just as I fill the slots, new drives come out that are large enough to replace current drives. Notes: I really love the trayless sata design. They can use MOLEX or SATA power connectors. I choose this model in particular because the vents are larger then other models. Some of the other models have the vents blocked a little by the drive. This one has slightly more leeway for air flow. I also choose this model because they are fanless. I use the large 120MM fans to suck the air from the front. This requires taping up open vents on the case to keep the drives cool. I choose this design so I did not have a bunch of tiny fans spinning and used the larger fans instead. It's working well for me. These trayless are not perfect though. When you open the key, it does not power down the drive. So you probably want to replace drives while the machine is off. Also, the drives tend to get sort of stuck on the inside, so you wiggle them a little and they pop out. Still for me I like the idea of swapping bad (or upgrading) drives like they are floppies. No tray, no screws, Just unlock, open, wiggle, replace, close, lock, boot, click, click, and you are up and running. Also, The SATA cable does not lock into the trayless. in theory, the cable could eventually loosen due to drive vibration. Only happened once in the last 6 months. I plan to upgrade eventually to the 5in3 trayless modules, but for right now I do not need more then 9 drives. I have more pictures here, which shows different views and where to tape up the case. http://www.cotrone.com/rob/gallery/Atlas

What does the front look like? How small is it?

You could change the home directory of the user in question to a physical drive on your machine. Then the files placed there will continue to be there with customizations after each reboot. However, you will probably have to reinstall the package upon each reboot or possibly alter the bzroot image for a more permanent change.

do a cat /etc/resolv.conf Chances are your name is not resolving You can substituite the IP address for the hostname as in the example below root@Atlas:/mnt/disk2/bittorrent# ifconfig eth0 eth0 Link encap:Ethernet HWaddr 00:50:8D:9D:7B:AA inet addr:192.168.1.179 Bcast:192.168.1.255 Mask:255.255.255.0 UP BROADCAST NOTRAILERS RUNNING MULTICAST MTU:1500 Metric:1 RX packets:263331527 errors:0 dropped:0 overruns:0 frame:0 TX packets:255971791 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:4217610711 (3.9 GiB) TX bytes:3314289142 (3.0 GiB) Interrupt:16 root@Atlas:/mnt/disk2/bittorrent# /usr/bin/rsync rsync://192.168.1.179/ backups Backups vmware VMWare Backups music Music pub Public Files boot /boot files mnt /mnt files Videos VIDEOS bittorrent BITTORRENT

Try these root@Atlas:/mnt/disk2/bittorrent# whereis rsync rsync: /usr/bin/rsync /usr/bin/X11/rsync root@Atlas:/mnt/disk2/bittorrent# ls -l /usr/bin/rsync -rwxr-xr-x 1 root root 276688 Jun 14 23:04 /usr/bin/rsync* root@Atlas:/mnt/disk2/bittorrent# /usr/bin/rsync rsync://Tower backups Backups vmware VMWare Backups music Music pub Public Files boot /boot files mnt /mnt files Videos VIDEOS bittorrent BITTORRENT Also do echo $PATH