tillkrueger

if you click on your "Shares" link in the unRAID Management Console (the web GUI, or Graphic User Interface, as it's called), you will see three drop-down menus...the second one is for Disk-Shares, and allows you to choose it's export method...in effect how they are seen - or not seen - on the network...if you choose "Export read/write - hidden", then you can still mount them on your client machine (such as your HTPC, or Home Theater PC) if you know their name(s), which will always be disk1, disk2, disk3, etc. that may be the easiest way for you to get started...i can't find the thread right now which shows a method of mounting only one disk, and putting some symbolic links to a subset of the other disks into that mount, so that you have a neater way of accessing your many disk in the unRAID from a client...this method simply involves a few lines (mount commands) that you add to the "Go script" on your USB Boot Stick...it's located in the config folder, can be edited with a simple-text editor and is to go-to place (pun intended) to tell your unRAID system what to do while booting up...it's kind of like the "StartUp" folder on WindowsXP, or the old config.sys and autoexec.bat files in Win95. hope that helps a little bit.

i know you can hide *all* disk shares, but to hide individual shares might require a bit of Linux/command-line knowledge and modification of the go script (maybe hiding all disk-shares and then creating a set of mount-points of only those disks you want to expose? heck, there are others here that know their stuff in the Linux world a lot better than i do).

thanks guys. Rob, you may be right...it could be risky while taking even longer...no point in doing that if your prognosis is even remotely accurate...i might just do what i have been doing and do the USB drives (connected to a server and shared via 1Gb LAN)...the bottleneck here is the bad USB implementation of the Western Digital MyBook Essential drives...they just can't do much more than about 10MB/sec...my LAN is capable of over 35MB/sec, sometime even more. so, looking at Billped and bjp999's numbers, i'm looking at 14-20 hours...that's fine...i was worried it would be days. you know, i was wondering about the PCI bandwidth bottlenecks, and was close to holding off on the mobo choice and Promise controllers because of that...as it stands, i use almost exactly what Tom uses as his MD-1500/LL: > Lian-Li PC-A16B chassis with 15 removable hard drive trays > PC Power & Cooling Silencer 610 EPS12V power supply > ASUS P5B-VM DO motherboard with Dual Core E2180 and 2 GB DDR2-800 RAM > 2 Promise SATA300 TX4 PCI controller cards > 1GB USB2.0 Flash device (Cruzer) containing unRAID Server Pro OS i have the first 6 drives and the parity drive hooked up to the mobo controllers, the other 8 drives are on the Promise controllers. not to hijack my own thread, but does anyone know of a PCI-X based mobo that's known to work well with unRAID? i wouldn't be opposed to exchanging the ASUS for another mobo, exchange the Promise controllers for two Supermicro 8-port PCI-X controllers, and call it a day...i'd still want LG775, though...anyone?

since i have 4TB of data that are on 1TB external USB drives (OSX HFS+ formatted), and need to copy that data over to unRAID, i was wondering whether it would be possible to use something like Acronis Disk Director to convert the disks to ReiserFS, then take them out of their enclosure, insert them into unRAID, do a "Restore" of the array (i am not using parity yet), and get the data into the pool that way? just wondering...copying from the USB drives is painfully slow (about 10MB/sec), so it would take me another 6 days or so to copy it the same way i copied the current 4TB. btw, how long will it roughly take to calculate parity on 8TB of data?

i am not sure, but this may be the same behavior that has been mentioned in various other threads, and is related to the fact that parity has to be calculated at regular intervals...so what you see is supposedly expected behavior. i am still populating my 15-disk array with data, and since i have parity turned off until i copied all 8TB i currently have to my unRAID, i don't seem to be seeing the data-transfer drop that you are seeing...this would be consistent with my guess above. please , somebody, correct me if i am wrong.

no, i haven't spent more time on that either as i quickly realized the same thing as you...i'm not command-line savvy and will just have to wait until we get an Apache server on unRAID, at which point we probably could get the web-based proFTPd Admin interface working...until then i'll just have to use one of my other computers to manage FTP...sorry i can't be of more assistance for the moment.

haha...i hear you. yes, the tape thing has served me OK for a couple of years, but now that the thing just keeps giving me cryptic errors at totally unpredictable times/intervals, i might have to ditch that method. ditto on the MP3/Lossless-AAC/DVD/DivX collections...it's really taken years to collect it all, but i would/could survive without it...but my self-created photos/videos/animations/productions are priceless, and have to be protected at all cost...well, without the *all* part of the cost, if possible...of course, even "only" that part takes up in excess of 4TB already...oh well, at about $250/TB, i suppose it would be worth $1K to keep a second version of them on external disks and put them away at someone else's place...San Francisco can be a surprising territory, as i have known for sure since the 1989 earthquake.

so how do some of you deal with backups of your unRAID system? admittedly, it's pretty tempting to not back up with this level of redundancy, but most of us probably know that no system is fail-safe. the reason i am asking is this: i just built my first unRAID system to almost exactly mirror the MD1500/LL...i am now in the process of copying the 4TB of data from the RAID5 in my Powermac G5 to the unRAID...i had first copied all of the data from the RAID5 onto external USB Western Digital 1TB MyBook drives (this was before i decided to put together an unRAID)...i have collected 5 of the 1TB MyBook's over the past few months, with another 3 coming next week...frankly, had i known just how slow the USB interface behaves, i would have spent the extra money for firewire/eSATA...but who's to complain with that many drives. as soon as i moved the data to the external 1TB MyBook drives, i moved the 8 x 500GB Seagate 7200.10's from my G5 into the unRAID...for the past 3 days i have been copying the data from the external 1TB drives back onto the disk-shares...again, slow as molasses...about 10MB/sec, which is all the fault of that freakin' USB implementation of those drives (and mine for not leaving the data on the RAID5 and doing a direct copy via LAN)...since i just found out that those WD Essential Edition 1TB drives contain the WD "Green" drive, the same one i just purchased as a parity drive for my unRAID (because of its low energy/heat characteristics), i may pull them out of their enclosures and populate the other slots in my unRAID with them...hell, if the unRAID could read HFS, i'd do it now and connect them via eSATA. anyway, i'm digressing...back to the topic: when i had the RAID5 in my G5 i used a SCSI U320 card connected to a DELL 124T tape-library...this LTO-2 unit needs to be fed data at about 40MB/sec, or else the tape will keep spinning up and then down to let the data-cache catch up...but now that the data resides on the unRAID, i may not be able to get enough transfer speed to feed the data from unRAID via 1Gb networking to the G5 to the 124T. tape really is a two-edged sword...it's pretty cost-effective (the 200GB LTO-2 tapes can be found for about $20/ea) and it's pretty fast at about 40MB/sec...and with a tape-library like the 124T that holds 16 tapes, you can have automated backups going for quite a while (until you generate about 3.2TB of data). on the other hand, i find the unit to be pretty unreliable, and lately it's been giving me too many headaches. so, what do some of you do? if it were "only" 1 or 2 terrabytes, it would be fairly easy to just back up to a couple of external MyBooks...but i don't wanna have to keep buying big external drives...kind of defeats the purpose of the unRAID excercise. anyone with clever/cost effective ideas that aren't tape-based? or do you just trust your unRAID to the point of sayin': if my house doesn't burn down, i'll be fine.

i have had PSU problems on my Powermac G5 after i put an additional 8 drives in a RAID5 configuration in it (on top of the two system drives)...the symptoms were that on a cold start (after having been turned off for a while), it would first turn on and then turn itself off again while all disks began to suck power at once...if i turned it back on almost immediately (while some disks were still spinning a bit), i would often be able to "trick" it into booting...however, quite frequently i had one or two disks in the array stop spinning due to power not being sufficient (i suppose), which - needless to say - is a realy pain in the ass in a RAID5...it would go into a critical state, if only for a couple of seconds, and then start a few hours of parity checks...i ended up having to use a second power-supply on top of my G5, running the molex cables inside and powering just the disks off that PSU...and that was just too messy for my taste, which is part of the reason why i just put together an unRAID system. so, to answer your question more directly: - random HD spin-downs - it is unlikely that fans will fail, because they draw very little power. - most problems will likely make themselves known during boot-up, as your system draws by far the most power. - if you have a high-power processor and graphics card, then the other problems would rear their ugly head during game-play and rendering/transcoding (crashes and system shutdown) that's my experience, anyway, although PSU problems have made up a disappearing small percentage of my problems in 20 years of professional computing. JRS's advice to only buy from reputable manufacturers with conservative power-ratings is probably your best insurance against random shut-downs due to PSU problems.

that's what i just put into my rig...it's definitely not underpowered, as i have no problem with any of my content so far (HD or otherwise)...and once unRAID supports the second core, it'll be even more useful...at $30 i'd say it's a no-brainer...BUY!

yeah, i saw that you had posted about the mouse-awareness via putty in another thread...i tried that, because i also wanted to use MC like that, but found that neither my OSX Leopard nor my XP laptops have putty installed...i googled it and saw all this stuff about Darwin-Ports, and decided to just re-visit the idea of FTP instead...i'll get putty figured out as well, i'm sure, but in the meantime - especially while migrating almost 10TB of data to the unRAID - i can work with side-by-side FTP windows open. btw, is there a GUI for administering proFTPd (such as creating users/groups)? i found a couple online, put they seem to require Apache/PHP.

yes, i guess a re-appearance of the re-scan button would make sense then...i foresee a lot of "manual" arranging of folders in the disk-shares (as opposed to the user-shares), simply because it makes more sense to me...or at least it's more predictable until i see the light of user-shares and split-levels in its full glory.

ha! it works!! as per Joe's suggestion i typed: root@unRAID:~# echo nameserver 192.168.0.1 >/etc/resolv.conf root@unRAID:~# echo 192.168.0.6 unRAID >>/etc/hosts as you correctly noticed, i use a static IP on my unRAID. i haven't executed this per the Go script yet because i'm in the middle of a big copy action, but i guess if it works at the telnet prompt, it should work as part of the system startup. very, very cool...thanks so much xbit and Joe...the main reason i think FTP will be great is because with such good FTP clients like "Transmit" on OSX or "SmartFTP" on the Windows, one can easily move/copy/change files and directory structures on the unRAID while avoiding one of those unRAID -> Source -> unRAID bottlenecks some people keep running into while arranging their files...not everyone feels comfortable with the terminal...i'm not, for one. i'm lovin' this thing mo by the day

aha! i think i'm beginning to get how this works...although i'm kind of approaching it from the other end, i guess...as i said, i have eight 500GB disks for starters, and since last posting about this i have copied various DivX, H.264, MPEG, etc. folders to the first two drives' root...the first drive has a 400GB DivX folder, the second drive has 380GB worth of files in MPEG, DVD and HD folders... after some experimentation i found that if i first move all those folders into a freshly created "Movies" folder in the root (disk1/Movies/DivX and disk2/Movies/H.264 and disk2/Movies/MPEG) and *then* create a user-share called "Movies", it automatically contains all those sub-folders...logical if you know what to expect, but not so much when you first start wrapping your mind around it...my mind, anyway. so i'm beginning to think that in my case it's best to *first* copy all of the files and folders to the individual disks in the hierarchical arrangement that is logical to me, and *then* create user-shares that address the root-level folders i created, rather than the other way around (first creating user-shares and then copying folders to them, not always knowing where they will end up. in my mind it's a lot easier to plan how to copy files/folders to an array of drives, and then sort/share those folders in a logical manner, than to copy them to a "bin" that is linked to various disks that have some sort of "split-level" hierarchy that is determined by high-water and most/least-full levels... your mind may vary.

naw, same thing...after a reboot i get the same error message: root@unRAID:~# proftpd - getaddrinfo 'unRAID' error: Temporary failure in name resolution - warning: unable to determine IP address of 'unRAID' - error: no valid servers configured - Fatal: error processing configuration file '/etc/proftpd.conf' root@unRAID:~# so i suspect that the proftpd.conf file needs to be further modified for my particular setup? i looked at all the parameters, but nothing stuck out pertaining to url/ip addess...any ideas?

xbit, i really appreciate the help...i tried what you suggested but ran into a snag...since i don't want to hijack your thread any further, i started a new "proFTPd" thread...would you have the time to take a look at what my hang-up is and maybe tell me what you think causes it? i'm really close, i think.

since i don't want to hijack xbit's "hellanzb + lynix n00b = frustration" thread, i started a new thread about how to get proFTPd working on unRAID. his instructions are as follows: ...here's what I did. I have given root a password via the web interface and log in with that account. Download proftpd version. The latest version didn't work but this version did. http://packages.slackware.it/package.php?q=current/proftpd-1.3.1-i486-1#download Add the following to the end of your go script: installpkg /boot/packages/proftpd-1.2.10-i486-4.tgz cp /boot/packages/proftpd.conf /etc proftpd Copy the proftpd tgz file to /boot/packages Copy the proftpd.conf to /boot/packages (the "packages" directory I created to store all my customization packages, you can put it someplace else if you prefer. If you do, be sure to change the above go script as well.) reboot unraid via web interface (stop, reboot) and you should have ftp access. I've attached the modified proftpd.conf file you'll need to be able to log on using root. Basically the needed changes from the default config is: ServerType standalone UseFtpUsers off RootLogin on thanks, xbit, i have followed these instructions (nice form creating the "packages" folder...i did the same, and moved my Midnight Commander package in there as well, changing my Go script accordingly). the link you included is for the 1.3.1 download list...i downloaded and installed it via telnet, but when typing the proftpd command i got an error: root@unRAID:~# installpkg /boot/packages/proftpd-1.3.1-i486-1.tgz WARNING: pkgtools are unstable with tar > 1.13. You should provide a "tar-1.13" in your $PATH. Installing package proftpd-1.3.1-i486-1... PACKAGE DESCRIPTION: proftpd: proftpd (FTP server daemon) proftpd: proftpd: ProFTPD is the Professional File Transfer Protocol (FTP) server proftpd: daemon. ProFTPD grew out of the desire to have a secure and proftpd: configurable FTP server, and out of a significant admiration of the proftpd: Apache web server. proftpd: Executing install script for proftpd-1.3.1-i486-1... root@unRAID:~# proftpd - mod_tls/2.1.2: compiled using OpenSSL version 'OpenSSL 0.9.8g 19 Oct 2007' headers, but linked to OpenSSL version 'OpenSSL 0.9.8d 28 Sep 2006' library - Fatal: unable to load module 'mod_tls.c': Operation not permitted is that what you referred to when you said "The latest version didn't work", because in then realized that in your Go script you had installpkg /boot/packages/proftpd-1.2.10-i486-4.tgz. so i downloaded the 1.2.1 version...since i'm in the middle of a big copy operation i couldn't reboot to clear the install...instead i just pushed my luck and did the telnet install of 1.2.1 over the new version, but got this response: root@unRAID:~# installpkg /boot/packages/proftpd-1.2.10-i486-4.tgz WARNING: pkgtools are unstable with tar > 1.13. You should provide a "tar-1.13" in your $PATH. Installing package proftpd-1.2.10-i486-4... PACKAGE DESCRIPTION: proftpd: proftpd (FTP server daemon) proftpd: proftpd: ProFTPD is the Professional File Transfer Protocol (FTP) server proftpd: daemon. ProFTPD grew out of the desire to have a secure and proftpd: configurable FTP server, and out of a significant admiration of the proftpd: Apache web server. proftpd: Executing install script for proftpd-1.2.10-i486-4... root@unRAID:~# proftpd - getaddrinfo 'unRAID' error: Temporary failure in name resolution - warning: unable to determine IP address of 'unRAID' - error: no valid servers configured - Fatal: error processing configuration file '/etc/proftpd.conf' root@unRAID:~# is this because i didn't reboot and the mix of 1.3.1 and 1.2.1 are now screwy, or is there something in the config file i have to change to make proFTPd aware of my unRAID server's IP address (it's static and set to 192.168.0.6)? i'll have to wait another hour until i can reboot, so maybe it will fix itself after running the Go script on a clean system. thanks so much for that pointer so far...this is a lot easier than i thought (i anticipated the whole slackware/modding path). hope someone gets you taken care of on the hellaNZB end soon.

sorry i can't give you any pointers, as i'm totally new to Linux and unRAID as well...but would you mind telling us how you got proFTP to work? did you mod the slackware portion of unRAID as per one of the posts suggestions? i was able to do the Midnight Commander and Twonky Media Server installs, but those were very basic. hope you get one of the Linux cracks on the forum to help you out with hellanzb soon.

ok, it's not that i haven't done a dozen searches and read and experimented for the past 10 hours straight...but i am really having a hard time grasping the implications of user shares...i think i get the methods/parameters and what they do, but there just seem to be too many unknown factors in how my data is going to end up on the platters and what happens when folders grow past their original disk. yesterday i received all the parts (almost exactly like the MD-1500/LL, but with a dual-core proc and 2GB of memory in case a future updates will enable those)...by midnight i had everything running with 8 x 500GB Seagate 7200.10's occupying slots 1-8, and was a happy camper going to sleep. today i picked up a WD 1TB green drive and put it in slot 15 where i hooked it up as parity drive...and then i sat down and starting reading...and reading...experimented with a few folders, then kept on reading...and now it's midnight again and i feel like i know less than before. ideally, i'd like to just assign all disks to an "unRAID" user share (a tried that and ended up with a nice 3.65TB "drive" that i mounted both on OSX and XP) and fill it up, then start adding 1TB drives, and finally - if necessary - upgrade the 500's with 1TB's when all slots are filled. some of my questions: - if i do that, will all disks always start spinning when i access *any* of the data? - will all disks keep spinning (past the spin-down delay) once the data has been found on any one of the drives? - can i first copy my data to the individual disk1-disk8 shares, using my own directory structure/organization, *then* create top level user shares (i.e. a "Movies" folder on disk1,2,3) and *then* move the data i already copied into those folders? see, say in regard to movies, and/or music, i have a directory structure like this: Chris/Movies/DivX/_split/Alien/disk1.avi Chris/Movies/DivX/_split/Alien/disk2.avi Chris/Movies/DivX/complete/Aliens.avi Chris/Movies/DivX/complete/Babel.avi Chris/Movies/H.264/AppleTV/Episodes/Lost/Season1/Premiere.avi Chris/Movies/H.264/iPod/Movies/Rocky Horror.avi Tom/Music/MP3/Air/10,000Hz Legend/Caramel Prisoner.mp3 Tom/Music/Lossless/Air/10,000Hz Legend/Caramel Prisoner.mp3 long story short...the same type of "category" might sometime be 2 levels deep, sometime 4 or 5 levels deep (or even deeper)...so i feel pretty confused about what split-level to choose, and once i choose it, what it would do for any given scenario that i haven't even yet considered. again, if i could just treat all disks as a growing media-vault and do my own directory-structure arrangements and re-arrangements, i would be totally cool with that...but if that means all disk are always spinning, double clicking on a file will initiate a 8-disk spin-up, etc., then i think i should have just stayed with a RAID5...although i love the prospect of individual per-disk file-systems. sorry for the rant, but i'm tired and a bit frustrated about my inability to fully grasp split-levels and planning my data-storage strategy. any advice to easy my entry into this next phase would be great! g'nite.

over the past few weeks, and before coming across unRAID again (i first discovered unRAID three years ago), i happened to have evaluated the whole PM (port-multiplier) issue ad nausea...it is my understanding that a SATA chipset - on the mobo or host-controller - has to be PM capable in order to work...the SIL3124 chipset is, so using one of their inexpensive PCI controllers with 4 external SATA ports, one can hook up 20 external drives... as was correctly stated already, all the drives on one port share the bandwidth of the port...but since a single SATA II port can theoretically do 3Gb/sec, even 5 drives hooked up to one PM port can still operate at a combined transfer rate of over 300MB/sec...given the limitations of current drives and the PCI/PCI-X bus, this is not really something to worry about. btw, the SATA II and PM specs allow for a total of 15 drives per PM...i have not found a card/bracket that allows for that many hardware-connections, but anyone who is into that many disks would probably lean towards SAS attached storage, where up to 128 drives can be connected by means of SAS repeaters (expensive). if u use the SIL 3124 cards, which have gotten great reviews for allowing for very inexpensive RAID5 setups (again, you can hook up 20 drives via PMs), i found the limitation to be that only 5 drives can be included per array...so if you were thinking about an external 20 drive array (as i was), you would have to get 4 PM boxes with 5 drives each, and end up with 4 separate drive arrays (potentially giving up 4 disks, rather than 1 in a RAID5 scenario)...of course, if you're at home in the Linux world (and i am not), you can probably circumvent this limitation by not using the SIL software and building the RAID5/6 array purely in Linux's software facilities. still, if you add up the cost of the PM's alone (about $100/ea. for a 5-to-1), you're still looking at a lot of money, and it begs the question whether you wouldn't be better off getting something like a HighPoint controller with 4 multi-lane connectors, that can support 4 drives each for a total of 16 drives, but without some of that PM voodoo (much higher bandwidth, and frankly i trust HighPoint, or Arca, etc. more than a $70 SIL controller). long story short, after weeks of my head smoking from reading through hundreds of threads, i came across unRAID again and found that it might offer the best cost/capability/headache ratio for a setup that focuses mostly on storing huge amounts of data, which is usually: music, video, image collections for entertainment purposes...all my critical visual-productions workflow files (captured HD video, design data, etc.) will probably have to stay on a much faster RAID5 array in my workstation(s)...and when i say fast, i mostly mean how quickly can i back up a 200GB editing session to tape.

>> How much simultaneous trans coding do you really plan to do? well, of course there is only so much to do, and it won't be a continuous need, but when i do need it, i need it quickly and without tying up my work machine(s) for hours or days...my career is in visual productions, so i do a lot of rendering and transcoding...one of the first tasks would be to convert about 500 DivX movies to h.264...i know Elgato has a hardware-based USB rendering stick, but the quality is not all that great. back to the unRaid topic, though, i can see why you went back to offloading the unRaid machine onto something small and manageable (ITX)...my exprience is that one cannot expect to do everything on one machine and ado it well...that's why i have a half-dozen machines/notebooks in my studio that do single-tasks like DTP/Design, another one FCP editing, another one for surfing/dicking-around, etc... maybe the Lian-Li case with a decent mobo like the Asus P5B-VM DO and a dual-core proc (support for the second core will likely happen this year, i guess) is a good stand-alone solution with nothing else needed. the main reason, really, why i am partial to the thought of a dual-boot into Linux or Windows 2003 Server is an even more pragmatic one: i have about 10TB of data residing on a RAID5 in my Powermac G5 right now, which also houses a U320 SCSI card hooked up to a DELL 124T tape-library...the thing backs up once a day, and for LTO-2 tape to not have to wait for the computer, you have to be able to feed it data at about 40-60MB/sec...from what i am reading, the unRaid can read at maybe 10-20MB/sec, which means i won't be able to back up from it to the tape-library...if i were able to be able to dual-boot, i could at least boot into Linux/Win once a week and do the backups. in case you are wondering why i don't just stay with the Powermac G5 RAID5 solution: i am running out of space...all the disks require me to use a second external power-supply (feels too hackintosh-ish)...the OSX backup software and tape-library support isn't as elegant as some of the Linux/Win solutions...i wouldn't mind getting my Powermac G5 back as a FCP editing workstation...yada yada. well, thx for responding...

does anyone know of a server-type mobo that fits into the Lian-Li case and has as many of the unRaid requirements as possible? i do have a Asus P5B-VM DO on order, but i'd like to build a dual-boot monster that can server some transcoding/rendering applications that i also need to do (DivX to h.264, m2t to h.264, AAC lossless to MP3 VBR, etc.). i was looking at something like this ASUS DSBV-D Dual 771 Intel 5000V SSI CEB 1.1 Server Motherboard: http://www.newegg.com/Product/Product.aspx?Item=N82E16813131152R but have no idea whether it fits or even would work? i don't mind having to add a PCIe Gbit card if necessary, or even a PCIe X16 graphics card, but if it doesn't work with unRaid, then i'd rather stick with a Q6600 and call it a day...the mobo i linked to above i like because it has 3 PCI-X slots that would make a nice home for the SUPERMICRO AOC-SAT2-MV8 8-port SATA cards. yeah, i know...it's a bit out there, but you know it goes sometime. if i must let go of the Dual Quad-Proc. idea, what would be the most "modern" (non-discontinued) mobo that works with a Q6600 and still fulfills all/most unRaid requirements? i have read through most threads, but most of the time i am not sure whether they pertain to mobos that people had laying around and are happy to see them actually work, or whether they are particularly well working mobos that are still within their life-cycle. any pointers would be greatly appreciated.