Bye Bye and thanks for all the fish


Ford Prefect

Recommended Posts

...some time ago my no. 1 feature I was looking for in a NAS changed from ease-of-use to encryption.

After doing a lot of research and testing to get this thing going, I finally made my peace with unRAID,

retired my licenses and put the flash-sticks back on the shelves.

 

My box is now running the ESXi (all in one) version of this: http://hardforum.com/showthread.php?t=1573272,

based on Solaris 11 Express.

 

So I take the opportunity to say thanks for all the great help I received from this wonderful community.

I will continue to roam and pass by sometimes.

My unRAID licenses will possibly be revived when the kids grow older and want to run their own parts  ::)

 

regards,

        Ford

Link to comment

ZFS is an amazing raid/file system without a doubt.  I have looked at napp-it too.  I have even downloaded it along with solaris 11 express and openindianna..not sure which Ill go with.

Not going to run the all in one though.  I'm going to keep using unraid also though but in a different manner.

I had to buy new sas cards for solaris though, solaris is way more fussy on hardware support but then I guess you get a rock solid OS.

 

Still a good market for unraid though, it serves its purpose of ease of use and flexibility but I think more work needs to be done on "what do i do when errors start appearing"

The assumption that the data drives are correct and the parity drive should be corrected is wrong - as i found out today.

Needs to be ways of easily telling unraid to correct the data drive based on the parity for those errors instead of having to rebuild the entire drive again..

Should also log which files are affected by these parity errors so people can double check them etc.

double parity...

Email alerts etc etc etc

 

Hopefully development on unraid will continue with some big features....

 

Anyway good luck with ZFS, why did you choose express over openindianna? just cause it has newer versions of zfs etc?  no support...

update: ahh you need encryption.. fair enough :)

Link to comment

ZFS is an amazing raid/file system without a doubt.  I have looked at napp-it too.  I have even downloaded it along with solaris 11 express and openindianna..not sure which Ill go with.

 

Does zfs allow only spinning up the one drive that has your needed data?

 

That's golden to me :)

 

as far as i understand it and also the reason why i didn't try it... is that it is a normal raid system.... with a fancy filesystem....

this means all disks in the raid need to spin...

there is no easy way of expansion ... you need to add a second/ third array .....

if unraid came with the selfhealing zfs filesystem that would be golden...

Link to comment

ZFS is...OK.  It doesn't have the expansion capabilities of unRAID.  I tried the whole Solaris 11 situation before, and didn't like it.  No point in complicating things when you just need cheap, efficient, expandable storage.  Unraid is king in those departments.  On top of that, ESXi on SATA drives has the worst freaking disk performance ever.  I was using a Perc 6 with 8 freaking 1TB HD's in raid 10, and the performance was 40MB per sec (terrible).  For people that want to play and don't care about performance, sure, go esxi.  For everyone else that wants the best OS for storing media files, UNRAID.

 

UPDATE: I guess if you only care about encryption, then unraid isn't for you. EDIT: Truecrypt?

Link to comment

unRAID has been the answer to my storage needs for many years.

 

When I need real RAID I set that up. RAID1/RAID0/RAID5, etc, etc.

When I need an on demand file storage solution that can expand, provide separation based on spindle. provide a consolidated view when needed and is easily expandable. I use unRAID.

 

Since I access via nfs or smb, I do not care any more what the underlying filesystem is.

Just that it's protected and easily repaired or expanded.

 

Different strokes for different folks.

 

Link to comment

I'm getting to the point where I feel like I may be outgrowing my unraid boxes.  4224 maxed to capacity and a 4020 half full already of 3TB drives.  What does my furute of storage needs hold?  Just seems likes its growing faster and faster

 

Just this week i've removed all my services from both unraid boxes and just running powerdown and apc.  Moved everything else over to a new ubuntu server and things have been running much smoother, not that unraid wasn't smooth in the first place, but everything is now much more snappier

 

On ubuntu i installed a ZFS stripe with 5 old HHD's as my "workspace" (as soon as the dl's are done error checking and unraring, they get moved to the array so no need for data protection) and let me say i'm very impressed, especially how easy it is to setup.  At first I didn't think I did it right because it litterely took seconds, then after testing, etc.... was in awe.....

 

Which got me thinking, maybe turn my two norco cases into JBOD boxes and run all from a norco-470, can put 4 of those high-point 32 port HBA controllers ( 32 x 4 = 124 HHD's, currently at 3TB minus parity, = 336TB of space), on a supermicro motherboard, thow a lot of ram at it, create pools of raidz1 with 8 HHD's each, very tempting.  Also lower power because will run with two less motherboards, memory and processors

 

As a test I setup a raidz1 with those 5 HDD's and got a consistent 67MB/s write speed with spikes of 100MB/s across the network, on old sata drives, really got me thinking how slow unraid's copying really is.  I've always wanted to get more performance but never wanted to fork out the money to run other raid systems, now i'm thinking its not completely un-reachable anymore

 

just need to be willing to fork up $800 or so every time you need to upgrade hard drives, if you stay with pools of 8, (Solaris recommends nothing more than 9 per pool)

 

Am I smoking crack?  maybe............ A lot more research to do............

 

Josh

Link to comment

ZFS is an amazing raid/file system without a doubt.  I have looked at napp-it too.  I have even downloaded it along with solaris 11 express and openindianna..not sure which Ill go with.

 

Does zfs allow only spinning up the one drive that has your needed data?

 

That's golden to me :)

 

 

For me too! unRaid it spinning up only one drive, and it all the possibilites too use diferent disks is real unique!

Link to comment

I'm getting to the point where I feel like I may be outgrowing my unraid boxes.  4224 maxed to capacity and a 4020 half full already of 3TB drives.  What does my furute of storage needs hold?  Just seems likes its growing faster and faster

 

Just this week i've removed all my services from both unraid boxes and just running powerdown and apc.  Moved everything else over to a new ubuntu server and things have been running much smoother, not that unraid wasn't smooth in the first place, but everything is now much more snappier

 

On ubuntu i installed a ZFS stripe with 5 old HHD's as my "workspace" (as soon as the dl's are done error checking and unraring, they get moved to the array so no need for data protection) and let me say i'm very impressed, especially how easy it is to setup.  At first I didn't think I did it right because it litterely took seconds, then after testing, etc.... was in awe.....

 

Which got me thinking, maybe turn my two norco cases into JBOD boxes and run all from a norco-470, can put 4 of those high-point 32 port HBA controllers ( 32 x 4 = 124 HHD's, currently at 3TB minus parity, = 336TB of space), on a supermicro motherboard, thow a lot of ram at it, create pools of raidz1 with 8 HHD's each, very tempting.  Also lower power because will run with two less motherboards, memory and processors

 

As a test I setup a raidz1 with those 5 HDD's and got a consistent 67MB/s write speed with spikes of 100MB/s across the network, on old sata drives, really got me thinking how slow unraid's copying really is.  I've always wanted to get more performance but never wanted to fork out the money to run other raid systems, now i'm thinking its not completely un-reachable anymore

 

just need to be willing to fork up $800 or so every time you need to upgrade hard drives, if you stay with pools of 8, (Solaris recommends nothing more than 9 per pool)

 

Am I smoking crack?  maybe............ A lot more research to do............

 

Josh

 

Definitely smoking crack running zfs with linux, only one OS worth running it with and that is solaris, be it express, OI, or nexenta.

 

Unraid is never going to compete with write speeds compared to those kinds of setups (raid, raidz etc), everyone knows that?

Unraid isn't going to appeal to all - obviously, its all based on what you want/need etc, plenty of people that don't care too much about fast write speeds etc and would rather be able to slap in one drive at a time etc.  Lets not forget its really aimed at being a media server and being flexible, in that area I think its still number one.

 

 

 

 

Link to comment

Besides unRAID, I also have 3 Thecus N7700 NAS boxes with 7x2TB drives in each running ZFS.  ZFS is the only file system supporting partitions >16TB (XFS is limited to 16TB).  The linux ZFS port is NOT perfect, I have some corrupted files that cannot be read or corrected.  There is no file system check/repair in ZFS sure it does most things automatically but if something serious goes wrong, you lose everything.  RAID5 is also very dangerous as the chances of an unrecoverable read error on a parity rebuild are high (especially with large arrays), then it really is Bye Bye to all the fish - permanently.  For media storage, unRAID is VASTLY safer, better, cheaper on power, more expandable.

Link to comment

yeah don't see myself making a switch anytime soon, just promising in the fact that I know I can migrate another way in the future and won't be tied down with unraid forever, not that its not great, but just how large can it really get and still perform?  Figure in a year to a year and a half will be building a third server, need a better way for the machines to communicate with each other.

 

Actually looking at all of my hardware, win7 workstation, ubuntu server and two unraid servers, if I did make a switch I could do it right away and all i'd need to get a are new hard drives to build my first pool

 

Agreed on Solaris, just messing with zfs on ubuntu to get familiar with it.  I have a ton of reading and testing to do before I would ever trust my 60TB+ worth of data to another system, unraid hasn't let me down yet in that sense

 

 

Link to comment

ZFS is an amazing raid/file system without a doubt.  I have looked at napp-it too.  I have even downloaded it along with solaris 11 express and openindianna..not sure which Ill go with.

 

Does zfs allow only spinning up the one drive that has your needed data?

 

That's golden to me :)

 

 

For me too! unRaid it spinning up only one drive, and it all the possibilites too use diferent disks is real unique!

 

+1

Link to comment

Besides unRAID, I also have 3 Thecus N7700 NAS boxes with 7x2TB drives in each running ZFS.  ZFS is the only file system supporting partitions >16TB (XFS is limited to 16TB).  The linux ZFS port is NOT perfect, I have some corrupted files that cannot be read or corrected.  There is no file system check/repair in ZFS sure it does most things automatically but if something serious goes wrong, you lose everything.  RAID5 is also very dangerous as the chances of an unrecoverable read error on a parity rebuild are high (especially with large arrays), then it really is Bye Bye to all the fish - permanently.  For media storage, unRAID is VASTLY safer, better, cheaper on power, more expandable.

 

scrub is their repair tool :).  You have to remember ZFS was built with data integrity at the foremost with big enterprise systems in mind and to fix the flaws in tranditional raid as well.  Simply put for data integrity there is no better file system out there at the moment.

 

"ZFS ensures that data is always consistent on the disk using a number of techniques, including copy-on-write. What this means is that when data is changed it is not overwritten — it is always written to a new block and checksummed before pointers to the data are changed. The old data may be retained, creating snapshots of the file system through time as changes are made. File writes using ZFS are transactional — either everything or nothing is written to disk."

 

The file system uses a 256-bit checksum, which is stored as metadata separate from the data it relates to, when it writes information to disk. Unlike a simple disk block checksum, this can detect phantom writes, misdirected reads and writes, DMA parity errors, driver bugs and accidental overwrites as well as traditional "bit rot."

 

With a raidz2 or raidz3 you are pretty well protected.. as well are you are going to be, thats why enterprises use it..

 

Don't get me wrong, I am using both.. I love zfs and love unraid but precious data I'd only trust zfs with.  Media is fine with unraid.

Link to comment

Ford Prefect

The file system uses a 256-bit checksum, which is stored as metadata separate from the data it relates to, when it writes information to disk. Unlike a simple disk block checksum, this can detect phantom writes, misdirected reads and writes, DMA parity errors, driver bugs and accidental overwrites as well as traditional "bit rot."

 

Thanks for that, interesting post and it explains my data corruption, the checksum must disagree from my data and the Thecus box simply drops out when I get to that part of the file (a movie), preventing me watching or copying it.  There is no SCRUB utility on a Thecus box, it's simply gone forever, which is a shame when movies are compressed "lossy" and you can accept a bit of corruption.

 

I chose ZFS because it was the only future proof file system when I ran out of space and replaced 2TB drives with 3TB drives.  I can tell you from first hand experience, it can get corrupted, probably before I put new batteries in my UPS's, we get lots of power outages in the rainy season in Thailand.

Link to comment

Ford Prefect

The file system uses a 256-bit checksum, which is stored as metadata separate from the data it relates to, when it writes information to disk. Unlike a simple disk block checksum, this can detect phantom writes, misdirected reads and writes, DMA parity errors, driver bugs and accidental overwrites as well as traditional "bit rot."

 

Thanks for that, interesting post and it explains my data corruption, the checksum must disagree from my data and the Thecus box simply drops out when I get to that part of the file (a movie), preventing me watching or copying it.  There is no SCRUB utility on a Thecus box, it's simply gone forever, which is a shame when movies are compressed "lossy" and you can accept a bit of corruption.

 

I chose ZFS because it was the only future proof file system when I ran out of space and replaced 2TB drives with 3TB drives.  I can tell you from first hand experience, it can get corrupted, probably before I put new batteries in my UPS's, we get lots of power outages in the rainy season in Thailand.

 

Don't bother with ZFS on linux, its a port and poorly done.

Anyway,  aside from that, I actually lived in Thailand for a year :) and was there off and on for 4 years on contracts.  Contracted via my company to TOT.  Definitely an experience, they weren't using Unraid... :)

Link to comment

I am new to unRAID, Rajahal hand delivered my first server order from him 2 weeks ago with 15 of 3TB drives in it, I am in the process of copying all my data onto it at the moment, it's for a new home I am building in the mountains north of Chiang Rai.  When I retired here I became a director of Loxley who own 5% of TT&T and CSLoxInfo.  TOT must be the worst internet service in the world ;-)  I used to get all of 4KB/s downloading torrents some times

Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.