Beta-3 Requests

[BRiT]

How about a 64-bit version of unRAID?  Can we start testing this with the 5.0beta series?  I'm sure many of us would like to start running x64 apps on our processors to speed up CPU intensive applications.

 

Although it's not supported, I am testing out unRAID 5.0 beta 2 on a 64bit Slackware 13.1+ (Current) distro that is MultiLib'ed to run 32bit code with kernel 2.6.35.3. So far so good. However, I have run into an issue when trying to clear and add a disk via the management console (emhttp). With so much bleeding edge, it's hard to tell if its a 5.0b2, or Slackware 64Bit, or Kernel 2.6.35.3 thing.

 

There are two approaches that could be taken with this. The emhttp and shfs could remain the same 32bit executables and run under a 64Bit Slackware 13.x that is MultiLib'ed to allow for running 32bit code. Or the emhttp and shfs could be compiled under a pure 64bit environment. Even if emhttp and shfs are native 64bit, I have a feeling the users will want a MultiLib'ed OS. This will allow them to run any 32bit community generated packages.

 

In the background to make things easier for myself I've also created an unRAID 5.0beta2 package that's used to install onto a full Slackware distro. It's used to install and copy over all the unRAID needed files, deal with the additions and changes in /etc/rc.d and /etc, deal with the fstab, create mount points, and create /boot symlinks. There are 2 items it does not handle yet. The first was an oversight, which is the changing of 'short_open_tag = On' in /etc/httpd/php.ini. The second is much larger, dealing with the creation of the custom Kernel. Even if I never get the second item handled, it greatly simplifies the process of running on a full Slackware distro. 

[BRiT]

Nicely done and flushed out WeeboTech.

 

Did you come across anything that would need to be done or changed to have / on a tmpfs straight out of the boot? I haven't looked into the Linux booting or initrd realm at all.

[Tyson]

Can we get support for more controllers on the upcoming release?

 

Specifically, I would like to be able to use my LSI MegaRaid controllers under JBOD mode.

 

As things sit right now, without megaraid_sas kernel module support, drives attached to these controllers will not be presented to the underlying OS.

 

Specific Model Number of my controllers: LSI Megaraid 84016E

 

I have 4 servers housing 15 drives a piece under these controllers and would like to migrate away from Nexenta following the fall-out created by Oracle's purchase of Sun.

 

I understand that I *could* always go to Hardware RAID as the controllers support it, however I would much rather have a solution such as yours where I can recover data from a single drive in the event of failure.

[BRiT]

Do you know which of the following Linux Kernel config options must be enabled to support your card at the Linux level?

 

CONFIG_MEGARAID_NEWGEN,

CONFIG_MEGARAID_MM,

CONFIG_MEGARAID_MAILBOX,

CONFIG_MEGARAID_LEGACY

CONFIG_MEGARAID_SAS,

 

I suspect you might only need the CONFIG_MEGARAID_SAS option. It's description is:

 

Module for LSI Logic's SAS based RAID controllers.

To compile this driver as a module, choose 'm' here.

Module will be called megaraid_sas

 

Prompt: LSI Logic MegaRAID SAS RAID Module

  Depends on: SCSI_LOWLEVEL [=y] && PCI [=y] && SCSI [=y]

  Location:

    -> Device Drivers

      -> SCSI device support

        -> SCSI low-level drivers (SCSI_LOWLEVEL [=y])

[bubbaQ]

How about a 64-bit version of unRAID?  Can we start testing this with the 5.0beta series?  I'm sure many of us would like to start running x64 apps on our processors to speed up CPU intensive applications.

 

 

64-bit will have no appreciable effect on speed, except in rare situations.  

[WeeboTech]

Nicely done and flushed out WeeboTech.

 

Did you come across anything that would need to be done or changed to have / on a tmpfs straight out of the boot? I haven't looked into the Linux booting or initrd realm at all.

 

I have worked on having / on tmpfs with a modified version of switch_root.

The standard source and pivot_root will not work. There's a test that prevents the switch_root from occurring.

pivot_root will not work for initramfs. switch_root works with some source modifications to the source & dest stat.

There is a multiline if statement and by dropping a couple tests it works (if I remember correctly).

 

In any case, it requires some modification to the booting procedure and I'm not sure we're ready for that.

When we are, it opens the door for unionfs or aufs which will allow static install overlays on root.

In the meantime I'm happy to discuss this further in another thread if you want to know more about my experiments.

[ftp222]

How about a 64-bit version of unRAID?  Can we start testing this with the 5.0beta series?  I'm sure many of us would like to start running x64 apps on our processors to speed up CPU intensive applications.

 

 

64-bit will have no appreciable effect on speed, except in rare situations.  

 

I am not looking to increase the speed of unRAID, only the other applications running on top of it.  The main reason for unRAID 5.0 is to make it easier to install and have other applications run on top of it, so I believe support for a 64-bit environment should be near the top of the list to help performance of these other applications.  I understand this may not initially seem like a big deal, but there are many people running a myriad of CPU intensive applications (PAR2, ZIP/RAR, VMware of course, etc.).  In a Windows environment, it is pretty simple to compile for 64-bit, I'm just hoping it's similar for Linux.

 

 

 

 

[bubbaQ]

I am not looking to increase the speed of unRAID, only the other applications running on top of it.

 

And 64 bit will have no appreciable effect on speed of those applications.

 

If you have an app that is memory bound, and 2GB is not enough for its code (none I know of that big) and another 2GB is not big enough for its data, then 64-bit lets them use a flat memory model... assuming you have more than 4GB of RAM.  But unless you are thrashing the hell out of swap space, going to 64-bit will not improve speed of practically ANY application, except in some rare circumstances (e.g. obscene number crunching with 64bit ints)

 

What makes you thing PAR, ZIP, RAR, will run faster on 64-bit?

[WeeboTech]

I believe support for a 64-bit environment should be near the top of the list to help performance of these other applications.

 

I'm on the fence about 64bit being on the top of the list.

 

We still need to finish out the event support and package management of these applications.

I'm not so sure 64bit is going to improve much in the line of cpu performance.

We always tout, you don't need a fast cpu for an unRAID appliance.

Where it does help is allowing the kernel direct access to more memory, so the cache can be larger without using PAE.

 

The plugin methodology is still a moving target.

I spent the day retooling my locate & syslog monitor plugins.

 

There may be some driver gottcha's in the way of pointers when going from 32 to 64 bit.

[ftp222]

You are correct that the CPU performance may not be huge, but a 10-20% gain can be a good boost.  WeboTech is right that memory access will be the big benefactor.  I didn't mean to jump into the thread and demand a 64-bit version, I just think this is where software is headed and if it is an easy re-compile and post a 64-bit version, then great lets start down that path too.

 

As I understand unRAID 5.0 is the next evolutionary step to the software and getting the hooks into it is the most important.  But once we have those hooks, it would be great to take advantage of everything we can throw at it.  unRAID has been instrumental in the way it handles and protects our data and nothing else out there exists that accomplishes the task in the same way with the same confidence; however other devices are catching up quickly and I just want to see unRAID do everything else the major competitors are.  5.0 will be a massive leap forward, but with memory getting cheaper and bigger, 64-bit is the next logical step so why not start testing and working out the bugs now if it is easy.

 

[WeeboTech]

I just think this is where software is headed and if it is an easy re-compile and post a 64-bit version, then great lets start down that path too.

I agree, this is where software is headed. However, keep in mind, Once we go to 64bit, it eliminates a large segment of hardware.

 

5.0 will be a massive leap forward, but with memory getting cheaper and bigger, 64-bit is the next logical step so why not start testing and working out the bugs now if it is easy.

 

Let's not ask "why not", let's ask when, and for how long. I surmise the beta period will be a bit lengthy.

I don''t think it's just a recompile. There's a suite of programs and tests that have to be performed.

 

I thought Tom mentioned somewhere moving forward to Slackware 13. Perhaps it's already on the table.

 

I believe little steps need to be performed.

Change too much all at once and it becomes a bear of a test.

5.0b is a big step to a whole management interface upgrade.

[bubbaQ]

64-bit is the next logical step

 

Huh?  what makes it so?  I can think of many things more "logical" for a NAS appliance like unRAID before 64-bit.... such as Q-parity.

 

You are correct that the CPU performance may not be huge, but a 10-20% gain can be a good boost.

 

It is not 10-20%... it is zero.

[BRiT]

While official 64bit support would be nice, I'll have to agree that as far as a NAS appliance goes, there are desired features on the list that have a higher priority. I would go so far as to say 64bit shouldn't be done until 5.1 at the earliest, and more likely around the 5.2 era. I desire Q-Parity or Multiple-Array support more than official 64bit support.

 

It would be nice to have the switch to Slackware 13.1 during the 5.0 beta period. Hopefully we won't need to expand the beta period so far as to hit upon a Slackware 13.2 release.

 

What we need is to nail down the event and plugin structure. This alone will be fundamental in allowing unRAID to progress further with community development.

 

Items which I think a NAS appliance should have, if not built in there should be options for addons, plugins, or upgrades to provide the functionality. Some of this was taken from looking at existing NAS Appliances' featuresets.

 

• Simple Web-based Configuration
  
• Secure Management (https, ssh, policy-based ip blocking)
  
• Simple Web-based file manager
  
• Support for Win OS, MAC OS X, NFS Clients
  
• Pseudo-Realtime Notification of events - email, pager, sms
  
• Proactive Monitoring of HDD SMART
  
• UPS Support
  
• Network Recycle Bin
  
• Backup Server for client machines
  
• FTP Server
  
• Print Server
  
• Active Directory Integration
  
• UPnP / DLNA Media Server / Media Streaming for popular devices
  
• Supports remote Syslog Servers
  
• Torrent Client
  
• Usenet Client
  
• iSCSI Target Service
  
• Community Software Package Platform (Plugins and Addons)
  

[WeeboTech]

# Pseudo-Realtime Notification of events - email, pager, sms

# Proactive Monitoring of HDD SMART

 

I can help with this through the use of NAGIOS.

In using ssmtp or esmtp we can push the messages offline.

One of the gotcha's is NAGIOS uses regular CGI for the browser interface.

if emhttp can be updated to do CGI, then we're good. Otherwise it requires lighttpd or some other lightweight browser.

[np]

+1 for Multiple-Array support

[tmp31416]

While official 64bit support would be nice, I'll have to agree that as far as a NAS appliance goes, there are desired features on the list that have a higher priority. (...) I desire Q-Parity or Multiple-Array support more than official 64bit support.

 

(snip)

 

 

i'll have to chime in, here.

 

i'm with BRiT on this (quoted above).

 

just having recovered from a catastrophic failure at work (long story, can't give details about it, etc.) where it turns out that one of our storage sub-systems was not as robust as it is/was supposed to be, i am now even more sensitive about drive failure.  i.e., the more drives you add, the higher the probability that one or more of the drives will fail, somehow.

 

maybe i'm over-simplifying things, maybe i'm still a bit rattled by what happened at work, but the current single parity drive system used by unraid makes me uncomfortable when you start having servers with more than 12 disks, as the probability of a multiple-drive failure is definitively not null.

 

since it appears that adding a more elaborate data-protection scheme ('q-parity', right?) is not something that can be done easily/quickly, a quick and simple alternative could be to make an unraid server able to deal with multiple arrays.  splitting my server's 12 drives into 2 arrays of 5+1 disks (or, if i went over the top, 3 arrays of 3+1 disks) would be acceptable price to pay if i have a better chance of having my data survive a multiple drive failure.  i mean, the chance that just one array suffers a multiple data drive failure should becomes smaller, right?

 

anyway, i'd vote for the addition of multiple-array support (in lieu of q-parity)  to unraid v5.0  before quite a few other features.

 

cheers.

 

[bubbaQ]

Q-parity is ALWAYS safer than multiple arrays, given the same number of drives and size.

 

A 20-drive array, with both regular parity and Q parity, can recover from ANY 2 drives failing.

 

Two 10-drive arrays can survive a 2-drive failure ONLY if the two drives are on different arrays... a 50-50 chance.

[BRiT]

Can Q-Parity be maintained without spinning up all the other data drives (outside of the data drive, parity drive, and q-parity drive) ?

[tmp31416]

Q-parity is ALWAYS safer than multiple arrays, given the same number of drives and size.

 

A 20-drive array, with both regular parity and Q parity, can recover from ANY 2 drives failing.

 

Two 10-drive arrays can survive a 2-drive failure ONLY if the two drives are on different arrays... a 50-50 chance.

 

point noted, though i have to ask:

(1) wouldn't multiple arrays still be somewhat better than having only one parity drive for, say, 20 drives?

(2) whilst q-parity is still the best upgrade for unraid, how quickly can it be implemented?  people have been talking about it for quite some time and i am not aware it will be "baked in" v5.0...

 

cheers.

 

[nia]

QParity - as brilliant as it would be for some - should at least be made an optional feature, not to force us regular Joes into a world more complex than it needs to be for home use.

[NAS]

we have strayed somewhat

 

Tom wanted to know if anything should be added to beta 3. These would be small things. Debates on big new features should be forked.

 

Also "this is more important than this" discussions are pointless. Everything is more important to some and less to others. We should discuss features and let Tom decide that. Or start a thread and get people to vote.

 

 

 

[BRiT]

Agreed. Although some of the debate started with something that they were hoping would be simple enough to include.

 

I redirected the feature list over here: http://lime-technology.com/forum/index.php?topic=7089.msg73581#msg73581

 

[NAS]

yeah. i didnt intend to be the forum police just polite comment.

[ajeffco]

While official 64bit support would be nice, I'll have to agree that as far as a NAS appliance goes, there are desired features on the list that have a higher priority. (...) I desire Q-Parity or Multiple-Array support more than official 64bit support.

 

(snip)

 

 

i'll have to chime in, here.

 

i'm with BRiT on this (quoted above).

 

just having recovered from a catastrophic failure at work (long story, can't give details about it, etc.) where it turns out that one of our storage sub-systems was not as robust as it is/was supposed to be, i am now even more sensitive about drive failure.  i.e., the more drives you add, the higher the probability that one or more of the drives will fail, somehow.

 

maybe i'm over-simplifying things, maybe i'm still a bit rattled by what happened at work, but the current single parity drive system used by unraid makes me uncomfortable when you start having servers with more than 12 disks, as the probability of a multiple-drive failure is definitively not null.

 

since it appears that adding a more elaborate data-protection scheme ('q-parity', right?) is not something that can be done easily/quickly, a quick and simple alternative could be to make an unraid server able to deal with multiple arrays.  splitting my server's 12 drives into 2 arrays of 5+1 disks (or, if i went over the top, 3 arrays of 3+1 disks) would be acceptable price to pay if i have a better chance of having my data survive a multiple drive failure.  i mean, the chance that just one array suffers a multiple data drive failure should becomes smaller, right?

 

anyway, i'd vote for the addition of multiple-array support (in lieu of q-parity)  to unraid v5.0  before quite a few other features.

 

cheers.

 

 

^^^ This above all else ^^^

 

The ability to be able to split into two arrays, or some kind of double parity (q-parity?)

 

+1

 

 

[bbrodka]

Any possibility of eventually having a hot spare for an array

 

Hot spare would obviously have to be as large as your largest drive (like parity drive)

 

hot spare is spun down and waiting for a drive to fail

 

Should unraid detect a failed drive, it spins up the hot spare drive, removes the failed drive from the array, substituting the hot spare drive in its place and starts a parity rebuild.

 

When the failed drive is eventually physically replaced, it can take on the role of the new hot spare drive