SSD only Unraid

[tweedyc]

QUESTION:

I would like to run Unraid with only SSDs.  I request feedback regarding the following 2 options:

1. SSD only with no parity (#1 Unraid box).  Data recovery to be accomplished by rsync backups to #2 Unraid box.
2. SSD only with SSD parity.  If SSD trim works in 6.12, will this work?  Or will parity updates fail because of parity events conflicting with SSD trim events?

 

UNRELATED TO SSD QUESTION:  How do you improve data I/O to an Unraid box from other boxes (Dell servers, Macs, and WIN) on the network?  What is a list of things to do? 

UNRELATED TO SSD QUESTION GIVENS:

1. All HD and backplanes are 6 GBS. I do not consider 12GBS worth the money at this time. 
2. Files by % of Unraid Array are mostly photos and videos (80%) with the next being documents (15%) and the last being files (5%) compressed/uncompressed.
3. Original NICs (can add fiber card to server(s)).
4. Network is Ubiquity ethernet + fiber.

I am considering:

1. Fiber backbone (already complete)
2. Newer and larger SSD cache drives
3. ?

 

HW ENVIRONMENT

I have multiple 12 bay Dell servers to play with.  My original Unraid was on v4 on a Dell c2100 (still works fine after house fire, moving, and electrical outages) which started on 3TB HD and now has 12TB.  There are no VMs.  It has a few Dockers for rsync, Filezilla, etc.  I have 5 Unraid max licenses running.  The c2100s are running IT Mode LSI cards.

 

Options to play with include 5 Dell r720s, 4 Dell r730s, and one NVME Dell r740 for a SSD only rig.  I have 4 Samsung EVO 8TB SSDs, 10 Samsung EVO 4TB, and some others.

 

Also, I have a fiber backbone in the house.  Although Unraid is not RAM intensive, I have plenty to use; especially on a box running a lot of VMs.

 

SSD ONLY BOX INTENDED USE

1. Multiple VMs for running different OSs (WIN, OSX, Linux)
2. Experiment with low electrical consumption

 

-------------------Thanks in advance for your time in responding.  My search of the forums did not help me.  Either I used the wrong search terms or I did not understand what I was reading.--------------------em

[Jclendineng]

Following, in the exact same boat. I have a r630 I want to move to but don’t want to fill with hdds as ssds are more power efficient and a lot faster plus less moving parts. Unsure how unraid handles ssd’s in an array and if I can even use ssd’s with parity. Need to figure it out, or I’ll just use bare metal podman or rancher or something to manage docker. I couldn’t find anything conclusive on the forums about ssd’s and parity. 

[JorgeB]

Recommend waiting for v6.12 and then use them in a raidz pool.

[Zonediver]

1 hour ago, JorgeB said:

Recommend waiting for v6.12 and then use them in a raidz pool.

 

...and when will it come?

(soon 🤣)

Edited February 1, 2023 by Zonediver

[apandey]

26 minutes ago, Zonediver said:

and when will it come?

I would say "very very soon" 

Soon is what I reserve for things like zfs support for raidz expansion

[tweedyc]

Can someone clarify how raidz answers the mail?  I assume the point i for the raidz pool to be setup for SSDs.  If so, is there a need for a parity drive in this setup?  Also, do all other Unraid functions work normally?  Finally, in raidz do all drives have to be identical?  Can different size drives be used?

[JorgeB]

10 hours ago, tweedyc said:

how raidz answers the mail?

Hmm?

[denishay]

On 2/1/2023 at 9:15 AM, JorgeB said:

Recommend waiting for v6.12 and then use them in a raidz pool.

 

Exactly that. Plus, given the equipment you have + lots of RAM, ZFS would come as the ideal solution to maximize performance/redundancy here. The "current" unraid parity system would only slow down I/O a lot (while still probably being bottlenecked by your 6GB backplanes anyway).

 

Get onto the 6.12 beta "test" branch to try and experiment before you put any real data in there.

[pras1011]

Is there anymore progress on ssd unraid?

[gyto6]

47 minutes ago, pras1011 said:

Is there anymore progress on ssd unraid?

It works. My own system runs only with SSD. No "progress" to be made, it's a simple storage as any other.

[pras1011]

One day I want to get 8 x 8tb once the price drops far enough.

[gyto6]

18 hours ago, pras1011 said:

One day I want to get 8 x 8tb once the price drops far enough.

 

Maybe I should put the question that should have been written in your previous post to keep a link with the thread's title.

Does SSD works with any Unraid configuration?

1 - If you're using a ZFS Pool, no trouble. ZFS manages any Parity/Cache operation. Leave a "dead" storage in the array to start container.

2 - If you're using the SSD within the array, the TRIM shall break the parity, so it's not functionnal.
 

Quote

Trim doesn't work with SSDs assigned to the array, so there might be some performance degradation over time, but parity doesn't break.

AFAIK, no logs about Unraid 6.12 mentionned the SSD TRIM and Parity operation if dedicated to the array.

Edited April 30, 2023 by gyto6

[JorgeB]

18 hours ago, gyto6 said:

2 - If you're using the SSD within the array, the TRIM shall break the parity, so it's not functionnal.

Trim doesn't work with SSDs assigned to the array, so there might be some performance degradation over time, but parity doesn't break.

[tweedyc]

Thanks to everyone who have responded to my questions so far!  Thank you, gyto6 for your input.  I would appreciate some info or guidance about your SSD-only array (e.g., Unraid version, drive sizes, cache drive?, and how you set it up--thanks in advance if you reply).

 

The raidz or ZFS solution prompts new questions:

 

1. Accepting that such a configuration will take care of parity, how is raidz setup? 
2. Is a drive manually dedicated to parity as in typical Unraid?  (see note below regarding gyto6's response)
3. Would someone be kind enough to attach a link with raidz setup steps?  (redundant to #1)
4. Does raidz make cache drive(s) obsolete and/or redundant?
5. Will raidz allow an array that is composed of unmatched drives such as 12 different drive sizes (e.g., 6-1TB drives, 4-2TB droves, and 2-4TB drives) or must all drives be the same size (and specifications)?  Besides price, this was the #1 attraction I found for Unraid which was the ability to grow a drive at a time and as drives got larger (and prices dropped) my first Unraid machines utilized a pile of unused drives that I had from older machines.
6. Has ZFS changed to allow mixed drive sizes WITHOUT IGNORING THE EXTRA VOLUME OF THE LARGER DRIVES?  My current understanding is that ZFS in #5 above would treat all drives as 1TB for a total of 12TB not 18TB.  Please correct me if I'm wrong.

FYSA, If I can get a SSD array working correctly, I will probably use the older Unraid machines as a periodic backup location wherein I start the (spinning drive) Unraid machine(s) periodically, backup from the SSD Unraid, and then shut down the spinning drive Unraid machine. 

 

I understand the limitation of a 6 GBPS backplane and intend to slowly migrate to 12 GBPS backplanes however as we all know, that is an investment in computers and drives.

 

I find gyto6's note interesting, "1 - If you're using a ZFS Pool, no trouble. ZFS manages any Parity/Cache operation. Leave a "dead" storage in the array to start container.".  That sounds like there may be some instructions somewhere that would be helpful to people like me with respect to ZFS so direction to where we can learn would be appreciated.  The "dead storage" sounds like a drive that functions like a parity? drive or maybe not.  How would the dedicated "dead" drive be formatted?

 

I had considered trying a setup with Trim turned off because I saw some other posts that seemed to use such a solution but eventual corruption of a lot of SSDs would be cost prohibitive in the long run.  Thanks to all of you for explaining a bit about raidz.

 

 

[PitRejection2359]

Did you ever get the answers to your questions above?

 

I'm currently running an array that is 100% spinning disks (5 disks all different sizes) and a spinning parity, as well as a 100% SSD BTRFS cache. I'd like to start / transition away from spinning disks if I can, but will likely end up doing this slowly over time as I can afford and when SSDs are on offer, so knowing whether a ZFS array can accept multiple different disks (both different sizes and HDD / SSD combination) would be really helpful.

[pras1011]

I am thinking about this too. Something about no Trim on Unraid. Would having no Trim really be problem on a data drive. Would it be a problem if I write once and read many?

[tweedyc]

PitRejection, I have 2 responses to your email. 

1.  You should be able to see the responses that I've received in this thread. 
2.  The written responses, unwritten responses, and other research has lead me to the following conclusions to resolve this question for me that I will list below (note that my situation may not work for you):

2a)  Since Unraid protects against data loss (on a single machine) by use of 1 or 2 parity drives, a totally SSD based Unraid machine is not very useful since SSDs require trim.  Trimming a SSD parity drive will usually cause loss of parity data (thus defeating the value of having a parity drive).

 

2b) You could have all SSDs for data and a spinning drive for parity but that defeats the purpose of an all SSD machine.

 

2c)  You could do like I do and have 1 or 2 additional Unraid clone machines that rsync from A to B and A to C. (B is located 800 miles away).  This approach makes parity redundant to the clone.

 

2d)  You could abandon Unraid and run Rocky Linux and have a pure SSD machine and either have a clone or typical Raid to protect against data loss.  I am headed in this direction for a pure SSD machine.  I'm lucky since I have lots of servers laying around and can afford to have Unraid and Typical Raid boxes that are independent of each other.

 

2e)  Consider the purpose of a 100% SSD setup.  Such a setup shouldn't be for speed purposes since the backplane limits you to 6 or 12 GBPS.  If the purpose is to reduce electricity consumption consider this:  I noted that my ancient Dell c2100 was only drawing about 120-150 watts once I put it on a Kill-o-watt.  Occasionally, it would spike to 300 watts but that was infrequent.  When it was idle, it dropped to about 80 watts.  That's a light bulb.  The power consumption just didn't matter (the heat was nice in the winter before I put it in the garage).  I did throttle the fans so I didn't have to listen to them roar.

 

FYSA, I do not have my Unraid boxes connected to the internet to destinations for Sonaar, Radaar, etc.  I have separate boxes in separate network DMZs for that purpose.  While that can be done internally on a single Unraid box, I don't need to have one machine do everything.  Also, note that after sync between A and B, I simply shut down B and disable sync for security concerns.  It is not much trouble to turn it on for the next sync.

 

I hope this response was helpful.  Everyone's use case is different and everyone's resources are different.  I'm fortunate to have many servers that I can play with.  I started my NAS journey with QNAP equipment.  That was very expensive.  When I found out that I could have many 2U servers for the cost of a single QNAP (8 bay), I moved to Unraid on Dell c2100 12 bay boxes because of cost and I could take all of my old hard drives and use them as one big disc.  My original c2100 Unraid box is still running after 7 years.  It even survived a house fire (and the fire department).  I'm slowly migrating to Dell R720xd 12 bay machines now that they are so cheap. 

 

 

 

 

 

[itimpi]

16 hours ago, PitRejection2359 said:

Did you ever get the answers to your questions above?

 

I'm currently running an array that is 100% spinning disks (5 disks all different sizes) and a spinning parity, as well as a 100% SSD BTRFS cache. I'd like to start / transition away from spinning disks if I can, but will likely end up doing this slowly over time as I can afford and when SSDs are on offer, so knowing whether a ZFS array can accept multiple different disks (both different sizes and HDD / SSD combination) would be really helpful.

ZFS is not very flexible when it comes to expanding a ZFS pool as it requires all drives in a vdev to be of the same type (if you use mixed sizes then all drives are assumed to be the smallest size).  Also every vdev within a pool has to be of the same 'width' (i.e. no of drives) which makes adding single drives not an option.  No idea if there are any timescales for lifting such limitations.

 

If you want to use mixed drive sizes and add them on demand like you can with the current Unraid array then you should consider using BTRFS instead which DOES already support this.  Ideally this would be done with the 6.13 release in mind when it is expected pools will be able to act as both primary and secondary storage.

[pras1011]

Is it a matter of time until unraid supports ssd drives?

[JorgeB]

SSD drives are supported, the only caveat is that they won't be trimmed if used in the array, so some write performance degradation can occur, alternatively and if you don't require much flexibility expanding/removing devices, use a zfs pool for the SSDs, better performance since it can stripe the devices and has trim support.

[tweedyc]

I forgot to mention ZFS.  I do not consider ZFS as an elegant solution for a 100% SSD Unraid machine.  ZFS has strict constraints that I think are antithetical to Unraid.  One of the major attractions for me to Unraid was the ability to take all of the drives I have laying around and use them with no constraints regarding size or storage capacity.  With 24 bay 2U boxes so cheap nowadays it is attractive to take one and load it up with all of the old SSDs I have laying around and use them. 

 

However, the SSD requirement for trim and the data loss potential due to trim on a SSD parity drive are at cross purposes.

 

I will be using Unraid forever.  One experiment I plan to do is to setup up a 100% SSD Unraid box (Dell 24 bay 720xd) WITHOUT PARITY.  As mentioned in my post above I have 3 Unraid machines (one is 800 miles away) with a complete copy each of the critical data (using spinning drives BTW).  In case of a catastrophic failure, I will clone back to the SSD machine.

 

Again I feel compelled to mention the reason I have a NAS is data access and storage.  While the awesome other features and capabilities available on Unraid can be used to do so many things, I use Unraid almost exclusively as a NAS only.  The capability of the other Apps, Dockers, etc. on Unraid I perform on other boxes.  The data from those are copied to the NAS after thorough scanning. 

 

Once a no-trim SSD comes out, this whole issue will be moot.  Someday....

 

[JorgeB]

34 minutes ago, tweedyc said:

data loss potential due to trim on a SSD parity drive

Not sure what you mean here, trim doesn't work for array devices, so there's no data loss potential because of it.

[PitRejection2359]

Thanks for all the replies. Given all the info, for my particular use case I want to be able to make maximum advantage of the unraid usage efficiency of different sized drives, I can't see a way ahead in Unraid to achieve it with SSDs.

 

Until such a time as ZFS allows 100% usage of drives of different sizes (which seems very unlikely to ever happen), or Unraid develops a way to allow TRIM with SSDs in an array with parity, then I'm going to have to stick to either 1. spinning disks of different sizes with parity, or 2. purchasing same-sized SSD drives for ZFS redundancy, or 3. BTRFS with SSDs of different sizes (effectively RAID 1, which is not as efficient use of space and against the main premise of me using unraid arrays with parity protection).

 

Unless I've completely mis-understood it....!

[pras1011]

Can Unraid eventually support Trim? fstrim command has been available since 2020.

[JorgeB]

2 hours ago, pras1011 said:

Can Unraid eventually support Trim?

Unraid supports trim, just not for the array, since currently that would make parity out of sync.