What is the future of unraid?

[opentoe]

Sorry- this can be moved into the appropriate section. Wasn't thinking straight!

 

Eventually mechanical drives will be phased out. DVD's will be scarce just like the way of cdroms and track tapes. Is unraid going to continue development at this change in the industry? If I'm correct unraid is designed and writen for mechanical drives, correct? Eventually when it will be difficult to even find mechanical drives and SSD's are only available will unraid be ready to support an only SSD system? I read somewhere in the forums that the kernel doesn't support the trim function? Also, doesn't SSD drives wear down quicker than mechanical drives? I'd love to have an all SSD unraid system and not worry about degradation and take advantage of the speeds that we all want.  Been using unraid since version 3, and I hope to be using it for another 10 years. Is there a timeline for the public to view? Recently I just installed an M.2 upgrade in my laptop and it was smaller then a standard memory chip. I'd love to see a 200TB unraid box the size of a mini PC running cool and fast. I hope I'm alive to see it all happen!

 

Edited June 9, 2020 by opentoe

[BRiT]

The SSD drives will/should support deterministic trim, so a slight tweak to the md drivers used by unRaid and all will be fine.

[Storx]

I have been curious of this myself, SSDs are slowly dropping in price and we are starting to see enterprise used drives pop up on the used market. 

 

I am fairly new to Unraid, i started out my testing of Unraid with 2 x M.2 NVME drives (Parity and Disk 1) to test things out on the trial while i shopped for sales on the shuckable WD drives and after i replaced the parity and disk 1. I noticed there was practically no difference in transfer speeds between using the M.2 as parity and disk vs mechanical HHDs. I downloaded the trim plugin and asked questions about it on reddit, but noone could answer why i was seeing similar speeds. It was only after adding one of the M.2s as a cache drive that i saw nearly 3x the transfer speeds. So it seemed like the parity ran as a set speed non-dependent of which type of drive slotted in place. 

[itimpi]

On 6/9/2020 at 11:56 PM, Storx said:

I have been curious of this myself, SSDs are slowly dropping in price and we are starting to see enterprise used drives pop up on the used market. 

 

I am fairly new to Unraid, i started out my testing of Unraid with 2 x M.2 NVME drives (Parity and Disk 1) to test things out on the trial while i shopped for sales on the shuckable WD drives and after i replaced the parity and disk 1. I noticed there was practically no difference in transfer speeds between using the M.2 as parity and disk vs mechanical HHDs. I downloaded the trim plugin and asked questions about it on reddit, but noone could answer why i was seeing similar speeds. It was only after adding one of the M.2s as a cache drive that i saw nearly 3x the transfer speeds. So it seemed like the parity ran as a set speed non-dependent of which type of drive slotted in place. 

There should be a difference if you have NO mechanical drives in the array.    However the moment you introduce mechanical drives into the mix then much of the speed benefits of SSD will disappear as any write to the array which involves a mechanical drive as one of the participants in a write operation it will be slowed to the performance op that mechanical drive.

[opentoe]

I'd love to see a roadmap. I know cloud based services are getting more and more popular, hey why not, throw our little USB 2.0 stick that boots our system and use PXE.

[meep]

2 hours ago, opentoe said:

I'd love to see a roadmap. I know cloud based services are getting more and more popular, hey why not, throw our little USB 2.0 stick that boots our system and use PXE.

Wouldn’t be my cup of tea. As an option, maybe, but just because lots of services are moving to the cloud doesn’t mean that every service has to.

 

if I have a problem with my internet, I can’t boot or access my storage? No thank you! Cloud is good, and has benefits in lot of cases. Local is better for some things still. 

[TSM]

My 2 cents, is that even with streaming services and cloud storage becoming more and more popular, there will always be a use case for having your own media stored on a storage medium that you own and have physical control over.

 

As far as SSD's over mechanical drives are concerned, the technical reasons mechanical drives are preferred has been discussed in this forum before.  As for me though, even if those technical issues were overcome tomorrow, I'd probably stay with mechanical drives until the cost\storage ratio of SSD's came in line with mechanical drives.  

 

[opentoe]

I know exactly what you guys are saying but I'm thinking a little more ahead. 15 years ago users data probably was only saved where they live. Consumers I'm talking about. Now 15 years later thin about how much of that data never touch their computer. We will have the data that only controls our devices that access that data in the cloud. To turn it into a silly metaphor we will have all our drive maps, shares, even computer will all be running and accessed in the "cloud" without us even knowing it. It will happen.

 

[sota]

not for me it won't.

I will NEVER trust the cloud for anything of importance.  hell I won't even trust the cloud for things I don't care about.

I recently just started a "cloud" backup service for some of my clients, but only because *I* host the cloud server and *I* know exactly what human needs to be contacted when things blow up.... ME.

[mfwade]

2 hours ago, sota said:

not for me it won't.

I will NEVER trust the cloud for anything of importance.  hell I won't even trust the cloud for things I don't care about.

I recently just started a "cloud" backup service for some of my clients, but only because *I* host the cloud server and *I* know exactly what human needs to be contacted when things blow up.... ME.

I would be curious to know what you have done. I support multiple non-profits and set up a few Minio - S3 instances that they can use. Seems to work ok however, I am always looking to refine what I can provide to them.

 

-MW

[PeteAsking]

My Unraid setup only has SSD drives in it which is why I am using the beta with Kernel 5. Works fine if the filesystems are all btrfs. Im guessing this will be incrementally improved as each new version comes out, so not really sure what the issue is since it already works fine if you want it to.

 

Edit: one disk is an M.2 also, not sure if that helps. I only use the one since I want my reads for VM's on it to be fast.

Edited June 17, 2020 by PeteAsking

[sota]

17 minutes ago, mfwade said:

I would be curious to know what you have done. I support multiple non-profits and set up a few Minio - S3 instances that they can use. Seems to work ok however, I am always looking to refine what I can provide to them.

 

-MW

There's been a couple products to come out, but the one I'm using currently is called Syncrify.  It fit the best with my needs and concerns, although the company can be ... interesting ... to deal with sometimes.

[tr0910]

Tom just hinted a roadmap for storage that might include ZFS as a native filesystem in the 6.9.0 beta 22 release notes.

 

A future release will include support for multiple "unRAID array" pools.  
We are also considering zfs support. 

I have been playing around with the ZFS plugin, and have been fascinated with the possibilities.  The multiple cache pools included in beta22, and multiple unRaid array pools promised for a future release all are linked to keeping unRaid relevant.  We have a few more years before SSD will be cheaper per TB than spinning disks.  I would expect both SSD and mechanical drives to continue to decrease in price per TB in the foreseeable future.  What it would take to eliminate mechanical drives is a disruptive technology or a new "Dark Age" to be thrust upon us.  If that happens, more than hard drives will become paperweights.

Edited June 17, 2020 by tr0910

[hernandito]

If an 80TB cloud service storage would be affordable, I would only use it for a backup.... and that means that bandwidth would have to increase 10x what it’s currently. I think we are looking at 20+ years out. 20 years we were still on dial-up modems at home.

[hernandito]

On 6/17/2020 at 4:57 PM, tr0910 said:

We have a few more years before SSD will be cheaper per TB than spinning disks.  I would expect both SSD and mechanical drives to continue to decrease in price per TB in the foreseeable future.  What it would take to eliminate mechanical drives is a disruptive technology or a new "Dark Age" to be thrust upon us.  If that happens, more than hard drives will become paperweights.

I look forward to that day. A much quieter server would be a cool thing.

 

i also think that 4K would have to be more prevalent. Does this mean better compression tech? Or dynamically cheaper storage. A 1080 movie weighs average around 10GB. A 4K weighs in at around 25GB. Please correct me if I’m wrong.... not an expert on H265. So the per TB cost would really have to be low.

 

 

[sota]

I have long advocated for someone to create an "archival" SSD storage device that's just fast enough to stream 4K video, it gigantic, uses as little power as possible (NVRAM?), is as reliable and resilient as possible, and is as cheap as HDD in the $/TB factor.  I'm willing to sacrifice write speeds to the tune of up to 1/2 read speeds, to make the price point happen.  I pretty much want SSD near-WORM storage.

[BRiT]

I for one am waiting on the molecular data cubes shown ages ago.

[CrashnBrn]

I was going to post a similar question but I'll just add it here. @limetechI'm curious what business direction unraid is heading. Initially I always felt that unraid was a consumer/prosumer product but it feels like they are starting to target small/medium business with the new features planned. If so, is that something that came about due to more business interest after LTT's videos? Or am I just overthinking things?

[limetech]

On 6/8/2020 at 7:22 PM, opentoe said:

Sorry- this can be moved into the appropriate section. Wasn't thinking straight!

 

Eventually mechanical drives will be phased out. DVD's will be scarce just like the way of cdroms and track tapes. Is unraid going to continue development at this change in the industry? If I'm correct unraid is designed and writen for mechanical drives, correct? Eventually when it will be difficult to even find mechanical drives and SSD's are only available will unraid be ready to support an only SSD system? I read somewhere in the forums that the kernel doesn't support the trim function? Also, doesn't SSD drives wear down quicker than mechanical drives? I'd love to have an all SSD unraid system and not worry about degradation and take advantage of the speeds that we all want.  Been using unraid since version 3, and I hope to be using it for another 10 years. Is there a timeline for the public to view? Recently I just installed an M.2 upgrade in my laptop and it was smaller then a standard memory chip. I'd love to see a 200TB unraid box the size of a mini PC running cool and fast. I hope I'm alive to see it all happen!

 

SSD support in Linux and associated file systems has been in place quite a while and is quite mature at this point.  However don't be so quick to discount advances in spinners - that industry is not going away quietly and probably will be several years before we see the last of them.

 

With 6.9 we have introduced "multiple pools".  At present this only supports btrfs pools but much of the underlying work has been done to support other types of pools (that is, formatted with a file system other than btrfs).  Along with this, it gives us a path to generalize  pools further and let you define multiple "unRAID" pools.  This work will require changes in the unRAID kernel driver, and is naturally the time to address SSD devices in the unRAID array, as well as a few other improvements.  How this work is phased into future releases is T.B.D.

[gacpac]

On 6/23/2020 at 3:16 PM, CrashnBrn said:

I was going to post a similar question but I'll just add it here. @limetechI'm curious what business direction unraid is heading. Initially I always felt that unraid was a consumer/prosumer product but it feels like they are starting to target small/medium business with the new features planned. If so, is that something that came about due to more business interest after LTT's videos? Or am I just overthinking things?

don't trust everything you see at LTT. He's good at the consumer and prosumer stuff but not necessarily the business side. Unraid is a solid product, but there's several vendors that might be hard to compete with. It would be awesome to see Unraid at the enterprise level and be a certified Unraid expert stuff like that, I'm thinking years down the road.