Jump to content
JamesAlexander

Setting up NAS and need to know how Unraid works

18 posts in this topic Last Reply

Recommended Posts

Hi,

I'm in the process of building a NAS at home for my storage needs.
I take a lot of photos and want redundancy to prevent file loss.

It would also be great to have a media server and backup my smartphone remotely.

 

Now, some questions.

  • How does Unraid save data exactly?
  • Does it use the principle of drive pooling?
  • Can a single drive be taken out and read like a normal one?
  • How is data loss prevented?
  • What advantages would it have over Stablebit Drivepool?

 

Thank you in advance for your insights.

Edited by JamesAlexander

Share this post


Link to post

I'll answer in a slightly different order since it makes more sense that way.

29 minutes ago, JamesAlexander said:
  • Can a single drive be taken out and read like a normal one?

Yes each disk is an independent filesystem containing complete files that can be accessed on any linux.

29 minutes ago, JamesAlexander said:
  • How does Unraid save data exactly?
  • Does it use the principle of drive pooling?

Each disk is an independent filesystem, so each disk contains complete files. But folders can span disks, and these spanned folders are presented as network shares.

31 minutes ago, JamesAlexander said:
  • How is data loss prevented?

1 (or 2) parity disk allows Unraid to calculate the data for 1 (or 2) missing disk to be calculated from parity plus all remaining disks. Here is the wiki on single parity:

https://wiki.unraid.net/UnRAID_6/Overview#Parity-Protected_Array

Dual parity is similar but the parity2 calculation is independent and somewhat more complicated.

 

But, parity is NOT a substitute for backups. Plenty of ways to lose files besides disk failure, including simple user error. You must always have another copy of anything important and irreplaceable.

38 minutes ago, JamesAlexander said:

What advantages would it have over Stablebit Drivepool?

Not familiar with that solution so I will just mention some advantages Unraid has over RAID.

 

In addition to each disk being an independent filesystem which can be accessed by itself on any linux, Unraid also allows different sized disks to be mixed in the parity array, and you can easily replace or add disks without rebuilding the entire array.

 

The tradeoff vs RAID is exactly that each disk is an independent filesytem, there is no striping so each file can only be accessed at single disk speed instead of striped RAID where multiple disks are accessed in parallel. Writes are actually somewhat slower than single disk speed since Unraid maintains parity in realtime.

 

There is also cache or cache pool, optionally SSDs, which can allow faster I/O, including writes which are later moved to the slower parity array.

Share this post


Link to post
6 hours ago, JamesAlexander said:

Hi,

I'm in the process of building a NAS at home for my storage needs.
I take a lot of photos and want redundancy to prevent file loss.

It would also be great to have a media server and backup my smartphone remotely.

 

Now, some questions.

  • How does Unraid save data exactly?
  • Does it use the principle of drive pooling?
  • Can a single drive be taken out and read like a normal one?
  • How is data loss prevented?
  • What advantages would it have over Stablebit Drivepool?

 

Thank you in advance for your insights.

Did a little write up w while back hope it helps:

 

https://www.makeuseof.com/tag/unraid-ultimate-home-nas/

Share this post


Link to post

@techsperion, thank you for all this very useful information.

Would I be correct in saying that Unraid stores a given file as copies on multiple drives?

Also, is it possible to skip copies or redundancy for some unimportant files that I can afford to lose?

Edited by JamesAlexander

Share this post


Link to post
50 minutes ago, JamesAlexander said:

Would I be correct in saying that Unraid stores a given file as copies on multiple drives?

No. There is only one drive that holds any particular file.

Share this post


Link to post
50 minutes ago, JamesAlexander said:

Would I be correct in saying that Unraid stores a given file as copies on multiple drives?

Also, is it possible to skip copies or redundancy for some unimportant files that I can afford to lose?

No - a file is only ever stored in one place.

 

Redundancy against hardware failure (in particular disk failures) is a separate topic and functions independently of individual files.

Share this post


Link to post
@techsperion, thank you for all this very useful information.

Would I be correct in saying that Unraid stores a given file as copies on multiple drives?

Also, is it possible to skip copies or redundancy for some unimportant files that I can afford to lose?

 

Unraid stores a single file. However there’s enough data on the parity drive alone to recover that file if it got destroyed as part of a drive failure.

 

I’m not familiar with ignoring files but there’s some things to note:

 

1) you could insert a drive and not add it to the array. Then store all your not so useful files here. I believe you can do this as part of the unassigned devices plugin.

 

2) the cache is also a drive that you can choose to have unprotected files on. When you create a share you choose weather you want unRAID to write to the cache first then move it to the protected array later. Or skip the cache altogether, or only use the cache.

 

There’s way more experienced people that patrol this forum who will have better insight than me smile.png&key=d7bc0bbbfa2056a2a63af437c1f4a1188d2d7eb0deb5aa11e8f34aa88765e668

Share this post


Link to post
1 hour ago, techsperion said:

 

Unraid stores a single file. However there’s enough data on the parity drive alone to recover that file if it got destroyed as part of a drive failure.

 

Sorry - but the second sentence is not correct and is actually misleading.  @trurl already put it very succinctly...

 

13 hours ago, trurl said:

1 (or 2) parity disk allows Unraid to calculate the data for 1 (or 2) missing disk to be calculated from parity plus all remaining disks. Here is the wiki on single parity:

https://wiki.unraid.net/UnRAID_6/Overview#Parity-Protected_Array

Dual parity is similar but the parity2 calculation is independent and somewhat more complicated.

 

To rewrite your original statement... However there’s enough data on the parity drive in conjunction with the data on all of the the other data drives which do not contain that file to recover that file if it got destroyed as part of a drive failure.

Edited by S80_UK
Typo corrected

Share this post


Link to post

To be pedantic, it can not recover at the file level, it is at the device level that just happens to contain files. Unraid and Parity is not able to recover previous versions of files. One still needs to plan for backups accordingly.

Share this post


Link to post
 
To rewrite your original statement... However there’s enough data on the parity drive in conjunction with the data on all of the the other data drives which do not contain that file to recover that file if it got destroyed as part of a drive failure.


Ah so it’s like traditional RAID in that sense. You learn something new every day :) ta!

Share this post


Link to post
4 hours ago, techsperion said:

However there’s enough data on the parity drive alone to recover that file

And if you think about it for a moment, it would not be possible for parity by itself to recover a file from any possible disk that might fail. For that to happen, parity would effectively be a backup of the complete array. There is no magic that would allow it to have the capacity for that.

Share this post


Link to post
12 hours ago, jonathanm said:

No. There is only one drive that holds any particular file.

Ok, so this is like software RAID5? This is where parity data fits into the picture?

11 hours ago, techsperion said:

 

Unraid stores a single file. However there’s enough data on the parity drive alone to recover that file if it got destroyed as part of a drive failure.

 

I’m not familiar with ignoring files but there’s some things to note:

 

1) you could insert a drive and not add it to the array. Then store all your not so useful files here. I believe you can do this as part of the unassigned devices plugin.

 

2) the cache is also a drive that you can choose to have unprotected files on. When you create a share you choose weather you want unRAID to write to the cache first then move it to the protected array later. Or skip the cache altogether, or only use the cache.

 

There’s way more experienced people that patrol this forum who will have better insight than me smile.png&key=d7bc0bbbfa2056a2a63af437c1f4a1188d2d7eb0deb5aa11e8f34aa88765e668

Right, so there is a way to store but not protect against drive failure.

I will find these more experienced people.

10 hours ago, S80_UK said:

To rewrite your original statement... However there’s enough data on the parity drive in conjunction with the data on all of the the other data drives which do not contain that file to recover that file if it got destroyed as part of a drive failure.

How does the data on each drive on top of parity function?

How many drive failures can the system handle? One or more? I suppose this depends on what resources are available for parity?

9 hours ago, BRiT said:

To be pedantic, it can not recover at the file level, it is at the device level that just happens to contain files. Unraid and Parity is not able to recover previous versions of files. One still needs to plan for backups accordingly.

My question comes from what I understand of drive pooling which is just file level copies on separate drives.

So no versioning like with Freenas/ZFS.

I have dropped FreeNAS because it is apparently necessary to setup the final system and it is also difficult to grow storage.

Unraid seems more lightweight and flexible.

7 hours ago, trurl said:

And if you think about it for a moment, it would not be possible for parity by itself to recover a file from any possible disk that might fail. For that to happen, parity would effectively be a backup of the complete array. There is no magic that would allow it to have the capacity for that.

I will read more about parity.

This is a kind of error detection?

 

 

Edited by JamesAlexander

Share this post


Link to post
I will read more about parity.
This is a kind of error detection?
 
 

RAID5 splits the data over the array. A single file can belong on many drives. That’s not the case with Unraid. Unraid stores a single file On a single drive.


Your array can suffer the loss of however many parity drives you have.

Scenario: you have a total of 5 drives with the parity drive equal to or larger in size than the largest drive.

Eg:
Parity >= 3TB
Drive1 = 3TB
Drive2 = 3TB
Drive3 = 3TB
Drive4 = 3TB

If one drive dies and you slot in another, you’ll recover your data.

If you have 2 parity drives:

Parity1 >= 3TB
Parity2 >= 3TB
Drive1 = 3TB
Drive2 = 3TB
Drive3 = 3TB

You can lose two drives.

Parity in a nutshell is a mathematical computation of the data stored across the array. It uses just binary but to make it easier to understand:

If you have an equation:

1 + 1 + 1 + x = 3

What will x be equal to? I’m over simplifying it but you get the picture. Just Imagine those as your drives.



FreeNAS can grow the array size but it’s recommended that you do all the drives which is expensive. So if you had the scenario above you’d unplug the first drive, then insert a new larger one and let the array resilver / rebuild. Then unplug the next drive, plug in a larger one and let the array resilver. Rinse and repeat.

In contrast Unraid doesn’t really care that you have mixed drive sizes, as long as your parity is your largest drive you’ll be golden.

A lot of this is covered in the article I published above

Share this post


Link to post

Also you may have come across this statement before:

RAID be it on FreeNAS / Unraid / the millennium falcon is not a substitute for a good backup strategy.

RAID is more of a high availability / performance thing. If you have a treasured repository of cat memes that you absolutely cannot lose, make sure you have thought about on-site / off-site backups.

Parity must not be relied on to keep your data safe.

Sorry to come across as a little alarmist but this must be said :)

Share this post


Link to post
4 hours ago, techsperion said:

as long as your parity is your largest drive

Doesn't have to be the largest, each parity drive has to be at least as large as the largest single data drive.

Share this post


Link to post
9 hours ago, JamesAlexander said:

Right, so there is a way to store but not protect against drive failure.

I will find these more experienced people.

Don't really understand that first sentence. And it was in response to a quote that had some things wrong as far as how parity works.

 

As for those more experienced people, some of them have already replied on this thread.

 

9 hours ago, JamesAlexander said:

I will read more about parity.

This is a kind of error detection?

Parity can detect and correct a single bit error. Dual parity can detect and correct 2 bit errors.

 

Parity is very common in computing and communications, and it is always conceptually the same though the specific implementation may differ in some respects. If you have a group of bits, determine if their sum is even or odd then include an additional bit that reflects that even or odd. Then if a bit is missing you can figure out what it was based on whether it should be even or odd.

 

I already posted a link to the wiki on parity above.

 

 

 

Share this post


Link to post

And I missed this one

22 hours ago, jonathanm said:

No. There is only one drive that holds any particular file.

9 hours ago, JamesAlexander said:

Ok, so this is like software RAID5? This is where parity data fits into the picture?

No it is not like RAID5, what he was describing is one of the things that makes Unraid very different from any RAID.

Share this post


Link to post

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.