My New Unraid NAS Discussion


Recommended Posts

Hello all, 

 

I am new to the forum and I am investigating creating a NAS system by reusing some old hardware and buying some new stuff.

 

I am in the UK and we don't have anything like micro center, there is Currys but they aren't always the best for price.  

 

Old Kit, 

CPU: Intel Xeon x5670 or i7 920

Ram: 12gb (6x2gb) Patriot Viper DDR3 1600

GPU: Sapphire Radeon hd 5850 2gb

MoBo: Gigabyte GA-X58A-UD7

PSU: EVGA GQ 750

SSD: Kingston A400 120gb

Case: Coolermaster MasterBox 5 (with some drive caddies from my other cases)

External drives: 1x WD 4tb 1x Samsung 2tb

 

New Kit, 

HDD's: Unraid 4x 4tb drives with 2 of these for parity. I was expecting to need 6x 3tb RAIDZ2 drives for Freenas. 

 

My current setup is 3x desktops (me, the wife, my son) 2x smart tv's, 2x laptops one belongs to the wife and one is used in the workshop. Currently the data is split between the desktops with the wifes one of acting as a Plex and photo server. I have a backup on this desktop onto one of the external drives and my desktop backs up onto the other external drive but it is a bit of a pain checking that these have run and I have duplicate information on the systems. I want to bring this all together into one system so it is easier to maintain and accessible from all the machines with some restrictions for some users. I use Google cloud to back up my photos in addition to the local backup but it is expensive to increase the storage capacity. 

 

I currently have about 7tb of data over the machines but some of this is duplicated, my amount of data doesn't increase very rapidly and the 1tb overhead will last probably 8-12 months. 

 

I have a couple of questions that will help with my decision as to which way to go. 

 

Do you think the hardware above is ok for running Unraid?

Can I set up the system to sleep over night and in the day when there is minimal usage? 

How do the two systems compare in relation to drive failure?

How does windows see the drives, can I set it up as a single drive and have folders within this that have some restrictions?

How is the data written to the drives for example if I put a folder with 4 documents inside are these split across the drives? 

How easy is it to expand the pool?

How easy is it to handle a drive failure? 

Can I run a backup to an external drive over USB? 

What should I do with the SSD (I was going to install Freenas on it) could I use is as a cash drive or should I get another one for raid 1 before I do this? 

How does a cash disk affect the system? 

What is the recommended way to get disks, should I but some NAS drives (I am looking at Ironwolf drives) or get them from enclosures?

Can I run 2x USB sticks and set one as a backup for the other?

 

Sorry for the mass of questions, I am currently thinking that Unraid might be better because I can expand the pool and that I can get some of my information back even if there is a total failure of the system. 

 

Thanks in advance. 

 

Edited by MattC69
  • Like 1
Link to comment
42 minutes ago, MattC69 said:

How does windows see the drives, can I set it up as a single drive and have folders within this that have some restrictions?

User Shares are the preferred way. They are folders that span drives. Each user share is a top level folder on one or more disks with the same folder name. Each user share has settings that control access and how Unraid uses the disks for that share.

 

44 minutes ago, MattC69 said:

How is the data written to the drives for example if I put a folder with 4 documents inside are these split across the drives?

Each array disk is an independent filesystem that can be easily read on any Linux. Each file exists completely on a single disk. Folders can span disks as explained above.

 

45 minutes ago, MattC69 said:

How easy is it to expand the pool?

Just add a new disk to a new slot. Or you can increase storage by replacing a disk with a larger disk and letting it rebuild the contents. Each data disk can be no larger than any single parity disk.

 

47 minutes ago, MattC69 said:

How easy is it to handle a drive failure? 

Just replace the disk and its contents will be rebuilt to the new disk.

 

48 minutes ago, MattC69 said:

Can I run a backup to an external drive over USB? 

Yes, the Unassigned Devices plugin.

 

49 minutes ago, MattC69 said:

What should I do with the SSD (I was going to install Freenas on it) could I use is as a cash drive or should I get another one for raid 1 before I do this? 

How does a cash disk affect the system? 

Cache was originally used to speed up writes to the server, since they are not affected by parity update speed. Then during scheduled idle time the cached writes are moved to the array. Cache is now used in other ways, to increase performance of dockers and VMs for example, with some things meant to stay on the faster cache. Each user share has settings for how it uses cache.

 

52 minutes ago, MattC69 said:

Can I run 2x USB sticks and set one as a backup for the other?

If you mean in some sort of failover role, no. In fact, I see no point in having another "boot" flash plugged in since it could confuse things and can't be used as failover anyway. Better to just keep a backup of flash on another computer. You can download a zipped backup at any time, and there are plugins to automate flash backup. Then you can prepare a new flash from the backup.

 

Have you read the Product pages linked at the top of all forum pages?

 

Here is a link to the wiki Overview:

 

https://wiki.unraid.net/UnRAID_6/Overview

  • Like 1
Link to comment
1 hour ago, MattC69 said:

I am in the UK and we don't have anything like micro center, there is Currys but they aren't always the best for price.  

 

Old Kit, 

CPU: Intel Xeon x5670 or i7 920

Ram: 12gb (6x2gb) Patriot Viper DDR3 1600

GPU: Sapphire Radeon hd 5850 2gb

MoBo: Gigabyte GA-X58A-UD7

PSU: EVGA GQ 750

SSD: Kingston A400 120gb

Case: Coolermaster MasterBox 5 (with some drive caddies from my other cases)

External drives: 1x WD 4tb 1x Samsung 2tb

 

New Kit, 

HDD's: Unraid 4x 4tb drives with 2 of these for parity. I was expecting to need 6x 3tb RAIDZ2 drives for Freenas. 

 

For PC parts, don't even touch Currys with your toe.

There are many reputable PC parts dealers such as Scan, More Computer, CCL, just to name a few.

Amazon is also fine (just don't buy HDD from them due to their environmental friendly packaging that lacks padding).

 

With regards to HDD, 4TB and 8TB are basically the same in terms of price / GB so you should get 2x 8TB instead 4x 4TB. HDD fails in probabilistic manner so the more drives you have, the more likely you will have a failed one.

 

Now your questions:

 

1. Do you think the hardware above is ok for running Unraid?

As a NAS - sure. I have had Unraid running on older stuff than i7 920.

 

2. Can I set up the system to sleep over night and in the day when there is minimal usage? 

Can't send it to sleep - I have never managed to do that.

Can script to shutdown at a certain time. Waking it up, however, depends on hardware support and your skill level (to set up Wake-On-LAN for example)

 

3. How do the two systems compare in relation to drive failure?

FreeNAS is a RAID architecture which has 1 fundamental issue - if you have more failed drives than the number of parity (e.g. 3 failed drive on a Z2, which has 2 parity drives), you will lose ALL your data.

Unraid, in contrast, if you have more failed drives than the number of parity, you will only lose data on the failed drive (i.e. whatever is saved on the good drives will still be good).

 

4. How does windows see the drives, can I set it up as a single drive and have folders within this that have some restrictions?

via SMB i.e. network share.

Each share is a folder that spreads across the drives (or not, depending on how you set things up) and has user access restrictions.

 

5. How is the data written to the drives for example if I put a folder with 4 documents inside are these split across the drives? 

It all depends on your split level settings, allocation method and include/exclude disk. It could be 1 file per drive or 2 on 2 drive each or 4 on 1 drive.

 

6. How easy is it to expand the pool?

Very easy. There's a wiki for that.

 

7. How easy is it to handle a drive failure? 

There's a wiki for that.

 

8. Can I run a backup to an external drive over USB? 

Yep.

 

9. What should I do with the SSD (I was going to install Freenas on it) could I use is as a cash drive or should I get another one for raid 1 before I do this? 

Cache drive is fine. You may not even need it for cache if you don't run VM / docker. Modern drives tend to be limited by the network speed.

 

10. How does a cash disk affect the system? 

It depends.

If you have docker / VM (docker for Unraid is like an app for your phone) then you should have a cache drive for the docker image and appdata. Without cache, things can get very slow.

As a pure NAS (i.e. just a network folder for people to save stuff on - without docker / VM), a cache drive will improve write speed but only up to the capacity of the cache drive (then you have to run mover to move data from cache to array, either as a schedule or manually). For most home networks, the network speed is the bottleneck and not the array write speed so you won't be able to utilise the faster write speed anyway; hence, there's no need for a cache drive.

 

11. What is the recommended way to get disks, should I but some NAS drives (I am looking at Ironwolf drives) or get them from enclosures?

For Unraid, just get the cheapest you can get for the desired capacity from a reputable dealer (except Amazon due to shipping issue I said above).

There is no need to get "NAS" or "Enterprise" or anything like that.

Toshiba 8TB HDD are quite cheap right now.

 

12. Can I run 2x USB sticks and set one as a backup for the other?

No. You can manually backup the USB stick (or automate it using plugin / script) to cache / array.

However, you can't have 2 USB sticks as live backup of each other.

Your license is based on the GUID of the USB stick so if your stick fails, you need to follow to stick replacement procedure as outlined by Limetech.

Edited by testdasi
  • Like 1
Link to comment

Hi all, 

 

I have had a quick look through the wikki pages but I am going to look in a bit more detail later. 

 

I would prefer having some parity of the disks, i would expect that 2x 8tb would be almost the same as a raid 1. 

 

I think I am going to try and get some disks this weekend and start the setup using the trial version. 

 

Do you know if the disks park when not in use?

Link to comment

With 1 parity disk assigned to parity1, and one data disk, yes, the operation is pretty much a RAID1. Any other configuration reverts to the general parity disk calculations, with the associated speed penalties.

 

Disks can be configured to spin down when not being actively accessed. However, with only 2 disks and no cache drive defined, any docker container or VM services will keep the array spun up constantly.

  • Like 1
Link to comment
28 minutes ago, MattC69 said:

I would prefer having some parity of the disks, i would expect that 2x 8tb would be almost the same as a raid 1. 

A 2-drive array (1 data 1 parity) behaves like a RAID-1 - it's the beauty of math.

It doesn't mean having a 2-drive array is NOT having parity.

 

You should not try to have more drives just for the sake of "having some parity".

Parity is there to help you when a drive fails.

It's better to have a lower probability of a drive failure so you don't need to use parity to begin with.

Backblaze also reported that newer larger capacity drives seem to have better reliability than older smaller capacity drives.

 

Start small with big drives will also help with future expandability.

If you start with 4x4TB now, when you next expand, you can only get 4TB or smaller drives. If you get a larger drive, it will have to be put in the parity slot (remember: no data drive can be larger than the parity drive).

SATA ports are limited. HDD mount points are limited. 4TB will eventually go way of the Neanderthals.

  • Like 1
Link to comment

Hi all, I have ordered my 3x4tb drives. I intend to add 2 more drives probably WD red pro drives and then use them as the parity drives. My drives should be arriving tomorrow and I have gone through and started the setup of Unraid without the array. 

I have come up with a few questions and I hope someone will be kind enough to point me in the direction of the answer. 

 

The first is about security and passwords, I have setup some user accounts each with a password on the server, obviously these don't have any shares associated with them yet because there is no array. I have also setup a password for the root account. Is the server visible to the internet and with the passwords set is it secure? I don't have anything like a firewall other than what is in the Virgin Media router. I don't currently plan on having access to the server remotely at the moment and only on computers on the LAN will have access. 

 

I haven't got to this point yet but how do I access the shares on windows 10, I haven't seen anything on the wikki about this. do i just map to the shares in win 10? If I do this how do the passwords work for the users which have access to these shares? Is there a single sign on/ can windows save the logon information? 

 

How do I setup backup to an external USB hard drive, and does this drive count towards the licence?

 

Finally is there a way to identify the drives in the array so if I get a failure I know which one it is, I was thinking or adding a disk and marking it, shutting down, adding a second disk and marking it and so on?

 

Are there any recommended addons I should install?

 

Edited by MattC69
  • Like 1
Link to comment
1 hour ago, MattC69 said:

The first is about security and passwords, I have setup some user accounts each with a password on the server, obviously these don't have any shares associated with them yet because there is no array. I have also setup a password for the root account. Is the server visible to the internet and with the passwords set is it secure? I don't have anything like a firewall other than what is in the Virgin Media router. I don't currently plan on having access to the server remotely at the moment and only on computers on the LAN will have access. 

The server should never be exposed directly to the internet as it is not hardened for that.   If you want to access it remotely then it is recommended that you use the built-in WireGuard VPN facility.

 

1 hour ago, MattC69 said:

I haven't got to this point yet but how do I access the shares on windows 10, I haven't seen anything on the wikki about this. do i just map to the shares in win 10? If I do this how do the passwords work for the users which have access to these shares? Is there a single sign on/ can windows save the logon information? 

As far as Windows is concerned Unraid is just a network device exposing shares.   There is no concept of a single sign on but Windows can remember passwords.   In fact with Windows one of its quirks is that you cannot have a single user signed on twice at the same time to the same server with different passwords.  If you have any problems in connecting from Windows then ask in the forum as Microsoft keep changing things at the windows end via their updates which can cause unexpected behavior

 

1 hour ago, MattC69 said:

How do I setup backup to an external USB hard drive, and does this drive count towards the licence?

Any drive plugged into the server at the time you start the array counts regardless of whether Unraid is using it or not.   Drives plugged in after array start do not count but it is convenient if your license is of a tier that allows you to leave such drives plugged in.

 

There is no (currently anyway) any built-in backup capability, but running your own script to do this is easy enough via the User Scripts plugin.  There are also a number of docker based solutions that many user use.

 

1 hour ago, MattC69 said:

Finally is there a way to identify the drives in the array so if I get a failure I know which one it is, I was thinking or adding a disk and marking it, shutting down, adding a second disk and marking it and so on?

Unraid identifies drives via their serial number.  so if this is not easily visible many users attach a label giving the last few digits of the serial.

 

1 hour ago, MattC69 said:

Are there any recommended addons I should install?

The one that is deemed essential is the Community Applications (CA) plugin as this adds App store type capability that can be used for finding and installing any further plugins and/or docker containers.

 

Unassigned Devices (UD) will help with managing the USB drives you mentioned

Fix Common Problems (FCP) will help identify mistakes that are frequently made in setup.

User Scripts provides an easy way to run scripts either on demand or on a schedule.

 

Their are plenty of other addons available and many users will have their own views of what they consider essential.   If in doubt ask in the forums.

  • Like 2
Link to comment
3 hours ago, MattC69 said:

Is the server visible to the internet and with the passwords set is it secure? I don't have anything like a firewall other than what is in the Virgin Media router. I don't currently plan on having access to the server remotely at the moment and only on computers on the LAN will have access. 

You have to purposely set things up with the Virgin Media router to expose your network to the Internet so unless you have done something, your server isn't visible by default.

  • Like 1
Link to comment
1 hour ago, testdasi said:

You have to purposely set things up with the Virgin Media router to expose your network to the Internet so unless you have done something, your server isn't visible by default.

That's great thanks, I was coming to that conclusion and I was going to check some settings. 

 

Again thanks for everyone's reply's

Link to comment
3 hours ago, testdasi said:

You have to purposely set things up with the Virgin Media router to expose your network to the Internet so unless you have done something, your server isn't visible by default.

1 hour ago, MattC69 said:

That's great thanks, I was coming to that conclusion and I was going to check some settings.

You should take that post by testdasi as a warning and not attempt to make any changes to your router without knowing exactly what to do. See this previous post in this same thread for more information:

5 hours ago, itimpi said:

The server should never be exposed directly to the internet as it is not hardened for that.   If you want to access it remotely then it is recommended that you use the built-in WireGuard VPN facility.

 

  • Like 1
Link to comment

Hi all, 

 

I am starting to get my head around the Unraid system and I am now onto backups and dockers. 

 

Can anyone point me in the direction of a method to backup the data on the full array onto a USB hard drive? 

 

Also can anyone recommend a way to backup my machines onto the Unraid server? 

I was thinking to create a user share called Backup for each machine then use the built in windows 10 backup to the network drive on the server. I might not do this if I can get my wife and son's to save directly to the server. 

 

Also I have setup a Plex docker which runs but it doesn't seem to have any media in it, I added the paths to the docker setup. Also is there a way to add more directory's to the docker?

Turns out I was just tired and I have now sorted it, I just hadn't setup the Plex side of things

 

Thanks again

Matt

Edited by MattC69
Link to comment
2 hours ago, MattC69 said:

I might not do this if I can get my wife and son's to save directly to the server. 

Not sure what you mean by this. If you are talking about having their files stored directly on the server instead of having them stored on their computer and then backed up to the server, then obviously that is not a backup plan.

2 hours ago, MattC69 said:

lso I have setup a Plex docker which runs but it doesn't seem to have any media in it

Post your docker run command for plex as explained at the first link in the Docker FAQ pinned near the top of the Docker Engine subforum.

Link to comment
43 minutes ago, trurl said:

Not sure what you mean by this. If you are talking about having their files stored directly on the server instead of having them stored on their computer and then backed up to the server, then obviously that is not a backup plan.

I understand that this isn't the best way to backup, I would have a local backup of the server onto an external USB hard drive. 

What is the best way to back up the array onto an external hard drive?

Link to comment
2 hours ago, MattC69 said:

What is the best way to back up the array onto an external hard drive?

I use the Unassigned Devices plugin.  There is a sample script included in the first couple of posts in the UD support thread.  That script automatically backs up the "Photos" share when the device is plugged into the server.  I have modified it for my purposes to back up several shares from my unRaid server when I plug in my 14TB USB drive.  

 

If the data you want to backup from the array is larger than the capacity of a single drive, you could have a script for one USB Unassigned Device that backs up shares A, B and C and another script for another USB drive that backs up shares D, E and F.

 

5 hours ago, MattC69 said:

Also can anyone reccomend a way to backup my machines onto the Unraid server? 

For backup of the three desktop/laptop PCs in my home, I run Acronis True Image (there are also some free backup solutions you could use - like Macrium Reflect) and have a Backups share on the server to which they point as the destination.  Each PC has its own folder within the Backup share.  Others have opted for a separate share for each machine to be backed up with backup user credentials for each of those shares.

  • Like 1
Link to comment
2 hours ago, Hoopster said:

I use the Unassigned Devices plugin.  There is a sample script included in the first couple of posts in the UD support thread.  That script automatically backs up the "Photos" share when the device is plugged into the server.  I have modified it for my purposes to back up several shares from my unRaid server when I plug in my 14TB USB drive.  

 

If the data you want to backup from the array is larger than the capacity of a single drive, you could have a script for one USB Unassigned Device that backs up shares A, B and C and another script for another USB drive that backs up shares D, E and F.

 

For backup of the three desktop/laptop PCs in my home, I run Acronis True Image (there are also some free backup solutions you could use - like Macrium Reflect) and have a Backups share on the server to which they point as the destination.  Each PC has its own folder within the Backup share.  Others have opted for a separate share for each machine to be backed up with backup user credentials for each of those shares.

That is great information, thanks for taking the time to post. I will look into it. 

Link to comment
6 hours ago, MattC69 said:

That is great information, thanks for taking the time to post. I will look into it. 

By the way, in addition to backing up the array to external USB drive(s), I also backup to a backup unRAID server and the cloud.  Just one backup is never enough especially if it is in the same location as the server.

Link to comment
14 hours ago, MattC69 said:

That is great information, thanks for taking the time to post. I will look into it. 

In terms of backup, it's easy to just remember the Four-O

  • Original copy
  • Online backup e.g. what's stored on the Unraid array (in addition to the original copy!)
  • Offline backup e.g. your external USB device
  • Offsite backup e.g. the cloud

So just work towards getting Four-0 and your data is about as safe as it will ever get.

Of course, it's costly to achieve the full Four-O so be selective as to how many O's each piece of data should get.

 

Link to comment
  • 2 months later...
On 1/17/2020 at 10:17 AM, testdasi said:

 

 

11. What is the recommended way to get disks, should I but some NAS drives (I am looking at Ironwolf drives) or get them from enclosures?

For Unraid, just get the cheapest you can get for the desired capacity from a reputable dealer (except Amazon due to shipping issue I said above).

There is no need to get "NAS" or "Enterprise" or anything like that.

Toshiba 8TB HDD are quite cheap right now.

 

 

Why are you suggesting any drive type vs using NAS type drives (out of curiosity). I'm looking at slowly expanding my current 4x4TB WD Red to an 8TB platform and finding it difficult due to cost. I found some 8TB Ultrastars for an ok price

 

Your comment has me interested - if I can use any ol' 8TB this drops my cost significantly. Also one would think that using non-nas drives would warrant a double fault tolerant setup now - two parity drives (ability to lose two drives in the array vs only one).

Link to comment
1 hour ago, sannitig said:

Why are you suggesting any drive type vs using NAS type drives (out of curiosity). I'm looking at slowly expanding my current 4x4TB WD Red to an 8TB platform and finding it difficult due to cost. I found some 8TB Ultrastars for an ok price

 

Your comment has me interested - if I can use any ol' 8TB this drops my cost significantly. Also one would think that using non-nas drives would warrant a double fault tolerant setup now - two parity drives (ability to lose two drives in the array vs only one).

NAS and Enterprise drives typically come with additional features such as:

  1. Longer warranty
  2. Better vibration / drop protection
  3. (Free / discounted) data recovery service
  4. Better rating for continuous (24/7) operation

 

Let's use ZFS as an example. It is RAID, which stripes data across multiple drives. That means:

  • All drives have to be spun up so a RAID system almost never spin down. That makes (4) rather important.
  • If more drives fail than the number of parity, ALL data is lost. That makes (3) critical if any data at all is to be recovered.
  • It usually deploys in enterprise / corporate envi which (a) staff don't care as much about handling things with care and (b) server racks are usually terribly designed with regards to vibration absorption and vibration is bad for moving parts. That makes (2) important.
  • Companies usually depreciate their assets and once fully depreciated, they will replace the assets, usually with a small accounting profit if the assets are resold. This depreciation is typically done over 5 years, 10 years, etc. Wonder why enterprise drives tend to have 5-year warranty? That's why.

 

Compare that to Unraid, which is NOT RAID.

  • There's no striping, each disk has its own file system. So:
    • Drives can be spun down when not in used. Making (4) not as important.
      • In fact, there's an argument AGAINST NAS drives with Unraid precisely because they are rated for 24/7 operation, not the up-and-down usage pattern of Unraid. Moving parts rated for continuous operation don't necessarily take kindly to being switched on-and-off regularly.
    • If more drives fail than the number of parity, only the failed data drives will lose data. That makes (3) less important because usually some data is recoverable. Depends on the data, "some" may be good enough.
  • Most Unraid users deploy their Unraid servers at home in consumer cases. That makes (2) less important.
    • Consumer cases usually house fewer drives than, for example, a 4U rack-mount case (or a storinator!) so less overall vibration. A lot of consumer cases also have vibration mitigation built-in because vibration means noise which is rather not appreciated at home.
    • Also if you own the stuff, you tend to be more careful than that IT guy who just broke up with his lady.
  • Home users don't replace storage at fix schedule but rather only when it stops working, which typically is WAAAAAAAAYYYYYY after the warranty has expired. That makes (1) less important.

 

Now all those points could be thrown out of the window if consumer drives are terrible and fail significantly more often than their enterprise brothers (including NAS types). Fortunately, we have Backblaze to the rescue with its annual HDD failure analysis. And in short, enterprise drives ARE better, but not by much (like 0.05% - 0.1% diff).

 

So considering you will be paying at least 20% more for NAS drives that at best are 0.1% less likely to fail, for features that are not as important to Unraid.

If money is not a concerned (e.g. Linus Sebastian) then OF COURSE go for the NAS and the enterprise drives because they ARE better.

But when value is important (e.g. the rest of us), I'd say the benefits don't justify the cost.

 

 

 

Edited by testdasi
  • Like 3
Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.