New home server


Recommended Posts

Hello, I have over a few years been hosting some kind of media server at home. Mostly for plex, but as the years has gone by and more storage has been added and the lack of interesting features to use in windows 7. I want something new, thats were I started to read about unraid, I really like the concept on how everything works, as well as the plugins. But before converting I do have some questions which I hope some of you can answer.

 

1) as for now I have a lot of random drives, one 1tb, one 2tb, one 4tb and 8tb. As I've read I need to have the largest drive as "redundancy" drive. Lets say I make an array with one parity drive. Then I will have about 7TB left. If I want to change out the drives as the time goes. Swap out the 1tb drive for a new 8tb drive. Would there be no problem building the array up again? or should I copy all the files form the 1tb drive to the new 8tb drive? Or let the array build it up? My main goal is to have 5x 8TB drives, with one parity drive.

(also will I kill my drives with re-building if I am swapping out the non 8tb drives over a few months? I've read that re building is stressing the drives a lot, and it usually kills them...

 

2) will 16GB ram be enough, its not ECC ram, but I will only be running plex and own cloud/mycloud as for now.

 

3) Does unraid run on the ram or the bootable usb when installed? Will my "server" get slow with high uptime since its running on the ram?

 

4) I usually download a lot of video, is there any way to use some kind of browser and torrent client in unraid? or do I need to use a separate pc to download files and store them directly on the server?

 

5) From a windows perspective will I see one big storage array? I have some problems using windows now, because the storage that is left is so spread. 50GB here and there, and if I were to download a large file on lets say 70GB, how will unraid handle it? will it spread it across drives? if so how does this affect rebuilding an array?

 

Really appreciate for all help I can get :) and a big thanks to the people that will take their time to answer!

Link to comment

1) Largest drive must always be parity drive in any RAID system. I myself just copy my data over and remove the old drive and recheck, or you could let parity rebuild the files.

2) 16GB should be plenty for what you listed you only need more RAM when doing VM's and maybe some complex docker setups.

3) Good question not 100% sure 

4) There's plugins for these things or use a docker image

5) Depends how you configure it, for the reason you list alone I configure all my drives as single store so data is not spread across (easier for me to offline backup/update and recovery if multiple drive failures)

 

FYI: SECURITY IS NON-EXISTENT so i'd suggest VPN to your (unraid)owncloud if thats what your looking to run, hopefully you have a newer router which has OPENVPN support to make things simple

Link to comment

Thanks for the answers to far. I have really only been watching own cloud, it seemed very simple, and it had an app for ios/android which also is handy. I do want to have access to my files from outside the network, so security is something I need to read a bit more about. The server is it self will be hoster at my parents house since I dont have any room for an extra computer while I am studying, so this will be a kind of hobby project, if all goes as planned, I am starting a computer engineering degree next year as well, and network and security is something I would like specialize my self in for future jobs etc. So remote control and remote access is something that I would love to have. An example scenario would be, if I record something with my Gopro and make own cloud take an backup from it, storing it at my server.  That way I would love to be able to "download" that file on my Mac book pro and edit it, remotely of course :)

Link to comment

1) The "redundancy" drive is more properly referred to as the parity drive. It doesn't actually contain any of your data, but it does allow the data from a single missing disk to be calculated from the data on all the other disks. You must let unRAID format any disk it will use, so if you are planning to use existing disks you will have to put their data somewhere else. You can do it one disk at a time if needed. Hopefully you already have backups anyway, and if not you should consider a backup plan since even with parity, unRAID is not a backup. The only thing that counts as a backup of your irreplaceable and important data is additional copies.

14 minutes ago, dent1337 said:

I've read that re building is stressing the drives a lot, and it usually kills them...

Can't imagine where you read that. Rebuilding is the normal way to replace smaller drives with larger drives, and there is no good reason to do it differently.

 

2) 16GB will be more than enough for what you have planned. Only if you decide to run multiple VMs will you possibly need more.

 

3) unRAID unpacks the entire operating system fresh from the bootable usb into RAM when it boots. Other than that some configuration data is kept on that usb but it isn't read or written very often or very much. Not sure why you think running in RAM might slow things down.

 

4) There are many usenet and torrent clients available as dockers to run on unRAID.

 

5) Each data disk in unRAID is an independent filesystem. Each file is stored completely on a single drive. But you can have user shares that span multiple disks and you can configure them so you don't really have to be concerned which disk is used for any particular file. Each user share will appear separately in Windows. Most people make specific shares for specific uses.

Link to comment
2 minutes ago, dent1337 said:

Thanks for the answers to far. I have really only been watching own cloud, it seemed very simple, and it had an app for ios/android which also is handy. I do want to have access to my files from outside the network, so security is something I need to read a bit more about. The server is it self will be hoster at my parents house since I dont have any room for an extra computer while I am studying, so this will be a kind of hobby project, if all goes as planned, I am starting a computer engineering degree next year as well, and network and security is something I would like specialize my self in for future jobs etc. So remote control and remote access is something that I would love to have. An example scenario would be, if I record something with my Gopro and make own cloud take an backup from it, storing it at my server.  That way I would love to be able to "download" that file on my Mac book pro and edit it, remotely of course :)

In addition to security, which is the most important concern for what you are proposing, you will also need to consider your connection speed, both up and down. Many ISPs give good down speed but much slower up speed. Good down speed is great if you are only getting data, but you must have good up speed if you are supplying data. That will be important on both ends, your parent's place and yours.

Link to comment
5 minutes ago, trurl said:

In addition to security, which is the most important concern for what you are proposing, you will also need to consider your connection speed, both up and down. Many ISPs give good down speed but much slower up speed. Good down speed is great if you are only getting data, but you must have good up speed if you are supplying data. That will be important on both ends, your parent's place and yours.

as for now I do have a "decent" internet connection with 72mbps down and 10 mbps up. (according to speed test)  And there are also some rumors going on about building fiber connection in my area.  Also have about the same speeds where I am studying. Don´t think I will be uploading big files, but a 10 minutes 1080p 60 fps go pro clip, should´t be that big.

Edited by dent1337
Link to comment
15 minutes ago, trurl said:

1) The "redundancy" drive is more properly referred to as the parity drive. It doesn't actually contain any of your data, but it does allow the data from a single missing disk to be calculated from the data on all the other disks. You must let unRAID format any disk it will use, so if you are planning to use existing disks you will have to put their data somewhere else. You can do it one disk at a time if needed. Hopefully you already have backups anyway, and if not you should consider a backup plan since even with parity, unRAID is not a backup. The only thing that counts as a backup of your irreplaceable and important data is additional copies.

Can't imagine where you read that. Rebuilding is the normal way to replace smaller drives with larger drives, and there is no good reason to do it differently.

 

2) 16GB will be more than enough for what you have planned. Only if you decide to run multiple VMs will you possibly need more.

 

3) unRAID unpacks the entire operating system fresh from the bootable usb into RAM when it boots. Other than that some configuration data is kept on that usb but it isn't read or written very often or very much. Not sure why you think running in RAM might slow things down.

 

4) There are many usenet and torrent clients available as dockers to run on unRAID.

 

5) Each data disk in unRAID is an independent filesystem. Each file is stored completely on a single drive. But you can have user shares that span multiple disks and you can configure them so you don't really have to be concerned which disk is used for any particular file. Each user share will appear separately in Windows. Most people make specific shares for specific uses.

https://www.youtube.com/watch?v=0wcplVZLyPs

 

 

regarding drives dies during re building, this is what video I saw. Might not be the most polite source? The explanation starts around 3:50 mark

Link to comment
5 hours ago, dent1337 said:

regarding drives dies during re building, this is what video I saw.

He talks about disks never getting used and so you don't know they are bad. Sounds like he never heard of scheduled parity checks and notifications. Videos like this are not really my cup of tea, since it is just a lot of words that could be better studied and understood by reading them. A video should show you something. Just my opinion.

 

Here is a good read that explains unRAID parity:https://lime-technology.com/wiki/index.php/UnRAID_Manual_6#Parity-Protected_Array

Link to comment

Thanks for the links aswell as the update regarding drives. Guess i can safely swap out one drive at a time without concerning to much about drive failure. Wish i could afford all the drives at the same time, but being a full time student makes that a big hard. I guess it will Take a lot of time rebuilding the array each time i swap out a drive, will it be possible to use plex while i rebuild?

Link to comment
4 hours ago, dent1337 said:

Thanks for the links aswell as the update regarding drives. Guess i can safely swap out one drive at a time without concerning to much about drive failure. Wish i could afford all the drives at the same time, but being a full time student makes that a big hard. I guess it will Take a lot of time rebuilding the array each time i swap out a drive, will it be possible to use plex while i rebuild?

And there is no way to swap out multiple drives at a time (with single parity) so doing it one at a time is what you would do anyway. And you can buy drives as you need them to increase capacity and so take advantage of the latest prices. I would say not buying them all at once is a good thing.

 

You can use the array while rebuilding but performance will be affected. I haven't needed to use 8TB disks so far, but my 6TB parity check only takes 15-16 hours, and a disk rebuild would be similar. Depends on your hardware since there can be bottlenecks with controllers, etc.

Link to comment
32 minutes ago, trurl said:

And there is no way to swap out multiple drives at a time (with single parity) so doing it one at a time is what you would do anyway. And you can buy drives as you need them to increase capacity and so take advantage of the latest prices. I would say not buying them all at once is a good thing.

 

You can use the array while rebuilding but performance will be affected. I haven't needed to use 8TB disks so far, but my 6TB parity check only takes 15-16 hours, and a disk rebuild would be similar. Depends on your hardware since there can be bottlenecks with controllers, etc.

Well that sounds good enough. Been trying to find a used enterprise raid card, but no luck so far. I Currently only have 4x sata 3.0 ports, so i might swap out the motherboard for a new one with 6 3.0 ports. Found a good deal on a used one, but it got sold so fast.

 

also one last question. Is it possible to remote control the "server" ? Tightvnc etc? 

Link to comment

There are ways to access from outside your network, but that is not builtin. There is an OpenVPN docker for example. If your router has VPN that is probably easiest. Even easier is to use Teamviewer but you would have to use that to remote control another computer on your network and use it to work with your server.

 

Many people (me) run their server headless, so remote control within your LAN is definitely possible and it is really designed for that. The main interface is by web browser, and if you ever need to go to the command line, telnet/ssh using puTTY or similar.

Link to comment
  • 1 month later...

one last question before I start shopping hard drives. I do have a 8tb drive from before, so planning to buy 4 new 8tb external drives, since they are cheaper. will open them and remove the HDD. But the one single 8tb drive I have from before is a bit slower than the ones I am planning to buy. Can I set the slowest drive as a parity drive? Or will that impact the whole system? The slowest drive have 100mbs,  and the other 4 have around 150mbs.

Link to comment
5 minutes ago, Zonediver said:

The Parity Drive should be a fast on. Use the slower one for Data storage.

okey, but the 8tb drive I have now, is pretty slow with a lot of stuff on it. But when its wiped or half full, it´s peaking at 150mbs. But as its only for mass storage for movies, it doesn't´t really Mather than much about speed. The local network is 1GB, so anything above 125mbs is pretty waste.

Link to comment

If the Parity Drive is slow it might impact things during data rebuild or parity checks. When you use a 8TB for Parity and some smaler drives (4-6TB) then it doesnt matter.

But in your case, all drives are 8TB so it can slow down things if the parity drive is slower then the rest of the System - just an FYI.

Parity drive "should" always as fast as possible otherwise you waste performance - but you can use the slower disk as parity too of corse.

Edited by Zonediver
  • Upvote 1
Link to comment

Parity faster than any data drive provides no advantage, but there is a disadvantage to having parity slower than a data drive.

 

If parity is slower than other disks it can have an effect on parity operations. Parity operations include parity checks, rebuilds of data or parity disks, writes to the parity array, or reads from an emulated "missing" disk.


Since you already have data on the "slower" drive I would just leave it that way.

  • Upvote 1
Link to comment
1 hour ago, trurl said:

Parity faster than any data drive provides no advantage, but there is a disadvantage to having parity slower than a data drive.

 

If parity is slower than other disks it can have an effect on parity operations. Parity operations include parity checks, rebuilds of data or parity disks, writes to the parity array, or reads from an emulated "missing" disk.


Since you already have data on the "slower" drive I would just leave it that way.

Thanks for explenation.  I do have data on the disk, but as i am going to buy 4 new drives. I will be setting up an array with 3 fresh 8tb drives first. And then start copying all my files from the existing drive over to the array. Before i wipe it and add the 2 remaining drives. I guess i can just put the old drive as a storage drive, as i need to wipe it anyway, and the parity drive has already been selected when i built the first 3 drives array. Can't wait till i get my tax money from last year.  Which i hopefully get in the beginning of june. 

 

 

Link to comment
21 minutes ago, gubbgnutten said:

No advantage? What happens when there are writes to two+ data drives at the same time?

Possibly. I have seen that mentioned, and I don't doubt it can happen with some configurations. I wonder how often it actually happens though, or how reliably an advantage can be demonstrated in testing.

Link to comment
  • 6 months later...

Well i guess singles day is good for something. There are some really cheap 8tb seagate archive drives now.  212 dollars, converted from the currency in Norway. For example a iron wolf drive is 338 dollars. My use is just a mass storage of movies, tv shows and pictures. I've read that the drive speed on these archive drives are super slow. But since i only need to write the file one time over to the drive, it shouldn't impact the performance that much i hope. I will be using a 250GB ssd as cache as well. Should i buy 4 drives now, the sale is over tomorrow, and i don't think prices will be much lower that in the future either. As mentioned before i already have a 8tb archive drive in my system from before. So i will be running 5 drives with 1 as parity and the rest as data drives.   I did read through the whole thread underneath. A lot of the discussion was a bit above my knowledge. But using the drives for plex mass storage i hope i can't go that wrong. As read speed is the only factor when watching a movie. Also should i wait for a new motherboard with 6 sata 3.0 ports before making an array? or is it just plug and play if i just swap out the motherboard.

 

 

 

 

Link to comment

The archive drives are actually pretty fast on sequential reads and writes. Considerably faster than WD REDS. They can get bogged down if you are doing random writes, which can fill the disks PMR buffer and sort of put you in a performance penalty box until the drive recovers. Heavy fragmentation could cause this. I have several and never seen it happen on mine.

 

But I remember reading that Seagate's newest budget drives have a low workload rate limit (TB/year) about a third of earlier versions (170TB vs 550TB). I don't put anywhere near that mileage on my drives, but you might do some research.

 

In the US, WD external 8T units have been well priced - around $170-$200, in particular a model exclusive to Best Buy called EasyStore. They had contained WD REDS but hearing reports some may be containing white label drives.

Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.