First time going unraid here need some advice!


Recommended Posts

Hello Guys

 

Since there are no introductions on the forum i would like to introduce myself ....

 

I am a complete noob when it comes to Linux/Terminal commands .. but i am not afraid to learn and i know i will have some hiccups along the way so please bare with me ... I also have experience in assembling computers and in the hackintosh world so i guess there isn't anything with the proper guidance can be obtained ...

 

I was planning to go with a Qnap/Synology solution but when i saw the specs they offered and the price i really felt i they are asking way too much for what they offer ...

 

My first idea was to use some old hardware i had laying around to build a server so did the research and then a deal came up for the Lenovo TS 140 the i3 version and i really could pass it up ... So bought it and here is everything i have in order to use for my server :

 

-Levovo Ts 140 server with the i3 version

-2TB WD green formatted NTFS which currently holds my media collection

-4TB WD green brand new

-64GB SSD Crucial M4

- I also have a free copy of Windows Server Essentials 2012

 

I have also spent some time reading on the forums and correctc me if im wrong but as of right now the best way to go for a new build  would be to use the last beta and dockerize all of the plugins right ????

 

And if i undersstood right the process should be :

1.- Creating the USB ( I plan on purchasing a key i dont know if in the forum you allow to pony up with someone else to buy 2 keys so it can be a little cheaper)

2.- Installing unraid and make sure everything works from the WEB gui

3.- Format all of the drives i currently plan on formatting the 4TB and use the 64 SSD as cache formatted as Bfrs in order to use docker.

4.-Start by creating the shares and then transfer my current media to the 4TB drive

5.- Install the 2TB drive and create or add it to the array

6.- Enable Docker

7.- Install gfjardim pluging

8.- Try to add all of the docker containers that i want to use withing the plugin (Sab, Sickbeard,NZbget,Torrent, Couchpotato,Plex) Pretty much here is where i will run into all of the problems but i hope with the help of the forum i could get things to work.

9.- Profit ???? ;D ;D ;D

 

Hopefully i could set everything and have it up and running over the weekend.

 

And then if everything works as it should i would like to experiment by adding a VM with the 2012 Essentials that i have.

 

Also if you guys are interested i could probably record the whole thing and upload it to youtube so we can have like a tutorial guide .

 

Let me know if i missed anything

 

Thanks all

 

Shremi 

Link to comment

Welcome to the unRaid forums!  Glad unRaid piqued your interest and I think you'll find our a great platform for a media server.

 

I'd like to better understand your goals. Most people are drawn to unRaid for its data protection and expandability. You indicate you have 4T and 2T drives, but you need at least there drives to have any protection. The third drive has to be as large or larger than your largest drive. Buying a third drive is not a big deal but the server you bought appears to only support 2 drives. You may be able to get a cage that converts those 2 slots into 3 3.5 inch drive slots, but thee drives in a media server is rather limiting. But again it depends on your needs.

 

Adding add-ons with your SSD is surely possible. But I think you should first focus on getting the array defined and protected. Then explore the add-on features.

 

You might look at the DS380 as a small server with more expandability.

 

Oh, LimeTech allows two people that know each other and want to split a 2 pack of licenses, but frowns heavily upon someone looking to split a pair with a stranger. And if you do share a pair, they are licensed as a pair to only one person. If you only need 2-3 drives though, you can use the free version.

 

Feel free to ask questions and learn about the options and possibilities.

 

Best off luck!

Link to comment

Thanks for the quick reply and the arm welcome :D

 

The main purpose for this server will be to be able to download and stream all of my media content that as of right now consist in 2 TB of Movies Tv Shows and Emulators and Roms... I understand how parity works and i know parity will be important but i am waiting on a good deal for a 4tb drive hopefully soon (before black Friday) so i can add a parity drive  i also have a pretty fast internet connection and if i loose some data as of right now i think everything is redownloadable .... But i have a portable 2.5 2TB disc which i will use to hold most of the data until i can get a hold of the parity drive .... The downside is that it doesn't have a proprietary connection in order to use it inside the box  :'( ...

 

Also the reason i wanted to switch to a dedicated server is that i currently have a combo of HTPC and Gaming rig on a Bitfenix Prodigy case which i modded the hell up in order to take a full custom water cooling loop so i have no more space left for another 3.5 Drive inside.... So i first thought just to replace the 2tb drive with the 4tb drive but i really didn't want my whole gaming rig to be up and running 24/7 since it really uses a lot of energy.And a low power server that is powered on 24/7 will be ideal... Also i want to use my gaming rig purely for gaming ...

 

I also considered myself a bit of a hardware junkie since i am constantly changing and upgrading my rig ....  So i had some spare parts lying around the house and thats where i first thought of building the server with the parts that i had laying around but i was able to make some money by selling everything and that's where i took the opportunity to buy the Lenovo TS140 for $210 you cant really beat the price... Much better than the offerings from a big nas company.

 

I think you have the wrong server here is the link  of the one i got and inside there are 5 sata ports on the mobo. As for HDD space, there are no spots for 2.5'' drives but its an ssd and i really know my way around computers so i think i can mount it anywhere i want there are 2 spots for 3.5'' drives, then a 3rd if you don't put a card reader or 3.5'' floppy in the slot. There's also space for one more 3.5'' drive, but I'll need a 5.25'' adapter. I also could pull out the dvdrom as well after the os install and get a 5th HDD in there with another 5.25'' adapter.So i think i can be safe for a little while until my media collection outgrows this server and then i will move on to another solution. I also bought 8GB of more ECC ram since i also want to play around with windows server essentials in a VM

 

I understand where you stand on group buying so as of right now that wont be a problem ..

 

I hope i could explained myself better

 

Thank you for you patience

 

Shremi

 

 

 

Link to comment

I misread the specs on the server. Still seems small and would not be my first choice (not criticizing but without options to load the drives in removable cages, which greatly reduces the chances of having issues rebuilding drives after a failure, it is not one I would recommend).

 

But if you can get up to 5 drives in there it has a reasonable capacity.

 

Almost everyone that starts with unRAID underestimates their capacity needs. With a smaller server I recommend going with largest drives, even if the price per TB is higher (within reason, obviously). With 5 slots and assuming using an SSD for cache, you have the ability to have 4 data drives. You already have a 2 and a 4. Adding two more 6TB drives would get you to 18TB before having to swap out the 2TB for a larger one, whereas adding two more 4TB would only get you to 14TB. (You'd also need a 6TB parity). Swapping out smaller drives for larger ones gets very pricey per TB. Imagine having to swap a perfectly fine 4TB drive to a 6TB today because you are full and have no more room for another drive. That would cost $150 per TB of extra capacity! Better to spend $10-15 extra per TB now to avoid such tough choices in the future.

Link to comment

You mentioned formatting drives. unRAID will format the drives as needed. What you need to do is preclear them.

 

Parts of the wiki are a little dated, but this Configuration Tutorial is worth reading.

 

Ask lots of questions.

 

Thank you for the link that is good read indeed ...

 

So far if i have it right the preclear script should be able to determine if a new drive has some failure right??? how about a used drive should i also run the preclear script on that drive???

 

Also 2 more questions :

 

I read that the process of pre clearing a drive is long so i want to save some time and start by doing it right now is it possible to preclear a drive lets say from windows or a live cd??? I want to be ready for the setup over the weekend so i figured this might save me some time.

 

My current media resides in my 2tb NTFS formatted drive .. what would be the best route to transfer everything into the array ?? I have another gaming rig with a hot swap bay that i can use I also count with a couple of spare 1TB disks but i really don't plan on using them in unraid BC of the limited disk options i have now so i was thinking of :

 

1 Transfer everything into the spare 1tb disk that i have

2 Format the 2 tb drive with the correct format type that unraid uses ( don't know which one or if i should pre clear this drive also )

3 Transfer everything back into the 2tb Drive with the correct format type

4 Pre clear the new 4tb File

5 Put the 4TB and the 2TB in boot unraid and create the array

6 Profit???  ;D ;D ;D

 

I misread the specs on the server. Still seems small and would not be my first choice (not criticizing but without options to load the drives in removable cages, which greatly reduces the chances of having issues rebuilding drives after a failure, it is not one I would recommend).

 

But if you can get up to 5 drives in there it has a reasonable capacity.

 

Almost everyone that starts with unRAID underestimates their capacity needs. With a smaller server I recommend going with largest drives, even if the price per TB is higher (within reason, obviously). With 5 slots and assuming using an SSD for cache, you have the ability to have 4 data drives. You already have a 2 and a 4. Adding two more 6TB drives would get you to 18TB before having to swap out the 2TB for a larger one, whereas adding two more 4TB would only get you to 14TB. (You'd also need a 6TB parity). Swapping out smaller drives for larger ones gets very pricey per TB. Imagine having to swap a perfectly fine 4TB drive to a 6TB today because you are full and have no more room for another drive. That would cost $150 per TB of extra capacity! Better to spend $10-15 extra per TB now to avoid such tough choices in the future.

 

Yeah i know this isn't the best setup ever but i think it will cover my needs for a few years .... The other thing i could do is mod the case in order to use a 5 in 3 hot swap cage.... or try to buy a cheap case and move existing hardware to that case... The thing that i love about this case is that it is dead silent... i don't know if i could achieve it with another case.

 

I totally understand your point in spending big the first time but as of right now i would like to start with what i have because i am using the logic that by the time i am running out of space the 4Tb disk will be much more cheaper and maybe the 6tb drives also will be more afordable...

 

I would like to point out that this is the media that i have created over 2 years so i figured 4 more TB should last me long enough and i will cross that bridge when i get there with the current pricing options of that time being.

 

 

It is worth pointing out that one can only have 2 data drives (plus optionally 1 parity drive) on the free license.

 

But how about a cache drive??? I have a spare one and i have read it really increases the speed of the server plus i also would like to run some VM to play with windows home server essentials.. I also think that you should pay for an amazing solution like unraid since it has all of the features that i was looking for. Mixed HDDs only one Parity drive Cache drive ... I first considered using Freenas but man that ZFS is a pain for someone like me who wants to expand little by little plus no mixed HDDs and the way parity works over there really pushed me away. So i would like to support unraid

Link to comment

A cache will work well for both speeding up writes and for providing a drive to save applications that are "outside" of the (usually protected) array (in your case it won't be protected until you add a parity drive).

 

But to use a cache, you'll have to buy a license, as the free version doesn't support a cache drive.

 

Link to comment

You mention a 2.5" 2TB drive. Is this separate from the 2TB green drive you mention? Also, you say you have a couple of spare 1TB disks. If I were you, I would go ahead put them all in the array, use the 4TB as parity, the SSD as cache. It is very simple to upgrade disks to a larger disk in unRAID. I have done it many times. My array was all 2TB drives when I started 3+ years ago and now it is all 3TB drives. I have found other uses for the old drives, such as offsite backups, building other computers, etc.

 

Since I am recommending a totally different approach than you have outlined in the steps in your post, I won't comment on them specifically, except to say that you should preclear anything you want to add to the array, let unRAID format the drives after you add them and don't try to format them yourself, and don't try to transfer any data to any drive before you have added it to the array.

 

You can add the 2TB drive after you have transferred the data from it to the other drives that you have already added to the array.

 

Rather than my going into further detail and speculation, I'll just suggest you take it slow and continue to ask questions as you go.

Link to comment

You mention a 2.5" 2TB drive. Is this separate from the 2TB green drive you mention? Also, you say you have a couple of spare 1TB disks. If I were you, I would go ahead put them all in the array, use the 4TB as parity, the SSD as cache. It is very simple to upgrade disks to a larger disk in unRAID. I have done it many times. My array was all 2TB drives when I started 3+ years ago and now it is all 3TB drives. I have found other uses for the old drives, such as offsite backups, building other computers, etc.

 

Since I am recommending a totally different approach than you have outlined in the steps in your post, I won't comment on them specifically, except to say that you should preclear anything you want to add to the array, let unRAID format the drives after you add them and don't try to format them yourself, and don't try to transfer any data to any drive before you have added it to the array.

 

You can add the 2TB drive after you have transferred the data from it to the other drives that you have already added to the array.

 

Rather than my going into further detail and speculation, I'll just suggest you take it slow and continue to ask questions as you go.

 

Ok sounds like a good idea i will start with 3 Hdds since i don't want to leave all of my computers without storage.

 

My only concern is that my 1TB HDD is 7200RPM so i don't really know if this might affect with temperatures inside the case with all the drives connected .

 

So my setup will be something like

 

4TB drive as parity

2TB and 1TB as the array and then ill add more HDDs or swap the 1TB for a larger one.

64 GB SSD as a cache drive. don't know if its big enough for the cache and a windows WM.

 

Ill try to set up the array tonight

 

 

 

Link to comment

As I noted above, you can set up this array with no issue using the free version of UnRAID, but you'll need to buy a Plus license to add the cache drive.    Up to you whether to buy a Plus or Pro license ... for a relatively small array (especially considering the size of modern drives), a Plus array will likely last you for a good while.

 

Link to comment

64GB SSD might be a little small for a Windows VM if you also intend to use it for other things, but since you could store any data the VM used on the array it might not be too bad depending on what else you wanted to do with the SSD. That sort of functionality is very much beta at the moment, but I think some have it working. Some people are mounting SSDs outside the array rather than as cache for their VMs. If you wanted to do that then you could use an HDD for cache now or later. You might just leave this capability for a while until you get everything else working and get more familiar with things, do more research, etc.

 

Also, I think it is not recommended to use preclear on an SSD.

Link to comment

Ok guys great news  ;D ;D ;D thank you all for the responses and advices you gave me.

 

I have unraid up and running as we speak ..... I still have to wait until i get the email with the license so i can start with the cache drive and all of the goodies.

 

So far i have done the following

 

1.-Arranging everything in bios so i can run the server as headless .... boot into unraid by default.... Added the DHCP reservation for the server in my router.

2.-Setting up the server changing the root password.

3.- Adding all of the drives and add the parity drive

4.- Setting up user shares properly with the right settings acording to the wiki and some guides i read online.

 

Am i missing something here i want to know what do you suggest i can do here ... thanks

 

Also about the parity drive

 

I have 3 options for a parity drive so i wan to know which of these drives will be better ...

 

64GB SSD  Mainly as a spare ssd

128GB SSD I currently use it as the boot drive in my test bench

500GB HDD 2.5 5900 RPM i took this drive out of my MPB and upgraded it with an SSD so its laying around in case i need a 2.5 drive.

 

I read about the cache drive and i really want it bc it improves transfer speeds ... also i could have all of the plugins with docker ... and i also could use it to boot a VM...

 

But i really want to know if a SSD drive would really be necessary to get the best transfer speeds ... i just want a good opinion on which drive should i use..

 

Thanks for your time again

 

Shremi

 

 

 

 

Link to comment

I have 3 options for a parity drive so i wan to know which of these drives will be better ...

 

I presume you mean the cache drive, since you've already assigned parity (and none of those would work for it, since it has to be >= the largest drive in the system).

 

As for the best drive ... an SSD is clearly faster; but for working purely as a cache drive it doesn't matter as long as the drive is as fast as your network.  As long as that's true, the limiting factor will be your network speed; not the speed of the cache drive.    An older 500GB laptop drive will NOT be as fast as your network (assuming you're on a Gb network), so I'd probably use the larger SSD ... but the laptop drive would still give you a very nice improvement in transfer speeds, so if you want the higher capacity, just use it.

 

Note that if you're running additional apps on the cache drive, then the network limitation doesn't apply, since these are locally run.  In that case you'll definitely benefit from an SSD.

 

 

Link to comment

I would suggest that when you are doing your initial load of data on the array, that you copy directly to the disk shares and not use the user shares. User share configurations can be tricky and if it isn't perfect, files that you want to stay together can get split across drives. Straightening it all out after the fact can be a giant (and time consuming PITA). If you just copy your files to the disks you won't have this problem, and when you start using user shares, if things don't work exactly as expected as you are copying a few new files to the array, it will be easier to fix.

Link to comment

I would suggest that when you are doing your initial load of data on the array, that you copy directly to the disk shares and not use the user shares. User share configurations can be tricky and if it isn't perfect, files that you want to stay together can get split across drives. Straightening it all out after the fact can be a giant (and time consuming PITA). If you just copy your files to the disks you won't have this problem, and when you start using user shares, if things don't work exactly as expected as you are copying a few new files to the array, it will be easier to fix.

 

Thank you for the advice ... i am starting to transfer data as we speak but i have a few problems ... Since i am making some test to se the performance speed i am transfering my movies one my one .... the problem is that its starts great arround 100MB/s and then slows down all the way to 30-40MB/s isnt this supposed to be faster with the cache drive and a GB network ????

 

I have 3 options for a parity drive so i wan to know which of these drives will be better ...

 

I presume you mean the cache drive, since you've already assigned parity (and none of those would work for it, since it has to be >= the largest drive in the system).

 

As for the best drive ... an SSD is clearly faster; but for working purely as a cache drive it doesn't matter as long as the drive is as fast as your network.  As long as that's true, the limiting factor will be your network speed; not the speed of the cache drive.    An older 500GB laptop drive will NOT be as fast as your network (assuming you're on a Gb network), so I'd probably use the larger SSD ... but the laptop drive would still give you a very nice improvement in transfer speeds, so if you want the higher capacity, just use it.

 

Note that if you're running additional apps on the cache drive, then the network limitation doesn't apply, since these are locally run.  In that case you'll definitely benefit from an SSD.

 

LOL you are right about that i meant cache drive... I went ahead and installed the largest SSD since i have a GB network .. And my test bench is only for testing hardware so ill probably can live with the 64GB drive.

Link to comment

You might try the "SNAP Plugin" (make sure you get the V6 version, do not use the V5 version). When I set up my system for the first time I didn't use a Parity drive during the initial copy. It makes the transfer much faster. Then you can add the parity drive to the array after and just let it calculate parity in a few hours. Depending on how much you have left, this could be faster for you in the long run.

Link to comment

30-40 MB/sec is nomal to a protected array, but copying to cache should be a bit faster.

 

Using the cache disk for the initial load is not recommended. It may seem faster because the copy from the workstation finishes, but the move to the protected array is still to come overnight. Adding together the time to copy to cache to the time it takes to move it to the array is slower than just copying to the protected array.

 

Be careful that you don't copy files to an unprotected array and delete the backups. The array is new and untested. You don't want a failure due to hardware, software, or user error to cause you to lose data. The initial load is a somewhat risky time IMO. You want it to go fast and the stakes are high. Be patient!

Link to comment

A few thoughts ...

 

=>  Your copies start fast (~ 100MB/s ... reasonable for a Gb network) and "... then slows down all the way to 30-40MB/s ".    After how long?    Note that at 100MB/s you're doing ~ 6GB/minute.  That will fill a 128GB cache drive in about 20 minutes -- and that's assuming it doesn't have anything else on it (apps, docker containers, etc.).    If it's got less space than that, then it will take even less.

 

=>  Are you copying to a user share that's got cache enabled?  If not, then you're copying directly to the user array anyway, with no cache involved.  If that's the case, the initial 100MB/s copies will only be for a few seconds.

 

=>  In either event, I agree with Brian that for an initial load it's best to just disable cache and let the writes be directly to the protected array.  If you have any really significant amount of data, the copy is going to take long enough that caching a bit of it isn't really going to help ... in fact it'll take longer, as when the cache gets emptied in the middle of the night, the copy speed will be much slower since you'll have two sets of writes to the protected array during the time the cached data is being moved.

 

=>  It simply takes a while to do an initial load ... you can do roughly 2.5 - 3TB per day if you let it run non-stop.  Just be patient ... it'll finish.    When I switched from using external drives to a 2nd array for my backups, the initial backup took a bit over ten DAYS to run  :)    But it only took about one minute of my time ... to start the copy.

 

Link to comment

When I did my initial load (years ago), I ran the array unprotected. It was a risk and I didn't know what I was doing. Turned out I had a problem and came very close to losing a drive's worth of data.

 

So for a while I would recommend that people buy a few "seed drives" (usually do anyway) and initially set up the array with no parity disk (saving one disk to be parity). Copy data from wherever to the unRAID box, filling the data disks that are there. When full, then build parity. And only then start preclearing and repurposing old disks that have already been copied into the unRAID box to start receiving new data. In this way a large amount of data can be copied as much higher speed.

 

But now I recommend to set up the array with parity and empty data disks you have and start copying. Not only are you copying data, you are also testing out your rig. And this is the way data is going to be copied to your array for the rest of its life - so test the heck out of it. It may take some time, but in the end you will have an array you can trust.

Link to comment

Thank you so much for everyone that replied .... I finally was able to finish moving up my entire data in about a day and a half ... my only problem was that i had a few disconnections here and there but its finally done .... Now i need to move on to the plugins.

 

I have formatted  the drive accordingly to the guide and installed docker and the docker plugin ... I wanted to try out plex since i am familiar with it and everything installed perfectly but know i have my first problem its seems it doesn't recognize my library  i have tried a lot of things. maybe you can tel me what am i doing wrong .

 

I have all of me media in disk1 under a share called Media i have tried adding to the path this :

 

/mnt/user/Media/Movies/ and also like /mnt/disk1/Media/Movies/

 

Am i missing something here ????

 

 

 

Link to comment

When I did my initial load (years ago), I ran the array unprotected. It was a risk and I didn't know what I was doing.

 

While I agree it's best to copy to the protected array, it's really not that much of a risk PROVIDED you validate all of the copies ... either by using an automated verification utility to do the copies;  or by doing a full compare of the copies after they're completed.    THEN build the parity disk when you know the data's all good.

 

But now I recommend to set up the array with parity and empty data disks you have and start copying. Not only are you copying data, you are also testing out your rig. And this is the way data is going to be copied to your array for the rest of its life - so test the heck out of it. It may take some time, but in the end you will have an array you can trust.

 

Same as above ... be sure you VERIFY all of the data from your backup (source) disks after it's on the array.  With or without parity, the copies aren't "known good" until you've done that.  A bad parity-protected copy of a file isn't any better than a bad unprotected copy is  :)

 

 

... I finally was able to finish moving up my entire data in about a day and a half ...

 

I hope you mean "copying" instead of "moving".  You should NEVER "move" a file.  Copy it;  VERIFY the copy; and then delete it.  That's a FAR safer approach.    You should, of course, also have backups, but that's another story ...

 

Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.