Planned Unraid NAS config, looking for advice


Recommended Posts

Hi everyone,

 

I am a brand-new member, and happy to be here. As a beginner, I understand that some of my questions might be puerile, we've all been there, so please be gentle with me.

 

Without further ado, here's my current situation:

I have a homemade NAS running Windows 10 Pro x64 (I know, I know...) and despite the general consensus about such a choice, it has served me well. The main scope of the NAS is to hold large amounts of data (mainly media files) and PLEX. Its hardware configuration is nothing special:

Motherboard: Gigabyte B450 mATX
CPU: Ryzen 3 1200
RAM: 16 GB DDR4
A throwavay GPU (GT 210)
Storage:

  1. 2x 256 GB SSDs (nVME), one installed on the motherboard, the other in a PCI Express adapter. The first SSD is used for the OS and the other is used as a scratch disk for downloads (I have Gigabit Fiber and downloading at 100 MB/s consistently).
  2. HDDs:
  • 1x 4 TB, internal (empty)
  • 1x 8 TB, internal (full)
  • 1x 10 TB, external (USB 3.0) (full) - this is a WD drive, I don't plan on shucking it.
  • 1x 14 TB, external (USB 3.0) (full) - this is a Seagate drive, I will shuck it after emptying it.
  • 2x 14 TB, brand new, shucked (Seagate IronWolf NAS) (empty)

 

Obviously, current configuration reached its limit, therefore I have planned an intermediary upgrade path.
1st step (already complete) was to buy 2x LSI 9210-8i controllers. I will configure both, then replace the scratch disk (256 GB nVME SSD in the adapter) with one controller. I have also bought a larger case which can hold 10x HDDs and 2x SSDs. FWIW it's a Gamemax Titan Silent M905S. Another thing I have bought is 2x WD Blue™ SATA SSD 2.5” (500 GB each), which I will use as cache pool.

Yes, I am considering replacing the mobo/CPU/RAM, my personal preference would be an AMD 2400G (to free up the motherboard PCI Slot #1) with a X570 motherboard with as many SATA connectors as possible, but that's not going to happen right away (budget constraints), so for now I will use what I have.

One of the 14TB HDDs is going to be used as a parity drive, the plan is for all the other HDDs (except the external 10 TB drive) to be connected to the LSI controller and merged into one large storage pool. Because I have a large amount of data (almost 32 TB), I am going to have to split this into multiple steps, and here's where I would need some advice, aka sanity check:
 

  1. Create the array, with 1x 14 TB (parity), 1x 14 TB + 1x 4 TB (data), 2x 500 GB SSDs (cache pool).
  2. Copy over data from the full 14 TB drive to the array.
  3. Add the 14 TB drive to the array.
  4. Copy over the data from the 8 TB drive to the array.
  5. Add the 8 TB drive to the array.
  6. Copy over the 10 TB drive to the array.
  7. Remove the 10 TB drive from the array, maybe try to shuck it and take care of the 3.3V WD proprietary pin, then add it too to the array, or keep it as backup for important data, as an extra layer of protection. Haven't decided yet.

 

On to my questions, then:

 

  1. Is the above plan sound? Am I missing something basic which would ruin it?
  2. After all the above has been finished, I will be left with a 256 GB nVME SSD as an unassigned drive. I want it to be used for PLEX (and its media cache). From the documentation it looks to be possible, but I might have missed something, advice would be helpful.
  3. Anything else you might think of, that would help me in my endeavor, would be awesome!

 

Sorry for the very long thread starters, I like being thorough, even it it might look like "too much information" :)

And thanks a lot, in advance, for your answers!

Link to comment

I am not an expert, so this is from an enthusiast to another.

 

For your process of creating the array and transferring your data, it will take a lot of time, but will work. I would strongly suggest to have 2 parity drives in the future. This will add more protections, but it will need to be 14tb as well.

 

On your first 14tb copy, I would suggest to not have the parity enabled for better performance. The cache could be useful, but you will fill it really fast so I would also bypass the cache for the first data transfer. (In the share, just disable cache) It will be slower in the beginning for sure, but I had an issue in the past when transferring a lot of data and filling the cache. Some file transfer did not work because of that.

 

When creating the cache pool, be sure to select a raid 1 cache pool. This way all your appdata will be duplicated between each cache drive. This will include the data from plex.

 

You can use the NVME drive for plex (specify another location than the default appdata), but the performance boost from NVME vs SSD is not that big for plex, in my opinion, unless you have 100 simultaneous users. Also, the plex data will not be duplicated or backuped using the NVME drive. I would not use the other NVME drive for Unraid, I am not sure you can activate it using this. A modern USB drive should do the work. 

 

If you used VM for daily usages, I would suggest using the NVME drive as passthrough. It will be a lot faster for that VM, but you won't be able to use it for anything else.

 

The GPU is pretty much useless for plex transcoding. It is not fast enough to transcode anything other than audio. So you would not need to configure plex for that. But it could be used to run a VM with the GPU passthrough.

 

I hope this answer your questions and I just want to remind you I am not an expert :D 

  • Like 1
Link to comment

Thank you very much for your reply. I was hoping someone would :)
I will structure my reply by points, makes it easier to continue the conversation.

 

  1. Parity drives: Yes, there will be 2 parity drives, I will buy two more 14TB drives in April, one of which will be the second parity drive. Long term plan is to change motherboard and CPU, have two LSI Controllers (8 HDDs + 16 HDDs), with parity and everything.
  2. Copying: would the cache fill very quickly, given read speed from one HDD is roughly equal to write speed to the destination HDD? At any rate, I will enable the cache pool after finishing all transfers, so thank you for this advice. I was under the impression that parity needs to be enabled first, otherwise it might not work. Also, I don't mind it the transfer taking a long time, as long as they go well.
  3. Cache pool: Yes, RAID1 was the plan. The SATA SSDs are identical, and they were not expensive (120 bucks both), and to be honest I am strongly considering buying two more fort the cache pool. Maybe it's overkill... you tell me :)
  4. The nVME SSD has no other destination but to hold PLEX and PLEX cache, as well as some containers I will likely have to add (Torrents, Usenet) - this is something I will have to read about, investigate, howtos, tutorials, documentation...
  5. No VMs. My main PC is powerful enough for a couple VMs, and I plan on upgrading it anyway towards the end of the year.
  6. No transcoding. PLEX is only for in-house usage, through 5GHz wireless, "Disable video stream transcoding" option is enabled. The GPU card is there because current CPU has no integrated GPU. It will go away once I upgrade my NAS hardware and I will use an integrated GPU.

Once again, thank you very much for your reply, I appreciate you taking the time to do so.

Link to comment

Here are a few suggestions.  

 

First, get unraid running on your hardware.  

Then, preclear each drive that you will initially use - this ensures each is in good working order, and you should do this with all new disks.  In your case, i would preclear those two new 14 tb drives at the same time, and then create a 2 disk array without parity.  You have a 3rd 14 tb drive that you will shuck - that would become your parity.

with your 2 disk unprotected array, copy all of your data - will all of it fit on these two drives?  if you dont plan on shucking the 10 tb drive, i would leave it full and out of your plan until after you have parity.

 

Once all of the other data is on your two disk array, you will want to preclear the rest of the drives - shuck that 14 and preclear it first, make sure it is ready to be parity. 

 

At this point it's up to you.  you can first add the parity drive and protect the array, then add the other smaller disks, or add the disks first.  I dont think there's actually a difference, but you will want to preclear them all before using them in your array.  

 

After the array has all of its disks, you can copy from the 10 tb you will not shuck.  The only question is whether you build parity before or after that data gets transferred.  If it were me, i would build parity second but it will only save you a few hours.  

 

after that you are up and running.  I wouldnt experiment with any of the unraid toys until that data is in the protected array.  

 

dont forget to preclear your disks before adding them!

 

Link to comment

I am all in the kimifelipe. It is really important to preclear the disks. (DON'T DO IT ON THE EXISTING DRIVE IF YOU HAVENT MOVE THE DATA TO THE ARRAY).

 

7 minutes ago, war4peace said:

Copying: would the cache fill very quickly, given read speed from one HDD is roughly equal to write speed to the destination HDD? At any rate, I will enable the cache pool after finishing all transfers, so thank you for this advice. I was under the impression that parity needs to be enabled first, otherwise it might not work. Also, I don't mind it the transfer taking a long time, as long as they go well.

 

When the cache is full, the write speed should be the same as reading from the HDD. I am only suggesting to bypass the cache only because I had issue in the past when doing a really large transfer with big files. You can enable the cache pool at the begining, you can just skip it in the share settings. image.png.cad33ad026c2e9bfed2c7d32770c276a.pngimage.png.fd5abe50298133e8ad77159dace31abf.png

10 minutes ago, war4peace said:

I am strongly considering buying two more fort the cache pool. Maybe it's overkill... you tell me :)

 

It really depends on what you intend to do. If you think you will fill the cache really often (other than the initial transfer), then yes this could be something you could do, but if you don't really fill it up and use it only for fast appdata access then there is not really any use for it. You are better to put the price of toward a GPU for plex transcoding or more drives :D

 

12 minutes ago, war4peace said:

The nVME SSD has no other destination but to hold PLEX and PLEX cache, as well as some containers I will likely have to add (Torrents, Usenet)

 

The plex data, torrent, other apps you would install would go by default to the cache. It is not mandatory, but it is by default. If you really, really, really want the NVME for plex, you can do a second cache array with both NVME for plex. But don't use just one. If the NVME fail you lose all the data in plex. 

 

16 minutes ago, war4peace said:

Once again, thank you very much for your reply, I appreciate you taking the time to do so.

 

No problem, I needed help in the begining so I am glad to give help in return.

Link to comment

@kimifelipe
Thank you for the advice.
So the plan would become:

  1. Configure LSI controller.
  2. Install Unraid on USB stick, launch, run initial configuration.
  3. Preclear 2x14 TB HDDs, preclear 4 TB empty HDD, create unprotected array using all three disks. That gives me 32 TB of empty space to copy the data to.
  4. Connect 1x 14 TB HDD (the full one), copy data over to the array.
  5. Preclear the 14 TB HDD, set it as a parity drive (I don't mind subsequent data transfers taking longer).
  6. Copy the 8 TB drive data, preclear 8 TB HDD, add to array.
  7. Copy the 10 TB data, figure out what to do with it (shuck with WD fix, or sell, or use it for something else, e.g. DVR connected to a TV, etc.)
  8. Play with Unraid apps and whatnot.

@vinid223
Cache filling: yes, it could fill up really quickly. I am not sure how Unraid works regarding Internet downloads, but I am taking into consideration the fact that I can easily reach 8 TB a day through downloading. I could start by adding both 500 GB SATA SSDs as well as the 256 GB nVME drive to the cache pool and see how it behaves when downloading at max bandwidth, then decide if I should expand it or not. If it's enough, this means two more SATA connections available for large HDDs, which is way more helpful.
As for GPU: there is no transcoding happening in the foreseeable future. Streaming locally to TVs, one file at a time, two max in very rare occasions. But never say never; if the need arises, I could pry out one of my watercooled 1080 Ti cards and slam it in the Unraid server :)

 

Link to comment

W4P, yes that sounds like a good plan.  Steps 5 and 6 are interchangeable - again, having parity will slow the transfer but it is your choice; will make no difference whether you do 5 or 6 first.  

 

Keep in mind that when you add your parity disk, unraid will "build parity" immediately upon starting the array - it will "write" parity to the parity disk.  This will take something like 30 to 40 hours with a 14 tb parity drive ( i use a 10 tb double parity and it required 24 hours to build parity the first time).  So factor that time period into your schedule.  

 

You dont want to write any data to the array during a parity build.  IMO.  Strictly speaking it is fine but I never risk it, I am never in that much of a hurry.  

 

good luck with it.  

 

WRT your reply to vinid, I dont point anything at my primary array, for example, i download to a separate pc and then transfer.  In your case you will point your downloads to the cache and it will serve the same purpose but mover will put everything into the array automatically.  

 

I also dont transcode on my array - i use a pc at every screen and let them do the work.  :)  

Link to comment

I normally download directly to the cache, never to the array, unless I bust the cache size. 

 

I point to transcoding only depending on your needs. I do have a lot of 4k REMUX content, so the bandwith is heavy on the network, so I transcode to lower that size. I should encode it thinking about it :D

 

Good luck with your project. 

 

Let me know how it goes.

Link to comment

My TVs are connected to 5 GHz wireless, there is enough bandwidth for direct streaming.

On another note, I have decided not to bother doing the hardware upgrade later, so I ordered new Motherboard, CPU, PSU and RAM.

  1. Motherboard: ASRock X570 Phantom Gaming 4
  2. CPU: AMD Ryzen 5 2400G (I wanted a 3400g but it's nowhere to be found where I live)
  3. PSU: Corsair RM850 2019 - 12 SATA connectors
  4. RAM: Corsair Vengeance LPX 32GB (2x16), DDR4, 3000MHz
  5. A couple USB 2.0 sticks for Unraid.

They should all arrive tomorrow.

 

This way I will be able to use both nVME SSDs as well as the 2x 500 GB SATA SSDs I bought. The SSDs as well as the new HDDs are already installed in the case. The LSI controllers also arrived at customs yesterday, waiting for processing.

I'm pretty excited about the whole thing, it's my first time using Unraid, and I guess I'll mess up a few things, but in the end it's a learning process.

Thank you all for your replies, I appreciate it, you've helped me clarify quite a few things!

  • Like 1
Link to comment

Happy to report I have it running, installed a few dockers I need, almost finished copying all data from individual HDDs to the array (down to the last HDD now), and looking at ways to further tune/optimize the setup.

Big thanks to everyone who helped, and to the community in general, for being no less than AWESOME!

Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.