Planning Stages, Have Some Unique Ideas, Need Advice

[House Of Cards]

Greetings,

 

So I plan to make the jump from TrueNAS due to their ongoing problems with jails/plugins, and because adding storage space is an expensive task when drives have to all be upgraded to grow the pool.  It has been rock solid, but for home use, I think Unraid looks more flexible.

 

The not-so-unique is that I have a 12-bay rack-mounted case to use...  I plan to fill it up with drives and utilize more of the plug-in architecture of docker to replace all the other things I have into one server.  My general ideas are to use these cheap components, because I don't really have high-demands...

 

Motherboard...
https://www.amazon.com/dp/B0BMLJJXS3/?coliid=I3BUEQX5GYW8LF&colid=2D3CNHHDTB8MO&psc=1&ref_=lv_ov_lig_dp_it

SATA Expansion Card...

https://www.amazon.com/dp/B0B2WRDXQJ/?coliid=I1ETVYEGVPXPFN&colid=2D3CNHHDTB8MO&psc=1&ref_=lv_ov_lig_dp_it

 

The primary purpose of this machine will be storage of documents and mapping shares to the devices in my house.  One of the other primary functions will be an Emby server running in docker, maybe home assistant, grafana...  Basic stuff but all under one machine.  Nothing really demanding.

 

NOW, for the unique thing... 

 

One idea I've always kicked around in my head was to use Emby to automate my own media server in a more interactive way.  What I mean is that most people dump everything in there, and you play it when you want to.  I would like to have content change, and this is what I envision... let me know if you have any ideas how best to get started before I delve in and create my own roadblocks.

 

I will have shares for emby...  "Feature Films" and "Television Shows", for example...  And I will have other shares for Emby like "Limited Engagement Films" and "Limited Engagement Shows"...  These will be what is shown in Emby, but it will not be the primary collection of content.

 

I will have main shares of "Movies" and "TV Shows" where I store everything, but would like to make scripts to automatically modify transferring between the two.  So imagine December 1st happens...  A script would run that copies "A Christmas Story" from the main share ("Movies") to the Emby share of "Feature Films".  Now your Emby server would show "A Christmas Story" as recently added just in time for the holidays.  Then on December 31st, another script would run deleting "A Christmas Story" from the Emby ("Feature Films") share, and it would still exist in the main "Movies" share to reappear next December.

 

I realize that the scripts would be cumbersome to stay on top of, but my A.D.D. should be able to keep up.  What this would do is create something more like a curated server with content that changes constantly.  Imagine adding 9/11 documentaries in September, or Jaws every summer...  Throw in some campy horror movies in every October...  Movies about a tragedy around the time the tragedy happened in real life...  Your media center would constantly remain relevant.

 

So I guess the question is do you all have any suggestions on maybe a tool to use to help manage this, or caveats to look out for?  I should be ready to order the components shortly, then I'll build and test, then I'll get working on setting everything up... 

 

Any comments would be appreciated.

 

Thanks,

Steven

[ChatNoir]

22 hours ago, wormuths said:

SATA Expansion Card...

https://www.amazon.com/dp/B0B2WRDXQJ/?coliid=I1ETVYEGVPXPFN&colid=2D3CNHHDTB8MO&psc=1&ref_=lv_ov_lig_dp_it

That could be the cause of serious disk access slow downs. 10 drives on a 2 drives controller is not a great idea as it require port multipliers (x2 here I think) that are not recommended.

 

I would suggest looking at this thread :

 

[House Of Cards]

4 hours ago, ChatNoir said:

That could be the cause of serious disk access slow downs. 10 drives on a 6 drive controller is not a great idea as it require port multipliers (x2 here I think) that are not recommended.

 

I would suggest looking at this thread :

 

 

See?  That's why it pays to ask.  LOL

 

So I updated my expected motherboard to this one, mainly because it has 6 SATA ports, supports 3 M.2 which I will use PCIe M.2 drives in, and it has 2.5GB Intel LAN.

https://www.amazon.com/dp/B08SYWZW4S/?coliid=I38PA2BI6OWT7N&colid=2D3CNHHDTB8MO&psc=1&ref_=lv_ov_lig_dp_it

 

This with a 6 core CPU with on-board GPU...

https://www.amazon.com/dp/B086MN38Q2/?coliid=I7SLDV069BKG&colid=2D3CNHHDTB8MO&psc=1&ref_=lv_ov_lig_dp_it

 

So I intend to utilize all M.2 slots.  It appears as though I can use PCIe M.2 to not disable the onboard SATA ports, although I am admittedly not up on these latest technologies.  I'll possibly use a small M.2 for a "scratch" drive for speed in transfers.  The other two I envision having a BTRFS mirrored pair for docker and such...  

 

This should leave 6 SATA ports on the board, and my case supports 12 swappable drive bays for storage drives.  I essentially need to add 6 more SATA ports.  The board has 3 PCIe 4.0 x16 slots for expansion cards.  So with everyone's collective knowledge, what is the cheapest option to add the 6 additional ports using eaith a single or multiple expansion cards? 

 

Maybe this?  Or is there a better/cheaper option?

https://www.amazon.com/FebSmart-Expansion-System-Marvell-Controller-FS-S4-Pro/dp/B09BMBPZLZ/ref=sr_1_5?crid=1I7LT2KTSY5TF&keywords=asmedia%2BASM1064&qid=1675551200&s=electronics&sprefix=asmedia%2Basm1064%2Celectronics%2C148&sr=1-5&th=1

 

I appreciate the advice...

Steven

 

[House Of Cards]

This one is $33...

https://www.amazon.com/10Gtek-Profile-Bracket-Controller-Expansion/dp/B09Y577WJS/ref=sr_1_11?crid=9SHSFJS42EY9&keywords=asmedia%2BASM1166&qid=1675551591&s=electronics&sprefix=asmedia%2Basm1166%2Celectronics%2C202&sr=1-11&th=1

 

Uses the ASM1166.

 

[House Of Cards]

And I just saw I need an 11th Gen processor for the M.2 slots.  🤑

[Vr2Io]

6 hours ago, wormuths said:

So I updated my expected motherboard to this one, mainly because it has 6 SATA ports, supports 3 M.2 which I will use PCIe M.2 drives in, and it has 2.5GB Intel LAN.

https://www.amazon.com/dp/B08SYWZW4S/?coliid=I38PA2BI6OWT7N&colid=2D3CNHHDTB8MO&psc=1&ref_=lv_ov_lig_dp_it

Great with so much PCIe slot.

 

6 hours ago, wormuths said:

https://www.amazon.com/10Gtek-Profile-Bracket-Controller-Expansion/dp/B09Y577WJS/ref=sr_1_11?crid=9SHSFJS42EY9&keywords=asmedia%2BASM1166&qid=1675551591&s=electronics&sprefix=asmedia%2Basm1166%2Celectronics%2C202&sr=1-11&th=1

Good too, but SATA socket too close each other, some SATA cable won't fit due to SATA plug little bit bigger / wider.

[TimTheSettler]

I started with TrueNAS and then switched to unRAID.  They work a bit differently when it comes to the drives.

 

In TrueNAS the OS sits on one of the drives.  I had mine on an M2 NVMe drive.  You then define a pool of drives which could be anything.  If you have more than one M2 drive then it was common to set this up as the L2ARC or SLOG.

 

In unRAID the OS sits on the USB and stays there.  You then define two types of pools, an array and a cache.  Your HDDs make up the array and your M2 drives would be the cache.  The M2 drives are defined as one cache.  Just like the array this allows for redundancy to be built into the cache.  If an M2 drive fails then you replace it without losing anything.  You can have more than one cache and there are other kinds of configurations that you can do but this is the basic and typical setup (this is what I did and you're basically on your way to do what I did).

 

The idea here is that your VMs and docker apps run from the cache and your data sits on the array but the cache can be used as a buffer for file transfers so that file changes are saved first to the cache and then moved to the array.

[House Of Cards]

Yeah, I'm on TrueNAS now, have been for half a decade or more.  It's bulletproof file storage, but for the home user, it's just not flexible.  I'm planning to keep the TrueNAS server because it's a more than adequate machine for file storage, and use that for backup of the UnRAID machine.  This gives me the best of both worlds, like the long-term protection ZFS offers along with snapshots.  Just keep a copy offsite and I'm golden.

 

The main reasons for UnRAID are two things...  First, flexibility in adding storage.  I just can't justify buying five disks to add more storage to TrueNAS.  Second, the flexibility of docker being able to directly share storage and interoperate with one another.  The logistics of making BSD jails and trying to get them to share data with one another is a complexity beyond the average persons ability to fathom.  Try to port all your data into Grafana between multiple jails and you'll know what I mean.  LOL

 

I will make two cache pools.  One small single NVME drive as a scratch disk for accelerated writes, and one larger NVME pair for redundancy where all the default shares for docker and the like will live.  Beyond that, I have a rack case with 12 removable drive trays for plugging in 12 disks, making for easy swaps and upgrades. 

 

The needs of this machine are small, and the media server aspect is the most demanding.  It will have mild use mainly just with people at home.  This machine should outlive me.  😆

[mathomas3]

I dont know what your budget is and if you already have the case or not but... I would recommend leaning on a larger case with spare HHD slots...

 

I have been using unraid for 10 years now... and I have housed the hdds in 5 different cases now... Currently on a 24 bay DAS and I might have to get another one soon... 

 

I have had good luck with 8tb drives. I havent had a single one go bad on me. Before this 4,5,6tb drives failed right and left. 

 

something to think about... Oh and I would recommend using SAS controller. You can add a sata drive to a sas controller but you cant add a SAS drive to a sata controller. SAS drives have worked well for me as well

 

In short... using SAS enterprise drives is what I would recommend and giving yourself room to expand is a good problem to have

[House Of Cards]

1 hour ago, mathomas3 said:

...giving yourself room to expand is a good problem to have ...

Exactly why I'm here.  LOL

 

My current TrueNAS has 5 drives which with Raid Z2 and a spare only gave me 6 TB when I first built this 6 or so years ago.  Now I don't have too much stuff, so that worked for a long time.  Now I'd like to double that, and I have to go buy 5 drives to upgrade the storage.

 

For the price of those five drives, I could buy mid-range components and use all the spare stuff I've accumulated over the years and build this server.  I already had the 12-bay case I planned to switch into to add another pool to TrueNAS, so a motherboard, CPU, RAM and some small miscellaneous parts and I can throw all these "smaller" drives I have which are otherwise unusable into the 12 bays and upgrade them as they fail.

 

Not to preach to the choir in these forums, but that's what the benefit of UnRAID seems to be.  I don't have to throw out that 2TB drive.  I can just throw it in here and use it until it dies.  When it does, just throw a bigger one in to replace it.  For me, the storage needs are small.  I could buy two 6 TB drives for $200 and have plenty of room for what I have. 

 

I've been planning how to do the drive configuration, and I think I have a solution which works...  At least on paper.  Planning the caches, etc...  It's a combination of advice, which hopefully balances use of all 12 drive bays with power efficiency.  I've broken out the bundle of 10 disks (2 are parity, so 10 for data) into groups for shares which should allow for future upgrades of space available to each share, while only spinning up certain disks when, for example, watching media.  My dockers and VM's will all be running on a mirrored pair of NVME and backed up regularly.  One additional small NVME for accelerated writes to the shares.

 

Then, on a schedule which makes sense, I can backup to the TrueNAS machine which will maintain that bulletproof redundant copy of data and backup offsite periodically.

 

I'm still tinkering with components, but really trying to keep this inexpensive.  I'm going for 5400 RPM drives, since I don't think streaming a movie needs the extra power consumption of a 7200 RPM drive.  I'm trying to avoid watching my electric meter spin while 12 drives spin at 7200 RPM 24/7.  😁  The way I'm planning this, all my services (docker media server, etc...) will be running on the NVME, so under idle time, all the drives should spin down.  Drives should only power on when streaming, or when mover runs.

 

Which brings me to a question...  As far as using the cache drives.  Is there any real benefit to enabling the cache for something like a movie share?  Beyond the initial write, what benefit would the cache drive do?  Or should I save the cache for better suited uses?

 

Thanks...

[mathomas3]

I think you have a pretty good idea as to what you need/have... I would still suggest a sas controller to make things easier for you down the road should you need more space... I would think that your cache drives could be stored inside the case vs using a tray for it and allowing you more room for hdds... Having a spare is always nice and you can avoid the stress of preclearing/testing a drive

 

having a cache drive comes in handy when there are high r/rw operations. I think you could do without... 

 

Though having a app pool with two drives ssd would be a good idea 

 

One other suggestion I would make... when making the share permission and what not... I would suggest the allocation setting be set to 'fill-up'... reason being is that when you start transfer data to the pool... only 1 disk + parity drives are spinning and I think it's less wear and tear on the drives, because once the drive is full it only has to read from then on... it's a theory 

[JonathanM]

22 hours ago, House Of Cards said:

I can just throw it in here and use it until it dies.

Be VERY careful with that philosophy. Your 'nym very much applies here, as each data drive is equally important when it comes time to rebuild a failed drive. The last thing you want is for a rebuild of a different drive to fail because one of your rarely used array drives buckles under the strain of being read completely.

 

The ability to rebuild any single failed drive relies on the rest of the array drives performing flawlessly for the duration of the rebuild. Never leave a "failing" drive in the array. A "good" drive unexpectedly dying means all your questionable drives are suddenly very important, regardless of what data was on them.

 

Periodic parity checks help verify that all drives are still worthy of array participation. Setting up notifications and acting on any warnings promptly will help you keep the array healthy.

[House Of Cards]

3 hours ago, JonathanM said:

Be VERY careful with that philosophy. Your 'nym very much applies here, as each data drive is equally important when it comes time to rebuild a failed drive. The last thing you want is for a rebuild of a different drive to fail because one of your rarely used array drives buckles under the strain of being read completely.

 

The ability to rebuild any single failed drive relies on the rest of the array drives performing flawlessly for the duration of the rebuild. Never leave a "failing" drive in the array. A "good" drive unexpectedly dying means all your questionable drives are suddenly very important, regardless of what data was on them.

 

Periodic parity checks help verify that all drives are still worthy of array participation. Setting up notifications and acting on any warnings promptly will help you keep the array healthy.

Yeah, I understand that...  The drives I'm talking about aren't "failing", they were just retired because they were too small for a particular use, etc...

[trurl]

2 minutes ago, House Of Cards said:

retired because they were too small

I always recommend fewer larger disks instead of more smaller disks. Large disks perform better. More disks requires more hardware, more power, and perhaps most importantly, each additional disk is an additional point of failure.

[House Of Cards]

On 2/10/2023 at 1:29 PM, mathomas3 said:

I would still suggest a sas controller to make things easier for you down the road should you need more space...

I'm not familiar with SAS controllers, never used them.  Don't they need to be in IT mode or something?  And generally more expensive?

 

With the 12 bays, two for parity, I don't know if needing more would be an issue for quite some time.  If I went to the latest 20 TB drives, that gives me 200 TB of storage.  I don't envision needing more, and by the time I do, I would likely just buy a whole new system.  Do you know of a cheap SAS controller that comes ready to go, without flashing or anything?  I'm not opposed to it, just don't know if it's necessary.

[House Of Cards]

6 minutes ago, trurl said:

I always recommend fewer larger disks instead of more smaller disks. Large disks perform better. More disks requires more hardware, more power, and perhaps most importantly, each additional disk is an additional point of failure.

Yup, I get it...  It just offers an opportunity to use up something with a useful life remaining rather than tossing them out.  On the plus side, rebuilds will be quicker.  👍

[mathomas3]

The reason I like SAS controllers is that they are NOT consumer grade products... SAS is enterprise... 

 

I have had too many sata devices fail(over 10 disks) on me... I refuse to add them to my array due to failures... 

 

each sas contoller port can handle 8 disks each... normally they have two ports... thus they can handle 16hdd... this board was a quick search here... just searching google for sas+unraid should provide you with some good options...

 

I would like to counter Trurl's post... "fewer larger disks instead of more smaller disks" I have 2x5tb disks and 20x8tb disks... the 8tb disks have never failed on me... and should I want to upgrade to something like a 16tb or 18tb... I would have to buy 3hdd just to increase the space by 8tb or 10tb... for me it makes more sense to add another 8tb disk then buying something newer... 

 

My point being is... dont be afraid to use renewed enterprise SAS HDD, they are built to last... The prices on these drives tend to be lower because SAS is enterprise thus many are not interested/know about these drives yet there are many on the market

 

the sas controller should be a good fit for you due to your 15hdd requirement(breakout cables will be needed) and will provide you with better motherboard options(not needing 6sata ports on the MOBO) ... 

 

I would suggest this... find a good deal on a stack of hdds(used or not that are NOT consumer grade) 12tb or more and the build system around that... 

 

Im stuck on the 8tb drives as they are around 50 bucks a piece...

 

IMO in the long run... you will find SAS drives are cheaper and they will last longer... (if you willing to use renewed drives and or find a good deal) 

 

this was a long post... im drunk, sorry... but in the ten years of using unraid... I strongly recommend using SAS drives... if you would like me to count the number of my sata vs sas drive failures I can... but it would be something like this... SASx0 SATAx12+

[mathomas3]

I have a box of the failed drives

 

it's not light

Edited February 12, 2023 by mathomas3

[mathomas3]

Dont think about using WD green/red drives... If you are... you and I will have words

[mathomas3]

another good reason to have a sas controller... should you decide to upgrade the mobo/CPU combo... the sas controller gives you greater flexibility on the combo you decide on... 

[mathomas3]

Heck... I have a sas controller that I was using 8 months ago(with breakout cables) that I would  be willing to part with should you and I come to an agreeable arrangement... 

[mathomas3]

IE... you play for shipping plus a Starbucks GC? I would be willing to provide proof that the card works... I would need to use some of my older hdds+and under powered PSU to do so... 

 

the choice is yours

[House Of Cards]

See?  Who says you can’t get good advice from drunk people.  LOL

 

I don’t use the red drives since the SMR thing.  And I was planning 8TB drives just because of the price point.  Seems a good bang for the buck right now.

 

I could go SAS controller, no concerns there.  That card you pointed out is pretty cheap.  That and some breakout cables and I’m in business.  The only question is my case bays have SATA connections.  I think you can use SATA drives on the SAS controller, right?  But not the other way around?  
 

Thanks!

[JonathanM]

3 hours ago, House Of Cards said:

 I think you can use SATA drives on the SAS controller, right?  But not the other way around?  

Precisely. Just need to be careful when you transition connector styles, the cables are one way only, and they produce SATA->SAS and SAS>SATA cables that look identical but function only in the designed direction, forward or reverse breakout.

 

16TB SATA Exos are available as refurbs for $170 or so if that fits your needs.

[Lolight]

On 2/12/2023 at 7:19 PM, House Of Cards said:

Yup, I get it...  It just offers an opportunity to use up something with a useful life remaining rather than tossing them out.  On the plus side, rebuilds will be quicker.  👍

You might be facing a point where it's just not worth your time, electricity and effort to keep re-using certain old, obsolete but still functioning hardware.

A fresh start with new, efficient and up to date components will make your project simpler, less problematic and cheaper in the long run.

Edited February 15, 2023 by Lolight