Thinking about biting the bullet and going unraid

[reconstruction]

Looking to do some changes in my home network file storage and a buddy is pointing me towards unraid, but I'm curious if I'm headed the right direction

 

Current setup:
Old (OLD!) Xeon HP tower server running Raid 5 (4-4TB drives), Ubuntu 20.04.  Samba shares/Time Machine, mySQL, Plex (poorly), and a few other little things.

 

 

What I'm looking to do/use so far (several things are up in the air as I figure things out)

-i7 12th gen processor 

-64 GB ram

-LSI card to run 8 14tb NAS drives in Raid 5 (will have a spare HDD to swap in whenever failure strikes)

-Some kind of motherboard (Haven't decided on form factor yet)

-Case to support 8 HDDs (Haven't decided on form factor yet, but needs to be a tower... no racks)

-PSU sufficient to spin up everything

 

 

System goals (everything will be LOCAL access only, no internet exposure - I have a VPN server if remote access is needed)

-samba shares/permissions control & TimeMachine backups

-MySQL

-Apache/PHP

-Plex (only in-house sessions, not looking to run a dozen transcodes at once)

-2-4 Linux VMs accessable via VNC or xRDP

 

I would anticipate the Unraid being mostly headless/CLI - Having a web guide accessible from the primary use systems would be nice.  This seems like a more elegant solution than my current ubuntu server.

 

 

Thoughts/suggestions/feedback/jokes?

[trurl]

1 minute ago, reconstruction said:

run 8 14tb NAS drives in Raid 5

Unraid is NOT RAID. Unraid will do everything you mention and much more

 

https://wiki.unraid.net/Manual/Overview

[reconstruction]

Yeah, that was poorly communicated on my part.  unraid would manage the storage... I wouldn't be using any form of raid controller anymore when this system goes online.

[TimTheSettler]

Your proposed setup is very similar to the servers that I built.  Be sure to get a UPS, a mobo with two M2 NVMe slots for a nice fast cache pool, and with 8 large drives I recommend that you use dual parity.  Note that it takes a very long time to do a rebuild of a large drive.  My 18TB drives took about 28 hours.  The 10TB drives took about 13 hours.  Below is a screenshot of a recent parity check (18TB).  Clearing the drive (pre-clear), building a drive (rebuild of failed drive or initial build of parity), and the parity check shown below all take about the same amount of time.

 

 

Also wanted to mention that the trial key is a good way to try it out without committing but I think you'll like it.

Edited January 25, 2023 by TimTheSettler

[reconstruction]

Appreciate the feedback.  I'm looking forward to the project, gunna take my time picking out the parts.

 

So far, I'm looking at a Meshify 2 full size case (the XL is too large for where it's going to live).  Loaded with fans it should stay nice and cool.  The system will go on a "pure sine wave" UPS that my current server is sitting on.  

 

Good to know it'll take a while to build - I won't be in a hurry when the time comes.  Going to spend a bit of time getting it secure as I can.... always paranoid about file security/data integrity.

[reconstruction]

Well, after doing some more reading... here's the current plan:

 

-Meshify 2 Case (full size, not XL or Compact)

-i7-12700K

-64 GB Ram

-ASRock Z690 (Newegg Link for Example)

-LSi Card that'll support 8 additional SATA Ports

-2 WD Black 500gb NVME Drives

-Several 140mm Fans (as many as I can fit without blocking room for HDDs)

-Corsair CP-9020235-NA Power Supply (supports 14 drives)

-Extra HDD Trays for t he case

 

I'll have Sata ports to spare at that point, enough ram to run the VMs that I want/need to run, and enough airflow to keep everything happy will be inside a cabinet that's setup with several AC Infinity fans that push/pull air across the cabinet connected to a temp controller.  Has worked great on the existing systems that are in that cabinet... one of which is going to be retired for this build)

 

 

Edit to add

 

Well, plan is moving forward

 

Meshify 2 Case, RAM, ASRock Z690, LSi Card, WD NVME, Power Supply are all ordered.

 

Waiting for some things to get here before I figure out what length SATA cables I'll need, how many 140mm fans I'm going to cram into the case, etc.  I'll order extra HDD brackets when I place the next order.


Haven't ordered the processor just yet, hoping to find a sale.

Edited February 5, 2023 by reconstruction

[TimTheSettler]

Sounds good.  It's basically the same as (but better than) my unRAID04 server.  When you connect the HDD be conscious of which drive connects to which port.  What I mean is that your parity drive should be the best and fastest of all the drives and have the most dedicated connection.  Some people recommend avoiding the mobo SATA ports but I haven't seen any problem so maybe it's just that crapier mobos have poor SATA ports.  Although the LSI card is nice and fast there's a post shown below where the speed drops when more drives are connected.  Do the mobo SATA ports do the same thing?  I'm not sure...

 

I think your new mobo seems to be pretty nice.

 

 

 

[Lolight]

1 hour ago, TimTheSettler said:

Some people recommend avoiding the mobo SATA ports...

Hmm...

Can you point me to a recent (or any) example of such recommendation?

Edited February 6, 2023 by Lolight

[-Daedalus]

LSI 9207-8i is a good bet. It's PCIE 3.0, so you'll get ~500MB/s to each drive, if you want SATA SSDs in there at any point, and it's only a HBA, so no flashing needed.

Or if you know you're only ever going to use HDDs with it, then a 9211-8i will do fine (you'll get around 330Mb/s max per drive)

[reconstruction]

I ordered a 9221 in “IT mode” - frankly speed isn’t an issue for the data access, with exceptions to the VMs which will have their primary drives in a nvme pool. 

[TimTheSettler]

19 hours ago, Lolight said:

Hmm...

Can you point me to a recent (or any) example of such recommendation?

 

Sorry, that statement was poorly worded and was very generalized and I should explain it better.  The onboard SATA ports are usually fine.  In fact I use them for all my servers.  However, it's possible that you have one of the following problems in which case a good onboard SATA controller card like the LSI models is better:

• The motherboard is old and uses older SATA standards.
• You're using an M2 SATA card which shares an onboard SATA port.
• The motherboard is cheap and uses a chipset that might not be recognized.
• The motherboard uses a chipset that only supports a limited number of SATA ports but the motherboard manufacturer has added more ports which use another controller chip.

Some links...

https://www.reddit.com/r/intel/comments/rxasm6/over_4_sata_ports_on_a_b660_motherboard/

 

https://www.msi.com/Motherboard/MAG-B660M-MORTAR-WIFI-DDR4/Specification

 

https://ark.intel.com/content/www/us/en/ark/products/218832/intel-b660-chipset.html

 

 

[TimTheSettler]

I liked this article.  Some people here might find it useful.

 

https://unraid-guides.com/2020/12/07/dont-ever-use-cheap-pci-e-sata-expansion-cards-with-unraid/

[Lolight]

3 minutes ago, TimTheSettler said:

 

Sorry, that statement was poorly worded and was very generalized and I should explain it better.

No problem, thank you for the clarification

[neal_is_king]

I got the 9207-8i pre-flashed to IT mode from some guy on Amazon. It has treated me well.

 

If 8 drives is all you plan to run, I'd look at the Node 804 too. The footprint is impressive for how much you can get in there. I switched from an Arc Midi 2 maybe 5 years ago, I love it.

 

Speaking of love it, unRAID is awesome.

[reconstruction]

I figure 8 drives will be the max, but I'll have an abundance of SATA ports.  The MB has 8 ports (one gets disabled if I install a 3rd NVME), and then add the 8 ports from the LSI card.

Doing the math on how much storage space - I should be good for a long time.  Server that's getting retired lasted us 10 years, I'll be thrilled to get 10 out of this one.

[Aran]

The MB has 8 ports (one gets disabled if I install a 3rd NVME)

I once worked with this bord.

One sata port (i don't know which one, check manual) is disabled if the second M.2 port gets occupied by a SATA-type ssd. If you install a NVME ssd, the sata port will still work.

Good luck with your build!

[reconstruction]

Well, looks like the last of the parts needed for the build will be here 2/27... so its time to start ironing out some things.

 

 

My initial game-plan:

-Build out system

-Configure BIOS

-MenTest86

-Install unraid

-configure HDDs & NVME (Cache/VMs.... will reach forums/wiki for best practices)

-configure shares/folder layout (I'll be hitting the forums/wiki for this... it looks like this is MUCH different than what I'm used to with ubuntu from what I've gathered so far.)

-Manage user level permissions/folder permissions (read only/write access/deny)

-Migrate Data from old system to new system (my poor router.....)

-Plex Docker

-Configure TimeMachine backups / other system backups

 

 

Then there's the part that I'm not 100% sure how to proceed.  Part of what kicked this all off and pushed me to upgrade away from my existing server was I was out of space to create a SQL database that's going to be somewhat large - I just didn't have enough space to work with the data.  Eventually, there will be a web access to the database, however it will NOT be internet facing.  Only local (or VPN) into my home network.

 

I figure I can either do a MySQL and Apache/PHP docker or I can do an ubuntu server VM and run both on that.  Any suggestions there?  I'm a HTML/PHP newb, but will muddle through it over time (projects keep me sane!).... I have a few PHP buddies I can ping as time goes on - once the system is up and running.

Edited February 22, 2023 by reconstruction

[TimTheSettler]

6 hours ago, reconstruction said:

I figure I can either do a MySQL and Apache/PHP docker or I can do an ubuntu server VM and run both on that.  Any suggestions there?  I'm a HTML/PHP newb, but will muddle through it over time (projects keep me sane!).... I have a few PHP buddies I can ping as time goes on - once the system is up and running.

I'm not sure what you're development level is at but starting at a simple level you should split your development into two parts.  Create a web server container and create a separate DB container.  A number of apps out there already use this approach (like NextCloud).
 

[reconstruction]

Well, we are up and running.  Got the system built the other day, last night powered it up and ran a memtest86 as a verification that everything was good.

I was 100% underwhelmed when booting unraid for the first time... was expecting a typical software installation.  Nope, it just booted up and started working.  Lot easier to get going than the ubuntu/esxi/other systems I've worked with in the past.

 

Created a mess of user accounts, a few shares/basic folder structure for everything.  Setup a Time Machine docker to keep my mac's happy - backed up very quick, that was nice.

 

Set the shares I plan on using to "private" as I'm a firm believer in only giving users access to stuff they need, rather than everything.  Having an issue mapping folders from my Mac right now, but I'm sure that's just an mis-configuration somewhere that I haven't found just yet.

 

So far, seems like a pretty legit file management system..... very intuitive.

 

 

Well, figured out my mapping issue.... search is my friend.  Samba needed to be set to "export" 

Edited February 25, 2023 by reconstruction

[TimTheSettler]

On 2/25/2023 at 12:04 PM, reconstruction said:

Created a mess of user accounts, a few shares/basic folder structure for everything.  Setup a Time Machine docker to keep my mac's happy - backed up very quick, that was nice.

 

I should've mentioned that the best thing to do when you're up and running is to plan out the shares and users BEFORE you create them.  Do this in Excel with users listed along the side and shares along the top then mark off which users need access to which share and what kind of access they need.  You might find a lot of duplication.  The more shares and users you have will mean more complication.  Also plan out why you need the shares and the users.  In my system I wanted somewhat open access to media (audio and video), more restricted access to family stuff, and a completely restricted "home" folder for each member in the family.  So, in the end, each user has their own share (private), there's a common share (private), and a media share (secure).

 

This is what my SMB settings look like:

 

[reconstruction]

I actually did that with pen and paper before I installed anything on the system. Figured out my file structure and user permissions. 
 

i do wish there was a simple way to control sub folder permissions, but I can make things work as-is. Rather than have /ShareA/Folder1 and /Folder2…. I’m just dropping everything to the base level. More overall shares, but can still make it work. 
 

had an issue transferring some large files (20+gb files) from ubuntu to UnRaid…. Connection time out errors right and left. Moving the files to an external drive on the old server and will manually move them on unraid once the copy process is done in a day or two. 

[TimTheSettler]

22 hours ago, reconstruction said:

i do wish there was a simple way to control sub folder permissions, but I can make things work as-is. Rather than have /ShareA/Folder1 and /Folder2…. I’m just dropping everything to the base level. More overall shares, but can still make it work.

 

I used to run a Windows domain so I got used to the sub-folder/file level permissions.  That was hard to leave behind but in the end I like what I have now.  When I was figuring all this out (on pen and paper) I asked myself, do I really need all this granularity?

 

22 hours ago, reconstruction said:

had an issue transferring some large files (20+gb files) from ubuntu to UnRaid…. Connection time out errors right and left. Moving the files to an external drive on the old server and will manually move them on unraid once the copy process is done in a day or two.

 

I had this problem in Windows and I narrowed it down to the anti-virus (Windows Defender).  It could be that something is grabbing the file you are trying to transfer and holding onto it which then causes the transfer to fail if the file is not let go in time.  (For example, let me transfer this file from here to there.  Anti-virus steps in and says, wait a second, let me check that file first.  It takes a while to check the file and in the end the transfer has timed out.)

 

Note that if you move or copy files from some external location (externally connected drive or docker app) then that file has the permissions of that external location (probably root).  If you then try to move this file in your share then you won't have permission (since the share uses "nobody").  In this case you'll need to run the "Docker Safe New Perms" tool.  This tool replaces all permissions on all shares to nobody.  You get this tool when you install the "Fix Common Problems" plug-in.

 

[reconstruction]

Thanks for the heads up on that, you just saved me a huge headache in the very near future!!!

 

I’m migrating from a 11-12 year old Ubuntu Server that I ran for a few different things.  Switching to unraid will simplify a lot of things and make maintenance easier.  Having a web gui to work with is really nice, previously I was setup for token based SSH sessions which I only had configured on 2 computers in the house…. Now I can use my phone for basic things, which is nice.

 

Part of what prompted me to go to UnRaid was the simplification of user and file management. I used multiple logins across multiple computers for various tasks and inevitably ran into file ownership issues between the systems - this was my own fault in the long run.

 

One thing I do like about the granular level permissions is its defense against crypto/ransomware.  Frankly, I’m less concerned about HDD failure and more concerned about getting my data encrypted by some kid with nothing better to do.  

 

The simple solution for me is to just use more shares on unraid and less folder nesting.  Not as pretty from a data Managment perspective, but it’ll work.  Each physical and/or virtual machine will only get access to write to folders needed for their task.  

 

Unraid is replacing the old ubuntu server I used for file storage and a MacMini that I used for a virtualization host (VMWare Fusion).  Both will be retired once I’ve verified that everything is happy on the new system.

 

I did consider migrating the VMs from fusion to the new system, but in the end it makes more sense to just rebuild.  I will miss the snapshot feature - but i can still physically backup the VMs for “disaster recovery” if needed.

[JonathanM]

1 hour ago, reconstruction said:

Frankly, I’m less concerned about HDD failure and more concerned about getting my data encrypted by some kid with nothing better to do.

Backup backup backup.

 

There are more stringent security things you can put in place for WORM data, here is an old thread (some things no longer apply) that discusses some of the strategies.

https://forums.unraid.net/topic/58374-secure-writing-strategy-for-unraid-server-using-write-once-read-many-mode/

 

[TimTheSettler]

7 hours ago, reconstruction said:

One thing I do like about the granular level permissions is its defense against crypto/ransomware.  Frankly, I’m less concerned about HDD failure and more concerned about getting my data encrypted by some kid with nothing better to do.

 

5 hours ago, JonathanM said:

Backup backup backup.

 

As JonathanM points out, it's good to have a backup.  My backup server (separate machine and at another location) runs daily and since all backup software nowadays uses deduplication it doesn't use much space and I can keep daily, incremental changes.  If someone encrypts my data with a ransomware attack then I just restore from a day or two back when the data was clean.  The following is a screenshot of my backup archive for my Pictures folder (I use Vorta based on Borg).