Hey, I'm new here. Is unraid for me?


eb.

Recommended Posts

Hi, I came across unraid recently and it definitely has my attention. I'm just curious if the community here thinks it would be good for my situation or not. I currently have a Windows 7 PC running 24/7 as a Windows Media Center server. WMC accesses live television from the network HD HomeRun device. XBox 360's on the network then act as WMC extenders to view the live programming or to schedule recordings. Any live television that is recorded by WMC the Plex server in turn adds to it's library. I also have iSpy running and acting as a DVR for a few network security cameras. Our family photos and videos are stored on this computer and are backed up to external on occasion. There is a program (MCEBuddy) running that removes commercials from the recorded TV and is fairly GPU intensive when it's processing a file.

 

I would like to separate the above functions that are running 24/7 from the everyday use of the PC just in case something nasty were to get downloaded. I'm stuck on Windows 7 because of WMC not being available on Windows 10, but if I could have both OS's running with Windows 10 being the one that my family could use for everyday stuff I think that could work well. For drives I currently have a 1TB m.2 SSD, 500GB SATA SSD, two 2TB 3.5" SATA HDD, two 4TB 3.5" SATA HDD. The m.2 was purchased this week before I came across unraid and is still in it's packaging unused, so I could still return it and use those funds elsewhere if it would make more sense. I would appreciate advise on how to layout the different drives if possible (cache, vdisk... etc, I'm still learning what these different drive designations mean) Thanks!

 

PC Stats:

CPU i7-6700K

16gb DDR4 memory

1050ti GPU

Link to comment
  • 4 weeks later...

Man do I miss MCEBuddy and WMC!!  It was actually the most polished solution for a set top device that could do it all.... I mourned it when it died.  It seems like unraid might be a good fit for your use case.  You could have a VM running 24/7 with your WMC up for all to enjoy.  The only major challenge I can see coming for you is physical layout and usb passthrough for the devices to those locations.  I would keep your m.2 for either a passthrough for a win 10 vm, or maybe for your cache drive.  The speed with be helpful to the VMs boot drives.

Link to comment
  • 2 weeks later...

Plex can also record from an HD Homerun and remove commercials and has DVR capabilities. Might be worth merging those devices into one.

 

You can pass through a video card to a VM for video crunching, I've had good luck with a Quadro P620 & P2000. I have a Win10 VM that runs my Plex server and haven't had any issues with it. But if you do go that route, I would recommend creating a separate VM drive for hosting the Plex database & metadata.

Link to comment

Sounds like a good match for unRAID.

 

Ok, let's start with my understanding unRAID disks. (Happy to have others chime in, with their experiences or opinions!)

You can read up more in the unRAID manuals online about this. There are some great tutorials by @SpaceInvaderOne on youtube. I recommend watching a few of them first before building your system.

 

When you install unRAID, you'll have a "drive array" that you can have parity error correction disk repair capability with. One thing to note, any drive going into the "drive array" will need to be pre-cleared and reformatted for unRAID to use effectively, so a back up is a "REALLY GOOD IDEA (tm)". The drives will be formatted into Linux formats, like xfs. The "Parity Drive" is a separate drive (or two) that can cover up to seven other "combined" drives with error correction drive recovery capability, such that if there is a drive failure in the array - you can rebuild the (entire) array by simply adding a new replacement drive for the one that failed. No muss, no fuss, just a rebuild and you're back in business. This is probably one of the biggest selling points for unRAID.

 

And as mentioned below, any drive showing errors are best replaced with that replacement handled promptly. For my system, I have a "hot swappable" 4TB drive sitting in standby (in a anti-stat wrapper, in a drive bay drawer) should that ever be the case.

 

And the parity drive needs to be specifically assigned, and the largest drive in system (or array). The only job of the parity drive is for error data protection, it cannot be used for other file storage. If you have more than seven drives, then you typically it has been suggested practice to just add another parity drive as you add more than seven drives. You can even add extra insurance with two parity drives covering the same (7-10 drive) "array". Having two parity drives is has also been recommended to me by some users of unRAID if your drives are getting older, or are of "questionable" lifespan, or you need to ensure that the data is not vulnerable to being lost. There is a lot more to learn about parity protection, but it seems a valuable feature that makes unRAID unique compared to many other systems available.

 

After that, the "drive array" that you would build/create - essentially merges the multiple hard disk drive spaces (even of varied sizes) into one "virtual drive" - but in such a way that you span drives, but can still assign specific areas of the drives for specific tasks. For my system, I have 3x4TB drives, with 2x600GB drives, totaling 13.2GB - (with one 4TB drive in parity) After the array is built, you can then have separate share folders in that array, that you can set size limits on, access as network drives (and assign as windows networked drives in VMs) - just like a Network Array Storage (NAS) device. For example, I have my photos in my array, in a protected user folder named "Photos" and have that folder assigned on my laptop and on my VMs as drive "P:" with my (retained) login information. And as long as I am on the network, I have access to those files - but no one else on the same network would.

 

You could also run each hard drive as an "unassigned device" (note, this is an added "pluggin" for unRAID) that you could then reassign inside your VMs as needed. It's a little tricky to do, but not hard either. Any "unassigned device" is also outside of the parity protection scheme, and for hard drives - is then treated like any other standard hard drive.

 

The unRaid Cache drive, is used for NAS file ingestion (uploads) to speed up the upload, and also contains the VMs and the Docker apps that you decide to install. This is typically made up of one or two SSD drives - and it's not protected by any parity protection like the drive array is. These drives area also not "spun down", as I understand it - in order to help with file upload times. The uploaded cache files are moved off at an interval you specify automatically by unRAID. You can also move them "manually" if you need to. The files are not protected by the drive array parity drive until they are moved.

There are also "unassigned devices" that are like separate drives, which can be either hard drives or USB drives. They can still be used for storage and backups, but are meant more for specific uses where parity protection isn't needed, and you might want them for only specific purposes (like as a VM system backup drive).

 

Now, let's get back to your potential set up.

 

From what I sense - you could run the Win 7 WMC VM in the background, with an Emby or Plex Docker server app running from unRAID. Plex could pull from the drive in your VM, and add the new WMC recorded video to your library - serving it back to the XBox. And you could also run a Win10 VM at the same time - doing other things.

 

You can even load balance those on the Cache with just the 500GB SDD - or with the VMs on the 1TB m.2 SSD, and still have plenty of room left over. (And in reality - you could run everything off one SSD and not be too cramped, so I'd try it first, then decide. For me, I'd want the speed from the m.2, especially if it's an NVME for my VMs.)

 

You might want to add a little more RAM to your system if you can. If you run multiple VMs at the same time, then it gets a little tight on 16Gigs. Not really bad, I've had a couple of 6GB VMs running at the same time. You'll just want to watch how you build them, and leave about 2 GB of RAM for unRAID to do "it's thing". If you have multiple VMs and Docker apps running, that would also be a good reason for 24-32GB total.

 

And the 1050Ti should let you do any video transcoding (to mp4 format?) you need to with Plex, and still have headroom for your VMs with passthrough. Plenty of good tutorials on setting those up. If you do decide to run Plex or Emby as a docker app, it will all go on the SDD Cache drive - so very low overhead with those running. There are Docker apps for capturing networked cameras feeds (like you do with iSpy) as well. Note, Dockers, Apps, VMs, and Tools are all separate items in unRAID. Which allows for more flexibility - and sometimes better (tool) options to be considered.

 

You can even have the external drive added as an external device any time you want, and even assign specific USB ports to the VMs for hot plugging, if you need to. Also easy to have a Win10 VM running any time you need it, as a "front cover" on the machine, while running unRAID in a "headless" configuration ("headless meaning without need for a monitor or keyboard). You can then VNC into your windows 7 or 10 VMs via any standard browser on the network, and your XBox 360 can access the Plex or Emby services - and access your disk array as if it was a secure NAS system.

 

So... did that help?

Edited by rollieindc
revised file handling explaination, and correcting misinterpretations.
  • Like 1
Link to comment
4 hours ago, rollieindc said:

you'll have a "drive array" that you can have parity error correction with. One thing to note, any drive going into the "drive array" will need to be pre-cleared and reformatted for unRAID to use effectively, so a back up is a "REALLY GOOD IDEA (tm)". The drives will be formatted into Linux formats, like xfs. The "Parity Drive" is a separate drive (or two) that can cover up to seven other "combined" drives with error correction,

Sorry, I can't leave this be. Parity in unraid really isn't error correction, it won't fix corrupt files or drives. It will allow emulation and rebuilding of a failed disk or two, depending on whether you have one or two parity disks. One parity disk can emulate one failed drive for the entire parity array, not limited to any specific number. At the moment a Pro license allows 28 data drives, so you could have 1 parity and 28 data if you wanted. 2 parity disks will emulate 2 failed drives, regardless of the number of devices protected. Parity works in concert with the data drives, it doesn't have any data by itself. The parity calculation works on an unlimited number of devices, the limits are imposed by the unraid license and programming.

 

Having 1 or 2 parity is a choice based on risk tolerance, not technology.

 

Drives assigned to a new array will be reformatted unless previously set up to match what unraid expects, there is no need to pre-clear for unraid's sake. After parity is calculated, any new drive slot additions will be cleared (not pre-cleared) automatically by unraid before they are added so parity is maintained.

 

Pre-clear is a routine developed many many years ago before unraid allowed the array to be accessible while a drive was being cleared when added. Now that the array stays available, preclearing isn't needed, but it provides a very good stress test to ensure a new drive is worthy. Unraid parity requires ALL drives be read perfectly over the entire capacity to rebuild failed drives, not just the drive portion that is occupied by files.

 

Do not keep questionable drives in the array. If one of your other drives fails unexpectedly, you will be relying on that questionable drive to rebuild the failed drive. All drives must be good.

Link to comment
16 hours ago, jonathanm said:

Sorry, I can't leave this be. Parity in unraid really isn't error correction,...

1

Ok, yes, my apologies - I over-over simplified. It's really a failed disk recovery method. Thanks for doing a better explanation than I did at the time (I blame a restless night and not enough sleep.)

 

But also to be fair, I've gotten repeated recommendations to add a parity drive for every 8th drive. Maybe it's to reduce the time required by the algorithms to compute a parity value, and then write it (either in parity creation or in recovery) - but that was the recommendation. And I've never seen where the algorithm can have one value for data on up to 28 drives - if it can, great! I'll have to go back and re-read that part of the documentation.

 

And well, there still seems to be a lot of talk in the forum about pre-clearing new drives - perhaps to avoid "start up" deaths. But I still got quite a few admonitions to pre-clear any and all new drives being added to my arrays. But if not necessary, then ok... good to know that too! Thanks.

Link to comment
6 hours ago, rollieindc said:

I've never seen where the algorithm can have one value for data on up to 28 drives - if it can, great! I'll have to go back and re-read that part of the documentation.

https://wiki.unraid.net/Parity#How_parity_works

This was written before parity2 was implemented. Notice the equations don't have a limit, single parity can apply to ANY number of drives, it's just at the moment 28 is the artificial limit on drive slots, not the parity math. It's generally agreed that at some point, drive failures are statistically going to be greater than 1 at a time which is what single parity can recover from. Where that point is, isn't a bright line. If you keep a close eye on your drives and test them regularly, replacing at the first hiccup, single parity can work for very large arrays.

 

Link to comment
1 minute ago, rollieindc said:

Could you better define your nexus of a "first hiccup"?

It's based on personal risk tolerance combined with experience. 1 pending sector event that is successfully reallocated is one thing that starts a watch, but not replace yet. Second pending sector event within a week or so, make sure a spare is in hand. 3rd event, make sure your ducks are in a row, prepare for a server down replacement at earliest convenience. Steadily climbing bad sectors, server is going down for drive replacement, screw everyone planning on watching TV tonight. There are other SMART or behaviour events that can start my spidey sense tingling, but many of them are just based on years of watching the progression of drive failure.

 

The guiding principal is always this. If you feel uncomfortable with trusting your most important data to a drive with those particular symptoms, replace the drive. Because of the way parity works, with all drives involved in a rebuild, you ARE trusting your most important data to every drive, regardless of whether the data exists on that specific drive.

 

Sometimes drives fail catastrophically with NO advance warning. Reduce your risk by ensuring all drives are perfectly healthy to the best of your knowledge.

  • Like 1
Link to comment

My DiskSpeed Docker app can also give hints on wonky drives that have odd performance but otherwise has no issues with the SMART values. I replaced a WD Red 6TB because it developed a "wave" in it's read speeds that the other like-model drives did not have. One user recently shared a screen shot of a drive that the read performance TANKED in the last half of his drive but no indication of issue on the SMART report.

 

Another thing to note about bad sectors, every drive has them. Every. Single. One. They're mapped out at the factory and that may be represented by a small speed anomaly but as long as it's repeatable without variation, it's no concern. There's no way to get that factory bad sector map.

Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.