I'm about to build my first unRAID box - parts for The Monolith arrived today - and would like some advice on which way to set up split level and user shares.

 

The main content for my unRAID will be:

Movies

TV series split by series and seasons

Photographs split by camera and month taken

MP3s split by artist and album

eBooks split by author

Misc (pdfs, receipts, etc.)

 

I'm not sure what is the best setup for split levels and user shares. Would you recommend setting up a separate user share for different types of content? Separate user shares mean differen drive letters when I access the box via my desktop? Or is it possible to set up everything above on one user share with a certain split level setup?

 

I prefer the lowest maintenance possible and as little tinkering with the contents as feasible after they're uploaded to the array.

 

EDIT: Ignore the bonus question, I'm an idiot: Bonus question: does the parity drive have to be the largest drive to the byte, and is there a risk of data loss if the drive is not the largest? All my 2TB drives are slightly different sizes.

does the parity drive have to be the largest drive to the byte, and is there a risk of data loss if the drive is not the largest? All my 2TB drives are slightly different sizes.

If the raw reported size of your drives are different, then there is a problem. All the 2TB drives I am familiar with are exactly 2,000,398,934,016 bytes as reported by smart status. Some motherboards reserve a part of the hard drive by changing the reported size and use the extra space to store BIOS backups. If your hard drives are actually different sizes, you need to do a search for HPA, (host protected area) and deal with it before you use them in unraid.

 

Yes, the parity drive must be equal to or larger than any data disk. If a drives HPA is altered while in use by unraid, yes, you could lose data.

 

As for the share settings, only you can determine the best use case for you. Perhaps you should load a small sample of all your data types and try accessing it from various devices to see how it acts. That way you can experiment with different setups before you spend lots of time copying all your stuff.

 

Drive letter mappings are only needed if the program can't use UNC paths.

 

If your drives currently have data on them, all that data will be erased when you add the drives to unraid. The best way to deal with this is keep your existing drives as a backup, and load new drives into unraid.

does the parity drive have to be the largest drive to the byte, and is there a risk of data loss if the drive is not the largest? All my 2TB drives are slightly different sizes.

If the raw reported size of your drives are different, then there is a problem. All the 2TB drives I am familiar with are exactly 2,000,398,934,016 bytes as reported by smart status. Some motherboards reserve a part of the hard drive by changing the reported size and use the extra space to store BIOS backups. If your hard drives are actually different sizes, you need to do a search for HPA, (host protected area) and deal with it before you use them in unraid.

 

Yes, the parity drive must be equal to or larger than any data disk. If a drives HPA is altered while in use by unraid, yes, you could lose data.

 

After a closer look, all my drives are the same exact size. Let's just say I'm an idiot, and leave it at that

 

As for the share settings, only you can determine the best use case for you. Perhaps you should load a small sample of all your data types and try accessing it from various devices to see how it acts. That way you can experiment with different setups before you spend lots of time copying all your stuff.

 

That's a very good approach, I'll do that.

 

From the conceptual point of view, if I read the wiki entry on split-levels correctly, it looks like I can have only one split level (1, 2, 3 or 4) on each user share. So if I want different split levels for different types of content, I would have to create (at least) one user share per each split level depth.

 

That complicates things, as my movies, tv-series, etc. have 0-2 sub-directories, ie. levels. Therefore it might make most sense to create a separate user share for each content type. Is that a feasible solution, and would that increase maintenance needs?

 

If your drives currently have data on them, all that data will be erased when you add the drives to unraid. The best way to deal with this is keep your existing drives as a backup, and load new drives into unraid.

 

Yes, I noticed that the drives will be wiped. I have three brand new drives which I'll use to seed the box with. My plan is then to start moving my existing content from older drives, and installing them to the unRAID drive as soon as the array reaches parity, or whatever the correct terminology is.

 

I'm a bit wary of using a brand new drive as parity drive. Maybe it would be safer to use one of my older and proven drives as a parity drive?

I'm a bit wary of using a brand new drive as parity drive. Maybe it would be safer to use one of my older and proven drives as a parity drive?

 

No drive is more important than any other drive.    They're ALL needed to provide the fault tolerance in the array.    What's "safe" is to run a pre-clear cycle or two on all your new drives before you use them, to ensure there are no "infant mortality" issues with them.

 

Yes, I noticed that the drives will be wiped. I have three brand new drives which I'll use to seed the box with. My plan is then to start moving my existing content from older drives, and installing them to the unRAID drive as soon as the array reaches parity, or whatever the correct terminology is.

 

I'm a bit wary of using a brand new drive as parity drive. Maybe it would be safer to use one of my older and proven drives as a parity drive?

What are you using as a back up for your data? Perhaps you would be better off copying the data and leaving the old drives intact as a backup, if you don't have a backup in place now.

 

The parity drive is the least important of all the drives. Unraid can only reconstruct one bad drive, if a second drive fails while the first drive is bad, you will lose the contents of both drives. All the good drives are unaffected, which is what makes unraid so different from most raid implementations. If 2 data drives fail, you will lose what was on those two drives. If the parity drive fails, and a data drive fails, you will only lose the data that was on the one drive.

 

In any case, you should probably search and read up on Joe L.'s preclear script, it's a good way to gain confidence in a new drive. It's primary purpose was to speed the addition of new drives to an already protected array, but as a side effect it works the whole drive to make sure there are no weak links that made it by the factory tests. It also is a good way to thoroughly erase a drive for security reasons.

Yes, I noticed that the drives will be wiped. I have three brand new drives which I'll use to seed the box with. My plan is then to start moving my existing content from older drives, and installing them to the unRAID drive as soon as the array reaches parity, or whatever the correct terminology is.

 

I'm a bit wary of using a brand new drive as parity drive. Maybe it would be safer to use one of my older and proven drives as a parity drive?

What are you using as a back up for your data? Perhaps you would be better off copying the data and leaving the old drives intact as a backup, if you don't have a backup in place now.

 

I believe I have a pretty robust backup system already in place. I have a backup of critical data in my desktop. I have also rotating weekly incremental backups and a monthly full backup on two separate offline drives. In addition, I have Crashplan with several TB up in the cloud.

 

My plan is to keep one or two backup HDDs in my desktop going forward.

 

The parity drive is the least important of all the drives. Unraid can only reconstruct one bad drive, if a second drive fails while the first drive is bad, you will lose the contents of both drives. All the good drives are unaffected, which is what makes unraid so different from most raid implementations. If 2 data drives fail, you will lose what was on those two drives. If the parity drive fails, and a data drive fails, you will only lose the data that was on the one drive.

 

That makes sense, thanks for the explanation.

 

I'll do pre-clear as suggested by you and garycase, hadn't heard of that!

The parity drive is the least important of all the drives.

 

I don't agree.  I suppose I should revise my earlier comment ...

 

No drive is more important than any other drive.

 

... to read:  "No drive is more important OR less important than any other drive."

 

The simple fact is they ALL collectively provide the fault-tolerance for the array.  If one drive fails, the you can rebuild it.  But if a second drive fails, you'll lose the data on both of those drives.    It IS true that the consequences of one of those 2 failed drives being the parity drive are less traumatic than a 2nd data drive ... since you'd only lose one drive's worth of data in that case.

 

But if any one drive fails -- parity or data -- the entire array is now "running at risk"

 

The parity drive is the least important of all the drives.

 

I don't agree.  I suppose I should revise my earlier comment ...

 

No drive is more important than any other drive.

 

... to read:  "No drive is more important OR less important than any other drive."

 

To me, a drive full of my data is more important than a drive full of meaningless 1's and 0's. The parity is only usable in concert with all the other drives, so if two failures occur, that parity drive just became totally useless.

Except, of course, the parity drive is NOT "... full of meaningless 1's and 0's" ==> without it, the entire array has no fault tolerance.    Those 1's and 0's are very specifically calculated to provide fault tolerance for the entire array.

 

You're thinking in terms of IF you were going to lose 2 drives, you'd like one of them to be the parity drive ... and that's certainly correct -- we'd all prefer that.

 

But a FAR more likely scenario is losing ONE drive ... and then rebuilding it.    And for that purpose, all drives are equally important.    That's why you should always have a spare drive handy ... so IF you lose a drive, you can immediately do a rebuild -- thus minimizing the likelihood of a 2-failure situation.    If you have a failure and don't have a spare drive, I always suggest shutting down the array until you obtain a spare, so the "running at risk" time is minimized.

 

 

 

You're thinking in terms of IF you were going to lose 2 drives, you'd like one of them to be the parity drive ... and that's certainly correct -- we'd all prefer that.

So if you were only able to save 8 out of 10 drives in your array, you'd agree that the parity drive is less important. Meaningless even. That is my entire point.

 

Too often, unraid is so good at masking drive failures unless you actively watch your syslog and smart reports, or install a third party utility to send alerts, you don't know your array is degraded until it's too late. A server in the closet quietly serving media doesn't give you many clues that a drive has died if all you do is access the shared files.

 

Except, of course, the parity drive is NOT "... full of meaningless 1's and 0's" ==> without it, the entire array has no fault tolerance.    Those 1's and 0's are very specifically calculated to provide fault tolerance for the entire array.

Of course. That's the whole point of parity protection.

I'm well aware of how unraid works, I don't need your condescending explanation.

This went to shit pretty quickly. Please take it elsewhere.

For the share settings you need to define 2 things.

 

1. The directory levels you are going to use.

2. Which level do you want each directory to remain together on a single disk.

 

For example. TV

 

Share

    -->TV Series

                  --> Season

 

Is the structure and saying you want to have each TV series remain on a single disk is the level. if you need help, provide this structure and level for each share and I can confirm the split level.

 

As for the shares. I recommend a share for each media type.

 

There is another option. You can simply just make a big share, put your files on it and not care which files are on which disks. It works unless you care to have everything structured disk by disk.

There is another option. You can simply just make a big share, put your files on it and not care which files are on which disks. It works unless you care to have everything structured disk by disk.

 

I imagine this option would be the easiest, since I wouldn't have to worry about levels. But I can see it being a total nightmare to recover from in a multi-disk failure.

 

Other than that, I frankly don't see any point in knowing or managing how the files are distributed on the array. To me a huge selling point of unRAID is that I get to just throw everything in it, and it will do the rest for me.

There is another option. You can simply just make a big share, put your files on it and not care which files are on which disks. It works unless you care to have everything structured disk by disk.

 

I imagine this option would be the easiest, since I wouldn't have to worry about levels. But I can see it being a total nightmare to recover from in a multi-disk failure.

 

Other than that, I frankly don't see any point in knowing or managing how the files are distributed on the array. To me a huge selling point of unRAID is that I get to just throw everything in it, and it will do the rest for me.

 

Logically it's certainly the simplest.    The reason folks like to keep specific media together is generally so only one disk needs to spin up when you're streaming a movie.  If, for example, you're streaming a movie and the component files are spread across multiple disks, then you may have a notable "freeze" in the playback when it needs to access the next file if the disk it's on isn't already spun up.    If you're storing your movies in a single file (e.g. an ISO or MKV) you don't need to be concerned about that;  but if, for example, you're saving DVDs in their Video_TS folders, then if different .VOBs are on different disks you could have this problem.

 

Logically it's certainly the simplest.    The reason folks like to keep specific media together is generally so only one disk needs to spin up when you're streaming a movie.  If, for example, you're streaming a movie and the component files are spread across multiple disks, then you may have a notable "freeze" in the playback when it needs to access the next file if the disk it's on isn't already spun up.    If you're storing your movies in a single file (e.g. an ISO or MKV) you don't need to be concerned about that;  but if, for example, you're saving DVDs in their Video_TS folders, then if different .VOBs are on different disks you could have this problem.

 

Ah, of course, hadn't occurred to me. That's actually a good reason Most of my movies are in one file, so those are not an issue. MP3s, on the other hand, I'd prefer to keep on one drive so the array doesn't have to spin up a new disk every three minutes when switching songs. Same with photographs.

 

Is there a way to have both on the same unRAID box? ie. movies and series on a user share with no split levels, while mp3s and photographs are on user shares with split levels? If that can be done, it might make most sense to make a user share for each media type, and choose # of split levels or no split level based on content type.

When you define the SHARE in unRAID (on the SHARE tab in the GUI),

use either the INCLUDE or EXCLUDE fields (but not both).

For example, in a 3 disk + parity system, you could define the following:

a 'TUNES' share with INCLUDE 'Disk 2' would allow the share to access only Disk #2.

a 'MOVIES' share with EXCLUDE Disk 2 would allow movies to be on Disk #1, and #3 but not on #2.

 

Again, for the same share, don't use both exclude and include.

(And my own experience is 'don't overthink it'. Except for the .VOB instance for movies, and an Apple Time Machine, a noticable pause in playback as a disk spins up is rare.)

Each share can be individually configured for both the split level it should use, and also which disks can be used by the share.  If you have separate shares for each media type then you can fine tune each of them.

When you define the SHARE in unRAID (on the SHARE tab in the GUI),

use either the INCLUDE or EXCLUDE fields (but not both).

For example, in a 3 disk + parity system, you could define the following:

a 'TUNES' share with INCLUDE 'Disk 2' would allow the share to access only Disk #2.

a 'MOVIES' share with EXCLUDE Disk 2 would allow movies to be on Disk #1, and #3 but not on #2.

 

Again, for the same share, don't use both exclude and include.

(And my own experience is 'don't overthink it'. Except for the .VOB instance for movies, and an Apple Time Machine, a noticable pause in playback as a disk spins up is rare.)

 

Thanks. I think that I'll force MP3s on one drive to avoid constant spinups when listening to music, while the rest of the content can go anywhere.

 

Doing pre-clear on the drives, getting excited here

If you want one big share then you pretty much have to allow for the files to go anywhere. Different split levels requirements for different media types forces that to happen. The only time you may need one big share if is you have a media playback device that can only access one network location at a time.

 

Overall, if you set each share up correctly and are using a structured approach to file storage then the data pretty much takes care of itself and you still end-up with structure in the storage. Very little internvention is required unless you try to pack the disks right full.

 

Overall from what you indicated, ebooks and mp3's woud likely would be fine with a split level of 1. That keeps each book author and each music artist on a single disk. TV and your pictures would likely work better with level 2, keeping each TV season and each month of pictures on a single disk. You didn't post a movie structure.

 

There is another drive usage option. Assuming your pictures, music, ebooks and/or misc won't ever fill a disk you could just dedicate one disk to those items and then use the rest for TV and movies.

 

One more question. I understand that I can only have one split level on each user share. But can I have different split levels on andy one disk? eg, if I allow both user share Movies (Split Level 2) and user share Series (split level 4) to use disk 1, will the user shares work properly, or will there be a conflict?

 

There is another drive usage option. Assuming your pictures, music, ebooks and/or misc won't ever fill a disk you could just dedicate one disk to those items and then use the rest for TV and movies.

 

Yes, this is what I'm leaning towards, as well. I could easily fit all those on one 2TB disk, with room to expand.

One more question. I understand that I can only have one split level on each user share. But can I have different split levels on andy one disk? eg, if I allow both user share Movies (Split Level 2) and user share Series (split level 4) to use disk 1, will the user shares work properly, or will there be a conflict?

Yes - this will work fine.

 

Another point to note is that Split Level and/or drive restrictions occur at file creation time.  This means you can always change split level for a share and that then applies for new files going forward.  Files will still be found in user shares at read time even if they are on a different drive to what the current settings mandate for new files.

I decided to go with the KISS principle: I'm not using split levels on any drive.

 

I created separate user shares for Movies, TV Series, and Misc Videos not on scrapers (documentaries, late night shows). I don't mind them being spread on multiple disks. And since I use high water, it is rarely when the (few) multi-file movies would be split across disks. These all exclude Disk1.

 

I'm keeping all the rest on Disk1, this includes photos, mp3s, personal files and temp files for torrents. I wanted to keep photos and mp3s on a single drive to avoid spin-ups - although even that is not really an issue most of the time due to my 4-hour spin down delay. They can fit easily on that disk, with room to grown.

 

Thanks everyone for clarifying and helping out, really appreciate it!

 

One more question. I understand that I can only have one split level on each user share. But can I have different split levels on andy one disk? eg, if I allow both user share Movies (Split Level 2) and user share Series (split level 4) to use disk 1, will the user shares work properly, or will there be a conflict?

Yes - this will work fine.

 

Another point to note is that Split Level and/or drive restrictions occur at file creation time.  This means you can always change split level for a share and that then applies for new files going forward.  Files will still be found in user shares at read time even if they are on a different drive to what the current settings mandate for new files.

 

That's good info, wasn't aware of that!

 

So the same is true for include/exclude files? For example, if I later change my mp3s to include disk2 (instead of disk1 currently), will all future mp3s go to disk2? If so, will the user share include all files on both disk1 and disk2, although I would only have disk2 "included"?

I decided to go with the KISS principle: I'm not using split levels on any drive.

 

I created separate user shares for Movies, TV Series, and Misc Videos not on scrapers (documentaries, late night shows). I don't mind them being spread on multiple disks. And since I use high water, it is rarely when the (few) multi-file movies would be split across disks. These all exclude Disk1.

 

I'm keeping all the rest on Disk1, this includes photos, mp3s, personal files and temp files for torrents. I wanted to keep photos and mp3s on a single drive to avoid spin-ups - although even that is not really an issue most of the time due to my 4-hour spin down delay. They can fit easily on that disk, with room to grown.

 

Thanks everyone for clarifying and helping out, really appreciate it!

 

One more question. I understand that I can only have one split level on each user share. But can I have different split levels on andy one disk? eg, if I allow both user share Movies (Split Level 2) and user share Series (split level 4) to use disk 1, will the user shares work properly, or will there be a conflict?

Yes - this will work fine.

 

Another point to note is that Split Level and/or drive restrictions occur at file creation time.  This means you can always change split level for a share and that then applies for new files going forward.  Files will still be found in user shares at read time even if they are on a different drive to what the current settings mandate for new files.

 

That's good info, wasn't aware of that!

 

So the same is true for include/exclude files? For example, if I later change my mp3s to include disk2 (instead of disk1 currently), will all future mp3s go to disk2? If so, will the user share include all files on both disk1 and disk2, although I would only have disk2 "included"?

 

Yes. The settings only apply to new file writes.