Hi folks

 

My unraid is up and running, all drives pre-cleared and I've run a full parity check just to confirm it.  Now I'm almost ready to transfer my data over, but I'm a little confused about shares.

 

My current RAID structure is as follows:

 

Photos

Music

Movies

 

Photos and music are pretty straight forward, but then within Movies I have the following folders:

 

HD

DVD

TV

Jukebox [jukebox folder for YAMJ on my popcorn hour]

index.htm [index file for YAMJ on my popcorn hour]

 

A high proportion of my HD files have been ripped as BluRay folder structures (so several folders within a folder), due to BD .ISOs not working properly on the PCH-200 at the time.

 

I'm really a little confused as to how to set this up.  Part of me loves the idea of just one giant virtual disc (this is what I'm used to with RAID), but then do I risk a movie pausing half way through as one movie's folder structure is split across more than one disc?

 

Additionally I like the idea of if a drive fails, it being easy to identify what data was actually lost, rather than it being sheer guesswork.

 

And finally, I want to keep my folder structures largely intact.

 

Although I'm dying to drag and drop a load of files across from my NAS, I'm holding off as I want to get this right - 10TB of data is going to take 2-3 days to copy?  So I'm really looking for some solid advice, please help!

 

I have read the wiki re: split levels and maybe it's just because I've had a long day, but it's not really sinking in!

 

Can anyone give me some advice on how exactly to set this up before I go....well a bit mad?

 

I'm on the home straight......almost!  

 

Thanks in advance

Matt

 

I'm really a little confused as to how to set this up.  Part of me loves the idea of just one giant virtual disc (this is what I'm used to with unRAID), but then do I risk a movie pausing half way through as one movie's folder structure is split across more than one disc?

 

in the unraid settings set Spinup groups to 'Yes'.

'Spinup Groups':  This feature permits you to group hard drives so that if an I/O request is received on any drive in the group, the unRAID driver will spinup any/all spundown hard drives also in the group. In other words, when accessing a file in a share, all disks in the same share will spinup. This way there will be only a pause before the beginning of a movie, until all disks spin up...

 

 

Additionally I like the idea of if a drive fails, it being easy to identify what data was actually lost, rather than it being sheer guesswork.

 

And finally, I want to keep my folder structures largely intact.

 

if you use split level equal to 1, that means that you mostly achieve both goals. if you have a share called movies, then, by using split level of 1, all files created under HD will be in the same disk and the same applies for DVD and TV. if, on the other hand,you use a split level of 2, then, for example,  the directory HD may be split between multiple disks but all contents of directories under HD will be in the same disk

Thanks for the explanation.  To clarify, if I use split level 1, all movies in "HD" will be on the same disc, but the contents won't fit on one disc - how does this work?

 

With split level 2, the folder "HD" will appear as virtual folders on every disc, but each individual movie folder which contains subfolders a la BD structures, will always stay on one disc? 

 

Can you also explain how this works in terms of dragging and dropping files?  I'm used to seeing one big mapped drive, will I have one for each disc, but as I add movies the "overall share" will also update?  not sure if that makes any sense, but it's half past midnight in the UK...I probably need to read this again in the morning!

Thanks for the explanation.  To clarify, if I use split level 1, all movies in "HD" will be on the same disc, but the contents won't fit on one disc - how does this work?

 

to be honest, i dont know what will happen in this case, my guess is that you will be notified tha disk is full.

 

With split level 2, the folder "HD" will appear as virtual folders on every disc, but each individual movie folder which contains subfolders a la BD structures, will always stay on one disc? 

exactly

 

Can you also explain how this works in terms of dragging and dropping files?  I'm used to seeing one big mapped drive, will I have one for each disc, but as I add movies the "overall share" will also update?  not sure if that makes any sense, but it's half past midnight in the UK...I probably need to read this again in the morning!

 

it depends on the settings of allocation method in the user shares section.

look here for more details (scroll down a bit to allocation methods):

http://lime-technology.com/wiki/index.php?title=UnRAID_Manual#User_shares

 

depending on the allocation method, the data will be split on the various disks of the user share. This way you have a minimum guarantee that the data will be split (as evenly as it is allowed by the split level) on the various disks belonging to a user share

I'm a bit confused now folks - I set up my "Media" share as split level 2, and set all discs to be included:  disk1, disk2, disk3, disk4, disk5 (is that right?)

 

Just looked tonight after dumping my "TV" folder onto disc2....but it's not showing up in the "master" drive    What have I done wrong, anyone know?

Navigate to \\tower\disk2 and see what you see. You'll probably see a Media folder and a TV folder. Move the TV folder to the Media folder, and you'll see it under your Media share.

Thanks, that worked

 

However, if I create a folder in the master drive, it doesn't show in the individual disks?  Do I just need to create the same folder in each disk share individually, and then when I drop content it will mirror it in the master drive?

I'm a bit confused now folks - I set up my "Media" share as split level 2, and set all discs to be included:  disk1, disk2, disk3, disk4, disk5 (is that right?)

 

Just looked tonight after dumping my "TV" folder onto disc2....but it's not showing up in the "master" drive    What have I done wrong, anyone know?

The problem is that when you copy directly to a disk you bypass all the settings you setup for a share.

 

My recommendation for you would be to setup three shares:

 

Photos (Split Level 1)

Music (Split level 1)

Movies (Split level 2)

 

Also to clarify ...

 

I'm really a little confused as to how to set this up.  Part of me loves the idea of just one giant virtual disc (this is what I'm used to with unRAID), but then do I risk a movie pausing half way through as one movie's folder structure is split across more than one disc?

 

in the unraid settings set Spinup groups to 'Yes'.

'Spinup Groups':  This feature permits you to group hard drives so that if an I/O request is received on any drive in the group, the unRAID driver will spinup any/all spundown hard drives also in the group. In other words, when accessing a file in a share, all disks in the same share will spinup. This way there will be only a pause before the beginning of a movie, until all disks spin up...

Spinup groups will not fix the problem MrGrumpie is describing.  Assume he's watching a 2 hr movie and it's split in equal parts across two disks.  Spinup groups will initially spin up all the disks in the group when he presses play to start the movie but if the spindown timer is set to less than 1 hour the other disk would be spun down when the second half of the movie is needed.  This would cause a delay in playback while the disk is spun up.  The only way to avoid this is to make sure that all the required files for a single movie are on the same disk OR never spindown any disk within the movie share.

 

Also to clarify ...

 

I'm really a little confused as to how to set this up.  Part of me loves the idea of just one giant virtual disc (this is what I'm used to with unRAID), but then do I risk a movie pausing half way through as one movie's folder structure is split across more than one disc?

 

in the unraid settings set Spinup groups to 'Yes'.

'Spinup Groups':  This feature permits you to group hard drives so that if an I/O request is received on any drive in the group, the unRAID driver will spinup any/all spundown hard drives also in the group. In other words, when accessing a file in a share, all disks in the same share will spinup. This way there will be only a pause before the beginning of a movie, until all disks spin up...

Spinup groups will not fix the problem MrGrumpie is describing.  Assume he's watching a 2 hr movie and it's split in equal parts across two disks.  Spinup groups will initially spin up all the disks in the group when he presses play to start the movie but if the spindown timer is set to less than 1 hour the other disk would be spun down when the second half of the movie is needed.  This would cause a delay in playback while the disk is spun up.  The only way to avoid this is to make sure that all the required files for a single movie are on the same disk OR never spindown any disk within the movie share.

 

You are right. But, then, all he has to do is  to adjust the timer to 2 hours. Since most movies are no longer than 2-2.30 hours, he will have some spare time for toilet breaks and to microwave some popcorn

The only way to avoid this is to make sure that all the required files for a single movie are on the same disk OR never spindown any disk within the movie share.

He could set the spindown timer to say 2 hours or longer and this would eliminate the delay/pause in the middle of the movie.  The downside to using that method is that drives will be spun up for much longer than needed.  So I'll clarify my previous statement ... The proper way to eliminate the delay/pause during playback is to properly configure the split level within the user shares so that all the files for any single movie are stored only on a single disk.

The problem is that when you copy directly to a disk you bypass all the settings you setup for a share.

 

 

Ahh ok, didn't realise that.  If I manually setup a HD folder on each drive and drag n drop files to each disc though, these will still show up in the master share under HD won't they?

 

Splitting movie files across two discs won't be an issue if I manually manage the file transfers by disc, will it?

The problem is that when you copy directly to a disk you bypass all the settings you setup for a share.

 

 

Ahh ok, didn't realise that.  If I manually setup a HD folder on each drive and drag n drop files to each disc though, these will still show up in the master share under HD won't they?

 

Splitting movie files across two discs won't be an issue if I manually manage the file transfers by disc, will it?

 

Practically. shares just look look for directories with their names at the root of each disk, they dont care how they are created, so you certainly will have no problems

 

Just do as wsume99 suggests and put all files of each movie in the same disk....

 

If I manually setup a HD folder on each drive and drag n drop files to each disc though, these will still show up in the master share under HD won't they?

No.  Creating a HD folder on a disk is not the same as creating a HD share.  However once unRAID places a file in the HD share and saves that file to a disk (say disk 2) a HD share folder will automatically show up on that disk.  Once the HD share folder is created on the disk you can then copy directly to that HD share folder on that specific disk.  I hope that makes sense.  What I have found is that when I create a new share (i.e. Movies) then a Movies folder will show up on Disk 1 but non of the other disks.  The Movies folder won't show up on the other disks until a file is written to the Movies automatically by unRAID.  At least that's been my experience.

 

Splitting movie files across two discs won't be an issue if I manually manage the file transfers by disc, will it?

I'm not sure what you are asking - there are two possibilities and perhaps we are just mixing terminology.  Let me see if I can resolve this with an example.  Let's consider the following using movie only rips of Cars and Monsters Inc. [two of my kids favorite movies]DVDs placed in a Movies share comprised of Disk 1 and Disk 2.

 

Scenario #1 - Using Split Level 3

Movies

  DVDs (spread across Disk 1 & Disk 2)

    Cars (placed on Disk 1 & Disk 2)

      VIDEO_TS.BUP (on Disk 1)

      VIDEO_TS.IFO (on Disk 1)

      VTS_01_0.BUP (on Disk 1)

      VTS_01_0.IFO (on Disk 1)

      VTS_01_1.VOB (on Disk 1)

      VTS_01_2.VOB (on Disk 1)

      VTS_01_3.VOB (on Disk 1)

      VTS_01_4.VOB (on Disk 2)

      VTS_01_5.VOB (on Disk 2)

      VTS_01_6.VOB (on Disk 2)

    Monsters Inc (placed on Disk 1 & Disk 2)

      VIDEO_TS.BUP (on Disk 1)

      VIDEO_TS.IFO (on Disk 1)

      VTS_01_0.BUP (on Disk 1)

      VTS_01_0.IFO (on Disk 1)

      VTS_01_1.VOB (on Disk 1)

      VTS_01_2.VOB (on Disk 1)

      VTS_01_3.VOB (on Disk 1)

      VTS_01_4.VOB (on Disk 2)

 

This is not what you want.  When you playback both movies there will be a pause when transitioning between VTS_01_3.VOB  which is on Disk 1 and VTS_01_4.VOB which is on Disk 2.  This happens because unless you set the spindown timer to a high time like > 1.5 hours disk 2 will be spun down.  So the media player has to wait for disk 2 to spinup before it can proceed with playing VTS_01_4.VOB.

 

Scenario #1 - Using Split Level 2

Movies

  DVDs (spread across Disk 1 & Disk 2)

    Cars (placed on Disk 1)

      VIDEO_TS.BUP (on Disk 1)

      VIDEO_TS.IFO (on Disk 1)

      VTS_01_0.BUP (on Disk 1)

      VTS_01_0.IFO (on Disk 1)

      VTS_01_1.VOB (on Disk 1)

      VTS_01_2.VOB (on Disk 1)

      VTS_01_3.VOB (on Disk 1)

      VTS_01_4.VOB (on Disk 1)

      VTS_01_5.VOB (on Disk 1)

      VTS_01_6.VOB (on Disk 1)

    Monsters Inc (placed on Disk 2)

      VIDEO_TS.BUP (on Disk 2)

      VIDEO_TS.IFO (on Disk 2)

      VTS_01_0.BUP (on Disk 2)

      VTS_01_0.IFO (on Disk 2)

      VTS_01_1.VOB (on Disk 2)

      VTS_01_2.VOB (on Disk 2)

      VTS_01_3.VOB (on Disk 2)

      VTS_01_4.VOB (on Disk 2)

 

This is what you want.  All the individual files that comprise a single movie are placed on a single disk.  It's fine to place different movies on different disks.

 

You can write directly to the disk shares and bypass all the user share hassles. It will be faster, and your movies won't get split between drives.  Once you get this bulk moved over, then experiment with your split level and share configurations.

Maybe we need to make this clearer or you are not making your actions clear.

 

To copy to a disk first go to that disk, eg go to \\tower\disk1. You want a share named "Movies" so that disk should have a directory named "Movies" in it as a root level or top level directory. If you then enter that "Movies" directory you are now at the location where you would create the "HD" or "DVD" or "TV" directory.

 

Every top level directory will appear as a user share. I'm just not sure when (immediately, after a reboot, after an array start/stop which is the most likely) but a newly created top level directory will apear as a user share. These top level directories are the user shares. unRAID will scan the top level directories in each drive to know what network share to create and also combines directories that repeat on multiple disks into single network shares.

 

To set-up a share use these split levels

 

My recommendation for you would be to setup three shares:

 

Photos (Split Level 1)

Music (Split level 1)

Movies (Split level 2)

 

Do not use Fill-up as the allocation method. The High Water method works well for the most part so I would just use it. It's hard to explain exactly but the High Water method will basically fill the disks so they maintain equal amounts of free space. Say you start with all 2T drives. It'll fill each one until they all have 1T free, then begin again filling them all until they have 500gig free, then begin filling them all again until they have 250gig free, etc. If you had a single 1T drive in the mix it would get no data until all the 2T drives were filled to have 1T free.

 

The Include and Exclude disks only need to be used if you only want to use certain disks for the share. IF you are using them all then just leave these blank. Otherwise, enter the easiest pattern. For example, if you want to use all disks except say disk7 then it's easier to enter exclude disk 7 then include disk1, disk2, disk3, disk4, disk5, disk6.

 

The Min Free Space is really only needed for the Fill-Up allocation method so you don't really need a value there either.

 

The spin-up group is best used to get all the disks spinning when you access the share. That way, you don't suffer additional delays as disks spin up. I would try it both ways with spin-up groups disabled and properly set-up and see how it works. I did run into issues where I could bring up the media and pick something to watch and then I'd run into a delay as either WMC or Mediabrowser would be indexing files and spin up another drive. I think one of the big culprits was WMC since it seems that music files are almost constantly being background indexed. My music was on a different drive and it's begin to index it a few minutes after I'd fired WMC up and began to watch a movie or TV show using it which then caused a short playback pause during drive spin-up.

 

Peter

 

Thanks for the help all, I've actually just manually allocated files to the specific disks - in each case I've made a "Movie Jukebox" folder (which is the main share name now), followed by HD, TV and DVD depending on which disk it is.

 

Even with making the folders manually, all folders and content are showing on the master "Movie Jukebox" share - happy 

 

If I switch spin-up groups on and start watching a movie, will the other disks still spin down at whatever time I've set, once unRAID detects they aren't being used?  I like the idea of everything spinning up when I first browse my jukebox pages, but once I've started watching a movie - realistically 2 hours long - I still want the other drives to spin down.

I'm sure the spin-up groups will just spin-up, not keep the drives spinning. So, the drives should spin down after being idle for the set length of time.

 

Peter