Unraided, have you ever gone back and put another file in one of those movie directories later?

 

Your level 4 means that the "Silver Streak" directory could be on as many different disks as the high water needs it to be. You need to use level 3.

 

Try this if you don't believe me, look for the most full disk and copy a file into one of the movie directories that is on that disk using the share. unRAID will create a new movie directory on the most free drive and put the file there.

 

I use the same directory structure as the origional poster uses. If I set that share to 2 and drop a file into any of the movie folders on my most full disk I end up with a duplicate movie folder on the other disk and the file is stored there.

 

What's saving you is the high water and the fact you are filling and then never changing the contents of that directory again. However, you might find a directories that are duplicated on 2 drives caused by a move right when the high water 50% full switch to the next drive happens.

 

If you ever go back and do something like add metadata and images to use a pretty front-end such as MediaBrowser you will end up with a complete mess of data spread out everywhere. If you allow WMC to create those thumb images (I can't remember the actual name right now) they could end up all over the place. Someone else here not too long ago ended up with that happening which created a bunch of duplicate directories just for the thumbnail images.

 

Peter

 

I'm not sure if it created a dir on the 1st disk, as I cleaned out all the empty dirs since then.. But I will create a new share and will try this out again..

 

Well, having empty directories indicates there was some kind of problem. You should have ended up with one empty one when the move failed, not a bunch of empty ones.

 

Since this is a new server I'd think you have fairly large drives. High water should have filled disk 1 50% full and then started on disk 2.

 

Knowing the sizes of the drives, how full the drives are, the size of the typical file and how many you are trying to copy at once could help.

 

Maybe MC does something odd like create the folder structure first and then fill it. If it did this and you tried to move 500gig all at once and the drive is 400gig then it would fail. I really don't know, but you had something odd happen.

 

Peter

 

 

 

So you're saying in my setup, it's best to have split level 3 to maintian that the last folder level created and the files with the last folder level stay on the same drive, correct? According to my testing with split level either 3 or 4 I was achieving this as well, as newly created folders was only being created on the one drive and the files copied with in it say on the one folder and that one driver to. It is getting a little confusing now.

 

Unraided, have you ever gone back and put another file in one of those movie directories later?

 

Your level 4 means that the "Silver Streak" directory could be on as many different disks as the high water needs it to be. You need to use level 3.

 

Try this if you don't believe me, look for the most full disk and copy a file into one of the movie directories that is on that disk using the share. unRAID will create a new movie directory on the most free drive and put the file there.

 

I use the same directory structure as the origional poster uses. If I set that share to 2 and drop a file into any of the movie folders on my most full disk I end up with a duplicate movie folder on the other disk and the file is stored there.

 

What's saving you is the high water and the fact you are filling and then never changing the contents of that directory again. However, you might find a directories that are duplicated on 2 drives caused by a move right when the high water 50% full switch to the next drive happens.

 

If you ever go back and do something like add metadata and images to use a pretty front-end such as MediaBrowser you will end up with a complete mess of data spread out everywhere. If you allow WMC to create those thumb images (I can't remember the actual name right now) they could end up all over the place. Someone else here not too long ago ended up with that happening which created a bunch of duplicate directories just for the thumbnail images.

 

Peter

 

I'm not sure if it created a dir on the 1st disk, as I cleaned out all the empty dirs since then.. But I will create a new share and will try this out again..

 

Well, having empty directories indicates there was some kind of problem. You should have ended up with one empty one when the move failed, not a bunch of empty ones.

 

Since this is a new server I'd think you have fairly large drives. High water should have filled disk 1 50% full and then started on disk 2.

 

Knowing the sizes of the drives, how full the drives are, the size of the typical file and how many you are trying to copy at once could help.

 

Maybe MC does something odd like create the folder structure first and then fill it. If it did this and you tried to move 500gig all at once and the drive is 400gig then it would fail. I really don't know, but you had something odd happen.

 

Peter

 

I tried copying via rsync fist and that did create all the folders first..  I have some time today, so I will create a new share with split level 1 and will use the same movie structure as before (diff share name of course) and will see what will happen..

The drives are all 1 TB in size.  As of this moment drive 1 is almost full (about 10g free) drive 1 50% and drive 3 & 4 are empty ( still copying from the old server as we speak) The .avi's are between 700mb to 4gb in size (varies) and all have various smaller support files in the dirs (.nfo. .tbn .txt .jpg)

mc just creates the directory it is currently copying not everyone that is in the batch..

I tried copying via rsync fist and that did create all the folders first.. 

 

That might have been the problem. If the folders were all created on disk1 then you'd end up with disk1 out of space as you filled the folders.

 

Disk1 should have filled to 50% then disk 2 should have started to fill.

 

Peter

 

So you're saying in my setup, it's best to have split level 3 to maintian that the last folder level created and the files with the last folder level stay on the same drive, correct?

 

Yes. try the experiment I suggested and see what happens.

 

Peter

 

You'll probably want to lynch me for saying this, but why don't you simply use user shares for reading and disk shares for writing?

 

 

I run programs that capture data to provide a nice visual experience when searching for movies and TV shows. So, I have to do it on each disk. Instead of picking one directory to look at I have to enter a directory for each disk. Not a huge problem but it's a bit of extra setup. Also, when I go to the server then it shows each disk as well which adds clutter unless they are hidden which then requires typing in the address.

 

Peter

 

I think he runs the program whenever he archives a new movie. It's during the running of the program that it WRITES new data about the movie next to where the movie is at. I could be wrong as I'm guessing and reading more into what was said than is perfectly clear.

thanks, I'll try with my current split level 4 and see whether adding more files into a last level folder which only resides on one disks and see if it recreates the same folder onto another disk with the newly added files. I'll try this with split level 3 as well and see what results between both level 4 and 3 comes up with. Yeah a little troublesome to understand when mixed advice is given or you have different beliefs on how it suppose to work. I for one believe that this needs to be properly clarified, not just for newbies but for those who what to understand it thoroughly. Thanks

 

So you're saying in my setup, it's best to have split level 3 to maintian that the last folder level created and the files with the last folder level stay on the same drive, correct?

 

Yes. try the experiment I suggested and see what happens.

 

Peter

 

 

I named the shared folder, set to high water, and Export read/write. And started with 6 1.5 TB drives. The 1st drive filled to approx half, then the next drive. Once all 6 drives filled to half, then it started over filling half again, then half again. Once my 7 TB of media was transferred over to my unRAID server, I installed one 2TB parity drive. Ran parity. All was good to go, so now I have added the 4 1.5TB drive to the array, they are  clearing.

 

  Other then being really slow, (But moving 7TB of media is a lot.) I am loving this server.

Thanks to everyone involved in this great project.

I think he runs the program whenever he archives a new movie. It's during the running of the program that it WRITES new data about the movie next to where the movie is at. I could be wrong as I'm guessing and reading more into what was said than is perfectly clear.

 

Exactly, and if you write the data to a disk that has hit the high water limit then that data could end up on another disk.

 

Peter

 

unraided - read the limetech post here

 

http://lime-technology.com/forum/index.php?topic=1928.msg13748#msg13748

 

It's the same number of directory levels as you and he says to use level 3. I'd think the developer knows.

 

If you number as per that post the split level is the number of the last directory level you want to split across multiple disks.

 

 

You need to set "split level" to a value greater than 1.  Otherwise, it will never use the other drive for any directory under Movies.

 

Joe L.

 

This post may be correct but it's could be confusing. Joe means that any directory stored under Movies will not split to another drive. He does not mean that a new directory can not be created on another drive.

 

Peter

 

Hi lionelhutz, so in your quote below and referring to this example below too:

 

 

Server                            Split        Split          Split

 Name                          Level 1     Level 2       Level 3

    |                                 |              |               |

\\tower\mysharename\music&moves\dvds\silver_streak\vob_file1.vob

                    |                                          |-----\vob_file2.vob

                    |                                          |-----\vob_file3.vob

             Share Name                                  

 

If the High Water Mark for the disk which had started with the 'silver_stresk' DVD reached its mark while copy the set of *.vob files for the 'silver_streak' DVD, in this case 'vob_file3.vob' going to 'vob_file4.vob', 'vob_file4.vob' would get written to the next disk which as a high disk capacity?

I was under the assumption that given the Split Level 3 folder (silver_streak), unraid would make certain that all files within 'silver_streak' stay on the same disk regardless of the High Water rule, but a newly created folder at the Split Level 3, like another DVD folder created at the same level as 'silver_streak' would automatically be placed on another disk which has a higher disk capacity.

 

 

I think he runs the program whenever he archives a new movie. It's during the running of the program that it WRITES new data about the movie next to where the movie is at. I could be wrong as I'm guessing and reading more into what was said than is perfectly clear.

 

Exactly, and if you write the data to a disk that has hit the high water limit then that data could end up on another disk.

 

Peter

 

You missed a level again. mysharename is level 1.

 

Yes, with the wrong share level setting if the disk containing Silver Streak has reached the high water mark then the next time you write a file to Silver Streak then there will be a new Silver Streak directory created on another disk to hold that file.

 

Just try it. With your level 4 write a file into a movie directory that is stored on the disk that has reached the high water limit. Find the directory by looking at the disk shares but write to the user share. The directory and file will be created on the disk presently being filled, which then means the movie directory will be duplicated on 2 disks.

 

Peter

Apologies lionelhutz, yeah I was half asleep I wrote that original posting/diagram . My finding demonstrate the following:

 

Split Level 4 Setup findings:

 

Example:

 

\\tower\disk6\tower\Movies&Music\DVDs\The Departed\file1.vob

                                                                             \file2.vob

                                                                             \file3.vob

                                                                             \file4.vob

or

 

\\tower\tower\Movies&Music\DVDs\The Departed\file1.vob

                                                                     \file2.vob

                                                                     \file3.vob

                                                                     \file4.vob

 

Under a Split Level 4 setup, following example above, if I copied another *.vob (say file5.vob) file into the 'The Departed' folder and seeing that 'disk6' has reached its High Water mark, a new folder automatically gets created under 'disk7', as this has more disk space. The additional *.vob file gets placed into the newly created folder, still known as 'The Departed' under now 'disk7', resulting in the following:

 

\\tower\disk6\tower\Movies&Music\DVDs\The Departed\file1.vob

                                                                             \file2.vob

                                                                             \file3.vob

                                                                             \file4.vob

 

 

\\tower\disk7\tower\Movies&Music\DVDs\The Departed\file5.vob

 

or

 

\\tower\tower\Movies&Music\DVDs\The Departed\file1.vob

                                                                     \file2.vob

                                                                     \file3.vob

                                                                     \file4.vob

                                                                     \file5.vob

 

 

Split Level 3 Setup findings:

 

Example:

 

\\tower\disk6\tower\Movies&Music\DVDs\The Departed\file1.vob

                                                                             \file2.vob

                                                                             \file3.vob

                                                                             \file4.vob

or

 

\\tower\tower\Movies&Music\DVDs\The Departed\file1.vob

                                                                     \file2.vob

                                                                     \file3.vob

                                                                     \file4.vob

 

Under a Split Level 3 setup, following example above, if I copied another *.vob (again say file5.vob) file into the 'The Departed' folder and seeing that 'disk6' has reached its High Water mark, the additional file5.vob file gets placed into the original created 'The Departed' folder on 'disk6' and won't be placed onto disk, regardless of the High Water mark, resulting in the following:

 

\\tower\disk6\tower\Movies&Music\DVDs\The Departed\file1.vob

                                                                              \file2.vob

                                                                              \file3.vob

                                                                              \file4.vob

                                                                              \file5.vob

 

or

 

\\tower\tower\Movies&Music\DVDs\The Departed\file1.vob

                                                                     \file2.vob

                                                                     \file3.vob

                                                                     \file4.vob

                                                                     \file5.vob

 

Note: If under the DVDs sub-folder, if I create a new folder for another DVD, say 'Get Smart', if 'disk6' again was at the High Water mark, the newly created 'Get Smart' folder and all of it's contents would be placed on only 'disk7', see below:

 

\\tower\disk7\tower\Movies&Music\DVDs\Get Smart\file1.vob

                                                                                  \file2.vob

                                                                                  \file3.vob

                                                                                  \file4.vob

                                                                                  \file5.vob

 

or

 

\\tower\tower\Movies&Music\DVDs\Get Smart\file1.vob

                                                                \file2.vob

                                                                \file3.vob

                                                                \file4.vob

                                                                \file5.vob

 

In my scenario and for my needs, I'll be sticking with Split Level 3 as this setup is to my liking and I'd rather have a group of *.vob files under the same disk then spanned across multiple disks. Phew! It was a bit of thinking and typing for a Sunday. Thanks for your help Peter and I hope this eases others into thinking how they should setup Split Levels.

 

57086#msg57086 date=1270946902]

You missed a level again. mysharename is level 1.

 

Yes, with the wrong share level setting if the disk containing Silver Streak has reached the high water mark then the next time you write a file to Silver Streak then there will be a new Silver Streak directory created on another disk to hold that file.

 

Just try it. With your level 4 write a file into a movie directory that is stored on the disk that has reached the high water limit. Find the directory by looking at the disk shares but write to the user share. The directory and file will be created on the disk presently being filled, which then means the movie directory will be duplicated on 2 disks.

 

Peter

Thanks for testing. That shows what happens when the split level is too high. It also shows that Msan with his single level directory structure should use level 1. Level 2 can allow individual movie directories to be created on multiple disks.

 

From later posts, I'm believing that rsync had created a huge directory structure on his first disk and then there was not enough space to be able to fill it. Just removing all those empty directories would fix it without using a split level of 2.

 

Peter

 

No worries lionelhutz. Although the principles, rules, setup of Split Levels have been heavily mentioned on this forum and Wiki, (to me) it never has been properly documented well enough, with well laid out diagrams and using multiple examples or even using different scenarios to give people (newbies or experienced users alike) a thorough understanding on how to properly setup Split Levels for their use.

When I got into unraid, my initial understandings of Split Levels took a little thinking to understand and my original setup was Split Level 3 for the sake of my folder structure on my rig seemed to do the job, but after reading other threads and mixed opinions, you start to feel doubtful on even your own thinking about Split Levels.

 

Thanks lionelhutz.

Thanks for testing. That shows what happens when the split level is too high. It also shows that Msan with his single level directory structure should use level 1. Level 2 can allow individual movie directories to be created on multiple disks.

 

From later posts, I'm believing that rsync had created a huge directory structure on his first disk and then there was not enough space to be able to fill it. Just removing all those empty directories would fix it without using a split level of 2.

 

Peter

 

Well I ran some more tests and I can confirm that split level 1 works for my folder structure..  Thanks for all the pointers everyone..

No worries, looks like we all learned something from this thread

Speak of the devil.  I just ran into this issue.  All my shares are set to "most free", "min free space = 0" and "split level = 1".  I have 2 drives with hundreds of gigs of space yet unraid is trying to put something on one of the almost full ones.  Any idea why my data isn't going onto the drive with the most free space?  My setup is as follows:

 

\\unraid\Movies\Moviename\moviename.avi

\\unraid\TV\Showname\Season x\Showname - SxEy.avi

\\unraid\Music\Band\Album\blah.mp3

 

The issue happened when trying to add a TV show.  I'd like all the files for one movie, album or TV show season to go on the same disk.  What would the recommended settings be based on my structure above?

Do you have the TV directory created on all your data drives? If not, the split level of 1 will prohibit the files from going any place other than where the TV directory already exists.

 

You should have at least a split level of 2.

Do you have the TV directory created on all your data drives? If not, the split level of 1 will prohibit the files from going any place other than where the TV directory already exists.

 

You should have at least a split level of 2.

 

TV, Movies and Music are user shares, not directories that I've manually created.  Doesn't unraid handle the creation of user share names on data drives on an as needed basis?

What the TV Share split level means;

 

Level 1 = TV can go split to every disk, Showname and Season X can only be on one disk

Level 2 = TV and Showname can split to every disk, Season X can only be on one disk

Level 3 = TV and Showname and Season X can split to every disk

 

You want each Season to be kept to a single disk so you need level 2 for the TV share. Your present setting means each complete show has to be on a single disk.

 

Peter

What the TV Share split level means;

 

Level 1 = TV can go split to every disk, Showname and Season X can only be on one disk

Level 2 = TV and Showname can split to every disk, Season X can only be on one disk

Level 3 = TV and Showname and Season X can split to every disk

 

You want each Season to be kept to a single disk so you need level 2 for the TV share. Your present setting means each complete show has to be on a single disk.

 

Peter

 

Perfect, that explains it then.  Thanks Peter!

TV, Movies and Music are user shares, not directories that I've manually created.  Doesn't unraid handle the creation of user share names on data drives on an as needed basis?

 

Not that I've seen, but I manually control where my directories and files are.