March 3, 201016 yr I started the preclear script on two drives last night. one will be parity and the other data. I have questions......Do I start to transfer data once it is done? Do I need to set up shares/split levels first? What would be a good way to do it for my data at the moment? I will be starting with 1 data drive and will be getting another 2TB data drive soon. *EXAMPLE* Videos/ Movies/ Amorres Perros/ *.jpg *.nfo *.mkv TV/ 24/ Season 1/ S01E01.mkv S01E24.mkv S01.nfo Photos/ Spain 2003/ *.jpg Thailand 2004/ *.jpg Australia 2008/ *.jpg Music/ Band Name/ Album Name 1/ song Album Name 2/ song
March 3, 201016 yr I started the preclear script on two drives last night. one will be parity and the other data. I have questions......Do I start to transfer data once it is done? Do I need to set up shares/split levels first? What would be a good way to do it for my data at the moment? I will be starting with 1 data drive and will be getting another 2TB data drive soon. See below. Each of your shares can have the level set based on its directory structure. If you have only 1 data drive, it really does not matter if you set the level at all. It also has no effect if you copy directly to the disk shares rather than the user shares. (That is what I do... that way I know where the data is stored and have complete control) If you want to set them, set them as I've shown with the "*" below. *EXAMPLE* Videos/ <-- level 1 Movies/ <-- level 2 Amorres Perros/ <-- level 3 * *.jpg *.nfo *.mkv TV/ <-- level 2 24/ <-- level 3 Season 1/ <-- level 4 * S01E01.mkv S01E24.mkv S01.nfo Photos/ <-- level 1 Spain 2003/ <-- level 2 * *.jpg Thailand 2004/ <-- level 2 * *.jpg Australia 2008/ <-- level 2 * *.jpg Music/ <-- level 1 Band Name/ <-- level 2 Album Name 1/ <-- level 3 * song Album Name 2/ <-- level 3 * song
March 4, 201016 yr Well put Joe L, that would have to be the easiest way to explain Split level's to newbies. Good Stuff!
March 4, 201016 yr To add to Joe's post, I would consider using one share level less than his * marked level. Look at the movie share. Level 3 means that the "Amorres Perros" directory can split between multiple drives. This means that the 3 files stored in the "Amorres Perros" directory could be stored on multiple drives instead of being kept together on the same drive. In the music share, using level 3 means each album could span multiple drives spreading the songs from one album all over the place. Keep that share at level 1 and all the albums and songs from each artist will be stored on a single drive or level 2 will at least keep each album together on the same drive. There is a possible problem with your layout. If the "videos" share is set to level 2 so that each movie stays together on a single drive then the TV series will be limited to having a whole series stored on a single drive. This means that a TV series could fill a drive and when you add more episodes the copy will fail. A seperate "TV Series" share set to level 2 could solve this, or worst case set it to level 3 so each season can be stored on multiple drives. Personally, I set-up and use the shares, but I limited the different shares to certain disks. My TV series are going to disk3 right now and I will add a new disk when the time comes. That way, each TV series won't get spread out over multiple disks. I use split level of 1 and use I believe it's high water so that the new empty disk will fill with any new series. I'll leave some room on the first disk for new episodes for the series stored there which are still running. Peter
March 12, 201016 yr Trying to get my mind around this split level thing. I just started with unRAID and like it a lot, but cant quite figure out the whole split level idea. My biggest concern is Music shares. I want to keep all albums and mp3s kept together on the same disk for an artist. This way, I dont lose a single song in the event of a lost disk where parity did not get it back. Music Albums Artist Album *.mp3 I have no problem if metallica is on disk 2 and Elton John is on disk 3. I just want to make sure I keep all the elton john albums and their songs on disk 3 and all metallica albums and mp3s on disk 2. I want to avoid having Metallica - Ride the Lightning on Disk 2 while Metallica - Kill em All is on disk 3. Worse yet would be to have half the Mp3s from an album on disk 2 with the rest on disk 1...So does this mean a split level of 3? As for my movies, similar thing but not as critical to keep things together: Movies Bourne Identity bourne.ts fanart.jpg Top Gun TopGun.ts Fanart.jpg I just want the files for a movie kept together, but movies can be on any disk. As I as disks, I am using high-water to spread things evenly, but while spreading the wealth, I want to keep some sort of order...So for movies, am I a split level 2 or is it a 1? I think 1, but again, I have no clue. Lastly, I have my pictures. Since there arent as many of these at the moment, I figure to just lock them to a single disk. So, am I best to just say include disk x? Or maybe split level 2, let it go all disk and have a structure of: Pictures year folder 1 *.jpg folder 2 *.jpg year folder With split level 2 my years and its content would stay on the same disk, but 2008 might be on disk 1 while 2009 is on disk 3. Or, for my pictures would I do split level 1? Man, it seems so easy, but yet it is so hard to grasp and understand. Any and all help is appreciated. Hopefully we can get this all straightened out. Also, can you change the split level after the share is created? I created my movies with a split level 0, but I assume I will want to change it according to what I have above, but just want to make sure moving forward it will follow the split level and high water rules. If not, I will delete and recreate the share accordingly and then copy my data again. Thanks!
March 12, 201016 yr edit - this is a different way of looking at the share numbering. Follow Joe's link posted further down for a full understanding of what level 0 means. The share starts with 0 being the top level directory or the share name, depending on how you look at it. Each directory level below that is +1. One you number them, you pick the level corresponding to first directory level you do not want splitting. For your music, I think you'd want 2 but then your folder structure seems odd. I'm not sure what's up with that first Albums folder. 2 means the Artist directory stays on a single drive. For movies, it appears you want 1. 1 means the Movie (Bourne Identity) directory stays on a single drive. For the pictures, I'd say 1 as well. 1 means the Year directory stays on a single drive. But then, I'd consider just assigning the pictures and the music both to a single include disk so they don't split at all. Just put the disk# into the box and the split level doesn't matter anymore. This will hopefully help clear it up. 0 Music 1 Albums 2 Artist 3 Album 4 *.mp3 0 Movies 1 Bourne Identity bourne.ts fanart.jpg 1 Top Gun TopGun.ts Fanart.jpg 0 Pictures 1 year 2 folder 1 *.jpg 2 folder 2 *.jpg 1 year 2 folder Peter
March 12, 201016 yr The share starts with 0 being the top level directory or the share name, depending on how you look at it. Each directory level below that is +1. One you number them, you pick the level corresponding to first directory level you do not want splitting. For your music, I think you'd want 2 but then your folder structure seems odd. I'm not sure what's up with that first Albums folder. 2 means the Artist directory stays on a single drive. For movies, it appears you want 1. 1 means the Movie (Bourne Identity) directory stays on a single drive. For the pictures, I'd say 1 as well. 1 means the Year directory stays on a single drive. But then, I'd consider just assigning the pictures and the music both to a single include disk so they don't split at all. Just put the disk# into the box and the split level doesn't matter anymore. This will hopefully help clear it up. 0 Music 1 Albums 2 Artist 3 Album 4 *.mp3 0 Movies 1 Bourne Identity bourne.ts fanart.jpg 1 Top Gun TopGun.ts Fanart.jpg 0 Pictures 1 year 2 folder 1 *.jpg 2 folder 2 *.jpg 1 year 2 folder Peter I think your numbering is off by one. 0 has a very specific meaning to unRAID. It does not refer to the top most level at all. See the release notes here: http://lime-technology.com/forum/index.php?topic=3760.0 I think that Level 1 would be the top most share. (Movies, Music, Pictures)
March 12, 201016 yr OK, it is making a little sense now. So split level is the directory level at which point I want all contents of that directory to remain on the same disk folder 1 (share) folder 2 folder 3 folder 4 In the above, split level 2 means all the contents of folder 3 will be kept on the same disk. Now, I can have a folder 3 On different disks, but every time I do, the contents of that specific folder 3 will be kept together on one disk. Am I making sense?
March 13, 201016 yr I think your numbering is off by one. 0 has a very specific meaning to unRAID. It does not refer to the top most level at all. See the release notes here: http://lime-technology.com/forum/index.php?topic=3760.0 I think that Level 1 would be the top most share. (Movies, Music, Pictures) Yes, it was. But I described it as the number on the first directory you do not want to split is the split level, which evens it out. If you read it again you'll see that it does work. It does work if you want to put in 0 so that the share does not split. Just create the share directory (ie Movies or Music) on one data drive and with a split level of 0 the share will not split. I'll add another post with it explained the other way - the split level is the last directory you want to split. Peter
March 13, 201016 yr Here is the other way to look at it. The share starts with 1 being the top level directory or the share name. Each directory level below that is +1. One you number them, you pick the level corresponding to lowest level directory that you want to split. For your music, I think you'd want 2 but then your folder structure seems odd. I'm not sure what's up with that first Albums folder. 2 means the Album directory can split but no directories below that can split. For movies, it appears you want 1. 1 means the "Movies" share directory can split but the movie (Bourne Identity) directory below stays on a single drive. For the pictures, I'd say 1 as well. 1 means the "Pictures" share can split but the year directories below that stay on a single drive. But then, I'd consider just assigning the pictures and the music both to a single include disk so they don't split at all. Just put the disk# into the box and the split level doesn't matter anymore. This will hopefully help clear it up. 1 Music 2 Albums 3 Artist 4 Album *.mp3 1 Movies 2 Bourne Identity bourne.ts fanart.jpg 2 Top Gun TopGun.ts Fanart.jpg 1 Pictures 2 year 3 folder 1 *.jpg 3 folder 2 *.jpg 2 year 3 folder Peter
March 13, 201016 yr OK, it is making a little sense now. So split level is the directory level at which point I want all contents of that directory to remain on the same disk folder 1 (share) folder 2 folder 3 folder 4 In the above, split level 2 means all the contents of folder 3 will be kept on the same disk. Now, I can have a folder 3 On different disks, but every time I do, the contents of that specific folder 3 will be kept together on one disk. Am I making sense? Yes, you've got it. Peter
March 14, 201016 yr I fully understand the way share level works, but I wanted to use this thread as an opportunity to mention something I noticed. I have a share setup like this (I also use a cache disk).. TV <---- level 1 The Simpsons <--- level 2 Season 1 <--- level 3 .. Season n.. <--- level 3 My "TV" share is set to Split level 1 with a high allocation method and a disk inclusion of disk2,disk3,disk4 and exclusion disk1. Now, with this setup, it would be my understanding that the entire show should be contained on 1 disk. I noticed however after having ripped a few DVD seasons that they ended up on a completely different drive than the rest of the seasons (onced moved by the mover script from cache -> array). The reason why this happened still alludes me. The disk where the show resides was definitely the lowest in terms of remaining capacity (about 40gb). I have a few other disks in my system that have 700GB and 1.1TB respectively. It's almost as though unRAID (4.5.3 fyi..) completely ignored the split level in favour of the allocation level. I would be grateful if someone had a theory as to why this happened EDIT: I just noticed this happened again after having written this message. on 2 occasions.. disk3 tv <--- level 1 Survivor <--- level 2 Season 1 <--- level 3 .. Season N <--- level 3 Rescue Me <--- level 2 Season 1 <--- level 3 Season N <--- level 3 copied two new seasons of each show (which were put on the cache drive).. I just checked and they both ended up on disk4 disk4 tv <--- level 1 Survivor <--- level 2 Season 11 <--- level 3 Rescue Me <--- level 2 Season 4 <--- level 3 disk3 still had about 170GB left disk4 has 1.1TB.. Definitely seems as though it's favouring the allocation method and ignoring the split level.
March 14, 201016 yr I fully understand the way share level works, but I wanted to use this thread as an opportunity to mention something I noticed. I have a share setup like this (I also use a cache disk).. TV <---- level 1 The Simpsons <--- level 2 Season 1 <--- level 3 .. Season n.. <--- level 3 My "TV" share is set to Split level 1 with a high allocation method and a disk inclusion of disk2,disk3,disk4 and exclusion disk1. Since you've excluded disk1, I'd not expect any of your episodes to end up on that disk. Edit: Oh sorry, I misread at first. when you said "Now, with this setup, it would be my understanding that the entire show should be contained on 1 disk" I took that as disk 1. Sorry about that. Ignore my response.
March 15, 201016 yr I'm not fully convinced shares are working correctly all the time right now. I had level 1 and "Fill up" set with 2 drives allowed. I then had subdirectories in my Movies share appear on both drives. Oddly, all of the subdirectories but one were empty which is strange since they should only be created if necessary to hold a file written to the array. I had one movie split to the second drive when it should not have. New movies written to the array directly under the Movie share were still going to the first disk - the disk was not full and had not reached my min free space setting yet. The directories in my share would look like this; 1 Movies 2 Old Dogs 2 Up in the Air 2 Planet51 2 James Bond Movies * 3 Die Another Day 3 Live or Let Die 2 Die Hard Movies * 3 Die Hard 3 Live Free or Die Hard ** etc The directories with a * were created blank on the second disk and the directory with a ** was created with the movie contents on the second disk. I changed back to level 1 and "High Water" and have not seen it happen again but only time will tell. I need to go back and do something like edit the metadata for those movies again and see what happens. I sent a message to Tom about this and am waiting to hear back. Peter
March 16, 201016 yr Let us know what he replies. I've been checking my array for rogue files and I'm seeing them all over the place. I don't have a min-free setting on my shares, but I'm still assuming the split level should override the allocation method and eventually spit an "out of disk space" error if I'm daft enough to let it fill a drive.
March 16, 201016 yr A thread like this one makes me wish Limetech desined user shares as read only. Even though I'm having these issues, if unraid didn't support user shares, I would not have purchased a license.
March 16, 201016 yr A thread like this one makes me wish Limetech desined user shares as read only. Ah... but you can set the user-shares to be read-only, do all your own file-management on the writable, but hidden disk shares. It is what I do and it works perfectly. Lime-tech has had a tough time making the allocation of space intuitive. Is is not.
March 16, 201016 yr A thread like this one makes me wish Limetech desined user shares as read only. Ah... but you can set the user-shares to be read-only, do all your own file-management on the writable, but hidden disk shares. It is what I do and it works perfectly. Oh, yes, I know that, and that's how I do it too. I just said I wish they were read only by design, to spare the people the confusions when they writing to their server.
March 16, 201016 yr I understand your point, Purko, but I respectfully disagree. One of unRAID's original draws for me was the ability to read and write to user shares. I like that I can throw a new movie into my 'Movies' share and not care about where it ends up. Some people like to know exactly where their data is at all times - I do not. That said, I still love that user shares can be made read-only for certain users. I use this feature so that friends and roommates can access my media from my HTPC or from their computers without the possibility of messing anything up.
March 18, 201016 yr Funny that you both pipe up hinting that it's a user problem yet neither of you have any input on what the problem could be. I changed this; Share name = Movies Comment = Allocation Method = high water min. free space = Split level = 1 Include disks = disk1 disk2 exclude disks = to this; Share name = Movies Comment = Allocation Method = Fill-up min. free space = 50000000 Split level = 1 Include disks = disk1 disk2 exclude disks = and the shares quit filling correctly. 50,000,000 gives around 50gig as the free space and I had something like 170gig free at the time on disk1. I also find it very odd that unRAID created a bunch of directories but never filled them. It creates the directory when it's needed yet it went and created the directory without using it. Peter
March 18, 201016 yr One other thing. When I started I was using an XP machine to copy the files over. Hidden disk shares remained hidden until the first time I accessed them and then XP put a direct "link" into windows explorer to each disk share. Defeats the hidden share, yes? Peter
May 25, 201016 yr Are you using rsync by chance? I was having the same issue. Turns out rsync was writing every necessary folder to the share before it began the actual copy. Then it started copying and ran out of room, so unraid redirected rsync to the next drive. None of the aforementioned folders exist on this next drive so it creates them all again, as it continues with the rsync.
May 25, 201016 yr One other thing. When I started I was using an XP machine to copy the files over. Hidden disk shares remained hidden until the first time I accessed them and then XP put a direct "link" into windows explorer to each disk share. Defeats the hidden share, yes? Peter You can delete those links if your pc is shared by other family members. Although it does behave this way, it still doesn't broadcast the hidden disk shares. I suspect most of us that are acustomed to windows hidden and administrative shares ($) probably just assume unraid handles them the same. Perhaps someone with more linux knowldge can clarify for us.
Archived
This topic is now archived and is closed to further replies.