ericft Posted October 7, 2010 Share Posted October 7, 2010 I recently transfered my movie library 2.1TB onto my unraid server under movies shares with high water and split level 2. i have the files in genres and then each genre has movies in individual directories and those directories have movie files...when i transfered everything i looked on teh disk and found out that several files are on different drives under the same directory of movie... for example Movies share-action genre-300 folder-movie,folder.jpg files...so i thought split level 2 would avoid the separation ofr the movie file from the folder.jpg data on different drives...however that hasn't been the case....now i want to get the data aggregated back so that rather genres would be spread across drives and keep movie directories/content intact.... so it would be movies share-action, drama,scifi on one disk and movies-comedy,cartoon,exercise on another disk....with all the movies in each genre on its own disk... what are my options? 1. copy genre folders onto the first drive until full? and then spread genres myself? (not efficient way) 2. change split level? would 1 be the number to use? will it update the location of folders automatically or are the file data locations set for the split level 2? 3. do i delete all teh movies completely, then change the split level, and recopy? Thanks for any assistance... Link to comment
Rajahal Posted October 7, 2010 Share Posted October 7, 2010 Changing the split level (or any other share disk allocation setting) will only affect future writes. unRAID will never move your files for you. Here's option 4: Create a new share called Movies2 with the correct split level setting (which I think is 3 in your case, but I always get mixed up when it comes to split level - lionelhutz is the expert in that realm). Then move all your data from the Movies share into the Movies2 share. If you do this from the command line, the move will be quick as long as the data is staying on the same drive. If you do it through the network (drag and drop) it will be much slower, but it will still work. Once you have moved all the data and Movies is empty, delete the Movies share and rename Movies2 to just Movies. Link to comment
ericft Posted October 7, 2010 Author Share Posted October 7, 2010 how do i do it from the command line? not familiar with linux? Link to comment
ohlwiler Posted October 7, 2010 Share Posted October 7, 2010 The command is mv a search in this forum will show syntax. Instead I suggest you use Midnight Commander which is built into unRAID. To invoke it type mc from the console or a telnet session. It takes a little time to get used to the user interface, but it is very powerful. I stopped writing to user shares for just this reason and only write to disk shares. It has the side benefit of being faster. I also shuffle all of my data from disk to disk using Teracopy under Windows explorer, it is easier, but a little slower and it gives me a warm fuzzy to run the CRC check that Teracopy supports. Link to comment
kizer Posted October 7, 2010 Share Posted October 7, 2010 Yep I as well use TeraCopy and copy files to disk shares opposed to user shares. //tower/disk1 However I let my users aka myself and my wife view the users shares since its all mapped nice and pretty. Link to comment
ericft Posted October 7, 2010 Author Share Posted October 7, 2010 for those using only disk shares, what do you do when a set of data ie Movies folder has more files than can be stored on one drive?such as 2.2tb when your largest drive is 1.8tb formatted? i thought that was one of the points of unraid that you could span disks...so if one writes only to disks, is it as secure? Link to comment
Joe L. Posted October 8, 2010 Share Posted October 8, 2010 for those using only disk shares, what do you do when a set of data ie Movies folder has more files than can be stored on one drive?such as 2.2tb when your largest drive is 1.8tb formatted? i thought that was one of the points of unraid that you could span disks...so if one writes only to disks, is it as secure? I write to the disk shares... I can control where files get stored. Typically, each physical disk has a specific subset of an alpha range of movie titles. I let my media players use the user-shares, to consolidate the "movies" for easier browsing on the older media players that only give a file listing. Link to comment
SSD Posted October 8, 2010 Share Posted October 8, 2010 I write to the disk shares... I can control where files get stored. Typically, each physical disk has a specific subset of an alpha range of movie titles. I let my media players use the user-shares, to consolidate the "movies" for easier browsing on the older media players that only give a file listing. Joe L. - the most experienced and knowledgable unRAID user out there, does not write to his disks with user shares. Why would a brand new user that doesn't fully understand how unRAID works, as their very first act, set up a user share and write to their array. Writing to the disk shares is faster and far easier to understand. Load up your array with disk shares, create your user shares but use them only for reading initially. Then, if you feel that writing to the disk shares is to much of a PITA, play around with writing to the user shares. Link to comment
ericft Posted October 8, 2010 Author Share Posted October 8, 2010 "Load up your array with disk shares, create your user shares but use them only for reading initially. " sorry about not getting this...does that mean i enable disk smb read/write access (already done and I can see the disks on my network browser) and then just copy my files to the drives in question..ie: copy several genres from the Movies directory to one drive, another few to another drive, documents and software to another drive.... when you say use them for reading initially is that just for having the unraid setup the directories on the disks...then i just write directly to those directories as i see fit...when when I need to read those files, i have the HTPC just map to the user share? but all copying will be directly to disks, the user shares just setup for mapping purposes? I use sagetv with My movies and I want all my my movies data to be linked to one directory/source...i would prefer not to have the files linked by disk1\movie, disk 2\movie, disk3\movie in the my movies database...rather just say movies\blu-ray movies\avatar....in case my selection of drives changes, i don't want to break my mapping of video destinations...also i don't want to now create a whole alpha betical listing of my movies from scratch...just want to keep them in their genres... if i'm missing something fundamental, please let me know... Link to comment
SSD Posted October 8, 2010 Share Posted October 8, 2010 Exactly right. Create a user share called "Movies", then manually create "Movies" directories on several of your disks, and then copy your movies to those disks. The movies will all appear in the user share directory. Split level doesn't matter. Even the disk configuration doesn't matter. If a disk has a "Movies" directory, it will be part of the user share. This approach puts you in control of which movies wind up on which disks and avoids the problem that many new users discover of having files from the same movie spread across several disks due to an incorrect split level configuration. Plus the copying goes faster to disk shares (used to be a lot faster but I haven't seen recent comparisons of user share bs disk share write performance. Link to comment
Rajahal Posted October 8, 2010 Share Posted October 8, 2010 Maybe I'm in the minority (at least compared to the other hero members), but I've been using user shares since first day I built my first unRAID server. They are one of the primary things that attracted me to unRAID in the first place. My method was to start with a really high split level, then work it down until I figured out the right level. I also use 'included disks' and 'excluded disks' to limit certain shares to a certain set of disks. I keep my disk shares exported but hidden, although I almost never use them. I also use a cache drive, so my writes are much faster than writing to disk shares. Honestly, I can't imagine using disk shares for writing. For me, that would be like going back to my old archival system of 6 hard drives installed in a desktop computer. It is such a pain to start copying files over only to realize that a drive is full and you have to switch to another. I also used to use the 'Movies A-E, Movies F-M, etc.' method of organization, but I gave it up as soon as I discovered unRAID. What do you do when you want to add a movie into a directory on a full drive? You now have to start a new directory on a second drive, and well, it is just a pain in my opinion. I would rather let unRAID take care of all of that for me through user shares. So ericft, I guess I'm just saying don't be afraid of user shares if you want to use them. Getting the settings right can take a bit of trial and error, but once you get them right you'll be set for life. Link to comment
kizer Posted October 8, 2010 Share Posted October 8, 2010 I personally like the User shares vs Disk shares because of the following. I use User shares because I set everybody to read only and no matter what content I add or if I add additional hard drives its percieved that its one drive or a few depending on the shares. I use Disk shares set to hidden read/write so I can play admin on any machine on any account in the house by using Run > \\tower\disk1 or whatever disk I choose Also I'm really paranoid when it comes to Virus's and knowing that I have an open mapped drive with read/write permissions even thou I was the only one with read/write and everybody else has read only. Running scripts on my windows machine made me nervous so writing to hidden shares aka drives made more sense to me. I really tried to figure out the split levels and honestly I got really frustrated because no matter how much information I gleemed from everybody there was always something I felt like I was missing and even thou I tried to ask it 10 different ways I never seemed to get it. Not that support is bad, but maybe my frustration out weighed my patience. Link to comment
lionelhutz Posted October 8, 2010 Share Posted October 8, 2010 Out of curiousity, tell us all of the share settings. I had an issue when I used split level 1, "fill-up" and a min free space of something like 10gig (I think that I used 10000000 but can't recall right now and not going to look it up). Then, unRAID completely ignored my split level setting with these being used. If I go back to the default settings plus a split level of 1 then it works. It sounds like you want the "Movies" to go onto every disk but the genre subdirectories below to stay on one disk. Use a split level of 1. However, here is another option with much more control. Go to the disk shares and create the top file structure, by this I mean the top level Movies directory on each disk and then the Genre sub-directories on each disk where you want them to be. Now, use a split level of 0. What this means is that the file must go onto the drive where the parent directory exists. I'll give you an example. You create say something like the following directory structure; Disk1\Movies\Action Disk1\Movies\Children Disk2\Movies\Horror Disk2\Movies\Action You write a "Harry Potter" sub-directory and it's contained files to the \\Movies\Children directory. In essence, you are creating a new directory called; \\Movies\Children\Harry Potter Since the parent part "\\Movies\Children" already exists on disk1 it will not be re-created onto another disk. Only the new "Harry Potter" can be created on disk1. Now, say you write a "James Bond" sub-directory and it's contained files to the \\Movies\Action directory. In essence, you are creating a new directory called; \\Movies\Action\James Bond Since the parent part "\\Movies\Action" exists on both disk1 and disk2, the "James Bond" directory will be created onto either disk1 or disk2. The decision is based on which drive is presently being filled due to the allocation method. In both cases above, once that "Harry Potter" or "James Bond" sub-directory is created onto a disk it also becomes a fixed "parent". A new sub-directory created under either of them must also exist on the same disk. Say you created a "James Bond" directory and then put a bunch of directories for each movie below that parent directory. They all stay on the same disk. See the value of this. If you run out of space for one genre you dublicate the genre onto another drive and then you get 2 drives for that genre. Peter Link to comment
SSD Posted October 8, 2010 Share Posted October 8, 2010 Maybe I'm in the minority (at least compared to the other hero members), but I've been using user shares since first day I built my first unRAID server. They are one of the primary things that attracted me to unRAID in the first place. My method was to start with a really high split level, then work it down until I figured out the right level. I also use 'included disks' and 'excluded disks' to limit certain shares to a certain set of disks. I keep my disk shares exported but hidden, although I almost never use them. I also use a cache drive, so my writes are much faster than writing to disk shares. Honestly, I can't imagine using disk shares for writing. For me, that would be like going back to my old archival system of 6 hard drives installed in a desktop computer. It is such a pain to start copying files over only to realize that a drive is full and you have to switch to another. I also used to use the 'Movies A-E, Movies F-M, etc.' method of organization, but I gave it up as soon as I discovered unRAID. What do you do when you want to add a movie into a directory on a full drive? You now have to start a new directory on a second drive, and well, it is just a pain in my opinion. I would rather let unRAID take care of all of that for me through user shares. So ericft, I guess I'm just saying don't be afraid of user shares if you want to use them. Getting the settings right can take a bit of trial and error, but once you get them right you'll be set for life. There have been so many new users that make mistakes copying terabytes of information to brand new arrays, I think they should consider just copying files to disk shares for the initial load to avoid time consuming rearrangement after the fact. Some people prefer the finer control for the long term, others like you prefer unRAID to manage the writing of files. But for the first week it does not seem like such an inconvenience to help ensure the array gets loaded properly to use the disk shares for writing. Plus it is faster, which really matters on the intial array load. As for security concerns, using custom Samba shares you can implement user-share type security settings on single disk shares. They are more flexible that unRAID shares because you can create them at the subdirectory level. I use them extensively. I never access the disk shares. Link to comment
Rajahal Posted October 8, 2010 Share Posted October 8, 2010 Very good points, bjp999. I didn't know that you can apply user level security to individual directories, that is definitely handy. Link to comment
SSD Posted October 10, 2010 Share Posted October 10, 2010 Very good points, bjp999. I didn't know that you can apply user level security to individual directories, that is definitely handy. Here is a LINK to a post on setting up custom Samba shares. And another LINK about a feature added to unRAID to allow you to create these custom Samba shares in a documented way. Note that Samba shares can peacefully coexist with user shares. Link to comment
Recommended Posts
Archived
This topic is now archived and is closed to further replies.