JustinChase Posted June 10, 2014 Share Posted June 10, 2014 I'm not sure why this is happening, or when it started, but my cache disk is filling up, so I started looking into what's on the cache drive. There's a lot that should have been moved, but is still there. I manually started the mover, but things remain that should not. I see the 'errors' in the syslog (attached), but I'm not sure why there's "No space left on device". Is it related to the fact that it's under "user0", which I'm not sure I understand (why is there another user?) Ideas? syslog.zip Quote Link to comment
sureguy Posted June 11, 2014 Share Posted June 11, 2014 User0 exists to facilitate the mover functionality. I would suggest ensuring there is enough free space on the destination drive, and to remember that is chosen based on split, allocation method and disk inclusion/exclusion. Sent from a mobile device, sorry for any typos. Quote Link to comment
JustinChase Posted June 11, 2014 Author Share Posted June 11, 2014 Yeah, that's what it seems like from the message, but I have 109GB free space on the drive with the least amount of free space, all the rest of the drives have more available (see attached). Quote Link to comment
sureguy Posted June 11, 2014 Share Posted June 11, 2014 You could try manually moving the files. The only other thing I can think of is the min free space setting on the shares. Quote Link to comment
JustinChase Posted June 11, 2014 Author Share Posted June 11, 2014 Manually moving the files seems like WAAAYYYY too much work to move each one. I just checked, and none of the shares has a minimum. Some had it set for 0, and others were just blank. Anyone else got any ideas on how to resolve? Quote Link to comment
DaleWilliams Posted June 11, 2014 Share Posted June 11, 2014 Grasping at straws... Run the fix Permissions script Reiserfsck --check on all the drives Have you checked to see if the files on the cache drive HAVE been moved? and just not erased from Cache Drive? Quote Link to comment
JustinChase Posted June 11, 2014 Author Share Posted June 11, 2014 That they may have been 'moved' but not deleted from the cache drive seems the most likely, but I'm not sure how to confirm, nor how to fix this situation. I would have to look at 10 different drives to see if a file on the cache is also on one of the other drives, which seems a lot of work to test a theory. I'd be willing to spend the time to do it, but even if I confirm this is the case, how would I fix this situation? I haven't looked at all folders, but in the few I have checked, there are dozens of files still on the cache that I've checked, so I assume there are hundreds of files 'stuck' on the cache drive. I don't want to try moving them one at a time, so how would I move them in bulk? Quote Link to comment
BobPhoenix Posted June 11, 2014 Share Posted June 11, 2014 If you use unMenu there is a duplicate file check option. Not at home so can't tell you exactly where it is at the moment. Quote Link to comment
JustinChase Posted June 11, 2014 Author Share Posted June 11, 2014 Thanks. I tried opening midnight commander, then moving whole folders from the cache to a user share, and that seemed to work. I assumed it would not, since normally sending to a user share just puts the files on the cache drive, but when I finished, I had over 400GB of free space on the cache drive again. Not sure how it got jacked up in the first place, nor why mover wasn't working, but I suspect it was duplicate files causing the issue. I don't have unmenu installed, so couldn't use that to check for duplicates. It would be a great addition to the core unRAID functionality though. Anyway, I'm calling this fixed now. Thanks for all the help. Quote Link to comment
DaleWilliams Posted June 11, 2014 Share Posted June 11, 2014 Mover *SHOULD* have been deleting things...calling this 'fixed' may be premature. You might run Reiserfsck --check on your cache drive. And run the permissions fix mentioned in the v.5 install instructions. Quote Link to comment
JustinChase Posted June 11, 2014 Author Share Posted June 11, 2014 Mover *SHOULD* have been deleting things...calling this 'fixed' may be premature. You might run Reiserfsck --check on your cache drive. And run the permissions fix mentioned in the v.5 install instructions. Agreed, but *my* issues are solved, even if mover may yet be broken. The lack of response from LimeTech tells me they are focused on other things right now, so if I call it solved or not is probably irrelevant at this point. How do I run Reiserfsck on the cache drive? What is the exact command, and do I run it from Putty? thanks Quote Link to comment
DaleWilliams Posted June 12, 2014 Share Posted June 12, 2014 Putty should do fine. I use Telnet from my Mac. See this link: http://lime-technology.com/wiki/index.php?title=Check_Disk_Filesystems Watch the cautions in Red...but basically, no harm will come if you ONLY '--check' with the ARRAY in MAINTENANCE MODE: 'Reiserfsck --check /dev/sdX1' where 'X' is the cache drive identifier. (on the webpage at the bottom, in 'Additional Comments' there's a discussion of the cache drive. Don't forget the digit '1' for the cache partition number) The 'X' can be figured out from your syslog. Because Cache drive is slightly different than the data drives, you also might want to read THIS thread: http://lime-technology.com/forum/index.php?topic=15942.msg147860#msg147860 where Joe L. explains it better. Quote Link to comment
SSD Posted June 12, 2014 Share Posted June 12, 2014 I would NOT move or even copy files from a disk share to a user share! It is a recipee for data loss!! If you did it you should check that the files arrived on an array disk. Quote Link to comment
JustinChase Posted June 12, 2014 Author Share Posted June 12, 2014 I would NOT move or even copy files from a disk share to a user share! It is a recipee for data loss!! If you did it you should check that the files arrived on an array disk. I copied from cache to user share, I'm not sure if this is the same thing. Regardless, it's done, so if I lost files, so be it. Maybe this thread will catch the attention of LimeTech, and they can/will investigate the issue with mover for future users. Quote Link to comment
SSD Posted June 12, 2014 Share Posted June 12, 2014 If the cache is in the share I think it is the same thing. Removing a disk from the share does NOT fix the problem. Please let me know if this happened to you. I have been trying to raise awareness on this issue with Limetech and a real user losing data would get more attention than the theoretical. Quote Link to comment
JustinChase Posted June 13, 2014 Author Share Posted June 13, 2014 If the cache is in the share I think it is the same thing. Removing a disk from the share does NOT fix the problem. Please let me know if this happened to you. I have been trying to raise awareness on this issue with Limetech and a real user losing data would get more attention than the theoretical. I must admit that I don't really follow you here. I have no idea if 'the cache is in the share'. For me user shares are shares I manually setup, and the cache drive is an unRAID construct. I also don't know what you mean by 'removing a disk from the share' in this context. I have no disks assigned, nor specifically removed from any of my user shares, and I did not change that during my testing, or as part of the moving I did in Midnight Commander. With all that said, yes, I lost files. I had a movie that I had recently finished downloading, which is no longer to be found anywhere on my unRAID server. I had hundreds of files, maybe thousands that were 'stuck' on the cache drive that mover would not move, and no (easy) way to figure out what those files were. Now that I realize what has happened, I'm not at all happy about it. If you can bring this to the attention of LimeTech, I'd appreciate it, and will be happy to help them test/find a resolution. Thanks. Quote Link to comment
SSD Posted June 13, 2014 Share Posted June 13, 2014 Let me at least explain what happened so hopefully all will understand. This is confusing, so please ask questions. Do not glance through anything as all of this is important to understanding. What are user shares? A user share is a construct by which the files and subdirectories in root directories on ALL of the disks in an array are logically combined to form a logical mounted filesystem exported as a network share. The thing that combines them is called FUSE I believe. If a directory on a disk contains a subdirectory of the name of the user share, it participates in the user share. You have no control over this. The user share settings DO NOT impact whether a disk participates in a user share. The only way to remove a disk from a user share is to literally rename the subdirectory so that it does not have the same name as the user share. You may have to stop and start the array afterwards - I am not sure. You may even have trouble renaming it - I've never tried as I have never been a user share fan. But I repeat, if the root of any drive has a subdirectory with the same name as the user share, its files are combined with the other files on other disks to form the user share. So what does the setting for user shares mean? The setting affects what disks NEW files are copied to if they are written to the share. So lets say you have a share called "Movies", and the "Movies" folder exists on the disks 1, 2, and 3. But the setting says only disk3, then new files will be written to disk3. So if you copy "NEWMOVE.ISO" to the user share, it will go on disk3. But the files from disk1, disk2, and disk3 are all in the Movies share. Now what happens if you do not create a new file, what happens if you are overwriting an existing file. Let's say that file lives on disk2. Well the file on disk2 will be overwritten. I do not believe it is written to the cache even if there is a cache disk but I may be mistaken. Perhaps you can confirm. (see ramblings below after reading the rest) Ok, so how does this result in losing my data? Ok, so let's say you try to copy or move a file from disk share "disk2" to the user share "Movies". The user share thinks you are overwriting an existing file. If that file were coming from a Windows box or some other directory, it would work just fine and overwrite the file just like you want. But coming from you disk2, unRAID will try to copy the file from disk2 to disk2. The operating system will usually have special protections to not let you copy a file on top of itself, but the user share somehow bypasses that protection. The net result is that the file is truncated. And if you were moving the file, it is then deleted from disk2, which also deletes it from the user share. Poof. You copied files from cache to the user share, and since the files were on the cache drive, they were copied overtop of themselves. Poof. I'm not trying to be cute. But this is what happens. Users should never copy files from a disk share to a user share. You either copy from a disk share to a disk share, or from a user share to a user share. But don't mix them. Ramblings If a cache disk changes the user share behavior, and instead of overwriting a file it stages it to the cache disk and plans to overwrite it via the mover, then a cache disk would protect you if copying from a disk share, except cache, to a user share. Again, I am not sure it works this way and a test would need to be performed to see. It sort of makes sense to me, given the very few situations that this has occurred, that this might be the behavior. But you were copying files from the cache disk - so they would go right back to the cache disk. And if they originated in the subdirectory for the user share, you would lose data as described. Quote Link to comment
DaleWilliams Posted June 13, 2014 Share Posted June 13, 2014 Bjp999, So following your logic, wouldn't this work: 1. Files are 'stuck' on the cache drive and may or may not have been written to user shares. 2. "movies" share is defined and is a top level folder on disk1, disk2, and disk3. 3. Using MC (or whatever) copy the top level folder contents FROM 'Cache/movies' into "DISKx Share/movies" (where x is 1,2,or 3) This is a specific copy from a specific physical disk (cache) to another specific physical disk. The problem is that the same file might end up in disk1/movies as well as disk2/movies, but that's usually better than losing a file. And there are tools to detect duplicate files. Quote Link to comment
SSD Posted June 13, 2014 Share Posted June 13, 2014 Absolutely. Copying from a disk share to a disk share (cache is a disk share in this context) is fine. Quote Link to comment
BobPhoenix Posted June 13, 2014 Share Posted June 13, 2014 Also if copying to a "user0" share folder is fine. That is how mover does it. I do that all the time with MC on unRAID. What doesn't work is a "user" share folder since "user" share folder's include the cache drive but "user0" share folders do not. So if I copy from "/mnt/cache/ATSC/Movies" to "/mnt/user0/ATSC/Movies" it works fine. But if I copy from "/mnt/cache/ATSC/Movies" to "/mnt/user/ATSC/Movies" it would delete the files as bjp999 points out. Quote Link to comment
SSD Posted June 13, 2014 Share Posted June 13, 2014 Also if copying to a "user0" share folder is fine. That is how mover does it. I do that all the time with MC on unRAID. What doesn't work is a "user" share folder since "user" share folder's include the cache drive but "user0" share folders do not. So if I copy from "/mnt/cache/ATSC/Movies" to "/mnt/user0/ATSC/Movies" it works fine. But if I copy from "/mnt/cache/ATSC/Movies" to "/mnt/user/ATSC/Movies" it would delete the files as bjp999 points out. It may be ok to move from the cache drive to the user0 share. But it is NOT ok to copy from another disk in the protected array to a user or user0 share. But I really suggest, when copying or moving files from one place to another on your array, that you stick with disk shares. It takes all of the danger out of it. I contacted Limetech on this issue and this is on the list of high priority issues to address. Quote Link to comment
BobPhoenix Posted June 13, 2014 Share Posted June 13, 2014 Also if copying to a "user0" share folder is fine. That is how mover does it. I do that all the time with MC on unRAID. What doesn't work is a "user" share folder since "user" share folder's include the cache drive but "user0" share folders do not. So if I copy from "/mnt/cache/ATSC/Movies" to "/mnt/user0/ATSC/Movies" it works fine. But if I copy from "/mnt/cache/ATSC/Movies" to "/mnt/user/ATSC/Movies" it would delete the files as bjp999 points out. It may be ok to move from the cache drive to the user0 share. But it is NOT ok to copy from another disk in the protected array to a user or user0 share. Correct. Sorry I should have pointed that out. Quote Link to comment
SSD Posted June 13, 2014 Share Posted June 13, 2014 (See update to my prior post) Quote Link to comment
JustinChase Posted June 18, 2014 Author Share Posted June 18, 2014 I just wanted to update the thread with a couple of updates. 1. I had originally created the share "download" as a share that uses cache, but not exclusively, then later changed it to use cache only. However, in the meantime, mover had moved some files to the download directory on 2 of my disks, which caused unRAID to be 'confused' and was including the 2 disks in the size calculation of the share called "download" and also caused mover to try to move files from this (now) cache only share to the array. removing the folders from the 2 disks resolved this issue. however, I still had problems with mover not working, and giving 'no room available' error messages, which I resolved by 2. Somehow, at some point in the fairly recent history, my user share called "video" had gotten changed from use cache: yes to use cache: only, which was preventing it from actually moving the files. I finally realized this issue, then changed it back to "yes", then mover started working again. I ended up with a couple of duplicate files, which i moved with Midnight Commander from the cache disk to the individual disk, and now it seems that I finally have a 'clean' cache disk. Thanks for everyone's help, and hopefully my (self-induced) issue has raised awareness with LimeTech of a potential problem, and leads to a better product for everyone. Quote Link to comment
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.