June 16, 200917 yr I installed a Cache-drive as part of my Lacie edmini disk recovery operation to use it as a fast temporary storage. The idea was to directly copy the data from the edmini disk to Cache-disk. All my User shares by default have Cache-disk usage turned off so I thought Mover-script would leave all the copied directories and files alone. I copied single directory-tree named "share" from edmini-disk to the root of Cache-disk (path being /mnt/cache/share). This was in total of 450GB so it took some time to finish and I let it run over night. When I went sleeping (at 1AM) it had copied ~100GB of data. When I woke in the morning (8AM) the copy command had finished without errors. I then checked the size of directory structure but it was showing only 90GB, way less than the expected 450GB and also less than it had already copied before I went to sleep. I refreshed the information and now it showed 89GB. First I thought there was some terribly wrong but then I remembered the Mover-script with default scheduling at 3.40AM. I checked UnRAID main screen and saw that the last data disk in UnRAID array had about 350GB less free space than it should have had. So the Mover-script was moving data from Cache-disk to Disk3. I then browsed to Disk3 and saw that directory named "share" had been added to the root of Disk3 (path /mnt/disk3/share) and the recovered data was being copied to that directory. Then I checked Disk1 and Disk2 which also had the share-directory added to their root and some data inside the structure but as these disks were almost full the free size difference was not at first noticiable. At this point I decided to powerdown the server and restart the recovery operation. After the reboot a user share named "share" had been added to the UnRAID webgui Share-tab with Cache usage turned off. I checked the the previous syslog and "share"-share was not present then and I definetely had no added it in the mean time. Single question arises; Is the above possible or am I loosing my mind? I have not used user shares to this day though they are turned on. New root-level directories have been automatically added to the list of shares on system boot. I have not changed their default configuration. I have the following line in my syslog (for each user share) which I assume is caused by not configuring the user shares in the web gui: Jun 16 09:43:53 Viper_UnRAID1 emhttp: get_config_idx: fopen /boot/config/shares/share.cfg: No such file or directory - assigning defaults May this have caused the situation? I can try to reproduce the problem but for now I'm using directory named ".unmoved" within Cache-disk to keep things really unmoved.
June 16, 200917 yr I installed a Cache-drive as part of my Lacie edmini disk recovery operation to use it as a fast temporary storage. The idea was to directly copy the data from the edmini disk to Cache-disk. All my User shares by default have Cache-disk usage turned off so I thought Mover-script would leave all the copied directories and files alone. I copied single directory-tree named "share" from edmini-disk to the root of Cache-disk (path being /mnt/cache/share). This was in total of 450GB so it took some time to finish and I let it run over night. When I went sleeping (at 1AM) it had copied ~100GB of data. When I woke in the morning (8AM) the copy command had finished without errors. I then checked the size of directory structure but it was showing only 90GB, way less than the expected 450GB and also less than it had already copied before I went to sleep. I refreshed the information and now it showed 89GB. First I thought there was some terribly wrong but then I remembered the Mover-script with default scheduling at 3.40AM. I checked UnRAID main screen and saw that the last data disk in UnRAID array had about 350GB less free space than it should have had. So the Mover-script was moving data from Cache-disk to Disk3. I then browsed to Disk3 and saw that directory named "share" had been added to the root of Disk3 (path /mnt/disk3/share) and the recovered data was being copied to that directory. Then I checked Disk1 and Disk2 which also had the share-directory added to their root and some data inside the structure but as these disks were almost full the free size difference was not at first noticiable. At this point I decided to powerdown the server and restart the recovery operation. After the reboot a user share named "share" had been added to the UnRAID webgui Share-tab with Cache usage turned off. I checked the the previous syslog and "share"-share was not present then and I definetely had no added it in the mean time. Single question arises; Is the above possible or am I loosing my mind? I have not used user shares to this day though they are turned on. New root-level directories have been automatically added to the list of shares on system boot. I have not changed their default configuration. I have the following line in my syslog (for each user share) which I assume is caused by not configuring the user shares in the web gui: Jun 16 09:43:53 Viper_UnRAID1 emhttp: get_config_idx: fopen /boot/config/shares/share.cfg: No such file or directory - assigning defaults May this have caused the situation? I can try to reproduce the problem but for now I'm using directory named ".unmoved" within Cache-disk to keep things really unmoved. The behavior you saw was the expected behavior. The setting on each "user-share" to use, or not use the "cache" drive is used when writing to the user-share It has no effect on the "mover" script which will automatically create a top level folder on your data disks as needed for each of the top level folders created on the "cache" drive. The exception are those folders on the cache drive that start their names with a leading period. "." So, even though you have not used user-shares, by creating directories in the cache drive you are "defining" user-share folders (unless their names start with a leading period) The lines you saw in the syslog exist if no explicit config exists for that user-share. (I see them in mine too, for the few folders I created at the top level by making the directory using "mkdir" rather than through the user-share web-interface. Joe L.
June 17, 200917 yr Author Coming back to this. I created a directory named ".unmoved" in the root of the Cache-disk as instructed in the Release-notes to make the directory to be bypassed by the mover-script. Now my syslog is filled with entries like below. Unfortunately I currently have a very large number of files in the directory structure on Cache-disk and I now have for each file a row in my syslog (~25000). Jun 17 03:41:24 Viper_UnRAID1 logger: mv: cannot move `./.unmoved/share/DVD Profiler export/Covers/6438044010152.16f.jpg' to `/mnt/user0/./.unmoved/share/DVD Profiler export/Covers/6438044010152.16f.jpg': No such file or directory Should it be like this? For the mean time I changed the mover schedule to be ran only on 1st of Jan...
June 17, 200917 yr Coming back to this. I created a directory named ".unmoved" in the root of the Cache-disk as instructed in the Release-notes to make the directory to be bypassed by the mover-script. Now my syslog is filled with entries like below. Unfortunately I currently have a very large number of files in the directory structure on Cache-disk and I now have for each file a row in my syslog (~25000). Jun 17 03:41:24 Viper_UnRAID1 logger: mv: cannot move `./.unmoved/share/DVD Profiler export/Covers/6438044010152.16f.jpg' to `/mnt/user0/./.unmoved/share/DVD Profiler export/Covers/6438044010152.16f.jpg': No such file or directory Should it be like this? It too is expected... (a result of some missing logic in the code, but expected) The mover script worked in two phases. First it created the needed folders on the target drives, and it includes the logic needed to ignore the folders with the leading "." Then, it moves the files to the new folders. Obviously, if the new folder does not exist, it fails... That is what you are seeing. The second phase of "mv" should have also had logic to exclude directories with a leading "." but on older version of the mover script it does not. See here: http://lime-technology.com/forum/index.php?topic=3171.msg26591#msg26591 The version of the mover script in 4.5beta6 I'm running here has the fixes. It is at /usr/local/sbin/mover and the lines of code in it moving the files now looks like this: (cd /mnt/cache ; \ find . \ \( -type d -regex '[.]/[^.].*' -exec mkdir -p /mnt/user0/{} \; \) , \ \( -type f -regex '[.]/[^.].*/.*' ! -exec fuser -s {} \; -exec mv -v {} /mnt/user0/{} \; \) ; \ find . -type d -regex '[.]/[^.].*' ! -exec fuser -s {} \; -empty -delete \ ) Either edit the file, or upgrade to a newer version of unRAID, or live with the log entries for now. Joe L.
June 18, 200917 yr Author I feel a bit silly right now, since this has been a known problem since July 2008, should have done better research first. Well atleast I didn´t start a new topic . I think I will modify the script to be identical to 4.5beta one.
June 18, 200917 yr I feel a bit silly right now, since this has been a known problem since July 2008, should have done better research first. Well atleast I didn´t start a new topic . I think I will modify the script to be identical to 4.5beta one. Use an editor that does not add carriage returns to the ends of the lines in the file. Or, process it through fromdos as described here: http://lime-technology.com/wiki/index.php/FAQ#Why_do_my_scripts_have_problems_with_end-of-lines.3F Joe L.
Archived
This topic is now archived and is closed to further replies.