MrLondon Posted June 19, 2015 Share Posted June 19, 2015 Is there a way to ensure that a drive is not fill up fully? Because so far I have seen that it has only moved data to 1 drive and that is is now down to 40gb of free space, will that drop to 0gb free space or will unBALANCE automatically stop and move to the next drive? Quote Link to comment
jbrodriguez Posted June 19, 2015 Author Share Posted June 19, 2015 Hi MrLondon, It will leave some room, currently set to at least 450Mb. How it works is - It sorts the target disks by free space available - Tries to fill up the first disk as much as possible (respecting the 450mb) - Moves to the next disk, tries to fill it up as much as possible ... - Repeat the previous step as many times as needed You can choose one disk (or various) as not elegible (uncheck the TO column). Quote Link to comment
MrLondon Posted June 19, 2015 Share Posted June 19, 2015 Thanks for the answer, yes I have about 550mb of free space left on that disk, it moved to the next disk, but now it seems to have hung itself as for the past 5min at least it has not moved from a file and I cannot see any activity on the system with unmenu or system stats. is there a way to see from your system if it's hanging? from your logs these are the last lines I: 2015/06/19 14:46:31 core.go:345: ./movies/Sports Movies/Field of Dreams (1989)/movie.xml I: 2015/06/19 14:46:31 core.go:345: ./movies/Sports Movies/Field of Dreams (1989)/.actors/Timothy_Busfield.jpg I: 2015/06/19 14:46:31 core.go:345: ./movies/Sports Movies/Field of Dreams (1989)/.actors/Dwier_Brown.jpg I: 2015/06/19 14:46:31 core.go:345: ./movies/Sports Movies/Field of Dreams (1989)/.actors/Amy_Madigan.jpg I: 2015/06/19 14:46:31 core.go:345: ./movies/Sports Movies/Field of Dreams (1989)/.actors/Ray_Liotta.jpg I: 2015/06/19 14:46:31 core.go:345: ./movies/Sports Movies/Field of Dreams (1989)/.actors/Art_LaFleur.jpg I: 2015/06/19 14:46:31 core.go:345: ./movies/Sports Movies/Field of Dreams (1989)/.actors/Kevin_Costner.jpg I: 2015/06/19 14:46:31 core.go:345: ./movies/Sports Movies/Field of Dreams (1989)/.actors/James_Earl_Jones.jpg I: 2015/06/19 14:46:31 core.go:345: ./movies/Sports Movies/Field of Dreams (1989)/.actors/Mike_Nussbaum.jpg I: 2015/06/19 14:46:31 core.go:345: ./movies/Sports Movies/Field of Dreams (1989)/.actors/Gaby_Hoffmann.jpg I: 2015/06/19 14:46:31 core.go:345: ./movies/Sports Movies/Field of Dreams (1989)/.actors/Burt_Lancaster.jpg I: 2015/06/19 14:46:31 core.go:345: ./movies/Sports Movies/Field of Dreams (1989)/.actors/Lee_Garlington.jpg I: 2015/06/19 14:46:32 core.go:345: ./movies/Sports Movies/Field of Dreams (1989)/.actors/Larry_Brandenburg.jpg I: 2015/06/19 14:46:32 core.go:345: ./movies/Sports Movies/Field of Dreams (1989)/.actors/Frank_Whaley.jpg I: 2015/06/19 14:46:32 core.go:345: ./movies/Sports Movies/Field of Dreams (1989)/Field of Dreams-poster.jpg I: 2015/06/19 14:46:32 core.go:345: ./movies/Sports Movies/Field of Dreams (1989)/disc.png I: 2015/06/19 14:46:32 core.go:345: ./movies/Sports Movies/Field of Dreams (1989)/metabrowser.dvdid.xml I: 2015/06/19 14:46:32 core.go:345: ./movies/Sports Movies/Field of Dreams (1989)/mymovies-front.jpg I: 2015/06/19 14:46:32 core.go:345: ./movies/Sports Movies/Field of Dreams (1989)/backdrop.jpg this is now 30min ago since last update. Now there hasn't been an update for 1 hour actually. still at the last line from above. Quote Link to comment
jbrodriguez Posted June 19, 2015 Author Share Posted June 19, 2015 ... this is now 30min ago since last update. Now there hasn't been an update for 1 hour actually. still at the last line from above. It definitely looks like it's hung. Is the web interface still running (the activity indicator still moving) ? If you're ok with dropping to the command line, could you post the output of these commands ps aux | grep unbalance ps aux | grep rsync From the last lines you're showing, there doesn't seem to be any error. Not sure why it stopped. The log file itself has a bit more detail. Let me know if you could see anything different in the last lines of the file. Quote Link to comment
MrLondon Posted June 19, 2015 Share Posted June 19, 2015 hi Iboregard, here are the output you asked for,but the log is still flashing from left to right, but no update in the list. root@Tower:/boot# ps aux | grep unbalance root 1381 0.0 0.0 176 4 ? Ss 10:52 0:00 runsv unbalance nobody 1606 0.0 0.1 10540 5976 ? Sl 10:52 0:01 /usr/bin/unbalance -c /config root 6475 0.0 0.0 5104 1636 pts/0 R+ 16:45 0:00 grep unbalance root@Tower:/boot# ps aux | grep rsync nobody 5162 0.0 0.0 17100 1792 ? S 14:46 0:00 rsync -dIWRpEAXogtl --numeric-ids --inplace ./movies/Sports Movies/Field of Dreams (1989)/backdrop.jpg /mnt/disk13/ nobody 5163 0.0 0.0 17100 1692 ? S 14:46 0:00 rsync -dIWRpEAXogtl --numeric-ids --inplace ./movies/Sports Movies/Field of Dreams (1989)/backdrop.jpg /mnt/disk13/ nobody 5164 0.0 0.0 17100 220 ? S 14:46 0:00 rsync -dIWRpEAXogtl --numeric-ids --inplace ./movies/Sports Movies/Field of Dreams (1989)/backdrop.jpg /mnt/disk13/ root 6923 0.0 0.0 5104 1676 pts/0 S+ 16:45 0:00 grep rsync nobody 15365 0.0 0.0 13356 2352 ? S 14:30 0:00 find ./movies/Sports Movies -depth ( ( -type f ! -exec fuser -s {} ; ) -o ( -type d -empty ) ) -print ( -exec [ -d /mnt/disk13/{} ] ; -o -exec [ ! -e /mnt/disk13/{} ] ; -o -exec echo Duplicate skipped: {} ; -a -exec false ; ) -exec rsync -dIWRpEAXogtl --numeric-ids --inplace {} /mnt/disk13/ ; -delete root@Tower:/boot# Quote Link to comment
jbrodriguez Posted June 19, 2015 Author Share Posted June 19, 2015 MrLondon, I sent you a pm. rsync seem to have stopped, just can't think of why. Quote Link to comment
jbrodriguez Posted June 19, 2015 Author Share Posted June 19, 2015 After discussing this with MrLondon, a failing disk seems to have caused the underlying rsync to get stuck. I will further review if it's possible to make rsync timeout, so that a meaningful error can be sent back to the UI (or via email). So this is mostly a heads up, to people that may have a S.M.A.R.T. test failing on an array disk. Quote Link to comment
B_Sinn3d Posted June 25, 2015 Share Posted June 25, 2015 I hag about a terabyte of data I moved from my disk 2. It calculated fine as it was supposed to move data to two other array disks and leave plenty of space. Kick it off last night. This morning got the email that it completed. I checked tonight and one drive was at zero bytes available and it appears it duplicated a lot of data. I still have 500GB on my drive 2. How can I find out what happened? Quote Link to comment
jbrodriguez Posted June 25, 2015 Author Share Posted June 25, 2015 I hag about a terabyte of data I moved from my disk 2. It calculated fine as it was supposed to move data to two other array disks and leave plenty of space. Kick it off last night. This morning got the email that it completed. I checked tonight and one drive was at zero bytes available and it appears it duplicated a lot of data. I still have 500GB on my drive 2. How can I find out what happened? Hi B_Sinn3d, what does the email say ? It should list which operations were completed and which weren't (due to space restrictions). Also, the log file should have more details about what went on. Quote Link to comment
B_Sinn3d Posted June 27, 2015 Share Posted June 27, 2015 I hag about a terabyte of data I moved from my disk 2. It calculated fine as it was supposed to move data to two other array disks and leave plenty of space. Kick it off last night. This morning got the email that it completed. I checked tonight and one drive was at zero bytes available and it appears it duplicated a lot of data. I still have 500GB on my drive 2. How can I find out what happened? Hi B_Sinn3d, what does the email say ? It should list which operations were completed and which weren't (due to space restrictions). Also, the log file should have more details about what went on. There were no errors, everything appeared to work correctly. I will show the beginning and end of the email. "Move operation completed. These are the commands that were executed:" ...here are all the diskmv commands in this section. "Started: 2015-06-23 21:42:55.184367525 -0400 EDT Ended: 2015-06-24 05:04:59.728098283 -0400 EDT Elapsed: 7h22m4.543730855s" That is how it ended, there were no errors and the logs did not show anything. Quote Link to comment
jbrodriguez Posted June 27, 2015 Author Share Posted June 27, 2015 How did you set up the folders in the settings ? There are no overlaps ? Like "movies", "movies/action", "movies/action/80s", something like that ? Quote Link to comment
B_Sinn3d Posted June 27, 2015 Share Posted June 27, 2015 How did you set up the folders in the settings ? There are no overlaps ? Like "movies", "movies/action", "movies/action/80s", something like that ? I just typed my share names in there. Movies, Music, Pictures... Etc. Quote Link to comment
jbrodriguez Posted June 27, 2015 Author Share Posted June 27, 2015 Hi B_Sinn3d, I've sent you pm. Pls check your inbox. Quote Link to comment
zivizivi Posted June 28, 2015 Share Posted June 28, 2015 hi thanks for this cool tool. tried it for reiserfs xfs convo needs but failed on me due to my diskmv lack of info ("/" stuff). so i had to use midnight commander instead it took ages and blocked my laptop but now i see other uses for unbalancer and want to use it time to time. if u accept i got a few suggestions pls auto add "/" to folder names in settings. without the slash marks its a nightmare many will have to find out soon. e.g. directory tree; disk6\- Movies ->\-- watched --\Hitchcock Collection\Frenzy\Frenzy (1972).avi now if i just say to unbalancer to get - Movies - folder folder from disk6 and move it to disk 3 it will fail to move most of the folders and files due to nonstandard characters. but i can make it work using /- Movies -/ folder in settings. another thing i would like to see is to be able to pick more from disk tick boxes. so i can organise my all around data to single disks as much as possible with ease . dunno if that is possible but would be cool to collect all the folders and data scattered around disks and combine them in a disk or two with just a few clicks inside unraid without telnet and such. or maybe user shares to disks option? that would be cool as well. like from xx share to disk2 and disk3 one last thing; if i move x folder from disk1 to disk2 with unbalancer the x folder is still in disk1 empty inside but folder is there dunno why. anyway i dont know if any of this is possible or worth your time to implement/fix but these would make the unbalancer much better and usable. p.s. yes i made a big mess when moving data around for xfs conversation now i want an easy way to organize my data inside the array Quote Link to comment
jbrodriguez Posted June 29, 2015 Author Share Posted June 29, 2015 Hi zivizivi, I think I follow what you're saying about the trailing slash thing. It has to do with the semantics of the underlying rsync command that is being used. I will check this further. With regards to organizing the data, you might want to take a look at consld8 (Github). It seems to do what you're mentioning. Not sure why the original folders are not deleted. The rsync command should be deleting them, after the copy has finished. I'll try to replicate this. Quote Link to comment
HAVOC Posted July 12, 2015 Share Posted July 12, 2015 Just installed the docker img awesome work my friend. Quote Link to comment
spencers Posted July 13, 2015 Share Posted July 13, 2015 Great work! Almost finished converting to xfs. Now if only there were a docker to re-balance! Quote Link to comment
jbrodriguez Posted July 14, 2015 Author Share Posted July 14, 2015 Thanks HAVOC and spencers ! Quote Link to comment
Russ Uno Posted July 27, 2015 Share Posted July 27, 2015 I didn't read thru this whole thread so this may have been brought up. When using mv to move files the creation date gets changed.. I don't know if there is a way around it. Instead I've been using cp with -a to keep dates intact then I delete the original after checking the copies. Quote Link to comment
zivizivi Posted July 29, 2015 Share Posted July 29, 2015 yes lboregard i red the diskvm info when i first found the problem and trying to fix it. then i come here to help others and let u know. in the end i had to use midnight commander. it was no biggy just a telnet session from a 7/24 open server/download laptop. was a long time ago cant really remember why unbalancer wasnt enough : ) Quote Link to comment
nuggrave Posted July 29, 2015 Share Posted July 29, 2015 This docker is awesome, it made my transition to XFS super easy!! Quote Link to comment
wgstarks Posted August 11, 2015 Share Posted August 11, 2015 Is there a way to stop unbalance in the middle of a mv? It's been moving the same jpg file for several hours. The file size is only about 175kb. I think something is stalled, but I don't know how to check this. Quote Link to comment
jbrodriguez Posted August 11, 2015 Author Share Posted August 11, 2015 Hi wgstarks, you could stop the internal rsync from the command line. Do a `ps aux | grep rsync` and kill the relevant process. It should pick up from there and move onto the next file, but to be honest I've never tried it Is it possible that your disk is failing, seems weird that it has been stuck for hours trying to move the file. Quote Link to comment
wgstarks Posted August 11, 2015 Share Posted August 11, 2015 Not sure what's going on. Haven't had any indications of a bad disk. Have noticed that this disk shows "Active" all the time. I've seen lots of similar reports though, so I wasn't really concerned about it. Currently, Dashboard is showing disk 1 (from) is active and disk 9 (to) is inactive. I think that's a good indication that nothing is moving. Quote Link to comment
wgstarks Posted August 11, 2015 Share Posted August 11, 2015 I ran the command you posted. Not sure it did anything though- Last login: Mon Aug 10 19:15:46 on console maggie:~ wgstarks$ telnet 10.0.1.20 Trying 10.0.1.20... Connected to 10.0.1.20. Escape character is '^]'. Password: Login incorrect Brunnhilde login: root Linux 4.0.4-unRAID. Last login: Sat Aug 8 16:17:05 -0400 2015 on /dev/pts/0 from 10.0.1.190. root@Brunnhilde:~# ps aux | grep rsync nobody 19528 0.0 0.0 13488 2620 ? S Aug10 0:00 find ./Media/XBMC -depth ( ( -type f ! -exec fuser -s {} ; ) -o ( -type d -empty ) ) -print ( -exec [ -d /mnt/disk9/{} ] ; -o -exec [ ! -e /mnt/disk9/{} ] ; -o -exec echo Duplicate skipped: {} ; -a -exec false ; ) -exec rsync -dIWRpEAXogtl --numeric-ids --inplace {} /mnt/disk9/ ; -delete root 25100 0.0 0.0 5100 1752 pts/0 S+ 10:04 0:00 grep rsync nobody 32765 0.0 0.0 17184 2492 ? S Aug10 0:00 rsync -dIWRpEAXogtl --numeric-ids --inplace ./Media/XBMC/Movies/Captain Ron (1992) HDTV/extrafanart/mNwDE6QY2VqWilIJSLk8Rj7tDgK.jpg /mnt/disk9/ root@Brunnhilde:~# Did i need to run another command also to kill the process? Quote Link to comment
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.