guyonphone Posted February 14, 2016 Share Posted February 14, 2016 I wish it were possible to have the GUI show how fast the mover script is copying like it does for Parity Checks and Data Rebuilds. Mockup: Thanks Quote Link to comment
NAS Posted February 15, 2016 Share Posted February 15, 2016 A nice idea my only worry would be the difference in speed moving of lots of small files vs large files causes and the support issues this would generate. Quote Link to comment
garycase Posted February 15, 2016 Share Posted February 15, 2016 For what purpose? You already know that writes to the protected array are MUCH slower than writes to a cache drive ... that's the whole idea of using a cache. The mover is going to write at essentially the same speed the writes would have been at if they hadn't been cached ... which will, as NAS noted, be appreciably slower for a lot of small files vs. a few larger files. But the speed shouldn't matter => that's why it's designed to run overnight (although you can, of course, schedule it for other times, or simply force it to run whenever you want). Quote Link to comment
trurl Posted February 15, 2016 Share Posted February 15, 2016 And if you really want to know for some reason you can see mover runs in the syslog very easily. Quote Link to comment
guyonphone Posted February 15, 2016 Author Share Posted February 15, 2016 For those asking why this feature would be nice. This feature would be nice for knowing how much work your cache disk is doing. If I know the mover is running and it's transferring at a certain rate. Perhaps I would wait to make a copy. But if it were going pretty slowly and I know that I have some bandwidth free for another copy to it. Maybe this idea isn't a winner, but I would like this feature. Thanks! Quote Link to comment
garycase Posted February 16, 2016 Share Posted February 16, 2016 Assuming the cache drive is either a modern high-density (1TB/platter or better) drive or an SSD, then writing to it at network-limited speeds (assuming you only have a Gb network) won't slow down any moves that are ongoing. No need to see mover activity to know that. Quote Link to comment
NAS Posted February 16, 2016 Share Posted February 16, 2016 IMHO there is nothing lost by including this request. Certainly it is way less useful than originally imagined but it is not without some use and anything that stops a user going to the console is a win. Quote Link to comment
RobJ Posted February 16, 2016 Share Posted February 16, 2016 Mover is just rsync right? So this request is for adding the --progress option to the rsync command, and then capturing its output and putting it into the GUI in a usable form. Technically, this number will be the speed of constructing the current single file at the destination, but should be close to the transfer speed. Trying to keep the GUI updated for each file will probably impact transfer performance, especially when transferring many small files, but since this is the Mover that is not important. Quote Link to comment
methanoid Posted February 16, 2016 Share Posted February 16, 2016 I dont give a **** how fast it is copying but a progress indicator would be useful. 10/20/30/40/50% etc complete... Would be informative Quote Link to comment
WeeboTech Posted February 16, 2016 Share Posted February 16, 2016 Some food for thought. FWIW, the mover writes it's pid in /var/run/mover.pid. If the gui was active it could grep lines from syslog matching the pid if it were tagged properly. The --progress option has useful information, but since it's designed for a terminal using \r instead of \n, it would not work well being piped into syslog via logger. In addition --progress tells you how many files and where you are in that list but that only works when doing one rsync to move a bunch of files. Currently the mover does a find and executes rsync many times, 1 rsync per 1 file. Changing mover to log to it's own log file and tailing that via the gui might work, but it's not necessary, The mover just needs to be tagged properly in the syslog with -tmover[pid] Also not all that difficult with a co-process in the mover depending on the bash version. I think the real gotcha for telling where you are in the list is the mover would need to be changed to capture how many files will be affected. Then provide some kind of indication of where it was in that list. I.E. a /bin/find capturing the file list, counting it, or iterating through it. It's much more complex then the current bash script that's in place using find and rsync. Quote Link to comment
RobJ Posted February 16, 2016 Share Posted February 16, 2016 Agreed. On the other hand, I've seen so many awful progress indicators, that I suspect most people's expectations are very low. We could create a rudimentary indicator by dividing 100 by the number of shares. So with an example of 5 shares, some with files to move, the indicator might jump to 20%, hang there awhile, then jump to 60%, hold briefly, then be done. Almost useful. But with that done, in future updates the developer could begin step-wise refinement, adding stepping updates to the number by either file counting or file sizes. By file count might be easier, but using file sizes would give a more accurate progress percentage. I recall one progress bar, that would stick at 0% for about 15 seconds, then jump to 73% and hang there for 2 minutes, then jump to 84% and hang there for a minute, then jump to 100% and hang there for 5 minutes. I don't know why the programmer bothered. Quote Link to comment
WeeboTech Posted February 16, 2016 Share Posted February 16, 2016 Another idea might be to let the mover do it's own thing with a co-process monitoring disk usage. When the mover ends it can send a signal to the co-process or remove the co-process's pidfile thus signaling it should exit. Or perhaps, (and again I'm thinking out loud so people get ideas) add another -exec line to the find that calls du -hs on the share. In these cases, using du the progress would not be going up, but would be going down as files are moved from the cache share. You would at least know how much data is left. If you know your array writes generally get from 60MB/s to 30MB/s you can guesstimate duration. Quote Link to comment
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.