SlrG Posted April 23, 2019 Share Posted April 23, 2019 (edited) advCPaMV Support the plugin? This is my Advanced Copy and Move Plugin for unRAID v6.8.3 and later. Please feel free to test it and report back if there are problems. To install it: v6.8.3 and later: Install it using CA or in the unRAID plugin manager by installing from github: https://raw.githubusercontent.com/SlrG/unRAID/master/Plugins/advCPaMV.plg Description: This plugin provides advanced cp and mv shell command alternatives named cpg and mvg that were created by applying the advcpmv-0.9-8.32.patch from here on the coreutils version used by unRAID. The Patch is a modified version of the one available here. But what does it do? cpg and mvg are the normal cp and mv shell commands extended to support the parameters -g or --progress-bar, which will show the progress of the copy or move commands. It will look like this while in progress: cpg -rg temp/ temp1 2 files copied so far... 3.0 GiB / 5.0 GiB [================================> ] 60.0 % Copying at 2.6 GiB/s (about 0h 0m 12s remaining) temp/file2.txt 1.0 GiB / 1.0 GiB [=============================================================> ] 99.9 % And afterwards it will show a summary of the copy process: 5 files ( 5.0 GiB) copied in 44.5 seconds (115.2 MiB/s). THIS SOFTWARE IS PROVIDED 'AS IS' AND WITHOUT ANY EXPRESSED OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE. THE AUTHOR SHALL NOT BE HELD LIABLE FOR ANY DAMAGES RESULTING FROM THE USE OF THIS SOFTWARE, EITHER DIRECTLY OR INDIRECTLY, INCLUDING, BUT NOT LIMITED TO, LOSS OF DATA OR DATA BEING RENDERED INACCURATE. SlrG Edited February 17, 2021 by SlrG rename commands 1 Quote Link to comment
ICDeadPpl Posted April 24, 2019 Share Posted April 24, 2019 Is it advisable to add 'cp' and 'mv' as aliases for 'gcp' and 'gmv'? Quote Link to comment
SlrG Posted April 24, 2019 Author Share Posted April 24, 2019 (edited) Well I have and I made the aliases to always use the -g switch too, as I really like to be able to view the progress of those operations. But I left that out of the plugin, so the users can decide for themselves. My aliases from my .bash_profile look like this: alias cp="gcp -g" alias mv="gmv -g" alias n="sudo -u nobody" alias ncp="sudo -u nobody gcp -g" alias nmv="sudo -u nobody gmv -g" The last three allow me to run commands, copy or move files from a root shell as user nobody, which is the default file owner on unRAID. Otherwise the new files will have root as owner, which might cause problems later on. Edited April 24, 2019 by SlrG added aliases 3 Quote Link to comment
wgstarks Posted August 11, 2019 Share Posted August 11, 2019 On 4/24/2019 at 2:41 AM, SlrG said: Well I have and I made the aliases to always use the -g switch too, as I really like to be able to view the progress of those operations. But I left that out of the plugin, so the users can decide for themselves. My aliases from my .bash_profile look like this: alias cp="gcp -g" alias mv="gmv -g" alias n="sudo -u nobody" alias ncp="sudo -u nobody gcp -g" alias nmv="sudo -u nobody gmv -g" The last three allow me to run commands, copy or move files from a root shell as user nobody, which is the default file owner on unRAID. Otherwise the new files will have root as owner, which might cause problems later on. I added the aliases to .bash_profile- # console coloring for kool kids PS1='\[\033[01;32m\]\u@\h\[\033[00m\]:\[\033[01;34m\]\w\[\033[00m\]\$ ' # if this is an xterm set the title to user@host:dir case "$TERM" in xterm*|rxvt*) PS1="\[\e]0;\u@\h: \w\a\]$PS1" ;; *) ;; esac # impersonate a user alias user="su -ls /bin/bash" alias v="ls -lA" # advanced copy aliases alias cp="gcp -g" alias mv="gmv -g" alias n="sudo -u nobody" alias ncp="sudo -u nobody gcp -g" alias nmv="sudo -u nobody gmv -g" How do I get them to survive a reboot? 1 Quote Link to comment
SlrG Posted August 12, 2019 Author Share Posted August 12, 2019 Copy the .bash_profile to your flash (/boot/config/) and add the following line to your go file: ln -sf /boot/config/.bash_profile /root/.bash_profile This will create a link overwriting the original file and pointing to the new one on the flash. Changing something in the original location will be put on the flash automatically and thus survive a reboot. 1 Quote Link to comment
wgstarks Posted August 12, 2019 Share Posted August 12, 2019 Thanks. After my previous post I was doing a little reading about .bash_profile vs .bashrc. Just want to confirm that the aliases need to be added to .bash_profile to work in the built in terminal window? Quote Link to comment
SlrG Posted August 12, 2019 Author Share Posted August 12, 2019 @wgstarks When trying just now, it works fine in the built in terminal when using .bash_profile. Quote Link to comment
Mantene Posted October 5, 2019 Share Posted October 5, 2019 On 8/12/2019 at 5:04 AM, SlrG said: Copy the .bash_profile to your flash (/boot/config/) and add the following line to your go file: ln -sf /boot/config/.bash_profile /root/.bash_profile This will create a link overwriting the original file and pointing to the new one on the flash. Changing something in the original location will be put on the flash automatically and thus survive a reboot. @SlrG Thank you for this!!! 2 Quote Link to comment
bubbaQ Posted March 15, 2020 Share Posted March 15, 2020 Have you tested it with *large* files? I get overflows with large files. See attached. Quote Link to comment
SlrG Posted March 16, 2020 Author Share Posted March 16, 2020 Sadly not with files that large. I guess it's better to withdraw the plugin then. I hope it did not corrupt files for you. Thank you for reporting the problem. Quote Link to comment
bubbaQ Posted March 16, 2020 Share Posted March 16, 2020 Don't be hasty.... I'm running hashes against them now to compare to backups. You might just need to use long integers and recompile it. Quote Link to comment
Squid Posted March 16, 2020 Share Posted March 16, 2020 19 minutes ago, bubbaQ said: Don't be hasty.... I'm running hashes against them now to compare to backups. You might just need to use long integers and recompile it. I've been asked to remove it from CA as a precaution. It can always get added back in at a later time. Quote Link to comment
SlrG Posted March 16, 2020 Author Share Posted March 16, 2020 Well, it might be hasty, but better save, then sorry. Until we know it does not damage data and how and if it can be changed to prevent this if it does, I wan't to minimize any risk as much as possible. Quote Link to comment
bubbaQ Posted March 17, 2020 Share Posted March 17, 2020 By "haste" I meant abandoning the project permanently. I completed my tests and all the hashes all match, so even on files over a TB, there was no corruption. Quote Link to comment
SlrG Posted March 21, 2020 Author Share Posted March 21, 2020 That's cool. Thank you very much for testing. I'll try to modify the patch to hopefully fix the overflows. Even if they are only graphical in nature, they could potentially unsettle people and need to be gone. I hope I'll able to fix them. It will take some time however. Quote Link to comment
cinereus Posted May 12, 2020 Share Posted May 12, 2020 You might also choose different command names. The repo suggests cpg/mvg or advcp/advmv it seems. gcp clashes with the ohmyzsh alias for "git cherry-pick" Quote Link to comment
SlrG Posted February 17, 2021 Author Share Posted February 17, 2021 @all Released a new version and renamed the commands from gcp and gmv to cpg and mvg. If you have shell aliases please update them accordingly! @bubbaQ If you still have those large testfiles and the freetime, could you give the new version a spin? I don't expect problems, but sometimes tests on one (my) system don't match others. Thank you very much! Quote Link to comment
BVD Posted March 8, 2021 Share Posted March 8, 2021 I just used it to migrate about 35TB of data from one unraid server to another without issue - usually use rsync, but it was a one time copy and didn't need the overhead I thought I'd incur. No issues on my end other than some graphical oddities that were more likely related to where I was transferring the data to than the tool itself - I had three concurrent streams going in tmux (so I wasn't just hitting one disk at a time) transferring via NFS over a 10G bonded pair, and I noticed whenever it went from copying many small files to the first in a series of larger ones, there was no progress noted for anywhere from 5 to 40 or so seconds. Looking at the network stats at the time, sure enough, things were dropping off significantly... Sure enough, it was due to network back pressure. If I end up having to do this kind of thing more often, I might finally have the justification needed for the time to implement RDMA, but for now, it's fine. Handy little plugin, thanks! Quote Link to comment
cinereus Posted April 8, 2021 Share Posted April 8, 2021 On 2/17/2021 at 9:08 PM, SlrG said: @all Released a new version and renamed the commands from gcp and gmv to cpg and mvg. If you have shell aliases please update them accordingly! Can I ask why? Quote Link to comment
SlrG Posted April 8, 2021 Author Share Posted April 8, 2021 @cinereus In your last post you mentioned, that Quote gcp clashes with the ohmyzsh alias for "git cherry-pick" and that the path repo suggested to use cpg and mvg as the command names. 1 Quote Link to comment
pbear Posted April 18, 2021 Share Posted April 18, 2021 Very nice plugin ! Thank you @SlrG 1 Quote Link to comment
SvbZ3r0 Posted October 4, 2021 Share Posted October 4, 2021 Not sure if this is a known issue, but the numbers don't add up. Version: 2021.03.17 Quote Link to comment
SlrG Posted October 5, 2021 Author Share Posted October 5, 2021 @SvbZ3r0 Yes, sadly the last change by me to solve the problems with quotes and double quotes in filenames is not perfect either and shows the shown symptoms under certain conditions. In all my test there was never data lost, but from time to time these "cosmetic" problems show up. If I have the time, I'll have a look at it again and will try to solve it. Sadly my times is very limited, so please don't expect fast results. Quote Link to comment
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.