I think I'm ready to rsync the files from my old NAS over to unRAID.

 

I don't have disk shares enabled, haven't used it. Can I make a user share DUMP and put everything in there?

 

I seem to read somewhere it's better to transfer into disk share but I don't have that enabled even though I can see disk1, disk2, and disk3 in ssh.

 

I'll run from the terminal of my old NAS:

 

rsync -av --stats --progress /mnt/disk1/ rsync://192.168.1.252/mnt/user/DUMP/

 

where 192.168.1.252 is my unRAID.

 

I've already tested it with one file and one directory transfer. Just want to confirm I can transfer all the files into a user share before I do it.

 

Once I sort the files into their respective directories, I plan to delete DUMP.

 

Why wouldn't you pre-create the shares and go share to share copy?  Also, did you setup the rsync server on the destination side?  If not, you should use rsync over SSH with minimal encryption something like:

 

rsync -av --progress -e "ssh -T -c arcfour -o Compression=no -x" <source_dir> user@<source>:<dest_dir>

 

Or create NFS mounts and do rsync "local" to avoid any encryption. 

 

 

Tim

 

Hi thanks for replying.

 

I think I created the rsync server on unRAID, I followed this post that came up in my research: http://lime-technology.com/forum/index.php?topic=13432.msg127670#msg127670

 

I'm basing what I'm doing on that post, kinda winging it.

 

Can you clarify what you mean by a share to share copy?

 

I also read about mounting it locally but  I couldn't get the mount to work.

I think KISS is your best bet here.  Pre-make all the shares so both sides have the exact same share names.  Then just rsync src to dest like this:

 

rsync -av --progress /mnt/user/Movies root@IP:/mnt/user/Movies (throw in the weaker encryption from earlier post)

 

The benefit to that is you can launch multiple rsyncs, all copying to their final resting spot.

 

Id recommend doing a permission change at the very end to make sure all is well on the unraid server with perms go.

 

Tim

 

Also if you want "local" mounts enable NFS sharing then do something like this:

 

All on source:

 

mkdir /new_server

mount IP_OF_OLD_SERVER:/PATH_TO_SHARE /new_server (mount 192.168.1.100:/mnt/user/Movies /new_server)

rsync -av --progress /mnt/user/Movies /new_server

 

Never use samba/smb to transfer linux to linux, is slow and shitty protocol.

 

You will have to remount the share after each transfer.

 

Tim

 

I think KISS is your best bet here.  Pre-make all the shares so both sides have the exact same share names.  Then just rsync src to dest like this:

 

rsync -av --progress /mnt/user/Movies root@IP:/mnt/user/Movies (throw in the weaker encryption from earlier post)

 

The benefit to that is you can launch multiple rsyncs, all copying to their final resting spot.

 

Id recommend doing a permission change at the very end to make sure all is well on the unraid server with perms go.

 

Tim

 

Yea KISS for sure, I don't know enough of the command line.

 

What's the difference between this rsync command and the one I was gonna do?

You dont have to add each and every one of the mount points into rsyncd.conf, it will also stop you from a double copy if you plan on dumping everything into the same directory. 

 

Tim

 

Should I undo what I did when I followed the steps in that post? The edits to inetd.conf and delete the rsyncd.conf file?

Should I undo what I did when I followed the steps in that post? The edits to inetd.conf and delete the rsyncd.conf file?

 

You can just leave it, when you reboot it should reset itself back to defaults since.

 

 

Tim

 

Oh I didn't know it won't survive a reboot, good to know.

 

Thanks for your help. I'll give it a shot later.

I want to try mounting the Buffalo NAS onto unRAID and do a rsync, but I can't do it with NFS.

 

I found these instructions here: https://lime-technology.com/wiki/index.php/Transferring_Files_from_a_Network_Share_to_unRAID

 

What I'm doing is:

 

SSH into unRAID.

 

mkdir /buffalo

mount -t cifs //buffalo_ip/share /buffalo -o iocharset=utf8,username=username,password=password

 

When I look in /buffalo I see the directories from my Buffalo NAS.

 

I'm not sure what to do next.

 

Do I:

 

mkdir /dump

rsync -av --stats --progress /buffalo/ /dump/

 

?

 

And if that's the right way, when the rsync is done, how do I move the files around in unRAID, to the user shares?

Only one thing I see that's a glaring problem is that you're creating /dump in RAM, so when you transfer the files it'll quickly error out.

 

Create a new share called dump and then rsync it to /mnt/user/dump  (or just rsync it directly to an already existing user share)

 

Alternatively, you can always just do a simple copy and paste via your Windows / Mac box from the Buffalo to unRaid.  Would be a little slower though...

Thanks for the headsup I did not know that will be in RAM! I'm new not just to unRAID but the entire Linux environment.

 

I wasn't sure if I can rsync to a user share and I remember reading about the disk-to-user-share bug which made me wary of trying anything like moving files around.

 

I'll do as you suggest with the new user share; I don't want to copy using Mac because as you said it'll be slower with a fair bit of data going through the Mac.

Thanks for the headsup I did not know that will be in RAM! I'm new not just to unRAID but the entire Linux environment.

 

I wasn't sure if I can rsync to a user share and I remember reading about the disk-to-user-share bug which made me wary of trying anything like moving files around.

 

I'll do as you suggest with the new user share; I don't want to copy using Mac because as you said it'll be slower with a fair bit of data going through the Mac.

With a few exceptions that aren't worth going into, any folder that not within /mnt/user is in RAM.

 

The so-called User Share Copy Bug (although I don't really consider it a bug when you dig down and realize what's actually happening) you won't get hit by because you're not going from a disk share on unRaid to a user share on unRaid.

 

rsync will work perfectly, but since you're new to linux, I would have to advise to stay away from it, unless you're very comfortable with working with command lines.  I would instead use midnight commander to move the stuff around.  (type mc at the command prompt).  It's a text based file manager GUI.

 

Beyond that, you should realize that by SSH'ing in and doing any of this, that if you happen to close the ssh session, then the transfer immediately aborts unless you install and use screen (you can get that from the NerdPack plugin)

 

 

Thanks for the reminder, I did install screen previously, after reading the instructions here.

 

Didn't think of mc, I'll give that a try, thank you!

The so-called User Share Copy Bug (although I don't really consider it a bug when you dig down and realize what's actually happening) ...

 

This is off-topic, but I have to agree with your point.  I wish we could rename it, without in any way lessening its significance or severity.  I started trying to come up with a better name, but so far, all I have come up with is "User Share Copy Problem".

 

There have been ideas raised in the past to get around it, to "fix" it, but they were not very workable, and possibly risky, hard to predict what would happen in unforeseen edge cases.  The only real fix is to not use User Shares, but they're a really nice feature.  At present, it's just not possible to try to safely have 2 different file systems managing the same files.  It's only safe if you only use one of the file systems, never both at the same time, for the same files.  And that's what this "Problem" is all about.

I wish we could rename it, without in any way lessening its significance or severity.  I started trying to come up with a better name, but so far, all I have come up with is "User Share Copy Problem".

Blame bjp999.  He named it    After this long, we're stuck with it (although whenever I have to post about it, always refer to it as so-called

I had screen rsync running over night and it's still going this morning.

 

The transfer is averaging at 11MB/s. Is this normal?

 

I tried to run ethtool on my Buffalo NAS but unfortunately ethtool is not available.

 

This is the ethtool result on the unRAID

 

Settings for eth0:
Supported ports: [ TP ]
Supported link modes:   10baseT/Half 10baseT/Full
                        100baseT/Half 100baseT/Full
                        1000baseT/Full
Supported pause frame use: Symmetric
Supports auto-negotiation: Yes
Advertised link modes:  10baseT/Half 10baseT/Full
                        100baseT/Half 100baseT/Full
                        1000baseT/Full
Advertised pause frame use: Symmetric
Advertised auto-negotiation: Yes
Speed: 1000Mb/s
Duplex: Full
Port: Twisted Pair
PHYAD: 1
Transceiver: internal
Auto-negotiation: on
MDI-X: on (auto)
Supports Wake-on: pumbg
Wake-on: g
Current message level: 0x00000007 (7)
		       drv probe link
Link detected: yes

 

 

Certainly looks like the buffalo is only connected at 100Mb/s

Hmmm yeah ... I guess the obvious thing would be the cable. 

Changed the cable and I'm getting better speeds. Seems to fluctuate quite a bit, from 21MB/s to 40MB/s.

 

But is this about right? Not really what else I can troubleshoot besides that cable.

Changed the cable and I'm getting better speeds. Seems to fluctuate quite a bit, from 21MB/s to 40MB/s.

 

But is this about right? Not really what else I can troubleshoot besides that cable.

More or less right.  Depends upon the speed of the drives installed, the size of the files, etc

 

I thought that might be the case. A huge improvement from the 11MB/s before. Thank you for the help!

After doing an rsync, is there a right way to compare/verify the files between two folders?

 

Would this command work?

 

rsync -rcnv a/* b/

+1 for salty brine's FolderMatch