(Solved) Copy from one unraid to another. Slow copy speed


Recommended Posts

Finally had all the parts arrive for a new unraid.  Decided to start fresh since my first unraid back from about 2009 has some pretty low specs.  Thought best to keep it for it's files and transfer to a new build.

 

SuperMicro X9SCM-F / Xeon E3-1230V2

 

FINALLY got the Ethernet port settings in bios correct, so I move on...  I set up Unmenu and Screen.  Started preclearing some 3TB WD Red drives (I thought I set up four).  I can't toggle between them.  This is my first concern.  Are they running?  The lights are all showing hard drive activity but I can only bring up one screen show drive status (and with eight hours of it running, it looks like not much is happening).

 

So I decided to run a console preclear command.  I should have done this from the start.  But then my next mistake, I didn't account for 4K alignment.  Not knowing how to stop the process, I run it again on the same drive accounting for the 4K.  I'm guessing it's in the queue to do next once the first process is done?  I started consoles on the other three drives and can toggle between them.  Now I'm happy but wonder if they are still running through 'screen' from my first attempt.

 

Should I exit out of everything and start again through a console since that way is working for me?  Can I just shut down the raid box to stop it all and just start again?

 

Man, I don't remember this from so many years ago...

Link to comment

It sounds like you were running screen from a remote telnet session, right? The screen command to go between screen sessions is "Ctl-a n" (ignore quotes); but you have to invoke screen first. Sorry, it's been a while since I had to run preclear or screen. Help see: http://lime-technology.com/wiki/index.php/Configuration_Tutorial#Preclearing_With_Screen

 

On your 3TB drives you don't need to worry about the 4k alignment switch; it's automatically set.

 

Once you log out of the console, preclear will stop if you have not started it within a screen session.

 

Entering "powerdown" (again, ignore the quotes) from the command line will terminate all processes and shut the server off and is probably better than holding down the power button.

Link to comment

Thanks, doorunrun -

 

I have a better handle on it all.  My concern, now is I do have preclear running using screen from the console (directly from the unraid box).

 

BUT I also have a session running without screen from the a console, as well (not through a telnet session). 

 

Preclear through screen is about done (80% in the second step for four 3TB drives).

 

Preclear without screen, but directly from the console, is at about 5% for those drives. 

 

CAN I TERMINATE the preclears that are not through screen without affecting the screen preclear process?  What would be the command at each CTRL-F1 (through F4 for each disk)?

Link to comment

Thanks, doorunrun -

 

I have a better handle on it all.  My concern, now is I do have preclear running using screen from the console (directly from the unraid box).

 

BUT I also have a session running without screen from the a console, as well (not through a telnet session). 

 

Preclear through screen is about done (80% in the second step for four 3TB drives).

 

Preclear without screen, but directly from the console, is at about 5% for those drives. 

 

CAN I TERMINATE the preclears that are not through screen without affecting the screen preclear process?  What would be the command at each CTRL-F1 (through F4 for each disk)?

 

Yes. And you should or they will takes several days to complete.

Link to comment

Yeah, that didn't go well!

 

Thanks guys for the info tho.  Just didn't understand what I was doing.  I should have just started over, but...  I must have gotten through the preclear because assigning disks and formatting too about five minutes instead of hours.

 

I have realized something good out of redoing all of this.  I had XBMC export the library files to each movie folder I have stored and that movie info was not placed on the disk where the original movie file sits.  My 'drive management' was a bit of a mess that this will put is straight (at least, for now).

Link to comment

Will have to get someone in to figure out why my network is so slow for copying.

 

All gig hardwired.  No parity enabled, yet.  I have full copies of everything being transferred.

 

I'm using  teracopy and only getting about 34 MB/s.  I was hoping for around 100.

 

Sorry for a stupid question - the speed of the computer I drag and drop from (also hard wired) doesn't effect copy speed, does it?

Link to comment

But I am not using my parity drive, yet.  Will transfer about 8TBs over then install parity.

 

At this speed, adding parity may be worth doing now then transferring the remaining data.  I'm only 1 TB in at this point.

Populating the array with data exactly how you will normally be using it is a very good idea to make sure everything is stable, and the data you are copying to it is intact. The miniscule increase in speed you may get from running without parity isn't worth it IMHO.
Link to comment

You could also try Robocopy instead of Windows File Manager.

 

List of options and examples can be found here http://ss64.com/nt/robocopy.html

 

I use variations of this in command prompt.

 

ROBOCOPY \\10.XX.XX.18\Folder \\10.XX.XX.20\Folder /COPY:DT /DCOPY:T /E /Z /XO /ETA

 

Command -  ROBOCOPY

Source - \\10.XX.XX.18\Folder

Destination - \\10.XX.XX.20\Folder

Source options (Must be used with UnRaid) - /COPY:DT  http://lime-technology.com/forum/index.php?topic=19829.msg176401#msg176401

Source options - /DCOPY:T : Copy Directory Timestamps

Source options - /E : Copy Subfolders, including Empty Subfolders

Copy options - /Z : Copy files in restartable mode (survive network glitch)

Advanced options - /XO : eXclude Older - if destination file exists and is the same date or newer than the source - don’t bother to overwrite it

Advanced options - /ETA : Show Estimated Time of Arrival of copied files (Not needed but nice)

 

Hope it helps

Link to comment

I can't move the data from one drive to another.  This is one reason this new build is a good idea for me.

 

I had my file structure pretty darn clean.  I decided to have xbmc export my library info to my unraid so rebuilding a database would always be quicker.  I found the movie info files  exported got written to other drives (different from where the movie file is). 

 

This slow process will win the race for me in the long run.

Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.