Transferring Files to unRAID

[Harblar]

Ok, So what is the best method for this?

 

I've pretty much always just gone into File Explorer on my PC, selected the files I wanted to to copy to my server, and then dragged them straight to the disk share I wanted them in on my unRAID server.

 

Since upgrading my server hardware I've noticed that transfers of large files (Bluray mkv's and such) hit higher speeds, but tend to be erratic and sometimes run into errors requiring me to "try again" on a file.  They usually start at around 100MB/sec, but then start jumping between anywhere from 1MBps to 80MBps and usually falling in the 30MBps range.

 

I'm running windows 10 (which until recently was having trouble seeing my unRAID server in File Explorer) and both my PC and server are less than 3 moths old, so I'm not entirely sure if it's a software issue, hardware problem, or something else altogether.

 

Both PC and Server are plugged into the same switch and aren't using more than 5 feet of cable between the two of them.

 

Is this something that could be taken care of more reliably through a separate program? I had a Killer Network Manager app that came with my PC Mobo (that I had to uninstall because it was full of memory leaks). Would something like this be useful in terms of ensuring these types of transfers have the highest network priority? I'm honestly not sure what's going on, exactly.

 

 

[danioj]

Ok, So what is the best method for this?

 

I've pretty much always just gone into File Explorer on my PC, selected the files I wanted to to copy to my server, and then dragged them straight to the disk share I wanted them in on my unRAID server.

 

Since upgrading my server hardware I've noticed that transfers of large files (Bluray mkv's and such) hit higher speeds, but tend to be erratic and sometimes run into errors requiring me to "try again" on a file.  They usually start at around 100MB/sec, but then start jumping between anywhere from 1MBps to 80MBps and usually falling in the 30MBps range.

 

I'm running windows 10 (which until recently was having trouble seeing my unRAID server in File Explorer) and both my PC and server are less than 3 moths old, so I'm not entirely sure if it's a software issue, hardware problem, or something else altogether.

 

Both PC and Server are plugged into the same switch and aren't using more than 5 feet of cable between the two of them.

 

Is this something that could be taken care of more reliably through a separate program? I had a Killer Network Manager app that came with my PC Mobo (that I had to uninstall because it was full of memory leaks). Would something like this be useful in terms of ensuring these types of transfers have the highest network priority? I'm honestly not sure what's going on, exactly.

 

There are many options available to you. However I will advise an option which is closely aligned with the content of your post.

 

So, using Windows 10 I would (and do) use a tool called teracopy:

 

http://www.codesector.com/teracopy

 

This is what "they" say about the need for their product.

 

One of the most common complaints about newer versions of Windows is the slow copying speed, especially when transferring lots of files over the network. If you want to speed up your copying or if you regularly transfer large amounts of data and have to stop the process to perform some other disk-intensive task, this program may be just what you need.

 

I have to admit I don't wholly disagree. I have used teracopy to flawless success (e.g. I used it to transfer over 10TB of data from one NAS to Another using an intermediary Windows 10VM which had terracopy installed) as I know have many other Windows users. It has features like resume and even a verify copy option which is excellent if you are paranoid (or get frequent network interruptions) and want to "be sure to be sure" that the transfer was a success.

 

Give it a whirl, I don't think you will be disappointed.

 

P.S. Make sure you you use the "Stable" version, their Alpha and BETA versions often have features missing and can be VERY buggy in my experience. The Stable versions however have been rock solid IMHO.

[JorgeB]

Are your disks reiserfs or xfs?

 

Reiserfs is known to cause erratic write speeds even with empty disks and timeouts with almost full disks.

[Harblar]

Yeah... Most of them are reiserfs.  I probably never noticed before since my old mobo and raid cards would only ever top out at around 30-35 Mbps

 

I should probably just take the plunge and start converting over to xfs a drive at a time. One of my newest 4TB drives got screwed up and write protected somehow to the point it absolutely wouldn't let me do anything other view or copy the files. If I tried to move or delete it just wouldn't go. I even tried resetting permissions  and a few other things a couple of times with no luck... Finally said to hell with it and formatted the dang thing (luckily there was only about one or two gig's worth of extremely non-essential data on it that I was trying to get rid of anyway.)  When Reformatted it I did it as xfs.  Next time I rip some movies I'll try sending them to that drive and see if it's any more consistent.

 

danioj,  I've used teracopy in the past and it seemed to work fine, though I wouldn't say any faster than file explorer.  Then again that was back on my old hardware and a windows 7 PC, So I might have to give it another shot. ;-)

[Adam64]

+1 for Teracopy.  Also, I'm guessing you've already done this, but make sure that SMB2&3 are disabled on your Win10 machine.

 

https://support.microsoft.com/en-us/kb/2696547

[Harblar]

+1 for Teracopy.  Also, I'm guessing you've already done this, but make sure that SMB2&3 are disabled on your Win10 machine.

 

https://support.microsoft.com/en-us/kb/2696547

 

I tried this a couple of times and never had any luck getting it to work. Still had to manually type in the server name to get it to show up in File Explorer, so I just reenabled them and forgot about it.  sometime within the last couple weeks Microsoft must have pushed an update that fixed this issue, because now it works just like it always did before.

[danioj]

 

danioj,  I've used teracopy in the past and it seemed to work fine, though I wouldn't say any faster than file explorer.  Then again that was back on my old hardware and a windows 7 PC, So I might have to give it another shot. ;-)

 

"I" wasn't saying that it was faster, that was their sales pitch. I said I couldn't whole disagree as I feel like it has been a "bit" quicker than Explorer file copy. That is however Anecdotal as I have no evidence to support such a statement.

 

That being said, I would STILL advise it over Explorer file copy operations for the "additional features" it gives you, which do make life much easier IMHO.

[Harblar]

Ok... So I really think the Reiserfs formatting must be the culprit. I just transferred a movie to an XFS drive and attached below is what the graph looked like this time. (I drew a red line to show what I was seeing before)

 

Much more stable and consistent. Other than one little dip/peak around the 50% mark it maintained 60MB/s (+/- 3MB/s) throughout.  Next time I'll try terracopy and see if that has any noticeable effect. In the meantime, I'll probably have to get started on converting to XFS. hahaha

[CSIG1001]

sorry had to revive this thread

 

Do you guys use the "verify" option in teracopy when copying over files ?  Not sure if this is required especially copying over movies i.e. ,mkv.

 

I hate to screw up a file

[Energen]

4 hours ago, CSIG1001 said:

sorry had to revive this thread

 

Do you guys use the "verify" option in teracopy when copying over files ?  Not sure if this is required especially copying over movies i.e. ,mkv.

 

I hate to screw up a file

I do not.