Jump to content

Issues transferring big files


Recommended Posts

  • Replies 51
  • Created
  • Last Reply

Renewing a DHCP should not be disruptive so unless something is very broken I don't think that is relevant.

 

Actually, in the 5.x candidate releases I found that it was.  When the IP renew occurred, it would reset SAMBA, even if it kept the same IP.  My solution was to hard code the IP, no more problems.

Link to comment
  • 4 weeks later...

Okay little update and this should probably be moved to the os version 6

 

I got all my drives finally pre-cleared and formatted.

I am starting to move my files over and I am still having the same issue. Funny part is it don't matter if it is a file or a folder. If I move a large folder with large files one of them will will make me skip it.

I am starting to wonder if it is a Ethernet switch problem.

 

The switch I have is the TP-SG1016D

I am on average getting on full good transfer 95 MB/second

The most I have seen is 104 MB/second

Link to comment

Forgetting the red ball issue, which is another problem entirely.

 

The problem you are describing is a well known bug in the implementation of the Reiser file system on some platforms, not just unRAID (but it fails "safe" so it's never been fixed).

 

You are probably finding

It occurs on a nearly full disk?

Once it times out and you restart it, it is ok some of the time?

If you have another copy (read or write) running from the same client, that fails also?

 

It will go away in unRAID 6 if you move away from the Reiser file system.

 

 

Link to comment

Forgetting the red ball issue, which is another problem entirely.

 

The problem you are describing is a well known bug in the implementation of the Reiser file system on some platforms, not just unRAID (but it fails "safe" so it's never been fixed).

 

You are probably finding

It occurs on a nearly full disk?

Once it times out and you restart it, it is ok some of the time?

If you have another copy (read or write) running from the same client, that fails also?

 

It will go away in unRAID 6 if you move away from the Reiser file system.

I did a pre-clear so no reball.

I am using unRAID 6.0beta7 now.

The drive is currently 56GB out of 4TB used.

I just hit skip and write the file name down and transfer the file separate and it will work.

I am only doing a transfer from one computer to the server. No other activity is being transfered or read from the server.

 

Link to comment

There is a problem like you describe in Reiserfs, the work around is you just copy the file again.

 

But that will become a bigger issue when I start transferring bigger file folders

I have seen the issue you describe if the network plays up for any reason.

 

I have had no problems copying very large amounts of files (~3TB) if I do it internally within the server.  I have found that I can get issues when copying over the network.  If at that point I reset the network hardware (not unRAID) and then tell the copy process to retry it normally proceeds OK.  If I try the same without resetting the network then on a retry the error repeats.  This suggests that in my case at least the issue is in the network and not unRAID.

Link to comment

There is a problem like you describe in Reiserfs, the work around is you just copy the file again.

 

But that will become a bigger issue when I start transferring bigger file folders

 

I have seen this on 3 unRAID servers now.  Yes, it's a pain.  But it fails "safe", you never see it on read, only on write.  I have just got used to going back and recopying all the failures.  I mostly see it when the disk is nearly full.

 

I have read posts on other Linux forums that say this disappears when you change your file system from Reiser to XFS or EXT4.  I should imagine BTRFS is OK too.  I am looking forward to testing it when stable 6 is released.

Link to comment

Since the symptoms seem to change when you mess with the network, perhaps you have a dodgy piece elsewhere in the chain. Does the symptom stay the same from multiple clients? What happens if you try large transfers from 2 clients at once? Do they both fail, or only 1?

 

Well the large transfer is coming right from my computer to the server. I could possibility understand if it was going from one server1 to computer to server2. I haven't had any issues like this before in my network. This is why I am so confused.

 

Let me see on transferring the Large file folder to a different server and see what happens.

Link to comment
  • 4 months later...

Archived

This topic is now archived and is closed to further replies.


×
×
  • Create New...