Issues transferring big files

[demonmaestro]

I am having this issue when transfering a large file folder. It goes for alittle while then says it cannot connect. I check to see and the server is running just fine and still connected. If I hit skip it will semi continue with its transfer. So why the issue? Also I had downloaded the file from my USB Hard drive with no issues at all.

 

I had seen that some was saying that it is heat. The drive is cool (Disk3)

 

[SSD]

Ive had something similar on 6.0b6 using TeraCopy, but copying from Windows explorer have not failed yet. But speeds will slow down to < 1MB/sec and sit there for over 20-30 seconds before climbing back up to 70MB/sec or faster, finally to settle down to speeds in the 40MB/sec range. I was copying a bunch of files that that were 1-2G in size. But as I said TeraCopy will hang.

 

If others are seeing this too I do think something may be wrong.

 

I've long had issues with copying very large files through Windows, especially when the disk starts to get full. I think I've heard others mention this too. I haven't noticed this problem copying files from disk to disk using Linux command line, but it definitely happens over Samba (disk shares).

 

 

[jonp]

Hmm...

 

I will review this with Eric and Tom.  I haven't seen that problem in our lab, but doesn't mean there couldn't be something simple that we're missing here...

[gfjardim]

Hmm...

 

I will review this with Eric and Tom.  I haven't seen that problem in our lab, but doesn't mean there couldn't be something simple that we're missing here...

 

If I can speculate about this problem, seems like or shfs fuse-fs or md driver hangs the read while a dormant drive is wake up and becomes writable. This behavior makes sense in write operations, but not in read ones.

[demonmaestro]

Well whats kinda funny. Is you were saying you were having issues with samba hanging. That is what i use on my other smaller file servers. And have moved TBs atones with no issues at all. Granted I'm just connecting to a USB external hard drive.

Now all a sudden went and made a batter server/NAS and getting issues.

Also it don't matter on type of files. It still happens. So i can recreate the issue.

[krazijoe]

Had this problem on Friday. THought it was heat, so I turned everything off for the night, and started the next morning and same issue. It was about 35GB of files I wanted to move. It would slow do to under 1MB for a bit, then it would speed up. to normal. Eventually it would work and run through. After it finished, I did a parity check and all was fine.

[mikeybunting]

I have had write errors with this release as well, and it has been on larger/large-ish transfers/writes as well. 

[demonmaestro]

Does anyone have anything for me?

It does the same thing even when transferring a 600mb file over to the server..

[mr-hexen]

syslog, please post it

[demonmaestro]

/usr/bin/tail -f /var/log/syslog

Jul 27 14:31:04 MainNAS dhcpcd[1345]: eth0: acknowledged 192.168.1.155 from 192.168.1.1

Jul 27 14:31:04 MainNAS dhcpcd[1345]: eth0: leased 192.168.1.155 for 7200 seconds

Jul 27 14:31:04 MainNAS dhcpcd[1345]: eth0: renew in 3600 seconds, rebind in 6300 seconds

Jul 27 14:31:04 MainNAS dhcpcd[1345]: eth0: adding IP address 192.168.1.155/24

Jul 27 14:31:04 MainNAS dhcpcd[1345]: eth0: writing lease `/var/lib/dhcpcd/dhcpcd-eth0.lease'

Jul 27 14:31:04 MainNAS dhcpcd[1345]: eth0: executing `/lib/dhcpcd/dhcpcd-run-hooks' RENEW

Jul 27 14:31:04 MainNAS dhcpcd[1345]: eth0: sending ARP announce (1 of 2), next in 2.00 seconds

Jul 27 14:31:06 MainNAS dhcpcd[1345]: eth0: sending ARP announce (2 of 2)

Jul 27 14:32:54 MainNAS kernel: mdcmd (89): spindown 0

Jul 27 14:32:54 MainNAS kernel: mdcmd (90): spindown 3

 

That is the short

 

 

[demonmaestro]

Any update on this?

[trurl]

Maybe a complete syslog would help, but that snippet you posted makes me wonder if the IP lease is expiring during the transfer. Something in your router perhaps?

[Chris Pollard]

Renewing a DHCP should not be disruptive so unless something is very broken I don't think that is relevant.

[demonmaestro]

Here is the system log.

 

and I am running PFSence for my router/firewall

sys.txt

[mr-hexen]

your DHCP lease is only 1 hour long. Change it in your router.

 

Also note the time during your transfer that it fails.. my bet is hh:10 on the clock when it hiccups.

[demonmaestro]

the thing that gets me is even though the DHCP lease is short the ip does not change so who does it still give issue?

[mr-hexen]

well the renewal takes 2 seconds each time and its asking for a lease. the DHCP server will simply give it the same one time after time unless it assigned it to another machine. My HTPC's are on auto-ip and they have the same IP every time for years.

 

Change the lease to 86,400 seconds (24hr) and try the transfers again.

 

This solution seems logical to me because small transfers are less likely to overlap the renewal period while large transfers are, which is why you are noticing issues on large transfers.

 

I'm not promising this will work, but seems like an easy thing to try.

[demonmaestro]

I changed it but it does not help at all. Still same issue. Also once it "disconnects" me I have to refresh the directory in order for anything to work but half way on the transfer it will say cannot connect again.

I have disabled all ethernet connections on my desktop to just one and still same issue.

There is multiple ethernet connections with 2 plugged in but only 1 is getting an ip and showing up for some reason. 

 

AND now there is a solid red ball...

 

Go to the system log and it says

Fatal error: Allowed memory size of 134217728 bytes exhausted (tried to allocate 70204804 bytes) in /usr/local/emhttp/plugins/webGui/include/myPage_content.php(69) : eval()'d code on line 2

 

and the Log button says this

/usr/bin/tail -f /var/log/syslog

Jul 31 19:09:31 MainNAS kernel: mdcmd (81): spinup 9

Jul 31 19:09:31 MainNAS kernel: md: disk4: ATA_OP e3 ioctl error: -5

Jul 31 19:09:31 MainNAS kernel: mdcmd (82): spinup 10

Jul 31 19:09:31 MainNAS kernel: mdcmd (83): spinup 11

Jul 31 19:09:31 MainNAS kernel: md: disk7: ATA_OP e3 ioctl error: -5

Jul 31 19:09:31 MainNAS kernel: mdcmd (84): spinup 12

Jul 31 19:09:31 MainNAS kernel: md: disk10: ATA_OP e3 ioctl error: -5

Jul 31 19:09:31 MainNAS kernel: md: disk11: ATA_OP e3 ioctl error: -5

Jul 31 19:09:31 MainNAS kernel: md: disk12: ATA_OP e3 ioctl error: -5

Jul 31 19:09:31 MainNAS kernel: program smartctl is using a deprecated SCSI ioctl, please convert it to SG_IO

Jul 31 19:10:01 MainNAS last message repeated 202 times

Jul 31 19:11:02 MainNAS dhcpcd[1349]: eth0: renewing lease of 192.168.1.156

Jul 31 19:11:02 MainNAS dhcpcd[1349]: eth0: rebind in 2700 seconds, expire in 3600 seconds

Jul 31 19:11:02 MainNAS dhcpcd[1349]: eth0: sending REQUEST (xid 0x30baad12), next in 4.56 seconds

Jul 31 19:11:02 MainNAS dhcpcd[1349]: eth0: acknowledged 192.168.1.156 from 192.168.1.1

Jul 31 19:11:02 MainNAS dhcpcd[1349]: eth0: leased 192.168.1.156 for 7200 seconds

Jul 31 19:11:02 MainNAS dhcpcd[1349]: eth0: renew in 3600 seconds, rebind in 6300 seconds

Jul 31 19:11:02 MainNAS dhcpcd[1349]: eth0: adding IP address 192.168.1.156/24

Jul 31 19:11:02 MainNAS dhcpcd[1349]: eth0: writing lease `/var/lib/dhcpcd/dhcpcd-eth0.lease'

Jul 31 19:11:02 MainNAS dhcpcd[1349]: eth0: executing `/lib/dhcpcd/dhcpcd-run-hooks' RENEW

Jul 31 19:11:02 MainNAS dhcpcd[1349]: eth0: sending ARP announce (1 of 2), next in 2.00 seconds

Jul 31 19:11:04 MainNAS dhcpcd[1349]: eth0: sending ARP announce (2 of 2)

 

 

update: I went and stopped the array and it said that most of all my drives were red ball and the #3 drive was missing...

[trurl]

Now we're getting somewhere. Sounds like you have one or more hardware problems, maybe just drive connection problems. Check all sata and power connections. Post a complete syslog.

[demonmaestro]

Here we go. I checked all drives and made sure all were seated properly.

 

Funny part is I can still navigate the directory that is listed on that drive.

sys.txt

[trurl]

A redball means that unRAID has made the drive read-only because writing to it failed. Also, unRAID will simulate a missing disk by reconstructing its data from parity and all other drives.

 

Did you reboot before getting this syslog? What is the webGUI showing now?

 

[demonmaestro]

A redball means that unRAID has made the drive read-only because writing to it failed. Also, unRAID will simulate a missing disk by reconstructing its data from parity and al other drives.

 

Did you reboot before getting this syslog? What is the webGUI showing now?

yes, and its still showing a red ball next to Disk3

[demonmaestro]

So any idea? Is it just drive3 that is causing my issue?

or do ya think its my addon card?

 

SUPERMICRO AOC-SAS2LP-MV8    <- that is what I have.

 

 

[JonathanM]

A redball means that unRAID has made the drive read-only because writing to it failed. Also, unRAID will simulate a missing disk by reconstructing its data from parity and all other drives.

Redball has nothing to do with the emulated drive being read only or not. It only means that unraid quit using the physical drive because a write to it failed. If the now emulated drive is read only, that points to corruption in the file system on that simulated drive.

 

Which power supply are you using?

[demonmaestro]

Its a Corsair 860ax

 

If you are saying its not the HD then how do I go about fixing the issue. The server is still empty besides what's on disk 3 and I still have all that data elsewhere still.