Issues transferring larger files from Windows PC to UnRAID <SOLVED>

[Roscoe62]

Hi there...I need some help.

 

Over the last 2-3 weeks I've noticed an issue transferring larger media files from my Windows PC to UnRAID. I don't use a cache drive yet, so there files can take a while to transfer. However, over the last 2-3 weeks I've had a couple of files (ranging in size from 20-28 GBs) seemingly stop part-way through the transfer, and finally fail the transfer due to timeout. Sometimes I've seen this happen (only for the file transfer to re-commence from where it was stuck after 30-50 seconds).

 

I had this behavior happen to me once before about 3-4 years ago, and the issue was resolved instantly by replacing the ethernet switch the UnRAID server is directly connected to in the room where the server is kept. However, I must have just been lucky last time. This time I replaced the switch again but it didn't resolve the issue, and I also replaced the ethernet cable running from the UnRAID server to the switch, and the issue still remains.

 

I'm able to transfer smaller files without any problems, but anything from that 20-26 GB file size is currently having this issue. I don't have a need to copy files larger than 20GB right now, but I'm assuming this may also be subject to this issue.

 

I've attached the diagnostics zip file in case this helps diagnose the issue. Any thoughts or insights appreciated.

server-diagnostics-20240128-1525.zip

[Frank1940]

Try one thing.  Spin up all of the drives  in the array before you start the next transfer of a big file that would normally fail.

[Roscoe62]

The bad news is - I almost always spin up all the drives prior to doing a file transfer (I'm a little OCD like that)...and I definitely spun them up prior to seeing this issue occur (every time).

[Roscoe62]

No suggestions? Seems to me this is a fairly fundamental issue - not being able to transfer files to the server.

 

Is it worth raising an issue formally with the UnRAID support team?

[Frank1940]

Make sure that you reboot all the LAN devices---router and switches.

 

Some folks have had issues if they attempt to use Jumbo MTU's  (Best to stick to 1500 which is the default.)

 

This type of problem occurs so seldom and I can't recall that a single thing being the cause/solution.

 

Tell us what is on the sending end of these failures-- OS, Ethernet NIC, file manager, etc.  It looks like you have an Intel 1Gb NIC in your Unraid server.  Have you tried a second PC to see if it has the same problems?  

[Roscoe62]

I'm sending files from a Windows 10 PC. Pretty standard Realtek NIC - part of the Gigabyte motherboard. Instead of using the bog-standard Windows explorer, I use Teracopy to manage file transfers. This is the same setup I've used reliably for years.

 

What I'm going to do is try to set up a shared folder on my wife's laptop, which is in the same room as the UnRAID server, and try to transfer the file that's not transferring successfully. Such a transfer would use all the same network components as me trying to transfer the file to the UnRAID server - except for the cable to the switch, which I've already replaced. That's all I can think of to try. If the file transfer fails (times out) then the fault is unrelated to UnRAID.

 

I'll post back when I've done this test.

[Frank1940]

5 hours ago, Roscoe62 said:

I use Teracopy to manage file transfers.

Try Windows Explorer as a test for these files rather than Teracopy. 

[Roscoe62]

A couple of things to report...

 

I set up a shared folder on my wife's laptop (same location as the UnRAID server). I was able to transfer 6 files - all more than 20 GBs, one after the other - from my PC without any issues.

 

Then, as an experiment, I took the main file I've been having issues with, transferred it from my PC to my wife's laptop, and then - after spinning up all drives on my UnRAID server - I attempted to transfer the file from my wife's laptop to the UnRAID server...using Windows Explorer (she doesn't use Teracopy). The file transferred successfully, but it's worth noting that during the file transfer I observed the file transfer paused 8 times. Each pause lasted less than 60 seconds but the transfer speed dropped to 0 each time.

 

I'm pleased the file transfer was successful, but I think there's still an issue with file transfers at the UnRAID end. I will try to disable Teracopy to see whether that improves a file transfer's chance of recovery if the transfer pauses, but I think whatever is causing the file transfer pauses is the real culprit.

[Roscoe62]

Just a quick update...

I transferred another file from my PC to the UnRAID server - this time using Windows Explorer. Again, there were several pauses, but the transfer recovered...until a final pause that lasted around 60 seconds, and the transfer failed "unexpected network error". I tried again, and it failed again - different part of the file but same failure reason "unexpected network error".

[Frank1940]

Many long years ago, I had a transfer problem from Windows to my Unraid server.  I spent the better parts of two months trying to figure out what was happening.  One of the things I finally tried was to update the Realtek driver on my Windows computer.  That fixed the problem!  This solution is one that it probably outside of the 3-Sigma limit but it is something to consider.  (Just be sure that you go to the MB manufacturer to check to see what version they recommended for your MB.  Update if they have a later version.)

 

RealTek has had driver problems for years.  Windows is generally better off than Linux but they are the low-cost supplier of NICs and their driver software fits that niche.

Edited January 31 by Frank1940

[Roscoe62]

Thanks for that. That was a good idea. I went digging and found, from the MB manufacturer's site, their recommended NIC driver was a couple of months newer than the driver I was using - so I updated it, rebooted my PC and, after confirming the updated driver was being used, I tried to transfer the file again. The file transfer ended up pausing several times before finally failing - "unexpected network issue".

 

I'm pretty confident the issue lies at the UnRAID server end, but I have no idea where to go from here.

[dlandon]

Several suggestions:

• Remove the Dynamix System Stats.
• Install Tips and Tweaks plugin and make the recommended adjustments to the NIC and disk buffering settings.
• Upgrade to Unraid 6.12.6.  You are currently on 6.9.2,
• Review your network settings.  You have two NICs, but only appear to be using one (eth0).  Set up a bond and bridge and use both NICs.

[Roscoe62]

Okay.....

 

2 hours ago, dlandon said:

Several suggestions:

• Remove the Dynamix System Stats.

 

Done. Rebooted UnRAID server and tried a file transfer - still paused several times and recovered, but transfer failed on the last pause.

Something I noticed, and didn't think to mention until now - I have the main UnRAID window up while doing the transfer, and even though the Windows file transfer shows the transfer has paused, for a little while after that I can see the 'Writes' numbers for both Disk1 and the Parity drive continue to increment. Once Windows confirms the transfer has failed, the write numbers continue to increment for a short time, but then they stop incrementing. This may or may not be significant, but thought I should mention it.

 

3 hours ago, dlandon said:

• Install Tips and Tweaks plugin and make the recommended adjustments to the NIC and disk buffering settings.

 

I had to look up how to install plugins, and I see this one is reliant on the Community Applications plugin. I checked on that in the Apps tab, and it looks like there's a problem with the CA plugin retrieving a list of current plugins (see attached screen-shot). In the 'Network Settings' I see the DNS server was set to the same IP address as the IPv4 default gateway - which I thought was correct. I stopped the Docker and VM services, and changed the DNS server values to those suggested (8.8.8.8 and 8.8.4.4), but that didn't seem to do anything, so I changed it back. Also, the screen-shot shows a message saying an update to Community Applications is available, but my current UnRAID version (6.9.2) is too old...which segue-ways nicely to the next point...

 

3 hours ago, dlandon said:

• Upgrade to Unraid 6.12.6.  You are currently on 6.9.2

 

This is something I WANT to do (see this post)

...but I need to sort security out first - for the Windows PC's I need to have access, and also for the media devices to which the UnRAID server provides content. I didn't want to "break" their ability to read/access content by updating the UnRAID OS version. I was just beginning to look at this when this current issue appeared and took the priority. I'm happy to take your advice - if upgrading the UnRAID OS version will definitely help, then I guess I'll take the risk of breaking the media players' access, and look at restoring access ASAP afterwards. Please let me know...

 

 

 

[dlandon]

CA is only compatible with 6.12 and later.  It doesn't work on 6.9.X any longer.  I don't know why it says an update is available.

[Roscoe62]

According to my plug-ins tab, I already have Community Applications installed, which may explain the notification. There must have been an earlier version of CA that was compatible with 6.9.2 - it's the only explanation.

 

QUESTION: So, it sounds to me from what you've said so far, the best way forward is to update the UnRAID OS to the latest stable version as a 'next step'. Is that correct? If so, I'll have to park the file transfer issue, sort out whatever security/access issues that arise, and THEN come back to look at this issue again (IF updating the OS didn't fix the transfer by itself).

 

Please correct me if I got this wrong.....

[Roscoe62]

[dlandon]

1 hour ago, Roscoe62 said:

According to my plug-ins tab, I already have Community Applications installed, which may explain the notification. There must have been an earlier version of CA that was compatible with 6.9.2 - it's the only explanation.

 

QUESTION: So, it sounds to me from what you've said so far, the best way forward is to update the UnRAID OS to the latest stable version as a 'next step'. Is that correct? If so, I'll have to park the file transfer issue, sort out whatever security/access issues that arise, and THEN come back to look at this issue again (IF updating the OS didn't fix the transfer by itself).

 

Please correct me if I got this wrong.....

You have a copy of CA, but it won't update on 6.9.2.

 

That is a good plan.

[Roscoe62]

Sorry this took a while to do, but it's done now.

 

Today I upgraded to unRAID 6.12.6. All access is working OK

I updated all my installed plugins - including Community Applications

I tried to transfer my larger test file, and it still fails with a pause and a timeout error

I browsed CA and found the "Tips and Tweaks" plugin, and I installed it

Your original post said to make the recommended adjustments to the NIC and disk buffering settings.  Looking at the output from the plugin - I don't see any "recommended adjustments" per se. There are a list of the relevant settings on the left, and a list of current NIC settings on the right...

I'm not particularly technical - especially in this area. Can you offer a little direction?

 

 

 

[dlandon]

6 hours ago, Roscoe62 said:

Can you offer a little direction?

Start with these:

And be sure that "Performance" shows for the CPU Governor on the right side.

[Roscoe62]

Hi dlandon.

 

Thanks for the help!

 

I changed all the settings that needed to be changed as per your instructions above, except for the 'Ethernet NICS' - I'll get to that in a moment.

 

I applied the changes and tried the file transfer again - still a pause followed by a timeout. That's OK though - I know there are a number of things that need to be tried before we find the culprit for this issue.

 

OK, so in an earlier post you mentioned setting up a bond and bridge and using both NICs as a solution. Your last post also shows this as a recommended change.

I'm very sorry, but I'm not sure how to do this so can you give some advice? I imagine I'd need to go into the server's motherboard BIOS to enable eth1, and maybe add another network cable from the eth1 NIC port back to the switch (same as eth0)? Please let me know if that's not correct - I've never done this before so any instructions are appreciated.

[dlandon]

1 hour ago, Roscoe62 said:

OK, so in an earlier post you mentioned setting up a bond and bridge and using both NICs as a solution. Your last post also shows this as a recommended change.

I suggested it because it can sometimes help with network issues.

 

You would need to enable all NICs in your BIOS and have cables for each NIC connected to your router or a switch.  Then go to Settings->Network Settings.  Click on the Help icon in the upper right side of the GUI for explanation of each field.

[Vr2Io]

Pls also check does disk have enough space for 20GB+ and check minimum space setting too.

[Roscoe62]

5 hours ago, Vr2Io said:

Pls also check does disk have enough space for 20GB+ and check minimum space setting too.

 

Good thought, but all data disks still have around 1 TB available on each of them.

 

I didn't find a minimum space setting for the disks, but the minimum free space setting for the share I'm writing to is currently set to zero.

[Roscoe62]

So I found the motherboard manual and attempted to find a BIOS setting to enable LAN port 2.

 

I didn't find what I was expecting to find.

 

The only BIOS setting I could find that referenced the LAN1/LAN2 ports seem to be in the context of BOOTING from LAN.

Also, there is a jumper setting on the motherboard, but this seems to only be used to have the onboard LAN1/LAN2 ports enabled (the default setting) or disabled. So it selects between having both of them enabled, or having both of them disabled - not what we're going for here. This is the way I'm reading it anyway. I snipped the relevant sections of the document, pasted them into a Word document and attached it here. If you're keen to see the manual instead - just ask me and I'll attach it.

 

So, if enabling eth2 isn't done in the BIOS, or making a jumper change physically on the motherboard, how is it done?

UnRAID Server motherboard dual LAN port data.docx

[Roscoe62]

Hi,

 

I'm sorry to raise this again, but I'm desperate to solve this issue, and I don't have enough experience/expertise to resolve it on my own.

I posted the place I was stuck (not knowing HOW to activate the unRAID server motherboard's 2nd LAN port) in the above post I put up almost a week ago. I figured if someone was going to offer any further help, they would have done so by now, so I'm posting again to nudge this thread again.

 

Earlier, @dlandon suggested activating the 2nd LAN port, and setting up a bond and a bridge, and this sounds like good advice to me, but I can't do this if I can't activate the 2nd LAN port.

 

I'm grateful for any suggestions you can offer.

MNL-1586_2.pdf