Unraid Crashes during large windows file transfer

[zacnaiman]

Hello guys, I just built my new computer and I’m new to unraid. I’m trying to transfer all my old data onto the new unraid machine using a basic SMB transfer. The problem is the system locks up and crashes after 1 min or 2 of transferring filles. GUI is unresponsive, server is not pingable  and requires a complete shutdown and restart to get working again. I have no idea how to fix this and hoping for any help I can get. I’m running a 12900k so that should rule out the issue I saw on here about c states as that was with AMD cpus. pool isn’t cached. Granted ik I’m trying to transfer allot of data, 20TB of movies and TV shows but I don’t understand why it completely crashes the system. Thanks in advance 

[trurl]

attach diagnostics to your NEXT post in this thread

[zacnaiman]

attached is the syslog. syslog i was doing a little troubleshooting and orignally when it crashed within a few mins i was using a multi threaded copy program. i tried to do it with traditional windows file system and while it was trasferring for a few hours it still crashed but last way longer. lmk what you think. thanks

[trurl]

Was crash between these entries?

Jan  9 22:30:21 MEDIASERVER webGUI: Successful login user root from 192.168.1.155
Jan 10 10:06:52 MEDIASERVER kernel: Linux version 5.10.28-Unraid (root@Develop) (gcc (GCC) 9.3.0, GNU ld version 2.33.1-slack15) #1 SMP Wed Apr 7 08:23:18 PDT 2021

Nothing obvious leading up to that.

 

On 1/8/2022 at 11:12 PM, trurl said:

attach diagnostics to your NEXT post in this thread

 

[zacnaiman]

ya crash was around 1:30 am i believe. did i set up syslog wrong or its just not recording this for some reason?

[zacnaiman]

its interesting cuz i was able to transfer my movies folder with traditional windows file explorer but tv shows keeps crashing. im thinking maybe thats because there are so many more files with TV shows cuz of each episode and maybe either computer (probs the old one) cant handle it?

[trurl]

Still waiting for you to

On 1/8/2022 at 11:12 PM, trurl said:

attach diagnostics to your NEXT post in this thread

 

[zacnaiman]

i thought i already posted them. they dont show anything as you said. ive tried different copying programs aswell and the server ends up crashing at one stage or another. attached are diagnostics. 

mediaserver-diagnostics-20220113-2001.zip

[trurl]

Have you done memtest?

[zacnaiman]

i havent yet. i can do that. at this point all try anything. the diagnostics didnt lead to anything did they?

[zacnaiman]

in order to run memtest i need to be in legacy and im booting in UEFI right now cuz my motherboard has a glitch where CSM wont stay and i have to flash the bios in order to fix it. i didnt bother cuz i assumed everything would be in in UEFI but if you think memtest is the next logical step ill do it. its just weird that it only crashes during SMB file transfers. it makes me believe that this is a software/driver glitch on either the old computer or the new unraid machine but i think it has something to do with the old one. i just dk what. am i able to access my share over FTP and maybe try it that way?

[itimpi]

You can download a UEFI compatible version of memtest from memtest86.com.

[kentb]

My unraid server is crashing in a very similar manner.

 

Brand new build (newly purchased parity SSD is a few hundred meg smaller than the other disks of the same brand, so reset and rebuilt the lot). First shares I created are the file shares, so I begin copying ... 

When copying a number of large files from Windows --> unraid, the network is fully utilised, then the network crashes. The entire ethernet network stopped working, but the wifi continued to work (I have some level of isolation - which seems to be pertinant)

 

I assumed it was a switch I have between the unraid server and the windows PC, so I connected diretly to the router (unraid server plugged into router as well). Nope - network down for all clients.

 

Assuming it was my windows PC, i eventually reset my network completely in Windows - blowing away all vpn, drivers and config etc. No joy.

Assuming it was my router (Asus beast with Asus-WRT merlin which I updated firmware a few days ago), I eventually factiry reset the router.

At this stage I was looking at buying a new router, or perhaps a new network adapter.

Plug in a laptop, network works, then goes down. then works, then goes down.Assume the Router has blown the ethernet chip and is overheated.

Pulliung out all network cables, restart router ... lappy only device ... it works. And is stable.  Plug in my PC ... still works. Wow. Run video on laptop and PC - still works.  Plug in the printer ... still works, plug in unraid ... CRASH ... entire network down.

 

Reboot unraid.  Network is working. Build parity and wait for parity to complete building.

 

Make some folders, start copying multiple large files win --> unraid (no switch, direct in the newly built asus router).

Copy about 6GiB, CRASH !!!  !!!!!!!!!!!!!!!!

Unplug unraid server ... all is working.

Plug un unraid, the array is down, and have to rebuild the parity disk all over again.  Machine looks to have rebooted.

 

SO ... when copying a large amount of files over SMB using win --> SMB, unraid crashes and breaks my entire network.  Confirmed it is unraid, and a very clean, recently built machine.

 

 

 

raider-diagnostics-20230129-0009.zip

[trurl]

14 hours ago, kentb said:

My unraid server is crashing in a very similar manner.

Please don't post in multiple threads about the same problem. It makes it impossible to coordinate responses.

 

New thread created, please go there to respond to this user:

 

[Brian Yuen]

Following this thread because I'm having the same issue. Transferring 2tb worth of RAW photos from my Mac to Unraid via rsync. System goes unresponsive, cannot login and just had to do a hard reset. This is the second time this has happened. The first time it happened, I let it sit overnight to see if Mover could do anything. Tried to login in the morning and was able to. This time it didn't work.

[trurl]

1 hour ago, Brian Yuen said:

see if Mover could do anything

You shouldn't be caching large transfers.

 

Attach diagnostics to your NEXT post in this thread.

[Brian Yuen]

Noted, thank you. I've attached the diagnostics zip

heavyd-diagnostics-20230201-0924.zip

[trurl]

Start the array and post new diagnostics

[Brian Yuen]

32 minutes ago, trurl said:

Start the array and post new diagnostics

Ah. sorry about that.. now attached with array started

heavyd-diagnostics-20230201-1200.zip

[trurl]

Which user share were you transferring to?

[Brian Yuen]

7 minutes ago, trurl said:

Which user share were you transferring to?

 

It was the one I created for myself 'brian'. The settings for that are:

• Use cache pool: Yes
• Enable copy on write: Auto
• Allocation method: High water
• Min free space: 5gb
• Split level: Automatically split any directory as required
• Include disks: All

[trurl]

And you left out which pool. Select cache pool: cache

 

You have no Minimum Free set for your pool named cache, so it won't overflow to the array if you fill it. You should set Minimum Free to larger than the largest file you expect to cache.

 

cache isn't full now, don't know if you filled it.

 

But in general, don't cache if you intend to transfer more than cache can hold. It is impossible to move from fast cache to slow array as fast as you can write to fast cache. And Mover is intended for idle time, trying to run it while transferring a lot of data just makes things worse.

 

 

 

[Brian Yuen]

1 hour ago, trurl said:

You have no Minimum Free set for your pool named cache, so it won't overflow to the array if you fill it.

Ah.. thanks for pointing that out.

 

1 hour ago, trurl said:

But in general, don't cache if you intend to transfer more than cache can hold.

So I'm assuming I can temporarily set 'Use cache pool' to 'No' in my Shares setting if I'm planning to move a ton of files?

 

Thanks for the help! I'm going to continue transferring the rest of my files after Parity check completes from the hard reset and will report back.

[trurl]

1 minute ago, Brian Yuen said:

temporarily set 'Use cache pool' to 'No'

correct

[itimpi]

13 hours ago, Brian Yuen said:

So I'm assuming I can temporarily set 'Use cache pool' to 'No' in my Shares setting if I'm planning to move a ton of files?

 

This is the recommended approach as it tends to be the most efficient.   

 

However if the Minimum Free Space value for the cache is set appropriately then you can leave it set to Yes and then when the free space falls below that new files will automatically overflow to the array.   Has the advantage you do not need to later set it back to Yes if this is how you intend to end up.