[SOLVED] UNRAID 6.6.5 drive swap crashes system

[Al313]

I'm running 6.6.5 and wanted to swap a 2 TB drive for a new 4TB drive. I have a simple system, just a bunch of disks. A while ago, I bought 4 Toshiba HDWQ140 NAS drives and have installed (via swap as I'm trying to do now) three of these with no issues. This last one is causing a kernel panic and system crash. The previous 3 swaps were done with earlier versions of unraid (6.6.1 and 6.5.3). I've been trying to solve this myself for the past couple of weeks by searching the forum and doing tests with no luck. First thing I did was run SMART tests on the drive. Did short and long test without errors. Then did a preclear on the drive and it ran fine. After the preclear, I tried to swap again and system crashed. I have two available slots left on the server so I then tried to add the drive as a new drive to the array and that worked fine. Now I'm confused...the drive works fine as a new drive but won't allow a swap? I really wanted to keep my last two server slots open and I need to know that I can do a swap if I have a disk failure in the future. I also tried to swap the Toshiba drive to a different 2TB drive on a different controller just to see if it was controller or cable related and the system crashed then also. I also ran the memory test for 80 hours to eliminate memory issues. It would seem that the problem is parity-sync related. When I do the swap, the system lets me assign the disk just fine, recognizes that it is unmountable and on start, begins a parity-sync/data-rebuild and crashes after a few seconds. The console becomes unresponsive and not always but often shows the message : Kernel panic - not synching : timeout : not all cpus entered broadcast exception handler shutting down cpus with NMI Kernel offset : disabled Rebooting in 30 seconds..        However, it does not reboot. If I force a warm reboot, the system starts, recognizes the new disk as being emulated and starts a parity-sync/data-rebuild and crashes. Since I have done lots of experiments, I have attached two sets of logs and diagnostics. One set show my baseline for a normal startup (saved flash contents prior to the first swap attempt) and the next set is after I have assigned the new drive to  the array and ready to initiate the START command. I don't know if there is a way to log what unraid does after "START" as the system hangs and I lose access. If unraid does write something to the flash before crashing, let me know what to get and I can attach it to this post. I appreciate any suggestions.

mserver1-syslog-20181203-1125-normal-startup.zip

mserver1-diagnostics-20181203-1130-normal-startup.zip

mserver1-syslog-20181203-1140-after-disk-swap.zip

mserver1-diagnostics-20181203-1143-after-disk-swap.zip

[trurl]

Haven't looked at diagnostics yet, but just to clarify your description.

 

Are you attempting to hot-swap drives?

[JorgeB]

Those old SAT2-MV8 controllers have issues with some new disks, though they usually crash on boot and you are already using some of the same model, still and if it were me, I would try doing the rebuild with the new disk connected to one of the onboard Intel SATA ports to see if there's any difference, it it still crashes connect the 4 Toshiba disks to the onboard ports and try again.

[Al313]

I'm not trying hot swaps. When I swap, I always power down. 

 

Just curious, if my issue is related to the controller, why does it come up fine as a new drive?

[JorgeB]

Like I mentioned it's strange since there are already similar drives working, still there are known issues with those controllers and it won't hurt to try.

[Al313]

I just tried to connect the Toshiba drive to one of the motherboard SATA ports. The system allowed me to assign it just fine, started the parity-sync/sata-rebuild, ran for 15-20 seconds and crashed.

 

You did suggest I connect all four Toshiba drives to the onboard ports, so I will try that next. 

[Al313]

Update: I have attached all four Toshiba drives to the motherboard ports and the system is doing the data-rebuild. It's been running for 30 minutes, although it is painfully slow... about 10 to 12MB/sec. Unraid reports it will take 4 days. I'll keep updating this post so those interested can follow my progress. My thanks to Johnnie for the suggestion. It looks like there is an incompatibility with the SAT2-MV8 controller and this drive series from Toshiba. 

[JorgeB]

8 minutes ago, Al313 said:

It looks like there is an incompatibility with the SAT2-MV8 controller and this drive series from Toshiba. 

I've see issues before with those controllers and multiple newer devices, both disks and SSDs, you'll probably need to think about upgrading the server.

[Al313]

Update 2: The data rebuild completed without error. The system is now working fine. Thanks to Johnnie for suggesting a possible controller issue. Based on this experience, it would seem to confirm that there is a compatibility issue with the SAT2-MV8 controller and the Toshiba HDWQ140 NAS series drives. I hope this post will help those with the same controller to be aware of this.