opixo

Hey, take a look here, might be interesting for you:

(2198) Unraid 7.3 Beta: Set Up a New Server with Internal Boot - YouTube
and this:
Unraid 7.3 Beta: Convert Your Server to Internal Boot

This is a very old post, but maybe will help someone in same situation as me. Today, i finally had some time to troubleshoot it and togheder with my friend the AI who is also writing the solution below we manage to fix it. Here is what we did:

🔧 [RESOLVED] Dell OptiPlex 7050 + QNAP TL-D800S JBOD — Complete Fix Guide
Hey everyone, follow-up to my original post from August 2024. After a LOT more troubleshooting (and almost losing my mind), I've fully resolved the cold boot drive failure issue with the QNAP QXP-800eS-A1164 PCIe card.
First — thank you to @Vr2Io and @Frank1940 for the suggestions in the original thread. You were both pointing in the right direction:
@Vr2Io: The PCIe AER Fatal errors WERE the key symptom. Couldn't force PCIe 2.0 in the Dell BIOS, but the fix was disabling power management features that were destabilizing the PCIe link.
@Frank1940: Disabling array autostart + waiting before starting was absolutely the right instinct. The go script solution below is essentially an automated version of that, plus NCQ disable.
Posting this in detail because I found very few resources for this specific combination and hopefully it saves someone else days of headaches.
🖥️ Hardware
Server
Dell OptiPlex 7050 Tower (i7-7700, 32GB DDR4, BIOS v1.27.0)
JBOD
QNAP TL-D800S (8-bay, SFF-8088 connection)
PCIe Card
QNAP QXP-800eS-A1164 (ASM2812 switch + 2x ASM1164 AHCI controllers)
OS
Unraid 7.2.3, kernel 6.12.54-Unraid
Array Drives
4x Seagate ST4000NE001 4TB (1 parity + 3 data)
Pool Drives
2x Hitachi 2TB (ZFS mirror) + 2x HGST 10TB (ZFS stripe)
💥 The Problem
Every cold boot, array drives on ASM1164 controller #1 would fail. Two types of errors depending on what was going wrong:
Type 1 — PCIe AER Fatal errors (from original post, Aug 2024):
pcieport 0000:00:1d.0: AER: Multiple Uncorrected (Fatal) error received from 0000:03:00.0 pcieport 0000:04:02.0: device [1b21:2812] error status/mask=00100000/04400000 pcieport 0000:04:02.0: [20] UnsupReq (First) ahci 0000:05:00.0: AER: can't recover (no error_detected callback) pcieport 0000:00:1d.0: AER: device recovery failedType 2 — SATA/AHCI errors (after PCIe stabilized with BIOS fixes):
ata5.00: exception Emask 0x0 SAct 0x7fffffff SErr 0x0 action 0x6 frozen ata5.00: failed command: WRITE FPDMA QUEUED ata5: hard resetting link ata5.00: failed to IDENTIFY (I/O error, err_mask=0x4)Both types would crash ALL 4 ports on the same ASM1164 chip simultaneously, cascade into hard reset loops, and disable all drives. The array would hang during start and require SysRq reboot (echo b > /proc/sysrq-trigger) because ZFS import processes get stuck in uninterruptible D-state.
Warm reboots sometimes worked. Safe mode always worked. 100% reproducible on cold boot under any I/O load.
🔍 Root Cause — THREE Issues Stacked Together
1. ASM1164 NCQ Bug
The ASM1164 has a known bug with Native Command Queuing (NCQ). Commands sent as WRITE FPDMA QUEUED (NCQ) would timeout and crash the controller. This is documented across multiple platforms — Raspberry Pi Linux, openSUSE forums, various NAS communities.
2. Dell BIOS Power Management
The OptiPlex 7050's default BIOS settings include aggressive power saving that destabilizes PCIe devices:
Deep Sleep Control: enabled by default (cuts power to PCIe in S4/S5)
C-States: enabled (CPU power states affect PCIe link stability)
Block Sleep: disabled (allows S3 sleep which kills PCIe)
AC Recovery: Off (no auto-restart after power loss)
3. PCIe Remove/Rescan Hack Breaking BIOS Initialization
This was the hardest to find. I had a workaround in /boot/config/go that did:
echo 1 > /sys/bus/pci/devices/0000:04:00.0/remove sleep 5 echo 1 > /sys/bus/pci/rescanThis was supposed to "reset" the card. What it ACTUALLY did was destroy the proper firmware initialization that BIOS performed during POST. The Linux PCIe rescan does NOT perform the full firmware handshake — it just re-enumerates the device. The controller appeared to work for light I/O but would crash under sustained mixed read+write workloads (like parity rebuild).
The smoking gun was in dmesg timestamps: all drives were detected at 3–9 seconds (BIOS init), then the go script removed the card at ~35s and re-scanned at ~147s, causing drives to be detected a SECOND time with degraded initialization.
✅ The Complete Fix
All three issues needed to be addressed:
Fix 1: JBOD Firmware Update
Updated TL-D800S firmware: 1.4.1 → 1.5.0
Download from QNAP website. I had to temporarily move the QXP card to a Windows PC to run the QNAP firmware update utility. The JBOD firmware update is separate from the QXP card firmware.
Fix 2: Dell BIOS Settings (F2 at boot)
Section
Setting
Value
Priority
Power Management
Deep Sleep Control
DISABLED
⚠️ Critical
Power Management
AC Recovery
Last Power State
Recommended
Power Management
Block Sleep
ENABLED
Recommended
Performance
Intel SpeedStep
DISABLED
Recommended
Performance
C-States Control
DISABLED
⚠️ Critical
POST Behavior
Fastboot
Thorough
Recommended
These settings prevent the motherboard from doing anything that could affect PCIe link stability during boot or runtime.
Fix 3: Go Script — NCQ Disable Only (NO PCIe hack!)
File: /boot/config/go
#!/bin/bash # === ASM1164 / QXP-800eS Workaround v4 === # BIOS fixes (Deep Sleep off, C-States off, ASPM off) + JBOD firmware 1.5.0 # now allow proper cold boot detection. PCIe remove/rescan REMOVED — it was # breaking the BIOS-initialized controller state. QXP_BRIDGE="0000:04:00.0" ASM_VENDOR="1b21:1164" if lspci -d $ASM_VENDOR &>/dev/null; then logger "go: ASM1164 QXP card detected" # Phase 1: Let drives fully spin up and settle (BIOS already detected them) logger "go: Phase 1 — 30s drive settle..." sleep 30 # Phase 2: Disable NCQ for all drives on the QXP card # ASM1164 has known issues with NCQ (WRITE FPDMA QUEUED timeouts) logger "go: Phase 2 — Disabling NCQ for ASM1164 drives..." for dev in /sys/block/sd*; do devpath=$(readlink -f "$dev") if echo "$devpath" | grep -q "$QXP_BRIDGE"; then echo 1 > "$dev/device/queue_depth" 2>/dev/null logger "go: NCQ disabled for $(basename $dev) (queue_depth=1)" fi done logger "go: ASM1164 workaround complete — starting emhttp" else logger "go: No ASM1164 detected — normal boot" fi # Start the Management Utility /usr/local/sbin/emhttpKey points about the go script:
🚫 DO NOT do PCIe remove/rescan — it breaks the BIOS initialization
queue_depth=1 forces commands to DMA mode instead of FPDMA (NCQ). Verify with: dmesg | grep "ata[5-8]" — commands should show READ DMA / WRITE DMA EXT, NOT WRITE FPDMA QUEUED
30s settle time gives drives time to fully spin up after BIOS detection
The script auto-detects drives behind the QXP bridge, so it won't affect onboard SATA drives
Kernel Boot Parameters
In syslinux config (append line):
pcie_aspm=off pcie_port_pm=offDisables PCIe Active State Power Management at the kernel level. Belt and suspenders with the BIOS ASPM disable.
🎉 After the Fix
Did a New Config (Tools → New Config → Preserve All → Apply) to get a clean super.dat, then started the array normally (not maintenance mode, parity NOT already valid).
Results:
✅ All 8 drives detected cleanly in 3–9 seconds during boot
✅ Zero ATA errors in dmesg
✅ Parity rebuild running at 224 MB/s with zero errors
✅ All ZFS pools mounted with zero checksum errors
✅ SMB shares working, Docker containers running
✅ Array fully stable under sustained I/O for the first time
❌ What Didn't Work (save yourself the time)
SATA link power management (max_performance) — no effect
PCIe device remove + rescan in go script — MADE IT WORSE
Longer delays alone (90s, 125s) — didn't help without BIOS fixes
Starting in maintenance mode — controller still crashed after ~30s
Direct dd write tests to diagnose — just kills drives when controller crashes
🔎 How to Diagnose If You Have This Issue
Check if errors say WRITE FPDMA QUEUED — that's the NCQ bug
Check dmesg for duplicate drive detection (drives appearing twice = PCIe rescan issue)
Check if ALL ports on one ASM1164 die simultaneously — that's the controller, not individual drives
Test drives individually with dd reads when array is stopped — if they read fine, the drives are healthy
Look for hostbyte=0x04 (DID_ERROR) — that means host adapter error, not drive error
ZFS checksum verification: zpool status shows 0 errors = your data is intact regardless of md errors
🛠️ Useful Commands
Check NCQ is disabled (should show queue_depth=1):
for d in /sys/block/sd*; do echo "$(basename $d): $(cat $d/device/queue_depth 2>/dev/null)" doneCheck for ATA errors:
dmesg | grep -iE "ata[0-9]+.*(error|fail|reset|frozen)" | tail -20Check parity rebuild progress:
mdcmd status | grep -E "mdResyncPos|mdResyncSize|rdevNumErrors"Emergency reboot when array hangs (D-state processes block normal reboot):
echo b > /proc/sysrq-trigger💡 Hardware Notes
The ASM1164 is a known problematic controller. If you're building new, avoid it and get an LSI 9211-8i or similar HBA instead. But if you already have the QNAP QXP-800eS + TL-D800S combo, the fix above makes it work reliably.
The Dell OptiPlex 7050 is a solid Unraid server otherwise — the i7-7700 handles transcoding, VMs, and Docker well. Just fix the BIOS power management settings.
Hope this helps someone. Happy to answer questions! 🤙