Vr2Io

Members
  • Posts

    2874
  • Joined

  • Days Won

    3

Everything posted by Vr2Io

  1. That's fine and agree, so I have test on another platform, it is AMD X399 due to it just free for test, all hardware is different and OS 6.10.1 with IOMMU enable. The test show even worst, it can't mount the NVMe after format ( I do it twice ). May 28 16:25:12 X399 unassigned.devices: Removing all partitions from disk '/dev/nvme0n1'. May 28 16:25:25 X399 unassigned.devices: Format device '/dev/nvme0n1'. May 28 16:25:25 X399 unassigned.devices: Device '/dev/nvme0n1' block size: 1000215216. May 28 16:25:25 X399 unassigned.devices: Clearing partition table of disk '/dev/nvme0n1'. May 28 16:25:25 X399 unassigned.devices: Clear partition result: 1+0 records in 1+0 records out 2097152 bytes (2.1 MB, 2.0 MiB) copied, 0.00448569 s, 468 MB/s May 28 16:25:25 X399 unassigned.devices: Reloading disk '/dev/nvme0n1' partition table. May 28 16:25:25 X399 unassigned.devices: Reload partition table result: /dev/nvme0n1: re-reading partition table May 28 16:25:25 X399 unassigned.devices: Creating Unraid compatible mbr partition on disk '/dev/nvme0n1'. May 28 16:25:26 X399 kernel: nvme0n1: p1 May 28 16:25:26 X399 unassigned.devices: Create mbr partition table result: write mbr signature done May 28 16:25:26 X399 unassigned.devices: Reloading disk '/dev/nvme0n1' partition table. May 28 16:25:26 X399 unassigned.devices: Reload partition table result: BLKRRPART failed: Device or resource busy /dev/nvme0n1: re-reading partition table May 28 16:25:26 X399 unassigned.devices: Formatting disk '/dev/nvme0n1' with 'btrfs' filesystem. May 28 16:25:31 X399 kernel: BTRFS: device fsid 1da2b2f0-b622-4cd3-83f3-1bfb840e7ba5 devid 1 transid 6 /dev/nvme0n1p1 scanned by mkfs.btrfs (43803) May 28 16:25:34 X399 unassigned.devices: Reloading disk '/dev/nvme0n1' partition table. May 28 16:25:34 X399 kernel: nvme0n1: p1 May 28 16:25:34 X399 unassigned.devices: Reload partition table result: /dev/nvme0n1: re-reading partition table May 28 16:25:41 X399 unassigned.devices: Adding partition 'nvme0n1p1'... May 28 16:25:41 X399 unassigned.devices: Mounting partition 'nvme0n1p1' at mountpoint '/mnt/disks/SSD'... May 28 16:25:41 X399 unassigned.devices: Mount drive command: /sbin/mount -t 'ntfs' -o rw,noatime,nodiratime,nodev,nosuid,nls=utf8,umask=000 '/dev/nvme0n1p1' '/mnt/disks/SSD' May 28 16:25:41 X399 unassigned.devices: Mount of 'nvme0n1p1' failed: 'NTFS signature is missing. Failed to mount '/dev/nvme0n1p1': Invalid argument The device '/dev/nvme0n1p1' doesn't seem to have a valid NTFS. Maybe the wrong device is used? Or the whole disk instead of a partition (e.g. /dev/sda, not /dev/sda1)? Or the other way around? ' May 28 16:25:41 X399 unassigned.devices: Partition 'P02848106820' cannot be mounted. After change to 6.10.2, all resume normal. But if I restore back 6.10.1 ( X399 platform ), problem can't reproduce even reboot again ...... anyway I may do more test on original problem hardware ( B365 ). ** Above problem should be UD reload partition table issue and mount the BTRFS in NTFS. ** During B365 mobo upgrade, I have test many item and many time and corrupt problem never solve .... both NVMe ( WD & Samsung ) were new.
  2. Use Broadcom NIC with tg3 driver and enable VT-d ? If yes, you can edit the config file ( "Syslinux Configuration" ) to add intel_iommu=off or unplug the USB to another PC to edit it. Or Create a blank tg3.conf file to un-blacklist the driver if you confirm no corrupt issue by tg3+VT-d. Data Corruption possible with tg3 driver when Intel VT-d is enabled. The combination of Linux 5.15 kernel, tg3 driver, and Intel VT-d enabled appears to be causing data corruption. This has been verified on several platforms which include a Broadcom NetXtreme Gigabit Ethernet NIC (note: there may be others). This release includes the following workaround: Very early in server startup (rc.S) if Intel VT-d is detected enabled, then the script will unconditionally create the file: /etc/modprobe.d/tg3.conf with following content: blacklist tg3 Hence by default if VT-d is enabled, which is to say, it has not been disabled in either bios or via kernel "intel_iommu=off", then we are going to blacklist the tg3 driver on all platforms. What if someone has a platform where tg3 does not give them any trouble with VT-d enabled? In this case they must create an empty file on their flash device: config/modprobe.d/tg3.conf When the startup sequence continues it will get to the point where it executes: install -p -m 0644 /boot/config/modprobe.d/* /etc/modprobe.d A blank tg3.conf file stored on the flash then effectively un-blacklists it. There will be users who will lose network connectivity because their NIC is blacklisted. If you are running on a problematic platform you should go into your bios and disable VT-d. If this is a platform without issue, then you will need to create the blank tg3.conf file on your flash config/modprobe.d directory. It may take some time to identify and integrate a proper fix for this issue, at which point we will remove the auto-blacklisting code. I want to thank @JorgeB for his incredible help in identifying and isolating this issue.
  3. I have retest on 6.10.2, problem seems gone even IOMMU turn-on, I could copy large file without error. ( In 6.10.0 or 1, rsync will show error in verify phase, even rotation disk to NVMe also got error )
  4. Mobo was Asrock B365M-HDV, no broadcom NIC, I use Intel 82599ES 10G NIC. NVMe will corrupt in onboard M2 or PCIe slot.
  5. Just confirm this also affect NVMe, during change mobo to Intel B365 and add NVMe also have corrupt problem ( WD SN550 & Samsung EVO 970 same ), if use MC or CP to copy file, it may not prompt error, but rsync will easy prompt you the error during file copy. By apply intel_iommu=off, this solve this serious problem. PS: Confirm even enable IOMMU, if not use NVMe, system stable solid. I have apply RMA for the mobo, anyway it is a good news that I don't need swap mobo after this fix.
  6. For NIC issue, pls try disable VT-d ( not VT-x ) for tg3 NIC until fix on kernel. You can disable VT-d by command line and set it in BIOS.
  7. Only eth0 can manage Unraid, so if you try another network port, you need set it be eth0. If you disable the problematic port and try next once, that's fine too.
  8. Ref. below post, check firmware, does in HBA mode or not, or disconnect all disk first ......
  9. eth0 have up/down several times, eth1 never have link up ( suppose no cable insert ) May 21 20:41:40 Tower kernel: tg3 0000:05:00.0 eth0: Link is up at 1000 Mbps, full duplex May 22 01:02:39 Tower kernel: tg3 0000:05:00.0 eth0: Link is up at 1000 Mbps, full duplex May 22 01:35:52 Tower kernel: tg3 0000:05:00.0 eth0: Link is up at 1000 Mbps, full duplex May 22 03:10:30 Tower kernel: tg3 0000:05:00.0 eth0: Link is up at 1000 Mbps, full duplex May 22 03:11:10 Tower kernel: tg3 0000:05:00.0 eth0: Link is up at 1000 Mbps, full duplex May 22 11:52:30 Tower kernel: tg3 0000:05:00.0 eth0: Link is up at 1000 Mbps, full duplex Usually it is cable problem, if above not help, pls try disable bonding on eth0 & eth1, then next swap NIC-1,2 be eth-0,1 and check again.
  10. Stop array first, from terminal cd /tmp wget http://download.qnap.com/Storage/tsd/bios/QW37AR36.zip unzip QW37AR36.zip cd /tmp/QW37AR36 chmod +x flashrom ./flashrom -p internal -w QW37AR36.BIN reboot
  11. Update BIOS will fix the problem https://forum.qnap.com/viewtopic.php?t=133898&start=15#p707545
  12. I never try iSCSI or ZFS at Unraid, just notice some performance issue when access share through virtual network bridge, no matter array pool / RAID pool, passthrough a NIC have lot better performance.
  13. You said array/data always update, so I don't understand the purpose of copy parity disk. Unraid can't remember the change and update the change in later.
  14. Then look like storage controller/subsystem issue, no more trouble shoot suggest as those were onboard.
  15. If DD only got 1.3GB/s, it look like CPU issue, my result 5.2GB/s was under J1900, md5sum test also easy reach 200MB/s. Suggest recheck CPU running clock rate or wrong pinning CPU core.
  16. Or use parity rebuild schedule plugin, only rebuild several hours in night time until fully rebuild
  17. Any written during live rebuild is ok , new and old parity disk will update ( of course not replaced disk ), but array read/write performance also seriously impact, almost unuseable. Copy parity wont have benfit
  18. Yes, but need have spare NIC. May be somone have other solution.
  19. Or access by passthrogh NIC instead VirtIO/bridge.
  20. Start array in maintenance mode then rebuild parity, if any fault happen, you can put back the original disk. Even not rebuild in maintenance mode, array still protect by other parity disk. The first step, should be check parity to ensure no error and well sync.
  21. No reason only got 30MB/s, you need dig why local also such slow. Pls test this by dd if=/dev/zero of=/dev/null bs=1M count=10000 10000+0 records in 10000+0 records out 10485760000 bytes (10 GB, 9.8 GiB) copied, 2.02103 s, 5.2 GB/s Try found a large file and perform ' md5sum *file* ' and check the read speed. In general, Unraid not a hungry OS.