unRAID hardware upgrade - is there a checklist? (also BIOS to UEFI in 2023)

[Derek_]

Hiya,

 

I'm about to update my hardware, from an AsRock Extreme/Intel i7-3770 16gb RAM, to an AsRock Z690 Pro RS/Intel i5-12500, 32gb RAM.

 

I'm running unRAID 6.11.5

 

The old mobo had flaky UEFI, so i stuck with BIOS at the time. The new mobo almost forces UEFI - i need to install a GFX card to enable CSM, and i've read that doing so disables the iGPU! I don't presently want a dedicated video card, but i may one day. So i guess i'll go with UEFI - it's probably time anyway.

 

• I understand i need to change a directory name on the USB from `EFI-` to `EFI`.
• My unRAID has a couple of VMs (Linux, Windows), but none have hardware pass-through. My research indicates i don't need to change anything.

 

It seems much has changed in the years since i last built a PC, and i have some questions:

 

1. Any negatives to using UEFI in 2023? My research indicates that hardware pass-through in years past was easier/more reliable in BIOS. Has the situation improved for UEFI?
2. TPM in the new motherboard UEFI settings. What do i do with that? My old motherboard doesn't even have TPM.
3. Moving the array. Any gotchas i haven't covered below? (XFS, encrypted)
4. Moving the cache drives. Any gotchas i haven't covered below? (mine are two SATA SSDs encrypted BTRFS mirrored)
5. Is "don't use USB3 for the OS stick" still a thing?
6. "XHCI Handoff" on the motherboard BIOS/UEFI settings - enable or disable? I don't understand the blurb on the screen:
    

   This is a workaround for OSes without XHCI hand-off support. The XHCI ownership change should be claimed by XHCI driver.

   
   It doesn't say whether it's enabled or disabled that applies the workaround, nor do i know if unRAID has the hand-off support. 🤯

 

 

Anything else i should be mindful of?

 

I've also found these tips while researching: https://www.reddit.com/r/unRAID/comments/sak4pb/moving_unraid_to_new_hardware/

 

Summarised:

 

• Take a screenshot of your drives and hardware config like networking. Take note of drives. Hard drives and SATA port order doesn't matter so the Main page is more of a just in case drives don't show up correctly or the new board doesn't detect a drive (I forgot to plug a power cable into one).
• Disable auto-start Array.
• Do a parity check before and after to make sure the data integrity was good through this process.
• If 2 network cards (e.g. 1 motherboard, 1 PCIe), make sure they're mapped to the correct interface e.g eth0, eth1.
• Set static IP address for server: Settings -> Network Settings.
• Set the correct UEFI or BIOS Settings which you used before or plan to use. As in Virtualisation or IOMMU Groups.

 

More tips thanks to the amazing SpaceInvaderOne:

 

 

The main takeaways i got from his video was:

 

1. Stop the VMs and Dockers, and turn off auto-start.
2. If any hardware pass-through on the VMs, change:
   a. Graphics to VNC
   b. untick any `USB Devices` and `Other PCIe Devices`
   c. Tools > System Devices > Untick any devices ticked for pass-through.
3. On new PC, reverse the disabling changes done in 2a,b,c.

 

This shouldn't apply to me as i don't presently have any pass-through set up (i guess?)

 

I also found this scary issue - serial numbers on HDDs changing after hardware change - it's from 2013 so hopefully it's not something that happens anymore: https://forums.unraid.net/topic/23982-solved-moved-drives-to-a-new-chassis-and-hd-serial-numbers-changed/

 

And, after typing all of the above, i found this post which has more tips

 

https://forums.unraid.net/topic/120929-solved-hardware-migration/?do=findComment&comment=1105283

 

Have i missed anything?

Edited May 27, 2023 by Derek_
XHCI question clarification

[Frank1940]

11 hours ago, Derek_ said:

I also found this scary issue - serial numbers on HDDs changing after hardware change - it's from 2013 so hopefully it's not something that happens anymore:

 

Yes, it can still happen today.  Just make sure you have the array set to not autostart.  The problem (today) usually happens when one goes from using a RAID SATA card (running in the JBOD mode) to IT-Mode SATA card.  All of the cases that I have seen, it is very apparent which disk goes where as the actual serial number of the disk is contained within the JBOD one.  All you have to do is scan the JBOD serial number to find it. 

 

You appear to have covered most of the bases.  Only thing that I see is that you want to make a backup of your flash drive before you start making any changes to the present setup to not autostart things.  That way, you can restore to a running system with the old hardware in few minutes.

 

If you have missed something, you should not have a problems getting things squared around.  (While I have a very simple setup, I have done hardware upgrades to both of my servers.  The problems I had were caused by actual hardware infant mortality (DOA) issues in the upgraded hardware.)    Just schedule the upgrade for a day when you have four to five hours free.  Warn everyone that the server may not be available that entire day and tell them to make other arrangements.  IT may only take a couple of hours but it something should happen, you have some breathing room to straighten things out. 

 

PS---- from your description, if you have an issue with the new hardware, you will be able to just swap the old hardware back in.

 

Edited May 26, 2023 by Frank1940

[Derek_]

Yep, the old hardware still works. It's merely an upgrade

 

I only have 3 drives in the array, all onboard SATA, plus 2 cache drives also on the motherboard's SATA. Plus one permanent Unassigned Device, also on the mobo SATA port. The new motherboard has 8 SATA ports (or 7 if i use all the m.2 sockets). Although it seems likely to be unnecessary, i'll put them in the same port number in the new build to further reduce the likelihood of issues.

 

I hope someone knows answers to my no.2 and no.6 questions. I'd rather have the right BIOS/UEFI settings before i even first boot on the new hardware.

[Frank1940]

On 5/25/2023 at 11:30 PM, Derek_ said:

TPM in the new motherboard UEFI settings. What do i do with that? My old motherboard doesn't even have TPM.

 

TPM is a requirement for Windows 11.  (Google is your friend here.)  I have never seen any discussion about the setting of it for Unraid so I assume it is unnecessary.  (I know that Unraid does not require it at this point.)  I suspect it does not make any difference if it off or on...

 

On 5/25/2023 at 11:30 PM, Derek_ said:

"XHCI Handoff" on the motherboard BIOS/UEFI settings - enable or disable? I don't understand the blurb on the screen:

Again, I googled this setting and found this article:

   

 

On 5/25/2023 at 11:30 PM, Derek_ said:

Any negatives to using UEFI in 2023? My research indicates that hardware pass-through in years past was easier/more reliable in BIOS. Has the situation improved for UEFI?

Probably most people are using UEFI today.  Windows requires it, so it is the default for most boards.  I use UEFI for both of my servers-- and the MB's are several years old now --and never had an issue.   I have not seen posts about issues with using it for several years now.   

[Derek_]

15 hours ago, Frank1940 said:

I googled this setting and found this article:

   

 

Oh i spent quite a while searching for the meaning of XHCI hand-off. I found many explanations, many contradictory. What i THINK it means is that it probably doesn't matter much except for old OSes (pre-Win7 - i wouldn't know which Linux kernel) which do not have driver support for USB3. For such systems, it should be disabled so the ports will operate as USB2 ports. For current OSes, I *think* having it enabled (i.e. handing off to the OS once the driver loads) is probably preferable to using the BIOS controller driver, as the software drivers are constantly being updated whereas bios firmware rarely, for things like usb3 controller.

 

Thanks for the post about XHCI for unRAID - it relates to booting the OS via USB3 port which i won't do anyway. I was curious as to whether the USB3 thing was still an issue, that post/thread is from 2018.

 

I think i've done enough research. I'm doing backups, parity check and whatnot (per initial post) and hopefully tomorrow i'll do the transition.

 

Thanks everyone (happy to hear any additional words of advice).

[Frank1940]

BIOS are minefields.  It seems like no two are identical and, for consistent nomenclature, forget it.   I have haven't even talked about different MB manufacturers.  Most of the time, the BIOS defaults work fine.  One should always start from there and make only changes as required.

 

One of the prime examples is the occasional problem of Unraid not being able to properly boot from USB3 ports.  The problem seems to only happen with certain motherboards.  No one has ever figure out the reason and I don't believe anyone has ever found a BIOS solution to make USB3 ports work properly.  That is why one of the first recommendations for people with boot issues is to put the boot drive in a USB2 port. 

 

If you have stopped the Docker and VM services as well as the autostart of the array, your new hardware will (probably) boot right up to a working Unraid OS.  (You should probably rename the EFI folder to permit UEFI as most modern MB's will default to it.  Unraid really does not care which method controls the bootup!)   Check disk assignments.  If correct, start the array.    (Actually, I would be worried more about a hardware problem with the new hardware than Unraid not coming up working...) 

[Derek_]

Well i'm happy to report that the transition seems to have gone well. Touch wood.

 

With all the research and notes, i was very careful and tested it on the workbench (in the garage, i don't like working on carpet). Moved it into the study and somehow during the process, one of the cache SSD SATA cables detached. Luckily i was double-checked 'Main' before starting her up, so noticed and fixed it. Put things back the way they were (Dockers, VMs - no passthrough) and it looks ok.

 

Thanks to everyone who offered advice. I'll jot down my step-by-step in the next few days for reference.

Edited May 29, 2023 by Derek_

[Frank1940]

3 hours ago, Derek_ said:

Moved it into the study and somehow during the process, one of the cache SSD SATA cables detached.

There was one item (which I should have remembered) that I forgot.  After working inside the case around the SATA connectors and cables, always double and triple check that the cables are all firmly seated on their connectors.  (I have said for years that the SATA connector design is the poster child for how not to design a connector system!)

[Derek_]

28 minutes ago, Frank1940 said:

(I have said for years that the SATA connector design is the poster child for how not to design a connector system!)

 

Worse than Micro-USB? 🤦‍♂️ 😄

[Frank1940]

40 minutes ago, Derek_ said:

 

Worse than Micro-USB? 🤦‍♂️ 😄

For causing problems in Unraid servers, Absolutely!   It seems that just disturbing a SATA data cable while working inside the case will cause the connector to unseat itself. (There is very small plastic nib in the cable end connector which serves a 'friction' lock.)  Then later, this unseating will cause the connection to be unstable.   The vibration from the drive rotating is enough to cause this situation to occur (or, in your case, moving it back into the house).

[Derek_]

Taking account of the various posts (including mine), this is what i did:

 

Old Server Hardware:

1. Run Parity check
2. Turn off Array auto-start
3. Stop the VMs and Dockers, and turn off auto-start.

4. If any hardware pass-through on the VMs, change:
   a. Graphics to VNC
   b. untick any `USB Devices` and `Other PCIe Devices`
   c. Tools > System Devices > Untick any devices ticked for pass-through.

   (On new PC, reverse the disabling changes done in 2a,b,c.)

5. Take screenshots/note of:
   1. Disk order (including cache disks) - note SATA ports with single parity drive doesn't matter, but the order shown in MAIN does matter (apparently the order of SATA ports used does matter if you have 2 or more parity drives - i don't have this).
   2. Network settings - also set static ip address if not already, and if multiple NICs note the correct one is selected on the new system.
6. Backup Data and USB stick
7. Shutdown
8. Rename `-EFI` directory on USB stick to `EFI` on PC.

 

New Server Hardware:

1. Memtest (this can also be done with the unRAID usb stick, but i did it before i transferred the HDDs).
2. Review BIOS/UEFI settings:
   a. Virtualisation is on, if there's IOMMU configurations, take note and adjust as appropriate (i don't have these in my new server).
   b. HDD set to AHCI (before installing HDDs)
   c. HDDs are detected (after installing HDDs), double-check AHCI setting.
   d. Boot from USB as first/only boot device.
   e. Check XHCI is enabled, per this post.
   f. Left TPM as enabled.
   g. Cleared SecureBoot keys, turned off SecureBoot.
3. Boot unRAID.
4. Check network settings (especially if connected to a monitor/keyboard)
5. Check disks are all detected and in the correct order (including cache).
6. Start array.
7. Start Docker service - test each docker and turn on auto-start as desired.
8. Start VM service. Check pass-through settings - test each VM and turn on auto-start as desired.
9. Check access to shares/files via usual access means.
10. Set array to auto-start if desired.
11. Reboot to make sure everything comes back as expected.
12. Parity check.
13. Backup USB stick

I think that's all i did. I did write notes, but as is often the case, they're a bit of a mess

 

Edited June 10, 2023 by Derek_

[Lolight]

On 5/27/2023 at 6:20 AM, Derek_ said:

The new motherboard has 8 SATA ports (or 7 if i use all the m.2 sockets).

A SATA port will be disabled only when using a SATA-type M.2 device in the M2_2 socket.

No SATA ports will be disabled if using a NVMe M.2 drive in that socket.

[Derek_]

1 minute ago, Lolight said:

A SATA port will be disabled only when using a SATA-type M.2 device in the M2_2 socket.

No SATA ports will be disabled if using a NVMe M.2 drive in that socket.

 

You're right, i noticed later that it only applies to the Ultra socket: https://www.asrock.com/mb/Intel/Z690 Pro RS/index.asp#Specification

 

Quote

If M2_2 is occupied by a SATA-type M.2 device, SATA3_7 will be disabled.

 

Though I don't know the difference between SATA and non-SATA device. How to tell?

 

As an aside, i HATE the way these names keep coming about. Like USB's "full speed" and "high speed". That is literally impossible to know which is faster without reference. Likewise "hyper" and "ultra" for nvme. It's so stupid! I had to go hunting. To my surprise "hyper" appears capable of faster speeds than "ultra" :eyeroll:

 

Quote

2 Hyper M.2 (PCIe Gen4 x4)
1 Ultra M.2 (PCIe Gen3 x4 & SATA3)

 

[Lolight]

7 minutes ago, Derek_ said:

Though I don't know the difference between SATA and non-SATA device. How to tell?

by looking at the M.2 drive's description when shopping.

The listing must mention the type - either SATA  or NVMe (PCie).