Intel Arc support

[kaares]

I have a A380 working just fine in unraid. All I had to do is install the thor kernel to /boot.  Basically just copy over four files. I run it without rebar as my mainboard doesn't support it.  Plex works great normally, but is a bit picky about subtitles. As long as I don't have to burn in subs during 4K AV1 transcoding i'm ok. 

 

Another cheap option is to get a Tesla P4 off ebay. They are like $70-80? Seemed like a no brainer. Just make sure to add a way to cool it. It doesn't have a fan. It NEEDS a fan. Seriously. It will get super hot and will damage either your card or your mainboard. I got a 3d printed shroud and a noctua fan for mine, and it works great. It isn't listed as supported by the nvidia plugin, but it was, and worked as soon as I installed that.

 

Just keep in mind that the thor kernel I used to get ARC working doesn't support nvidia, so you got to choose. I picked ARC. Having av1 support is nice.

[insomnia417]

On 11/24/2023 at 11:21 PM, unr41dus3r said:

Sorry, if i missed it in the thread, but has anybody information about the IDLE Power Draw of an A310/A380 or A750 ?

 

I am thinking to get an ARC but the most important thing would be the IDLE Power draw and maybe some power in reserve with the A750, but i dont know if there is any difference in the IDLE Power draw handling?

待机17-20W，转码峰值50W

[unr41dus3r]

6 hours ago, insomnia417 said:

待机17-20W，转码峰值50W

Yeah, i got it now and i also experience the 20W

It should be going lower with ASPM but i couldnt achieve it at the moment with Core Kernel.

ASPM is enabled in Bios.

[Harmani]

Hi all,

 

I picked up an Arc A770 during black Friday as an upgrade to my 1060. I was kind of hesitating because I couldn’t find definitive answers if the card would work for my use case or not, but I couldn’t resist the bang for buck 😊 Maybe I should not be as cheap in the future, because now I run into some deal-breaking-issues. Searching these forums, Reddit, etc gave me a lot of info, but it did not clarify everything unfortunately. So I’ll try my luck here:

 

First my relevant hardware and use case:

• ASRock Intel Arc A770 Challenger 16GB OC
• Gigabyte Aorus X570 Ultra
• AMD Ryzen 9 5900X
• Samsung 990 Pro 1TB
• Unraid Version: 6.12.5

 

The SSD is passed through directly as a (newly created) Windows 11 Pro VM. My goal is to use this VM for gaming. It worked flawlessly with my previous setup (Win10 & nvidia 1060). But now with the A770 I run into the following issues:

1. The VM boots up just fine with the A770 passed through (After setting Hyper-V to No). But when shutting down the VM, the Unraid host locks up and can only be reset the hard way... With a monitor connected to a second GPU, the last messages are something along the lines of:

   vfio-pci 000:0e:00.0: not ready 1234ms after FLR: waiting

   This happens ONLY when the sound card of the GPU is also passed through.

2. ReBAR is reported as disabled in Intel software and performance is sub-par. It is enable in system BIOS though (together with Above 4G something..)

Now googling these symptoms give a lot of hits. But not a definitive answer. I saw some tweaks etc in this forum, but not sure if they are still relevant to the current Unraid version. So my main question is:
Will the above issues be resolved when I use a custom kernel with Linux 6.2+ ?

Or am I better of returning the card and spend a few hundred bucks more if I want a seamless experience?

EDIT: is this also your usecase @SimonF ? And did you get it working?

Edited December 7, 2023 by Harmani

[SimonF]

1 hour ago, Harmani said:

Hi all,

 

I picked up an Arc A770 during black Friday as an upgrade to my 1060. I was kind of hesitating because I couldn’t find definitive answers if the card would work for my use case or not, but I couldn’t resist the bang for buck 😊 Maybe I should not be as cheap in the future, because now I run into some deal-breaking-issues. Searching these forums, Reddit, etc gave me a lot of info, but it did not clarify everything unfortunately. So I’ll try my luck here:

 

First my relevant hardware and use case:

• ASRock Intel Arc A770 Challenger 16GB OC
• Gigabyte Aorus X570 Ultra
• AMD Ryzen 9 5900X
• Samsung 990 Pro 1TB
• Unraid Version: 6.12.5

 

The SSD is passed through directly as a (newly created) Windows 11 Pro VM. My goal is to use this VM for gaming. It worked flawlessly with my previous setup (Win10 & nvidia 1060). But now with the A770 I run into the following issues:

1. The VM boots up just fine with the A770 passed through (After setting Hyper-V to No). But when shutting down the VM, the Unraid host locks up and can only be reset the hard way... With a monitor connected to a second GPU, the last messages are something along the lines of:

   vfio-pci 000:0e:00.0: not ready 1234ms after FLR: waiting

   This happens ONLY when the sound card of the GPU is also passed through.

2. ReBAR is reported as disabled in Intel software and performance is sub-par. It is enable in system BIOS though (together with Above 4G something..)

Now googling these symptoms give a lot of hits. But not a definitive answer. I saw some tweaks etc in this forum, but not sure if they are still relevant to the current Unraid version. So my main question is:
Will the above issues be resolved when I use a custom kernel with Linux 6.2+ ?

Or am I better of returning the card and spend a few hundred bucks more if I want a seamless experience?

EDIT: is this also your usecase @SimonF ? And did you get it working?

I still have issues with the Audio passthru. System dies when VM reboots or stops. 

 

Display is fine.

 

Later Kernel don't fix audio.

[Wvrent]

11 minutes ago, Harmani said:

Hi all,

 

I picked up an Arc A770 during black Friday as an upgrade to my 1060. I was kind of hesitating because I couldn’t find definitive answers if the card would work for my use case or not, but I couldn’t resist the bang for buck 😊 Maybe I should not be as cheap in the future, because now I run into some deal-breaking-issues. Searching these forums, Reddit, etc gave me a lot of info, but it did not clarify everything unfortunately. So I’ll try my luck here:

 

First my relevant hardware and use case:

• ASRock Intel Arc A770 Challenger 16GB OC
• Gigabyte Aorus X570 Ultra
• AMD Ryzen 9 5900X
• Samsung 990 Pro 1TB
• Unraid Version: 6.12.5

 

The SSD is passed through directly as a (newly created) Windows 11 Pro VM. My goal is to use this VM for gaming. It worked flawlessly with my previous setup (Win10 & nvidia 1060). But now with the A770 I run into the following issues:

1. The VM boots up just fine with the A770 passed through (After setting Hyper-V to No). But when shutting down the VM, the Unraid host locks up and can only be reset the hard way... With a monitor connected to a second GPU, the last messages are something along the lines of:

   vfio-pci 000:0e:00.0: not ready 1234ms after FLR: waiting

   This happens ONLY when the sound card of the GPU is also passed through.

2. ReBAR is reported as disabled in Intel software and performance is sub-par. It is enable in system BIOS though (together with Above 4G something..)

Now googling these symptoms give a lot of hits. But not a definitive answer. I saw some tweaks etc in this forum, but not sure if they are still relevant to the current Unraid version. So my main question is:
Will the above issues be resolved when I use a custom kernel with Linux 6.2+ ?

Or am I better of returning the card and spend a few hundred bucks more if I want a seamless experience?

Not much consolation, but I'm having pretty much the same experience running windows 11

My hardware:

• Intel Arc A750 Limited Edition
• Asus Strix x570-e gaming
• AMD Ryzen 9 3900X
• Kingston NV2 2TB (passthru)
• Unraid Version: 6.12

I am booting unRaid with CSM in the bios. Because the NVMe is passthru, I've booted up the system directly to Windows (switching to UEFI) to make sure everything works fine. GPU-Z recognizes the card, sees that ReBAR is set, all looks good.

I've also tried virtually mapping the soundcard onto the same virtual slot with no luck.  I think there may also be an Intel Bluetooth device that comes with the card so I've begun experimenting with passing that as well (Going back to @SpaceInvaderOne tutorial talking about Nvidia splitting soundcards causing problems)

Current test I'm running is a VM with Ubuntu 23.04. Hardinfo see an Intel i915 card but testing with something like blender not seeing the Xe-HPG or intel-compute-runtime

Dave

[Harmani]

53 minutes ago, SimonF said:

I still have issues with the Audio passthru. System dies when VM reboots or stops. 

 

Display is fine.

 

Later Kernel don't fix audio.

 

Ok thanks a lot for clarifying that! Most discussions I encountered where from like a year ago, so I hoped things got better by now.

 

I could opt for keeping the VM powered on all the time, but the GPU's 40W idle makes that a big no no 😅

[Harmani]

35 minutes ago, Wvrent said:

Not much consolation, but I'm having pretty much the same experience running windows 11

My hardware:

• Intel Arc A750 Limited Edition
• Asus Strix x570-e gaming
• AMD Ryzen 9 3900X
• Kingston NV2 2TB (passthru)
• Unraid Version: 6.12

I am booting unRaid with CSM in the bios. Because the NVMe is passthru, I've booted up the system directly to Windows (switching to UEFI) to make sure everything works fine. GPU-Z recognizes the card, sees that ReBAR is set, all looks good.

I've also tried virtually mapping the soundcard onto the same virtual slot with no luck.  I think there may also be an Intel Bluetooth device that comes with the card so I've begun experimenting with passing that as well (Going back to @SpaceInvaderOne tutorial talking about Nvidia splitting soundcards causing problems)

Current test I'm running is a VM with Ubuntu 23.04. Hardinfo see an Intel i915 card but testing with something like blender not seeing the Xe-HPG or intel-compute-runtime

Dave

 

Yeah seems we're on the same boat then. I've also tried booting "bare metal" and all is fine! The GPU does a pretty decent job in the games I play. Could have been the best €290 ever spend, if only I could get it to work in my Gaming VM...

 

What about that bluetooth? Is that maybe for RGB LED's or something? Never heard of a GPU with integrated BT 😂

Edited December 7, 2023 by Harmani

[Michael_P]

20 minutes ago, Harmani said:

I could opt for keeping the VM powered on all the time, but the GPU's 40W idle makes that a big no no

 

My A770 LE idles around 10W, have you tinkered with power management?

 

 

[Harmani]

1 hour ago, Michael_P said:

 

My A770 LE idles around 10W, have you tinkered with power management?

 

 

ok good to know that it's possible to get it lower! But unfortunately it's the least of my worries now Can't have the host crash on every reboot.

[don_diaz]

On 12/7/2023 at 11:26 AM, Harmani said:

I could opt for keeping the VM powered on all the time, but the GPU's 40W idle makes that a big no no 😅

 

This is what I do. Windows updates paused indefinitely. When I need to restart the VM, I pull the plug on the box and start back up. Feels rough. Looking at my options.

[DragonZeal]

I'm running latest unRAID on a Asus Rog Strix Z490-G Gaming WiFi with a i9-10900F. I'm using an Intel Arc A380.

Does this already work with Jellyfin Docker or do I still need to pass-through to a VM?

[TheIlluminate]

Just now, DragonZeal said:

I'm running latest unRAID on a Asus Rog Strix Z490-G Gaming WiFi with a i9-10900F. I'm using an Intel Arc A380.

Does this already work with Jellyfin Docker or do I still need to pass-through to a VM?

Unless you are running and can confirm unraid 6.13 has linux kernel 6.2 or higher OR using Thors little kernel hack to run linux 6.2 or higher on unraid 6.12 then no it doesn't work natively.  You would still need to pass it on through a VM. 

[DragonZeal]

Just now, TheIlluminate said:

Unless you are running and can confirm unraid 6.13 has linux kernel 6.2 or higher OR using Thors little kernel hack to run linux 6.2 or higher on unraid 6.12 then no it doesn't work natively.  You would still need to pass it on through a VM. 

I'm running latest stable unRAID so I don't think so. Would you recommend 6.13 or Thor kernel or passing through to a VM? Also Windows VM or Linux VM?

[TheIlluminate]

2 minutes ago, DragonZeal said:

I'm running latest stable unRAID so I don't think so. Would you recommend 6.13 or Thor kernel or passing through to a VM? Also Windows VM or Linux VM?

Personally I'm just waiting for 6.13.  If you need it now then yeah id run it through a vm.  I don't think it matters what VM but someone else could answer that better then me.  I'd do what  OS you are more comfortable with.  Windows has ARC drivers and Linux has them built in now.  So what ever is easiest for you. 

[DragonZeal]

7 minutes ago, TheIlluminate said:

Personally I'm just waiting for 6.13.  If you need it now then yeah id run it through a vm.  I don't think it matters what VM but someone else could answer that better then me.  I'd do what  OS you are more comfortable with.  Windows has ARC drivers and Linux has them built in now.  So what ever is easiest for you. 

I would rather wait for 6.13 also instead of tampering with kernels and so.

Keeping my seperate QSV box for now.

[Wvrent]

On 12/31/2023 at 1:09 PM, TheIlluminate said:

Unless you are running and can confirm unraid 6.13 has linux kernel 6.2 or higher OR using Thors little kernel hack to run linux 6.2 or higher on unraid 6.12 then no it doesn't work natively.  You would still need to pass it on through a VM. 

If you're like me, you might not even have much luck with passthru to a VM.  I seem to get the card to recognize on first (hardware) boot, but if I need to boot the VM for any reason the card isn't recognized.

[domrockt]

so i have my 2nd Unraid server nearly finished and updated the Unraid Kernel to 6.6 and now i can use my arc770 in dockers 👍 i went one step futher and use it at the moment as my Stable-Diffusion GPU inside that docker from the Hub nuullll/ipex-arc-sd:latest and it runs just fine 🤩

 

 

 

 

 

 

 

So after a few hours of testing and creating images i got this errors and the systems crashes and reboots in an instant

 

Jan 4 00:38:34 nvme kernel: i915 0000:47:00.0: Direct firmware load for i915/dg2_dmc_ver2_08.bin failed with error -2  ----------------------------------Done

 

Jan 4 00:38:34 nvme kernel: i915 0000:47:00.0: [drm] Failed to load DMC firmware i915/dg2_dmc_ver2_08.bin. Disabling runtime power management. ----------------------Done

 

Jan 4 00:38:34 nvme kernel: i915 0000:47:00.0: [drm] *ERROR* GT0: HuC firmware i915/dg2_huc_gsc.bin: fetch failed -ENOENT --------------------------------Done

 

 

*edit* so i recompiled the Kernel added the missing driver *edit* 

 

now i have no errors and test the docker again 🙃

 

 

fasten your Seatbelts and hold on on something.... Risercable....  i used an older one not working correct. I use another one right now with PCIE 3.0 and ordered a new one.... Now testing again.

Edited January 5 by domrockt

[martinkeat]

Those wanting intel arc support remember to vote: 

 

[DragonZeal]

3 minutes ago, martinkeat said:

Those wanting intel arc support remember to vote: 

 

Done

[J05u]

Hello, is it any way to see Arc 380 power draw? With nvtop I can see that GPU is there, without any stats

[Kuleinc]

On 12/2/2023 at 7:26 AM, kaares said:

I have a A380 working just fine in unraid. All I had to do is install the thor kernel to /boot.  Basically just copy over four files. I run it without rebar as my mainboard doesn't support it.  Plex works great normally, but is a bit picky about subtitles. As long as I don't have to burn in subs during 4K AV1 transcoding i'm ok. 

 

Another cheap option is to get a Tesla P4 off ebay. They are like $70-80? Seemed like a no brainer. Just make sure to add a way to cool it. It doesn't have a fan. It NEEDS a fan. Seriously. It will get super hot and will damage either your card or your mainboard. I got a 3d printed shroud and a noctua fan for mine, and it works great. It isn't listed as supported by the nvidia plugin, but it was, and worked as soon as I installed that.

 

Just keep in mind that the thor kernel I used to get ARC working doesn't support nvidia, so you got to choose. I picked ARC. Having av1 support is nice.

Interested in trying this. I have a super micro 846 4 U chasis with 5 chasis fans. would I still need a fan on the card itself?

[g.lyh]

On 12/7/2023 at 11:54 PM, Harmani said:

Hi all,

 

I picked up an Arc A770 during black Friday as an upgrade to my 1060. I was kind of hesitating because I couldn’t find definitive answers if the card would work for my use case or not, but I couldn’t resist the bang for buck 😊 Maybe I should not be as cheap in the future, because now I run into some deal-breaking-issues. Searching these forums, Reddit, etc gave me a lot of info, but it did not clarify everything unfortunately. So I’ll try my luck here:

 

First my relevant hardware and use case:

• ASRock Intel Arc A770 Challenger 16GB OC
• Gigabyte Aorus X570 Ultra
• AMD Ryzen 9 5900X
• Samsung 990 Pro 1TB
• Unraid Version: 6.12.5

 

The SSD is passed through directly as a (newly created) Windows 11 Pro VM. My goal is to use this VM for gaming. It worked flawlessly with my previous setup (Win10 & nvidia 1060). But now with the A770 I run into the following issues:

1. The VM boots up just fine with the A770 passed through (After setting Hyper-V to No). But when shutting down the VM, the Unraid host locks up and can only be reset the hard way... With a monitor connected to a second GPU, the last messages are something along the lines of:

   vfio-pci 000:0e:00.0: not ready 1234ms after FLR: waiting

   This happens ONLY when the sound card of the GPU is also passed through.

2. ReBAR is reported as disabled in Intel software and performance is sub-par. It is enable in system BIOS though (together with Above 4G something..)

Now googling these symptoms give a lot of hits. But not a definitive answer. I saw some tweaks etc in this forum, but not sure if they are still relevant to the current Unraid version. So my main question is:
Will the above issues be resolved when I use a custom kernel with Linux 6.2+ ?

Or am I better of returning the card and spend a few hundred bucks more if I want a seamless experience?

EDIT: is this also your usecase @SimonF ? And did you get it working?

The problems with ARC GPU passthrough also bothered me for quite a while (same "not ready XXms after FLR: waiting" issue).

 

My issues:

It is fine to simultaneously passthrough the entire card (video and audio) to a VM, no matter whether ReBAR is enabled or not.

But on shutdown of the VM, "not ready XXms after FLR: waiting" is reported in syslog. After waiting for some time, "device inaccessible" is reported and I lost the card. I must hard reboot the machine to get it back.

 

My server setup:

• MSI MAG B550M MORTAR MAX WIFI
• Intel ARC A380 index from GUNNIR
• AMD 5600G CPU
• Unraid Version: 6.12.4

 

ReBAR is enabled in BIOS. It can also be confirmed in a Win10 VM with the official Intel Arc Control software.

 

It is solved by the following scripts (so I run them with the User Scripts plugin and set it to run "At Startup of Array"):

#!/bin/bash
echo > /sys/bus/pci/devices/0000:12:00.0/reset_method
echo > /sys/bus/pci/devices/0000:13:00.0/reset_method

Note that these scripts must be executed before the shutdown of VMs. Modify the physical address to your own case.

 

I just need to run these scripts one time on each startup of the server, and I can flawlessly shutdown the VMs with perfect performance of the Intel ARC A380 GPU. No error lines are reported on logs.

 

Not sure if this is what you need. It seems that with Linux Kernel 6.2+, compatibility issues are resolved, but I would just wait for Unraid offical releases because stability is much more important to me.

[J05u]

1 hour ago, g.lyh said:

The problems with ARC GPU passthrough also bothered me for quite a while (same "not ready XXms after FLR: waiting" issue).

 

My issues:

It is fine to simultaneously passthrough the entire card (video and audio) to a VM, no matter whether ReBAR is enabled or not.

But on shutdown of the VM, "not ready XXms after FLR: waiting" is reported in syslog. After waiting for some time, "device inaccessible" is reported and I lost the card. I must hard reboot the machine to get it back.

 

My server setup:

• MSI MAG B550M MORTAR MAX WIFI
• Intel ARC A380 index from GUNNIR
• AMD 5600G CPU
• Unraid Version: 6.12.4

 

ReBAR is enabled in BIOS. It can also be confirmed in a Win10 VM with the official Intel Arc Control software.

 

It is solved by the following scripts (so I run them with the User Scripts plugin and set it to run "At Startup of Array"):

#!/bin/bash
echo > /sys/bus/pci/devices/0000:12:00.0/reset_method
echo > /sys/bus/pci/devices/0000:13:00.0/reset_method

Note that these scripts must be executed before the shutdown of VMs. Modify the physical address to your own case.

 

I just need to run these scripts one time on each startup of the server, and I can flawlessly shutdown the VMs with perfect performance of the Intel ARC A380 GPU. No error lines are reported on logs.

 

Not sure if this is what you need. It seems that with Linux Kernel 6.2+, compatibility issues are resolved, but I would just wait for Unraid offical releases because stability is much more important to me.

Are you using custom kernel? Still didn't try to pass through my 380 in my Frankenstein build

[g.lyh]

9 hours ago, J05u said:

Are you using custom kernel? Still didn't try to pass through my 380 in my Frankenstein build

I didn't use any custom kernel, just the default one with the Unraid OS release.

 

Just hapenned to find a possible solution on the Proxmox forum, and it worked!

Check this out:

https://forum.proxmox.com/threads/issues-with-intel-arc-a770m-gpu-passthrough-on-nuc12snki72-vfio-pci-not-ready-after-flr-or-bus-reset.130667/#post-573521