Intel Arc support

[dboris]

Some guys on the Proxmox forum were able to fix the A770M issue I encounter.
Any idea how to translate it on Unraid ?

 

Thanks,

[SimonF]

7 hours ago, dboris said:

Some guys on the Proxmox forum were able to fix the A770M issue I encounter.
Any idea how to translate it on Unraid ?

 

Thanks,

There are hooks in /etc/libvirt/hooks I would need to work out which status the proxmox one relates to. Suspect will be prepare

 

https://libvirt.org/hooks.html#etc-libvirt-hooks-qemu

 

        # Clear the reset methods before each start of the VM, to prevent the PVE host locking up
        # flr and bus methods dont work and re-appear after reboots.
        # bus method may still be attempted in subseqent VM starts, even if reset_method already cleared.
        echo "VM=$vmId - $runPhase : Clearing Intel dGPU and audio device reset_methods."
        echo > /sys/bus/pci/devices/0000:03:00.0/reset_method
        echo > /sys/bus/pci/devices/0000:04:00.0/reset_method

        # will appear as the following in journalctl:
        #  kernel: vfio-pci 0000:03:00.0: All device reset methods disabled by user
        #  kernel: vfio-pci 0000:04:00.0: All device reset methods disabled by user

 

[dboris]

As far as I understand, the ARC gpu won't crash if I clearing (both devices) by making sure that this file is empty using :

nano /sys/bus/pci/devices/0000:03:00.0/reset_method

same for the audio device : 0000:04:00.0

 

So, the first thing I will do is make sure this manual fix works, restart the VM, clear, etc.

I'll report my findings here tonight !

 

On the other hand, I don't understand yet where I should put the script / the uuid part.

I'll check the content of /etc/libvirt/hooks to get some clues tonight too.
 

Don't hesitate if you have any ideas of the directions to take, even if it involves testing. My arrays cries in bad blocks already. I already spent countless (worthless?) hours trying to get that NUC's GPU working as it's a perfect candidate for my use case. Wish I had found that post last month.

 

Edited August 1, 2023 by dboris

[SimonF]

4 hours ago, dboris said:

As far as I understand, the ARC gpu won't crash if I clearing (both devices) by making sure that this file is empty using :

nano /sys/bus/pci/devices/0000:03:00.0/reset_method

same for the audio device : 0000:04:00.0

 

So, the first thing I will do is make sure this manual fix works, restart the VM, clear, etc.

I'll report my findings here tonight !

 

On the other hand, I don't understand yet where I should put the script / the uuid part.

I'll check the content of /etc/libvirt/hooks to get some clues tonight too.
 

Don't hesitate if you have any ideas of the directions to take, even if it involves testing. My arrays cries in bad blocks already. I already spent countless (worthless?) hours trying to get that NUC's GPU working as it's a perfect candidate for my use case. Wish I had found that post last month.

 

Create file in /etc/libvirt/hooks/qemu.d/ if dir does not exist create. mkdir /etc/libvirt/hooks/qemu.d/

nano /etc/libvirt/hooks/qemu.d/"Window11_2-Arc_reset" with the lines below

and chmod +x /etc/libvirt/hooks/qemu.d/"Window11_2-Arc_reset" You can removing the logging if required.

 

Example hooks file. You can do something in bash if you want.

#!/usr/bin/env php

<?php


function log_to_log($m, $type = "NOTICE") {
        if ($type == "DEBUG" ) return NULL;
        $m              = print_r($m,true);
        $m              = str_replace("\n", " ", $m);
        $m              = str_replace('"', "'", $m);
        $cmd    = "/usr/bin/logger ".'"'.$m.'"' ;
        exec($cmd);
}

#Reset 
#      echo > /sys/bus/pci/devices/0000:03:00.0/reset_method
#      echo > /sys/bus/pci/devices/0000:04:00.0/reset_method
#
#		Note if this fails then the VM will not start.
#
$vmname = "Windows 11_2" ;
log_to_log("Hooks Arc Clear" .$argv[2]." VM:".$argv[1]) ;

if ($argv[2] == 'prepare' && $argv[1] == "$vmname"){
	log_to_log( "Clear /sys/bus/pci/devices/0000:03:00.0/reset_method") ;
	file_put_contents("/sys/bus/pci/devices/0000:03:00.0/reset_method", " ") ;
	log_to_log( "Clear /sys/bus/pci/devices/0000:04:00.0/reset_method") ;
	file_put_contents("/sys/bus/pci/devices/0000:04:00.0/reset_method", " ") ;
}
?>

 

Aug  1  computenode root: Hooks Arc Clear prepare VM:Windows 11_2
Aug  1  computenode root: Clear /sys/bus/pci/devices/0000:03:00.0/reset_method
Aug  1  computenode root: Clear /sys/bus/pci/devices/0000:04:00.0/reset_method

[dboris]

Sooo it worked   !

 

On the other hand I still randomly get the same error on : 

[8086:460d] 00:01.0 PCI bridge: Intel Corporation 12th Gen Core Processor PCI Express x16 Controller #1 (rev 02)

I modified the script you gave me accordingly but I still get the bug.
The UI of the server is unresponsive, but I was able to go check in the related reset_method file : it's empty.
However, after crashes, I noticed it's usually not empty ("bus").

 

GNU nano 7.2                                                         /etc/libvirt/hooks/qemu.d/Window11-Arc_reset                                                                    
#!/usr/bin/env php

<?php


function log_to_log($m, $type = "NOTICE") {
        if ($type == "DEBUG" ) return NULL;
        $m              = print_r($m,true);
        $m              = str_replace("\n", " ", $m);
        $m              = str_replace('"', "'", $m);
        $cmd    = "/usr/bin/logger ".'"'.$m.'"' ;
        exec($cmd);
}

#Reset 
#      echo > /sys/bus/pci/devices/0000:03:00.0/reset_method
#      echo > /sys/bus/pci/devices/0000:04:00.0/reset_method
#
#               Note if this fails then the VM will not start.
#
$vmname = "Windows 11" ;
log_to_log("Hooks Arc Clear" .$argv[2]." VM:".$argv[1]) ;

if ($argv[2] == 'prepare' && $argv[1] == "$vmname"){
        log_to_log( "Clear /sys/bus/pci/devices/0000:03:00.0/reset_method") ;
        file_put_contents("/sys/bus/pci/devices/0000:03:00.0/reset_method", " ") ;
        log_to_log( "Clear /sys/bus/pci/devices/0000:04:00.0/reset_method") ;
        file_put_contents("/sys/bus/pci/devices/0000:04:00.0/reset_method", " ") ;
        log_to_log( "Clear /sys/bus/pci/devices/0000:01:00.0/reset_method") ;
        file_put_contents("/sys/bus/pci/devices/0000:01:00.0/reset_method", " ") ;
}
?>

 

 

 

[SimonF]

8 hours ago, dboris said:

Sooo it worked   !

 

On the other hand I still randomly get the same error on : 

[8086:460d] 00:01.0 PCI bridge: Intel Corporation 12th Gen Core Processor PCI Express x16 Controller #1 (rev 02)

I modified the script you gave me accordingly but I still get the bug.
The UI of the server is unresponsive, but I was able to go check in the related reset_method file : it's empty.
However, after crashes, I noticed it's usually not empty ("bus").

 

GNU nano 7.2                                                         /etc/libvirt/hooks/qemu.d/Window11-Arc_reset                                                                    
#!/usr/bin/env php

<?php


function log_to_log($m, $type = "NOTICE") {
        if ($type == "DEBUG" ) return NULL;
        $m              = print_r($m,true);
        $m              = str_replace("\n", " ", $m);
        $m              = str_replace('"', "'", $m);
        $cmd    = "/usr/bin/logger ".'"'.$m.'"' ;
        exec($cmd);
}

#Reset 
#      echo > /sys/bus/pci/devices/0000:03:00.0/reset_method
#      echo > /sys/bus/pci/devices/0000:04:00.0/reset_method
#
#               Note if this fails then the VM will not start.
#
$vmname = "Windows 11" ;
log_to_log("Hooks Arc Clear" .$argv[2]." VM:".$argv[1]) ;

if ($argv[2] == 'prepare' && $argv[1] == "$vmname"){
        log_to_log( "Clear /sys/bus/pci/devices/0000:03:00.0/reset_method") ;
        file_put_contents("/sys/bus/pci/devices/0000:03:00.0/reset_method", " ") ;
        log_to_log( "Clear /sys/bus/pci/devices/0000:04:00.0/reset_method") ;
        file_put_contents("/sys/bus/pci/devices/0000:04:00.0/reset_method", " ") ;
        log_to_log( "Clear /sys/bus/pci/devices/0000:01:00.0/reset_method") ;
        file_put_contents("/sys/bus/pci/devices/0000:01:00.0/reset_method", " ") ;
}
?>

 

 

 

I cannot remember why I have it in on my test system, maybe was related to my ARC770 but maybe worth trying this, You will need to reboot for it to take effect. May also increase your power draw.

 

root@computenode:~# cat /boot/config/modprobe.d/vfio.conf 
options vfio-pci disable_idle_d3=0
root@computenode:~# 
 

[dboris]

On 8/3/2023 at 8:41 AM, SimonF said:

I cannot remember why I have it in on my test system, maybe was related to my ARC770 but maybe worth trying this, You will need to reboot for it to take effect. May also increase your power draw.

 

root@computenode:~# cat /boot/config/modprobe.d/vfio.conf 
options vfio-pci disable_idle_d3=0
root@computenode:~# 
 

Update : I was able to benchmark the GPU in a VM. The results were so underwhelming, probably because of the ReBAR requirements that I just stopped trying to improve the situations.

[kaares]

I just noticed that thor updated his kernel with (untested) ZFS support. Some of you might be waiting for that. 

[bubbadrk]

I have not had a chance to read through all the pages on this thread but was considering grabbing an Arc. Does it still take some tinkering to get it working properly on a docker container through Unraid? I have a working system now so I dont want to tinker with something stable for little benefit but if it is close to being supported I might just grab one now.

[kaares]

It runs as intended in a docker if you use the kernel by thor. This is how I use it. I get AV1 support in Plex so that is pretty cool. I'm not sure if it is as performant as the nvidia tesla t4. I get some buffering when transcoding a 4k hdr to lower resolution sdr with buring in subtitles. If I skip the subtitles it is smooth. I seem to remember the nvidia not buffering in this scenario. I'm currently having too much fun testing av1 to switch back to test.

 

If you use the official unraid release it will not support hardware video encode/decode. (You need the kernel to be 6.2 or later, the official Unraid is only 6.1.x.) Installing the kernel is as easy as copying four files to the boot folder. It will technically work as a video output in 6.1 but without hardware encode/decode.

 

If you have two gfx cards and want to combine the thor kernel with a nvidia gfx-card it will not work. The nvidia driver will refuse to load. If you want to use nvidia stuff stay on the official release.

 

ZFS is untested with the thor kernel. It's included, but none has mentioned testing it yet.

[Beaniiman]

On 8/18/2023 at 12:14 PM, kaares said:

It runs as intended in a docker if you use the kernel by thor. This is how I use it. I get AV1 support in Plex so that is pretty cool. I'm not sure if it is as performant as the nvidia tesla t4. I get some buffering when transcoding a 4k hdr to lower resolution sdr with buring in subtitles. If I skip the subtitles it is smooth. I seem to remember the nvidia not buffering in this scenario. I'm currently having too much fun testing av1 to switch back to test.

 

If you use the official unraid release it will not support hardware video encode/decode. (You need the kernel to be 6.2 or later, the official Unraid is only 6.1.x.) Installing the kernel is as easy as copying four files to the boot folder. It will technically work as a video output in 6.1 but without hardware encode/decode.

 

If you have two gfx cards and want to combine the thor kernel with a nvidia gfx-card it will not work. The nvidia driver will refuse to load. If you want to use nvidia stuff stay on the official release.

 

ZFS is untested with the thor kernel. It's included, but none has mentioned testing it yet.

 

Any luck with HW Tone Mapping, I had to disable tone mapping on HDR to get it to HW Transcode.  Other than that everything works great now.

[kaares]

1 hour ago, Beaniiman said:

 

Any luck with HW Tone Mapping, I had to disable tone mapping on HDR to get it to HW Transcode.  Other than that everything works great now.

Yeah, no issues. 

[requiRe]

Has someone tested ZFS with the Thor kernel yet?

Edited August 23, 2023 by requiRe

[requiRe]

 

On 8/23/2023 at 9:40 AM, requiRe said:

Has someone tested ZFS with the Thor kernel yet?

Decided to test it out myself, and thus far it works without problems.

[Riverfrome]

Look I understand it’s still in its infancy and I really respect and admire all the testing and work around people are doing…. Especially because in practice this/these cards show big promise to us… but looking at the data given I still see it’s far from a viable option. I run the Quadro P2000…. It idles at 6w and about 18++w  when transcoding… I’ll max have 4-6 users transcoding Plex at a time…

I run it along side a i7-7700.

I of course want quality and I care about power… that’s also why I didn’t upgrade to 13th gen and just use the iGPU ad if that’s running it’ll use even more power than the p2000… and at the end of the day the p2000 just works….

whats your experience for the others that run/ran the quadro? Have you switched to IGPU or the ARC gpu? Results? Feelings?

[ich777]

On 9/4/2023 at 6:53 AM, Riverfrome said:

I of course want quality and I care about power…

But then you should use something more efficient like a Nvidia T1000 and even consider switching to the iGPU from your 7700 since it is really, really more efficient (a few μW in Idel and depending on your iGPU up to around 12W max).

 

I don't want to start a war in terms of quality but some say that the Intel iGPUs are far superior than Pascal and older NVENC.

 

Sure the Nvidia cards are faster in terms of FPS but do you need that for HW transcoding...? Most applications like Emby and Plex throttle the transcoding anyways.

So to speak you even can transcode with a Intel iGPU 4-6 simultaneous streams.

 

Sure I'm the maintainer from the Nvidia Driver plugin and for people who have no iGPU or are in need of 10+ simultaneous streams I completely get the point but for people with only a few streams a Intel iGPU does the job good enough (of course if someone wants to do something else too ).

 

On 9/4/2023 at 6:53 AM, Riverfrome said:

at the end of the day the p2000 just works….

When 6.13 is released ARC will be natively supported and it will also just work, even easier than a Nvidia GPU because you don't have to install a driver that is about 200MB, add three additional parameters to the Docker template (okay, you also have to add one entry) and you don't have to restart the Docker service/entire Server, so I don't see this as a valid argument.

 

On 9/4/2023 at 6:53 AM, Riverfrome said:

Have you switched to IGPU or the ARC gpu? Results?

I've now run a few test with my Intel ARC A380 on a custom built Unraid version and these cards are fast, really fast (keep in mind that the official 6.12.x release doesn't support ARC) :

 

I haven't tested the power draw yet because this was not my main goal, it was just to investigate what's needed to get them running on Unraid back then. I saw that there where reports about high power draw in Idle but there is already a fix out there (I think).

 

Please keep also in mind that this is more or less a feature request thread and where people can communicate about their Intel ARC experience and custom Kernels, they are not really interested in Nvidia cards because they most certainly already have a ARC GPU on hand and don't want to buy a new one.

 

Hope you get my point and answered most (or hopefully all) of your questions.

[FlyingTexan]

If anyone is looking for an ARC card I'll cut you a deal. I have mine sitting in a drawer, got an RTX 4060 instead, Never once used probably still has the plastic on it. Bought it off newegg can give you a copy of the receipt for any warranty. I think I've had it since october. 

[domrockt]

Hey 👋 i got mine today and did a quick passtrough to an Win11 VM. Did a few VM reboots and forced shutdowns and have no reseat bug. 

 

*edit* when a Monitor is connected, i cant force close the VM* reseat Bug?
 

iam not at Home right now 

 

but iam on 6.12.4 and habe the Intel Arc a770 16GB Limited Edition 

 

i read the whole thread right now to be up to Date

Edited September 19, 2023 by domrockt

[adamgwaps]

Anyone can help me? it used to work on Ryzen now i migrated to Epyc maybe due to no Resizeable Bar in bios,the freaking Error 43 already tried every custom config on VM, Tried both machine types, tried vbios, i mean everything, i hope someone can help me how to make it work ill be willing to buy you a coffee if ever my problem gets resolved thanks!

[raysv]

Hi, have a small / big problem, just gotten an ARC A380, installed it in the unraid server and passed it thru to a ubuntu 23.10. 

 

VM boots fine, and im getting video out of the card! However, neither hevc_qsv og hevc_vaapi seems to be here, what is needed to get that running?

[firepong]

I do not know if this is possible, but can we see the temps of the intel arc card itself? I know gpu statistics does not report it with intel_gpu_top installed, so I have the feeling I will not be able to see it, but wanted to ask. I do not have any vms installed on my server, but I might have to get a barebone vm setup if this is the only way to do it. 

 

My setup:

I have a dell r730xd with an A380 installed without a fan (Passively cooled) because I did not have the room to install the card without a shroud (Have the Asrock Challenger A380). I know it was the wrong card to get as it takes up more room than I have due to the rear 2.5" drive caddies on the server. Just needed something that would work for plex transcodes that was cheap and had AV1 enc/dec support and the A380 fit the bill.

 

Now, After installing, I tested the gpu out with 3 transcode streams (4k > 1080p 12mbps) running in parallel and the card was hot, but was not blistering hot, so I think its fine with the amount of airflow that the 2u chassis pushes out. I just decided to raise my fan curve on the server to idle at 25% fan speed to keep a decent amount of air flowing through it instead of the normal 10-15% it sits at.

 

If anyone has any input on this and I need to boot up a VM to view temps, let me know and I will do that. If there is another way without a VM, I will be fine with doing that as well.

[domrockt]

i have no luck so far when i install the custom Thor kernel my Mellanox Nic is not detected anymore, i mean its there in the devices tab but not recognized as usable NIC🥲 can anyone relate?

 

i opened an Issue on Github, maybe its because the NICs are from NVidia

Edited October 26, 2023 by domrockt

[psych]

Hi all,

 

I want to purchase an Arc as it seems like a cost effective way to transcode. Unraid is currently working on my i5 6600k and I have a gtx 970 shoved in there temporarily. From what I can read the iGPU on this chip and the actual dedicated card cannot or is not efficient at transcoding HEVC/H.265.

Resizable bar is said to be needed for this the Arc to work at its best, but I've read conflicting information on whether it's needed if I only intend to transcode with it vs using it for gaming.

 

Any words of wisdom from those who have tried? I would hate to invest in a whole other system if a $200 gpu is on the market that can do what I need .

Edited October 22, 2023 by psych

[HAL777901]

On 10/22/2023 at 4:25 PM, psych said:

Hi all,

 

I want to purchase an Arc as it seems like a cost effective way to transcode. Unraid is currently working on my i5 6600k and I have a gtx 970 shoved in there temporarily. From what I can read the iGPU on this chip and the actual dedicated card cannot or is not efficient at transcoding HEVC/H.265.

Resizable bar is said to be needed for this the Arc to work at its best, but I've read conflicting information on whether it's needed if I only intend to transcode with it vs using it for gaming.

 

Any words of wisdom from those who have tried? I would hate to invest in a whole other system if a $200 gpu is on the market that can do what I need .

I'm pretty sure that you'll just need to make do or get a new GPU. A P1000 or 2000 Quadro would do the trick, however you could also try and find a 7700k or something from the next gen of chips. I'm pretty sure your board would support it.

[unr41dus3r]

Sorry, if i missed it in the thread, but has anybody information about the IDLE Power Draw of an A310/A380 or A750 ?

 

I am thinking to get an ARC but the most important thing would be the IDLE Power draw and maybe some power in reserve with the A750, but i dont know if there is any difference in the IDLE Power draw handling?