Keexrean Posted January 18, 2020 Share Posted January 18, 2020 (edited) [SEE THE END MESSAGE/UPDATE ON PAGE 2 for a TLDR and some infos] Hi! Right away: TLDR: Ploped a SFN5122F in a Z600. SFP+ DAC cables still incoming in the mail, not delievered yet, so nothing's plugged in it for now. Seems to boot gracefully with the card in. It appears in the list of PCIe devices. But nothing in the network interfaces. So my first concern is to know whether it's intended to not show up at all if not plugged to anything, or if I'm heading to an headache. Posting diagnostics just post boot (to avoid bugging you up with mile long logs) with PCIe ACS override off and also downstream, as I plan on a putting back my GPU passthrough. Same result though, no show in network interfaces. Long Talk and why the heck do I have purchased that: I built myself a pretty beastly workstation, but I would prefer my unraid server handling most of the video encoding (even though my workstation would be way more capable and quick at it, but it's as to not have it "busy". I'm less concerned about actual speed, and more about having my workstation free to do something else). I spared 4 threads of one of the two x5550 of my HP Z600 unraid running box and plan to passthrough a 1050TI to a VM (lent it to a friend because his MSI 5700XT Mech OC went FUBAR, will get it back in about 10 days). Having 1TB of RAID1 SSD cache and still some room on the array, I felt pretty confident that I could have the server ingest the footage directly, encode it to a timeline-friendly codec (cineform) + having the original rushs also rendered in h265 for spaceefficient-decentquality-safekeepingonarray, edit my stuff on the workstation off of the server though a 10Gbps direct link, export to the server's cache in cineform, and have the server do the final render. Eventually thinking about a similar workflow for C4D. I for now don't have any spinning rust in my workstation, and I'm not sure if I will ever put some inside, just 1TB nvme PCIe gen4 SSD for the system and softwares that needs it the most, and a 2TB sata SSD for other softwares and games. I would like to avoid storing ANYTHING on the workstation, as to not have it like my other computers are... with folders of thousands of random downloads and saved filed, unsorted, and with no redundancy what so ever. (before you make a comment, I'm actively trying to sort stuff atm, deleting or keeping, and archive it in well named zip files that can be moved around with nice sequential speed). Other issue: I tend to keep everything. I still have chatlogs from the early 2000s... more than a gig of the thing, all .xml files. Yes I'm sick. No I won't seek for a cure. I'm actively feeding my demons. Deal with it. Hence why this small investment, the idea being using a 10gbps link to make my server's redundant storage as accessible and (almost as) responsive as if it was internal to my workstation. The SFN5122F installation on windows went "alright" (did bsod when I innocently clicked "update driver" in the device manager, but going to the manufacturer's website and getting the drivers from there, the installation went like a breeze). Thing is, its twin doesn't show up on unraid as an interface. It's detected as PCI device in the system profiler, but not in the network interfaces. Cables are still incoming in the mail (was supposed to get them same day as the cards, but postmail services in my area are... meh). So my first concern is to know whether it's intended to not show up at all if not plugged to anything, or if I'm heading to an headache. Posting diagnostics just post boot (to avoid bugging you up with mile long logs) with PCIe ACS override off and also downstream, as I plan on a putting back my GPU passthrough. Same result though, no show in network interfaces. [ATTACHED DIAGNOSIS DELETED] Edited January 21, 2020 by Keexrean Quote Link to comment
JorgeB Posted January 19, 2020 Share Posted January 19, 2020 You'd need to make a feature request for that driver to be added to next Unraid release. Edit: Not the problem here, NIC is supported by Unraid. Quote Link to comment
Keexrean Posted January 19, 2020 Author Share Posted January 19, 2020 (edited) From what I gathered around before going for a pair of SFN5122f, it was supposed to be supported in unraid 6.4 rc15 as some said. Did they simply dropped support between then and 6.8.1 ? Edited January 19, 2020 by Keexrean Quote Link to comment
pm1961 Posted January 19, 2020 Share Posted January 19, 2020 Did you check out the video by Spaceinvader one? If I understand him correctly, he explicitly states that the Solarflare card does not work with unraid, hence his use of Mellanox cards.... Quote Link to comment
Keexrean Posted January 19, 2020 Author Share Posted January 19, 2020 (edited) Oh yah, oh yah I did. That video was exactly why I almost didn't bought it when it showed up cheap on ebay. Then silly me opened the video's description. If I understand him correctly, the same guy who explicitly states that the Solarflare card does not work with unraid actually went ahead and edited the description 7 days after to explicitly states that the Solarflare card does in fact work with unraid. Am I supposed to trust SpaceInvaderOne Circa nov 2017, or SpaceInvaderOne Circa dec 2017 ? Edited January 19, 2020 by Keexrean Quote Link to comment
pm1961 Posted January 19, 2020 Share Posted January 19, 2020 (edited) Hmmm, definitely room for confusion....... I’m waiting for a pair of Mellanox cards myself..... if I don’t come bleating on here, then it worked! I’ve asked this before, but never really got an answer, if we could have a list of supported hardware in unraid.... Lime must have that knowledge for each build......... they could put it as part of the wiki page maybe? I wasted a lot of money a few years ago on various RAID cards that either partially worked or wouldn’t load at all..... Now I just adopt “sheep mode” and follow what others have done...... Edited January 19, 2020 by pm1961 Quote Link to comment
Keexrean Posted January 19, 2020 Author Share Posted January 19, 2020 (edited) Yeah, if it turns out there is nothing to do about it, I'll have to cave for a Mellanox NC522SFP for 60€. (no, I won't go for a single port card, because I don't want to be stuck when <doing network upgrades / adding an other direct-attached computer> further down the line, and that's my LAST PCIe port available on the server.) I hope someone will be able to give me a more positive answer, because I can't believe that I'm the first one, and that it took 3 years to check whether or not @SpaceInvaderOne had a colorful party night before editing his video's description. Edited January 19, 2020 by Keexrean Quote Link to comment
BRiT Posted January 19, 2020 Share Posted January 19, 2020 TLDR: It's complicated. It is impossible for any company to know the exact state of support for all hardware as it is possible for bugs to be introduced on newer versions of the Linux kernel that could break hardware support, or a hardware company could stop supporting Linux entirely or not bother to port their hardware drivers to newer kernels. So many things can happen that is outside the control of LimeTech. A solid positive and benefit is LimeTech supplies trial versions so if you have the hardware already and are uncertain about hardware support you can try it out before hand. Quote Link to comment
BRiT Posted January 19, 2020 Share Posted January 19, 2020 (edited) The list of UnRaid 6.8.1 configuration for the kernel is located in /usr/src/linux-4.19.94-Unraid/.config and I attached it as a zip file to this post. unraid681.config.zip Maybe you need to load some modules? It has the following specified -- CONFIG_NET_VENDOR_SOLARFLARE=y CONFIG_SFC=m CONFIG_SFC_MCDI_MON=y CONFIG_SFC_SRIOV=y CONFIG_SFC_MCDI_LOGGING=y CONFIG_SFC_FALCON=m Edited January 19, 2020 by BRiT Quote Link to comment
BRiT Posted January 19, 2020 Share Posted January 19, 2020 Is the SFC module loaded? If not, try: modprobe sfc Quote Link to comment
Keexrean Posted January 19, 2020 Author Share Posted January 19, 2020 Pulled directly from my /usr/src/linux-4.19.94-Unraid CONFIG_NET_VENDOR_SOLARFLARE=y CONFIG_SFC=m CONFIG_SFC_MCDI_MON=y CONFIG_SFC_SRIOV=y CONFIG_SFC_MCDI_LOGGING=y CONFIG_SFC_FALCON=m Quote Link to comment
Keexrean Posted January 19, 2020 Author Share Posted January 19, 2020 Also, from the https://support.solarflare.com/index.php?option=com_cognidox&file=SF-103848-LS-39_Solarflare_NET_driver_source_RPM.rl.txt&task=download&format=raw&id=1945 page: Linux distribution support for Solarflare Adapters -------------------------------------------------- This driver package is supported on: - RedHat Enterprise Linux 5.10, 5.11 - RedHat Enterprise Linux 6.5, 6.6, 6.7, 6.8 - Redhat Messaging Realtime and Grid 2.5 - RedHat Enterprise Linux 7.0, 7.1, 7.2 - RedHat Enterprise Linux for Realtime 7.1, 7.2 - SuSE Linux Enterprise Server 11 sp3, sp4 - SuSE Linux Enterprise RealTime Extension 11 - SuSE Linux Enterprise Server 12 base release, sp1 - Canonical Ubuntu Server LTS 14.04, 16.04 - Canonical Ubuntu Server - - Debian 7 "Wheezy" 7.X - Debian 8 "Jessie" 8.X - Linux 2.6.18 to 4.7 This driver has been tested as a VF driver using KVM / ESXi 5.5 and ESXi 6.0 hypervisors on the following guest OSes: - RedHat Enterprise Linux 5.11 - RedHat Enterprise Linux 6.5, 6.6, 6.7 - RedHat Enterprise Linux 7.0, 7.1, 7.2 - SuSE Linux Enterprise Server 11 sp4 - SuSE Linux Enterprise Server 12 base release I'm there, contemplating DKMS or RPMbuild options, and which I'll try to forcefeed into my poor system. Some alternative in mind? Quote Link to comment
BRiT Posted January 19, 2020 Share Posted January 19, 2020 Did you try manually loading those kernel modules, at least the SFC? Quote Link to comment
Keexrean Posted January 19, 2020 Author Share Posted January 19, 2020 >insmod /lib/modules/4.19.94-Unraid/kernel/drivers/net/ethernet/sfc/sfc.ko.xz returns > insmod: ERROR: could not insert module /lib/modules/4.19.94-Unraid/kernel/drivers/net/ethernet/sfc/sfc.ko.xz: Unknown symbol in module Quote Link to comment
BRiT Posted January 19, 2020 Share Posted January 19, 2020 Was that you using insmod directly? Use modprobe instead. And use lsmod to see what is loaded. That SFC module loaded fine for me using modprobe. Of course I dont have the hardware. Also, are you using unraid or unraid-nvidia or unraid-dvb (one of those custom community version)? Quote Link to comment
1812 Posted January 19, 2020 Share Posted January 19, 2020 I run a SFN5122F in a z420 and actually split the ports: 1 for Unraid and 1 for a VM. so it works. post a screenshot of the following section in the network settings: 1 Quote Link to comment
Keexrean Posted January 19, 2020 Author Share Posted January 19, 2020 (edited) Yes, that was me risking a kernel panic, dabian habits. lsmod doesn't list sfc. It indeed isn't loaded, even though I did several >modprobe sfc I'm using bog-standard unraid 6.8.1, andso far never did anything to the kernel or drivers, never had to on this box. Edited January 19, 2020 by Keexrean Quote Link to comment
Keexrean Posted January 19, 2020 Author Share Posted January 19, 2020 @1812 I simply don't have this view, since it only detects the single rj45 port of the motherboard, and nothing else. 1 Quote Link to comment
1812 Posted January 19, 2020 Share Posted January 19, 2020 (edited) 13 minutes ago, Keexrean said: @1812 I simply don't have this view, since it only detects the single rj45 port of the motherboard, and nothing else. make a copy of the config/network.cfg file from the flash drive to your desktop (as a backup), then delete the one on the flash drive and reboot the server. this will force new settings and maybe it will pickup the solar flare card. if not, I'll dig through your diags a bit more looking at your logs, this prob won't help. Edited January 19, 2020 by 1812 Quote Link to comment
1812 Posted January 19, 2020 Share Posted January 19, 2020 Jan 18 21:50:43 Procyon kernel: Call Trace: Jan 18 21:50:43 Procyon kernel: ? efx_pci_probe+0x6ae/0x81d [sfc] Jan 18 21:50:43 Procyon kernel: ? pci_bus_read_config_word+0x44/0x66 Jan 18 21:50:43 Procyon kernel: efx_pci_probe+0x6ae/0x81d [sfc] Jan 18 21:50:43 Procyon kernel: local_pci_probe+0x39/0x7a Jan 18 21:50:43 Procyon kernel: pci_device_probe+0x13f/0x199 Jan 18 21:50:43 Procyon kernel: really_probe+0x13d/0x288 Jan 18 21:50:43 Procyon kernel: driver_probe_device+0x9c/0xd1 Jan 18 21:50:43 Procyon kernel: __driver_attach+0x89/0xb9 Jan 18 21:50:43 Procyon kernel: ? driver_probe_device+0xd1/0xd1 Jan 18 21:50:43 Procyon kernel: bus_for_each_dev+0x70/0xa6 Jan 18 21:50:43 Procyon kernel: bus_add_driver+0xe1/0x1c6 Jan 18 21:50:43 Procyon kernel: driver_register+0x99/0xcf Jan 18 21:50:43 Procyon kernel: ? 0xffffffffa014d000 Jan 18 21:50:43 Procyon kernel: efx_init_module+0x7a/0x1000 [sfc] Jan 18 21:50:43 Procyon kernel: do_one_initcall+0x71/0x147 Jan 18 21:50:43 Procyon kernel: ? _cond_resched+0x1b/0x1e Jan 18 21:50:43 Procyon kernel: ? kmem_cache_alloc+0x30/0xeb Jan 18 21:50:43 Procyon kernel: do_init_module+0x51/0x1c7 Jan 18 21:50:43 Procyon kernel: load_module+0x1890/0x1ec2 Jan 18 21:50:43 Procyon kernel: ? __insert_vmap_area+0x6c/0xb0 Jan 18 21:50:43 Procyon kernel: ? __se_sys_init_module+0xcf/0x10a Jan 18 21:50:43 Procyon kernel: __se_sys_init_module+0xcf/0x10a Jan 18 21:50:43 Procyon kernel: do_syscall_64+0x57/0xf2 Jan 18 21:50:43 Procyon kernel: entry_SYSCALL_64_after_hwframe+0x44/0xa9 Jan 18 21:50:43 Procyon kernel: RIP: 0033:0x14fdb343213a Jan 18 21:50:43 Procyon kernel: Code: 48 8b 0d 59 7d 0c 00 f7 d8 64 89 01 48 83 c8 ff c3 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 44 00 00 49 89 ca b8 af 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 26 7d 0c 00 f7 d8 64 89 01 48 Jan 18 21:50:43 Procyon kernel: RSP: 002b:00007ffe0a48c8d8 EFLAGS: 00000246 ORIG_RAX: 00000000000000af Jan 18 21:50:43 Procyon kernel: RAX: ffffffffffffffda RBX: 000000000065de00 RCX: 000014fdb343213a Jan 18 21:50:43 Procyon kernel: RDX: 000014fdb351184d RSI: 0000000000072d50 RDI: 0000000000e95000 Jan 18 21:50:43 Procyon kernel: RBP: 000014fdb351184d R08: 000000000064701a R09: 0000000000000006 Jan 18 21:50:43 Procyon kernel: R10: 0000000000647010 R11: 0000000000000246 R12: 0000000000e95000 Jan 18 21:50:43 Procyon kernel: R13: 000000000067b6b0 R14: 0000000000020000 R15: 000000000065de00 Jan 18 21:50:43 Procyon kernel: ---[ end trace 87b5401780146dfe ]--- Jan 18 21:50:43 Procyon kernel: sfc 0000:0f:00.0 (unnamed net_device) (uninitialized): could not map memory BAR at 0+ff0800 Jan 18 21:50:43 Procyon kernel: sfc: probe of 0000:0f:00.0 failed with error -12 Jan 18 21:50:43 Procyon kernel: kvm: VM_EXIT_LOAD_IA32_PERF_GLOBAL_CTRL does not work properly. Using workaround Jan 18 21:50:43 Procyon kernel: sfc 0000:0f:00.1 (unnamed net_device) (uninitialized): Solarflare NIC detected Jan 18 21:50:43 Procyon kernel: sfc 0000:0f:00.1 (unnamed net_device) (uninitialized): Part Number : SFN5122F Jan 18 21:50:43 Procyon kernel: sfc 0000:0f:00.1 (unnamed net_device) (uninitialized): could not map memory BAR at 0+ff0800 Jan 18 21:50:43 Procyon kernel: sfc: probe of 0000:0f:00.1 failed with error -12 the card isn't getting assigned memory. Jan 18 21:50:43 Procyon kernel: DMI: Hewlett-Packard HP Z600 Workstation/0AE8h, BIOS 786G4 v01.17 08/19/2009 I'd start by updating your bios first, as HP may have fixed this issue. 1 Quote Link to comment
Keexrean Posted January 19, 2020 Author Share Posted January 19, 2020 (edited) Now Jan 19 20:14:32 Procyon kernel: DMI: Hewlett-Packard HP Z600 Workstation/0AE8h, BIOS 786G4 v03.61 03/05/2018 Y'know the old saying: don't fix what aint broken, hence why I never touched to the bios until now. And so far nothing I was doing with it was concerning with Spectre stuff, so I was like "yeah, I prefer to keep the extra bit of performance instead". Bios updated successfully, reboot done, still same situation. procyon-diagnostics-20200119-2016.zip Edited January 19, 2020 by Keexrean Quote Link to comment
Keexrean Posted January 19, 2020 Author Share Posted January 19, 2020 Okay people, update. On old bios: I tried the card in every PCIe port I have. The ones on the Z600 are as follow x8 (-x4 electrically) gen 2 x16 gen 2 x8 (-x4 electrically) gen 1 x16 gen 2 And what do I have to plug inside? - PCIEx4 5port SATA3 card - PCIEx4 SFN5122F - PCIEdontcare GT735 for console display (I also eventally have a passive nvs295) - PCIEx16 GTX 1050TI (not actually plugged in, but will get it back in a week or so, lent it) We can all agree I can't have my SATA card on the x4 gen1, would be stupid. We can all agree I can't have the SFN5122F on the x4 gen1, would be stupid. We can all agree I can't have the 1050TI elswhere than very bottom slot, because else it would cover an other PCIe slot. The GT735 works perfectly in the PCIe gen1 slot. New bios, the SFN5122F was still in the top x16 slot, and still wasn't detected. I moved the SFN5122F to the bottom x16 slot (as a test) and NOW it works! detected in network interfaces and all! Now, drum rolls, it HAS to work in the top x8(x4) slot, or I'm one PCIe slot short. OOF Quote Link to comment
Keexrean Posted January 19, 2020 Author Share Posted January 19, 2020 Update²: the SFN5122F refuses categorically to work in any of the top two slots. I can take any graphics card I have at home and plug it into into the top x16, the x8(4) gen2 or even the x8(4) gen1 (not the 5700xt though) and it will pass through to a VM with no hassle. And this SolitarBollocks piece of foo-foo of a card refuses to work in any other slot that the bottomx16 that I need for GPU passthrough... Quote Link to comment
1812 Posted January 19, 2020 Share Posted January 19, 2020 46 minutes ago, Keexrean said: Update²: the SFN5122F refuses categorically to work in any of the top two slots. I can take any graphics card I have at home and plug it into into the top x16, the x8(4) gen2 or even the x8(4) gen1 (not the 5700xt though) and it will pass through to a VM with no hassle. And this SolitarBollocks piece of foo-foo of a card refuses to work in any other slot that the bottomx16 that I need for GPU passthrough... it's probably not the card since mine is in slot 4 of 6(counting pci gen 1 slot at the bottom) in the z420. my money says HP issue considering you have still have the memory map issue in your logs. look in the bios and make sure all network boot is disabled and that if you have pcie options, they are set to specifics and not auto (I don't remember what the bios options are for that gen, but I have a z400 collecting dust that if needed I can pull and take a look at since they should be similar enough.) 1 hour ago, Keexrean said: And so far nothing I was doing with it was concerning with Spectre stuff, so I was like "yeah, I prefer to keep the extra bit of performance instead". there's a plugin to disable mitigations and keep that performance. Quote Link to comment
Keexrean Posted January 19, 2020 Author Share Posted January 19, 2020 Well in fact, if I count the pci gen 1 slots, mine is NOW also in slot 4. Thing is, I would have liked it in slot 2, the top x16 slot. Every other PCIe cards I have on hand are detected correctly in this slot, just not the SFN5122f, be it Sata HBA or raid cards, usb3.1 card, graphics card, and all that while having the other 3 PCIe slots populated. 6 minutes ago, 1812 said: there's a plugin to disable mitigations and keep that performance. Oh nice, I'll check this out. Thanks! Quote Link to comment
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.