Bladedude
-
Posts
19 -
Joined
Content Type
Profiles
Forums
Downloads
Store
Gallery
Bug Reports
Documentation
Landing
Posts posted by Bladedude
-
-
13 minutes ago, bmartino1 said:
to use Mellanox ConnectX cards, you need to install the plugin and run the firmware:
while the infiniband for the ring form 1 dac to mutil doesn't work in unraid. a single interface direct sac to sac fiber line can still carry the full bandwidth when setup correctly.
Thanks @bmartino1. I'm not using infiniband, just 10GbE ethernet via DAC cable. Will that work as is? Or do I need fiber transceiver and cable?
-
Data when booting server and Mellanox ConnectX-3 Pro into TrueNas scale and ethernet comes up, but ethernet does come up when I reboot into Unraid.
admin@supermicro[~]$ uname -srm Linux 6.1.63-production+truenas x86_64 admin@supermicro[~]$ /sbin/ifconfig enp1s0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500 inet 192.168.X.201 netmask 255.255.255.0 broadcast 192.168.X.255 inet6 XX prefixlen 64 scopeid 0x20<link> inet6 XX prefixlen 64 scopeid 0x0<global> ether 9c:dc:X txqueuelen 1000 (Ethernet) RX packets 47076 bytes 49359473 (47.0 MiB) RX errors 0 dropped 1 overruns 0 frame 0 TX packets 16362 bytes 6274827 (5.9 MiB) TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0 admin@supermicro[~]$ sudo dmesg | grep -i enp1s0 [ 9.744788] mlx4_core 0000:01:00.0 enp1s0: renamed from eth0 [ 9.836760] mlx4_core 0000:01:00.0 enp1s0d1: renamed from eth0 [ 11.154076] mlx4_en: enp1s0: Link Up [ 48.685128] mlx4_en: enp1s0: Steering Mode 1 [ 48.740749] mlx4_en: enp1s0: Link Up admin@supermicro[~]$ netstat -i Kernel Interface table Iface MTU RX-OK RX-ERR RX-DRP RX-OVR TX-OK TX-ERR TX-DRP TX-OVR Flg enp1s0 1500 57271 0 1 0 20159 0 0 0 BMRU admin@supermicro[~]$ lspci | grep Ethernet 01:00.0 Ethernet controller: Mellanox Technologies MT27520 Family [ConnectX-3 Pro]
-
Yup. Fair enough. Let me 100% validate on my end again. I'll reboot into TrueNas Scale and grab a screenshot and dmesg on boot and see if that can help.
-
7 minutes ago, JorgeB said:
I'm sorry, I don't see how that can be an Unraid problem, driver is not detecting a link, and it won't work until it does, and find it very strange that TrueNAS would detected a link with exactly the same hardware.
Thanks, but help me understand: If I reboot the server, without touching the server or hardware, and only change the boot device, TrueNas Scale boots and detects the Mellanox NIC with DAC cables and receives a reserved IP. Unraid does not. Same HW, one OS brings up NIC ok, but another OS does not. Sounds to me like software.
-
8 hours ago, JorgeB said:
Are use using fiber with transceivers or DAC cables?
DAC cables
-
Diagnostics attached. Yes, it's in ethernet mode.
-
Ah gotcha. It wasn't crystal clear from the docs.
Quote7. Browse to the USB flash device to see the newly extracted contents from your Mac or PC.
I did that. Files were there.
Perhaps for clarity, it should say browse to the USB flash device, and complete the following steps from the flash device directory.
-
1 minute ago, itimpi said:
it looks as if you did not switch the flash drive before
I don't follow
-
# lspci | grep Mellanox 01:00.0 Ethernet controller: Mellanox Technologies MT27520 Family [ConnectX-3 Pro] # lspci -vv -s 01:00.0 | grep "Part number" -A 3 [PN] Part number: 779793-B21 [EC] Engineering changes: B-5718 [SN] Serial number: IL271700F5 [V0] Vendor specific: PCIe 10GbE x8 6WNew install of Unraid. Adaptor will not link up out of the box. I installed Mellanox Firmware Tools plug-in, and it's still not up. If I reboot into Truenas Scale (I'm wiping my existing Scale and installing Unraid), it came up from the default Truenas Scale installation.
-
Thanks. I tried manual install from both Mac and Windows with the errors detailed in OP. Mac USB Flash creator tool with deselected UEFI worked. Mac USB creator docs should include a link to open app from unidentified developer.
-
For Mac manual install, looks like it's failing in the last lines of the script:
mkdir -p /tmp/UNRAID/syslinux ; cp -rp /Volumes/UNRAID/syslinux/* /tmp/UNRAID/syslinux echo -n "To continue please enter your admin " sudo /tmp/UNRAID/syslinux/make_bootable_mac.sh $TARGET -
- Brand new metal Sandisk 32GB USB key, used the USB Flash creator tool on Mac with no issues. UEFI may have been enabled. Server errored out with USB not bootable.
- On PC, tried to reformat and reinstall OS to USB via PC Flash creator tool. Hit this known Unraid issue from 2 yrs ago, with USB device (incompatible).
- Followed suggestion on above page to use manual install.
- For manual install on Windows, downloaded Unraid.zip and verified MD5. Formatted USB drive as FAT32 with name UNRAID. Ran script and received the following error:
-
Quote
Make Bootable v1.5
ERROR - syslinux executable not found, expected: C:\syslinux\syslinux.exe
Press any key to continue . . .
-
For manual install on Mac, downloaded Unraid.zip and verified MD5. Formatted USB drive as FAT32 with name UNRAID. Ran script and received the following error:
$ /Users/XXX/Downloads/unRAIDServer-6.12.8-x86_64/make_bootable_mac ; exit; INFO: make_bootable_mac v1.3 INFO: The following drive appears to be the unRAID USB Flash drive: /dev/disk5 30.8GB Permit UEFI boot mode [Y/N]: n cp: /Volumes/UNRAID/syslinux/*: No such file or directory To continue please enter your admin Password: sudo: /tmp/UNRAID/syslinux/make_bootable_mac.sh: command not found logout Saving session... ...copying shared history... ...saving history...truncating history files... ...completed. Deleting expired sessions...1 completed. [Process completed]Please advise
-
Looking at the activation code notes at purchase:
QuoteNote: Activation Codes are one-time use only to generate the license keyfile for your server
Does this mean if I wipe and reinstall Unraid, I'll need a new key for the same server? I want to test configurations for HW and apps and make sure things are setup correctly. I was intending to document as I performed the setup, and then wipe and reinstall with lessons learned (because there always are) for the final install. Should I setup and use a 30-day trial?
-
4 hours ago, jit-010101 said:
there was an issue related to *arr with shfs (the underlying technology used as a basr for Unraid merged folders) so I'd highly recommend you to test the trial to the fullest to be able to face any issues and get yourself an idea what you might need to do
Based on the identical dozen spinners, I'll likely go with zfs. Is share filesystem (shfs) issue still a concern with RaidZ2? @jit-010101
-
Thanks for the additional comments @jit-010101 @mackid1993 and @SimonF. This is more feedback to this 1 post 30 hrs ago, than I've ever received from Truenas forums or their discord.
If I didn't have a dozen identical drives, it sounds like xfs would be the way to go given the primary use case of media + smb shares. Given the identical dozen drives, zfs sounds like the best use of the existing HW.
21 hours ago, Daniel15 said:The only caveat I've found is that currently you need at least one drive in the Unraid array (i.e. outside of a ZFS pool). I have an unused USB stick in mine as a workaround. I've heard they're going to make that more flexible in a future release.
@Daniel15 I'll need to find a solution for this. I do have 1 internal USB type A port that I will use for the boot/key USB stick. This post mentions using a "USB DOM" which might work since I have 2 free internal USB headers on the mobo.
7 hours ago, jit-010101 said:All that in mind: What the heck is wasting away those 200W? Both my servers use less then 60W together ...
@jit-010101PSU reports it's pulling 280W input at idle. 2 x E5v2 Xeons, 7 fans + 2 more in the PSU, and 12 3.5" spinners.
-
Thanks @Daniel15 @dopeytree and @HardwareHarry for the responses and examples of your setups.
Good info on a USB stick workaround for an array @Daniel15.
@dopeytree For power, I don't think the server can power down spinners, and the power is a sunk cost for me. With no drives it pulls ~200 watts, and ~325 watts with 12 3.5" spinners.
This vid All about Using Multiple Cache Pools and Shares in Unraid 6.9 from SpaceInvaderOne was helpful, as well as this one Overview ZFS for Unraid (Create, Expand and Repair ZFS Pool on Unraid)..
This ZFS documentation link mentions:
QuoteThis section should be completed once Unraid 6.12 has been released with ZFS support included as a standard feature
but maybe it's a backlogged item.
Use case is media server + smb shares. Given that all the drives will be spinning, zfs seems like the route to take. Maybe I'll try xfs+parity and see if the server powers down the unused drives.
-
3 hours ago, JonathanM said:
I'd recommend playing with 6.x
Not to be a debbie downer, but waiting for future versions from Unraid is not productive. They release when stuff is ready, and not a moment sooner. Ask anybody who's been around here for very long what Soon™ means.
If you watched through the video, you know that you can sidestep the requirement for having an array disk by using any old spare USB flash drive as Disk1 in the array slot, then build out your storage pool(s) any way you like. I would recommend adding a SSD or two to a different pool for system files, most notably the docker image and appdata for said containers. That will keep your media server GUI snappy.
The only real downside I see to not waiting is the lack of automated moves from pool to pool currently, but that shouldn't be an issue for you if you download and sort the media directly on the final storage pool instead of having a separate download pool.
Thanks for the response @JonathanM
Waiting for future versions: understood. USB flash drive for array disk, I'll have to rewatch the vid and do more reading. Download and sort the media on final storage seems like the optimum route using Trash guides / hardlinking.
HW available with original plans below:
- 12 x 10TB SAS for media in RaidZ2. Not sure whether to stick with zfs, or use a different method such as xfs + parity for future expansion. Though I could add another pool since the server has another 24 x 3.5" bays free
- 2 x 500GB for Truenas boot. Not sure what to use them for in Unraid
- 2 x 1TB NVME for apps
- 2 x 56GB Optane for zfs cache, slog etc
- I have 1 internal USB type A slot, and I've ordered a metal case 32GB USB stick for the license key
I'd welcome suggestions on how to arrange/optimize this HW for Unraid.
-
Hi there,
I have a Synology DS918+ 4 HDD server that I've outgrown. I tried Truenas scale on a SuperMicro SS847 with 12 x 10TB SAS drives, Optane, 10Gb DAC and Nvidia P600 GPU with the goal to use primarily for media serving and for *arr apps, and maybe as a Plex server. I struggled and wasn't able to get it to work, and community Truenas support was lacking imho ( 🤷♂️ it is free after all).
I had shyed away from Unraid due to cost, but was blown away, in a really positive way, by the conversation with the CEOs and specifically their talk of community engagement and support in this YouTube video: The Unraid Story: Lime Technology Co-CEO's Discuss the Past and Future of Unraid OS. So I immediately purchased an Unraid Pro license. If the community engagement is as strong as discussed in the vid and based on my limited interactions with Unraid users, I'd be willing to pay an annual support/updates subscription. Everyone's moving towards the subscription model, and it's a good predictable revenue stream for companies.
My question is: Should I install and setup the server now with Unraid 6.x, or wait for 7.x? I want to setup a RaidZ2, unless there is a better suggestion. I say that because for other OS, zfs seems to be the best solution for my requirements, but Unraid does have the ability to add storage and expand pools, like Synology SHR. Thoughts?
Thanks,
to use Mellanox ConnectX cards, you need to install the plugin and run the firmware:
Mellanox ConnectX-3 support
in General Support
Posted · Edited by Bladedude
@bmartino1Thanks. I had already installed the Mellanox Firmware Tools plugin. I assumed the interface would be enabled by default on install. It wasn't, but all I had to do was enable the interface. The board has 4 GbE + 2 SFP+, so I missed that on the network page. 🤦♂️ Once enabled, it came right up with no issues. Thanks again!
Feature request / bug fix would be to enable, by default, on install any/all interfaces.