Everything posted by Wody
-
Which Current Mainstream Supermicro Board?
No idea, the case is sold by a brand, but its not their design, they just buy the cases from China and market them as their own, without any modifications. It's a dumb backplane, so it only has power-connections and the SFF-8087 connector. Since I found out where it came from, I looked around and got a case out of China directly, also from a reseller but much more modern, that does have a brand-name on the backplane, which is OTT. The one I have comes with 8643 connectors for 12gbit, but they make a backplane with U2 as well now.
-
Which Current Mainstream Supermicro Board?
This picture isn't very clear because of all the cables, this is an X11SPL-F motherboard with 192GB RAM, 2xConnectX-4 Lx and my 9600-24i. There's also a USB with unraid and a TPM module installed. For cables, it's this ( from HBA to adapter without the baffle and then this to the backplane. Together it's all less than 1m/3 feet so it stays within SATA standards. This is a 2u case with 12 hotswap drives, so there's 2x3 cables and 3 adapters. I also use the same 8654 to 8643 in my other server which has a 8643 backplane. For price comparison, with a 9400 you can get away with these. But, I didn't use these because together with the 8654 cable it was longer than the SATA standard allows. Chenyang now also makes a 6843 to 6843 coupler which is cheaper, but I can't find shorter cables than 0.5m which means together it's already 1m so outside of SATA standards. If you only use SAS drives, you can go to 3m (10 feet) so then it is less of an issue.
-
Which Current Mainstream Supermicro Board?
Got the cables today, and unfortunately the results are as expected. So 6854-8i to 6843 or to 6843+6843-8087 adapter+8087-8087 cable works. I use that with a dumb backplane, and on one computer it detects my array and on the test-computer it detects the two drives I inserted, a 6TB WD drive, and a Samsung 870 SSD. These cable, so 8654-8i to 8087 do not work and detect no drives. Since it says compatible with SAS 4, even though I knew the outcome in advance, I tested with both my 9500-16i (SAS 3.5) and the 9600-24i (SAS 4), connectors identical, result identical. With a multimeter you can test cables, you connect it to one pin on one end, and to one pin on the other end, and if the pins are connected it shows 0, and if the pins aren't connected it shows 1. I've used this to verify that power-connectors were wired the same, and I should be able to do the same with these cables to figure out what is different about them, and I think I'll end up doing that eventually, but for now, the cables may support all kinds of things, but as tests show, they don't support Broadcom cards in the configuration we need at least.
-
Mobos without lane sharing issues
Consumer-processors usually have about 20 lanes or so, the latest about 24. Take away 16 for a GPU, one or two for the chipset, and there is almost none left, so you'll always have to share, either from slots, or because it has to go through the chipset-connection. You'll need server processors and boards to avoid most of those bottlenecks. Plus they are designed to run 24/7 and last longer, as in a decade minimum instead of 2-3 years.
-
Issue with LSI 9207-8i
What slot are you using? Only the E4 slot is usable (near the edge of the board) since that's x4 (which should run the card, but I haven't tested that model on x4), the others (E2 and E3) are x1 and most likely won't work. (Also assuming E1 is in use for the graphics card). If you have an SSD in the slot next to it, it'll only run at x2 which also may not be compatible.
-
Pfsense VM. Is a 2 port on board NIC enough?
Depends on the motherboard. Normally you want one port for the internet connection, and the other for your local connection. So if the motherboard has separate PCI-IDs you can pass through one of them for your internet and use the other for your local connection. During the VM creation you'd then select the network source as unraid, and pass through the port, and with pfsense you'd select the port as internet and the other connection as your lan.
-
Looking for advise on a new server chasis
Some manufacturers publish cooling lists, like Noctua, but I think in this case they are not reliable since they claim 14.5cm coolers work, but 2U is only about 9cm, so as long as you have a cooler that's 7cm in height or less it should work. You'd need to measure with the motherboard and processor installed to make sure There's coolers that have the fan on top of the heatsink instead of to the side and usually those are smaller but still have a big heatsink At least there's plenty of models available that fit the hight-limits. A lot will also depend on how you use the system. You'll have a lot more issues if you stick it full with drives so the air that comes to the cooler is already warm, but if there's less drives and it's just serving files it'll be fine.
-
Looking for advise on a new server chasis
From pictures it looks like there's a 2x4 for CPU-power (which can be combined into 1x8). But you can ask the seller to be sure, they also sell that board separately. As for 2U, in some ways it is a lot easier since there is less height so things are closer, on the other cable management is more important, and also you these use LP or 'low profile' for cards, which means you can only use half height cards, so they don't stick out (so most GPUs won't fit). For cards that do fit, they need to come with a LP-baffle, which is the metal part on the back of the card that you can remove and replace when needed. They are very different, for the full height the screw-hole is on the right, for the low profile one it is on the left, so it isn't a case of cutting a part off to make it fit. For many cards you can buy them separately, and you can 3D-print them yourself too. That is something to look out for though, when you install a card it is easy to get the wrong screw-hole in the chassis if you're not used to 2u.
-
Sata/SAS controllers tested (real world max throughput during parity check)
If you're picking a card to use with existing drives, something to keep in mind is also that the 9300 and newer no longer supports SATA 1 (1.5gbit) drives, and SAS 4 (so the 9600 and newer) no longer supports SATA 2 (3gbit) drives. The 9600 also has to work a lot harder, so it uses about 17 watt for the 16i. I haven't watched the videos, so I don't know if they measured temperatures but if not, most of the cards I have can get up to 60-65 celsius during startup, but usually they keep at 50-60 celsius during use. Exceptions are the 9211-8i, 9400-16i and 9500-16i which stay at about 40-45 celsius at most. This is all in a server with airflow, but all measured in the same condition (same slot etc). So keeping in mind power, heat and functionality, at this time the 9400-16i wins.
-
Any clue about this error on boot with my LSI card?
There are various manuals on the Broadcom site but not included in the previous link is the sas2flash manual which is here. -o is required for advanced commands, -e is erase, and level 6 erases everything except the manufacturing area which keeps the sas address, and -f tells it to flash firmware. You might be able to erase just the bios with -o -e 5, but I always do -e 6 and reflash so I'm sure about the integrity of the firmware.
-
Fatal Flash Drive Error
You can try a different USB port first, preferably USB 2, then if it happens again, you know you'll have to replace the USB drive, otherwise it can be the port. That said, I'd also get a new USB, just in case. And try to make sure you have a backup, or at least copy the folders (especially the config folder).
-
My Server is having a Stroke. :(
The cards normal have thermal epoxy which is not easy to remove, see here.
-
Any clue about this error on boot with my LSI card?
If Unraid detects the card, you can do it there, otherwise you need to use UEFI shell. From the broadcom site here you'll want to get the BIOS and UEFI files (in case you want them in the future), or the Installer P20 for UEFI or Linux and the P20 package under Firmware. You'll want to unpack the sas2flash and the 2118it.bin file, and then first erase the card and re-program it with sas2flash -o -e 6 sas2flash -o -f 2118it.bin Make sure your BIOS is also updated to the latest version. Don't remember if you need -o for the last one but it shouldn't hurt. As an alternative, setting the PCIe slot to Gen 2 in the BIOS also should work.
-
ELI5 How to manage multiple drives with vastly different storage?
The normal unraid way would be to put all HDD's as array with (one of) the biggest ones as parity for protection. Since that is 14TB, you'd have the leftover drives which is 14TB+3x2TB=20TB as storage. You could set up the media as a folder, or as a share, and the photos as well. Of course if you want to change, you'll need to empty one of the 14TB drives and convert the other because you can't/shouldn't use ZFS as a file-system in the array, so you'd need to find space for the files.
-
Which Current Mainstream Supermicro Board?
My understanding is that some of the pins are different between NVMe and SAS/SATA so it shouldn't work. But it is a good find, and now I want to know as well so I'll purchase some of them in a week or so and once I have them I'll let you know.
-
Which Current Mainstream Supermicro Board?
There's cables for SAS/SATA and NVMe. I know cables from 8654-8643 exist and work for SAS/SATA, but 8654-8087 are only for NVMe and as far as I know nobody makes them for SAS/SATA, at least all the ones I've bought that claimed to be for SAS/SATA did not work. There is one company that makes adapters (Chenyang), so Instead, you may be required to buy 3 of 8654-8643 or 8654 8i-8654 4i cables, and then use 6 of 8643-8087 or 8654-8087 adapters (they cost more than the cables). The adapters come as cards you're supposed to insert into slots, but you can just remove the baffle and use it without it. If you want to spend less money, you can also get 9400-16i or 8i and use 8643-8087 cables. Those also come in NVMe and SAS/SATA variants, but at least those work without expensive adapters.
-
7.1.2 - Hard crashes when starting the parity check
No. UEFI/BIOS boots the USB, and from that moment is no longer in control, the linux loader on the USB is, and that can run memtest just fine no matter if it started from BIOS or UEFI.
-
How to upgrade an LSI HBA firmware using Unraid
By the way, for the 9300-16i, if that's the only device installed, you can simplify flashing a lot by using the -fwall option instead of -f so: ./sas3flash -o -fwall 9300-16i_IT.bin -b mpt3sas.rom -b mpt3x64.rom
-
New system
Unraid makes System and Appdata because it uses them, but if they also exist on Cache and they are empty, you can delete them. This probably happened if you started unraid with the drives, but the cache-drives weren't there yet. There is no reason for it to go to BIOS every time, even if there is no OS there will be a message saying there isn't one, and it won't go to BIOS. See if the logs in IPMI show anything. You also don't need a monitor or keyboard attached, you can use IPMI to connect to it and control it remotely from a browser or IPMIview. If the fans are spinning up and down just sometimes it's because of heat, and normal. If they spin up and down all the time, you may be using fans that spin too slow for the motherboard, and you'll have to alter the trigger-amount of RPM's. This can be done using ipmitool.
-
LSI Card not Showing Up
All LSI cards (at least the 9200-series and up) can be flashed with unraid, there are Linux flashers available. To crossflash them, you'll need to use UEFI-shell though. Also verify in the manual of the motherboard that the slots are compatible, for older boards they often only accept graphics cards.
-
Which Current Mainstream Supermicro Board?
You'll need either a bunch of SATA-controllers or easier, a SAS-controller like the 9305-24i or 9600-24i, or use multiple controllers. The board itself has only 4 SATA-ports, and the MCIO ports are only for NVMe which the norco doesn't support.
-
How To Swap CPU with Lower Core Count? Xeon 2680 v4 to 1680 v3
Unraid normally only runs on 1 core, so you shouldn't have that many issues. The single-core speed of the 1680 is a lot faster, so you'll see a big improvement there. The memory-speed is slower, but in reality that doesn't make a lot of difference. Potential problems, the TDP is higher, most boards support up to 145W so it shouldn't be an issue, but make sure to check that. As for swapping, for Unraid, disable autostart of the VM's, and if cores are assigned, make sure they match what the new CPU has (I'd assign a single core to each VM from 1-7 so unraid has 0). You can of course already assign the proper number of vCPUs on unraid 7 but might as well wait to make sure everything works. After that, swap CPU's, and start it up again and it should work, you can then assign cores if you want to and start the VM's and verify they work one at a time. Nothing else should be required.
-
SAS846A + LSI 9750-24i4e
That is a RAID card, and you shouldn't use those for unraid if you can avoid it. It's product page is here with all the manuals, firmware etc.
-
New Motherboard/CPU combo for my setup
If those are the specific model, make sure you have enough airflow, since they have a different design from standard, which means air has to be forced through. Also you'll want to make sure they are all running the same firmware so I would flash firmware, bios and UEFI-bios on all of them, which you can all get here. It is possible your motherboard doesn't have the capability to run all 3 copies of the firmware/bios at once, so consider replacing them with a 9305-24i instead, or 9400 models for a 16i and 8i, or two 16i's.
-
Help Adding 10g Network Card to my Unraid
I'm not sure if it automatically trims, or if the filesystem has a similar function. So I would enable it anyway if it isn't enabled. How often you should let it run, depends on how intensely you re-write the contents, because trim lets it check what space is no longer used and can be erased, so if you have a lot of apps that change things often, you'll want to run it more often, and if you only have things that get read a lot but not changed, you can run it a lot less. Operating systems seem to default to weekly, so that's what I have set it to. Running it more often won't hurt anything, since trim just tells the drive that it can do this operation, but its firmware decides when it is needed to actually do it.