mikeyosm Posted March 18, 2016 Share Posted March 18, 2016 Just as a quick slightly off topic, I have a Samsung 951 256gb on the new beta. On my motherboard it is positioned under 2 gpus. What sort of temps are you guys reading? Mind tends to sit around 50's but I've seen it hit 58'c under heavy load. I know that very hot for a hdd but I think these are safe up to around 70'c anywhere between 50 and 60'c with my 950 pro 512GB. I also added heatsinks on the ram chips to drop the temps a few degrees since the m.2 slot is directly under my passive 750ti. Quote Link to comment
dAigo Posted March 21, 2016 Author Share Posted March 21, 2016 While the 6.2 beta has some issues for me, NVMe support seems solid. Currently running my Intel as a cache drive with VMs. No benchmarks yet, because 6.2 is to buggy. And yes, temps are showing (35°C) but no SMART, which is not unraids fault nvme is not yet supported by smartmontools.... And regarding temps of the 950: It seems at ~70°C it starts throttling, but some thermal pads should do the trick... http://linipc.com/blogs/news/65282629-samsung-950-pro-review-thermal-throttling-solved Quote Link to comment
bluepr0 Posted May 9, 2016 Share Posted May 9, 2016 As an additional, i modified by xml so the nvme image user the following <driver name='qemu' type='raw' cache='none' io='threads'/> it did make things faster. I have attached a crystal disk mark run on my nvme within 2 vm's booted and running. The read speeds aren't as high as possible but are pretty quick. I also added this to the bottom of my go file mkdir /mnt/nvme0 mount /dev/nvme0n1p1 /mnt/nvme0 I have my VM's set to autostart and the go file mounts the nvme on boot for me Regards, Jamie I will definitely try this when I get home I just picked up an Intel 750 400gb to play with. I posted something about tweaks I tried to improove perfomance in THIS thread. In short: I use "cache=unsafe" and "x-data-plane=on". <domain type='kvm' xmlns:qemu='http://libvirt.org/schemas/domain/qemu/1.0'> ... ... <devices> <disk type='file' device='disk'> <driver name='qemu' type='raw' cache='unsafe' io='threads'/> <source file='/mnt/nvme/Rechner/Rechner_vDisk1.img'/> <backingStore/> <target dev='hda' bus='virtio'/> <boot order='1'/> <alias name='virtio-disk0'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x04' function='0x0'/> </disk> ... ... </devices> <qemu:commandline> <qemu:arg value='-set'/> <qemu:arg value='device.virtio-disk0.scsi=off'/> <qemu:arg value='-set'/> <qemu:arg value='device.virtio-disk0.config-wce=off'/> <qemu:arg value='-set'/> <qemu:arg value='device.virtio-disk0.x-data-plane=on'/> </qemu:commandline> </domain> pardon my ignorance.... but from the benchmark image you posted what's the write and read speed I should consider? higher ones seems too high for the supported speed of the device itself (I think it was around 2.5 and 1.2) but lower ones are too slow... I'm confused! Adding your image for reference: Quote Link to comment
dAigo Posted May 10, 2016 Author Share Posted May 10, 2016 Well, its not "to fast" or "to slow", its just the whole system from the view of a VM. 2.5/1.2 are the theoretical/tested values in a "normal" system. Which means the driver of the OS can acces your pci-nvme device directly and that device is directly connected to the cpu with at least 4x PCIe3.0 lanes. In that system, latency and bandwith would usualy not be reduced or enhanced, so the measured results would be your disk. Changing anything will end up with other results. Adding multiple virtualized layers (vDisks, vCPUs, vMEM, etc.) will normaly increase latency and depending on the system reduce bandwidth. In that case, you would end up with ~50% to 60% of the raw power the disk alone would be capable of. As you can see in the "too low" values. On the other Hand, virtualization can add a lot of perfomance. For example changing the vDisks cache mode will result in the "too fast" values. The VM thinks that a block was written, but the VM-Host actually just put it in its RAM and writes it at a later time. So your VM can write to its Disk as fast as the host can put it in its RAM... Great to move big chunks of non critical data around, for example while doing a backup or installing a new system. That is obviously very unsafe in case of a powerfailure (or similar errors), which would result in dataloss an a potential corrupt filesystem If you want what you paid for (~2.4/1.2), you need to go with pci-passthrough. The overhead is very limited and you would achieve 95%+ of your nvme disk. But the disk could only be uses by one VM... Outside of any VM (as a cache device or "unassigned disk") would count as a default config and should give you ~2.4/1.2. As far as I know, VMware is currently the best vendor in keeping that overhead to a minimum, so you could achieve maybe 70%-80% on an ESXi host, but that comes at the cost of compatibility, consumer hardware won't get you far in VMware. Qemu/KvM is getting there and LimeTech is doing a good job of optimizing perfomance/compatibility/usability. My personal opinion is, 60%-70% of the raw speed of NVMe ist still more than enough for any workload and I can share it with many VMs and even unRAID itself. Its still faster than any SATA-SSD would be at 100%. I hope that answers your question, if not be more specific Quote Link to comment
bluepr0 Posted May 11, 2016 Share Posted May 11, 2016 Best explanation ever! Thanks a lot! Quote Link to comment
JustJoshin Posted June 25, 2016 Share Posted June 25, 2016 Just thought I'd chime in as another person interested in adding nvme to their unraid server. I'm looking to purchase the Samsung 950 Pro nvme with a pcie slot card. Quote Link to comment
JorgeB Posted June 25, 2016 Share Posted June 25, 2016 Just thought I'd chime in as another person interested in adding nvme to their unraid server. I'm looking to purchase the Samsung 950 Pro nvme with a pcie slot card. NVMe devices are already supported on v6.2-beta. Quote Link to comment
dAigo Posted June 26, 2016 Author Share Posted June 26, 2016 Just thought I'd chime in as another person interested in adding nvme to their unraid server. I'm looking to purchase the Samsung 950 Pro nvme with a pcie slot card. NVMe devices are already supported on v6.2-beta. Some things, like SMART Infos are still missing in the latest beta (6.2 b23). For critical data, maybe thats something to be considered. Quote Link to comment
JorgeB Posted June 26, 2016 Share Posted June 26, 2016 Just thought I'd chime in as another person interested in adding nvme to their unraid server. I'm looking to purchase the Samsung 950 Pro nvme with a pcie slot card. NVMe devices are already supported on v6.2-beta. Some things, like SMART Infos are still missing in the latest beta (6.2 b23). For critical data, maybe thats something to be considered. True, but that's because smartmontools does not yet support it, only temp info for now, nothing LT can do about that. Quote Link to comment
dAigo Posted June 27, 2016 Author Share Posted June 27, 2016 True, but that's because smartmontools does not yet support it, only temp info for now, nothing LT can do about that. Yes and and no, according to b23 notes, they use 6.5 smarttools, which states: 8 Date 2016-05-07 9 Summary: smartmontools release 6.5 10 ----------------------------------------------------------- 11 - Experimental support for NVMe devices on FreeBSD, Linux and Windows. 12 - smartctl '-i', '-c', '-H' and '-l error': NVMe support. 13 - smartctl '-l nvmelog': New option for NVMe. 14 - smartd.conf '-H', '-l error' and '-W': NVMe support. 15 - Optional NVMe device scanning support on Linux and Windows. 16 - configure option '--with-nvme-devicescan' to include NVMe in 17 default device scanning result. 18 - Device scanning now allows to specify multiple '-d TYPE' options. From what I read (could be wrong), to include nvme, you would need to add "--with-nvme-devicescan" in /etc/smartd.conf. Mine just states "DEVICESCAN" without any parameters. Not sure If anything should/could be done to activate that "experimental support". smartctl -A on nvme actually already has a result. However, format is completly diffrent, probably because "experimental". But I am sure it should be possible to extract some health info and put it into the gui or even create warnings on "error log entrys" or "Spare percentage used". I know that LT is watching the smarttools notes for nvme related stuff, so I am sure its somewhere on their priority list. Maybe quite low at the moment, which is fine for me. Others may think diffrently, but unless they say so, it will just come "eventually" rather than "soon". root@unRAID:~# smartctl -A /dev/nvme0 smartctl 6.5 2016-05-07 r4318 [x86_64-linux-4.4.13-unRAID] (local build) Copyright (C) 2002-16, Bruce Allen, Christian Franke, www.smartmontools.org === START OF SMART DATA SECTION === SMART/Health Information (NVMe Log 0x02, NSID 0xffffffff) Critical Warning: 0x00 Temperature: 36 Celsius Available Spare: 100% Available Spare Threshold: 10% Percentage Used: 0% Data Units Read: 13,711,975 [7.02 TB] Data Units Written: 13,529,802 [6.92 TB] Host Read Commands: 171,734,949 Host Write Commands: 191,229,783 Controller Busy Time: 0 Power Cycles: 129 Power On Hours: 5,441 Unsafe Shutdowns: 0 Media and Data Integrity Errors: 0 Error Information Log Entries: 0 root@unRAID:~# smartctl -A /dev/sdc smartctl 6.5 2016-05-07 r4318 [x86_64-linux-4.4.13-unRAID] (local build) Copyright (C) 2002-16, Bruce Allen, Christian Franke, www.smartmontools.org === START OF READ SMART DATA SECTION === SMART Attributes Data Structure revision number: 1 Vendor Specific SMART Attributes with Thresholds: ID# ATTRIBUTE_NAME FLAG VALUE WORST THRESH TYPE UPDATED WHEN_FAILED RAW_VALUE 5 Reallocated_Sector_Ct 0x0033 100 100 010 Pre-fail Always - 0 9 Power_On_Hours 0x0032 096 096 000 Old_age Always - 16289 12 Power_Cycle_Count 0x0032 097 097 000 Old_age Always - 2336 177 Wear_Leveling_Count 0x0013 091 091 000 Pre-fail Always - 293 179 Used_Rsvd_Blk_Cnt_Tot 0x0013 100 100 010 Pre-fail Always - 0 181 Program_Fail_Cnt_Total 0x0032 100 100 010 Old_age Always - 0 182 Erase_Fail_Count_Total 0x0032 100 100 010 Old_age Always - 0 183 Runtime_Bad_Block 0x0013 100 100 010 Pre-fail Always - 0 187 Uncorrectable_Error_Cnt 0x0032 100 100 000 Old_age Always - 0 190 Airflow_Temperature_Cel 0x0032 069 052 000 Old_age Always - 31 195 ECC_Error_Rate 0x001a 200 200 000 Old_age Always - 0 199 CRC_Error_Count 0x003e 253 253 000 Old_age Always - 0 235 POR_Recovery_Count 0x0012 099 099 000 Old_age Always - 411 241 Total_LBAs_Written 0x0032 099 099 000 Old_age Always - 38543025273 Quote Link to comment
SpaceInvaderOne Posted June 28, 2016 Share Posted June 28, 2016 I am thinking to buy a 256GB Samsung SM951, M.2 (22x80) PCIe 3.0 (x4) AHCI SSD, MLC NAND, Read 2150MB/s, Write 1200MB/s, 90k/70k IOPS and use this to store vm images on. I currently have a sandisk x400 512 ssd 540/520 as cache (were i store my vms and dockers) but was thinking maybe would i be better to buy another x400 and use a raid 0 btrfs cache pool. I know the speeds will not be as high as the samsung drive (but i guess should be faster than a single x400 cache drive) but the plus is i would have more capacity. I am a bit worried to change from my cache as xfs to btrfs. What are your thoughts guys? Quote Link to comment
dAigo Posted June 28, 2016 Author Share Posted June 28, 2016 I am thinking to buy a 256GB Samsung SM951, M.2 (22x80) PCIe 3.0 (x4) AHCI SSD, MLC NAND, Read 2150MB/s, Write 1200MB/s, 90k/70k IOPS and use this to store vm images on. I currently have a sandisk x400 512 ssd 540/520 as cache (were i store my vms and dockers) but was thinking maybe would i be better to buy another x400 and use a raid 0 btrfs cache pool. I know the speeds will not be as high as the samsung drive (but i guess should be faster than a single x400 cache drive) but the plus is i would have more capacity. I am a bit worried to change from my cache as xfs to btrfs. What are your thoughts guys? First of, I can't see any nvme related question, so you are probably in the wrong place ^^ Second, as far as I now, there is currently no raid0 (stripe) cache option, just a pool of disks. One file wont be sperated(striped) between disks, so read/write for one file cant be faster than your fastest drive. If you read/write diffrent files, that happen to be on diffrent disks in the pool, you could achive faster speeds in total. In addition to that, most tasks in your vm are probably 4k random read/write, so your limit would be IOPS rather than bandwith. And in that case, both share the ahci/sata limit of ~80-90k IOPS. If you are concerned about speed, I would recommend one of the disks outside of the array, that removes a software layer that your writes/reads have to go through and therefore should increase your IOPS. I dont see any workload where ONE ahci pci device could use its high bandwith, maybe loading stuff (games/videos) into ram, but even with 4 GB ram, you could reduce filling the ram from ~8s to ~2s... but thats probably unlikely to happen. To get back on topic, samsung released the successor to the oem SM951. Its the SM961 and only available with nvme. (see HERE and HERE) So you might get a good deal for the now obsolete 951, but dont expect nvme-perfomance from an ahci drive. Quote Link to comment
SpaceInvaderOne Posted June 28, 2016 Share Posted June 28, 2016 First of, I can't see any nvme related question, so you are probably in the wrong place ^^ Second, as far as I now, there is currently no raid0 (stripe) cache option, just a pool of disks. One file wont be sperated(striped) between disks, so read/write for one file cant be faster than your fastest drive. If you read/write diffrent files, that happen to be on diffrent disks in the pool, you could achive faster speeds in total. In addition to that, most tasks in your vm are probably 4k random read/write, so your limit would be IOPS rather than bandwith. And in that case, both share the ahci/sata limit of ~80-90k IOPS. If you are concerned about speed, I would recommend one of the disks outside of the array, that removes a software layer that your writes/reads have to go through and therefore should increase your IOPS. I dont see any workload where ONE ahci pci device could use its high bandwith, maybe loading stuff (games/videos) into ram, but even with 4 GB ram, you could reduce filling the ram from ~8s to ~2s... but thats probably unlikely to happen. To get back on topic, samsung released the successor to the oem SM951. Its the SM961 and only available with nvme. (see HERE and HERE) So you might get a good deal for the now obsolete 951, but dont expect nvme-perfomance from an ahci drive. Sorry didnt mean to post in wrong place. Was just considering the sm951, but meant the MZVPV256HDGL-00000 - 256GB Samsung SM951, M.2 (22x80) PCIe 3.0 (x4) NVMe SSD, MLC NAND, Read 2150MB/s, Write 1260MB/s, 300k/100k IOPS but posted wrong spec from other sm951 ahci drive! Then got off topic asking about the btrfs raid 0. I just thought you could set up a raid 0 stripe by using -dconvert=raid0 -mconvert=raid1 in cache balance. So am i right in assuming this only combines the drives into "one" drive with no data protection or striped data? Anyway I think i will take your advise and buy 256GB Samsung SM961 NVMe (as it isnt much more than the sm951 i was looking at) and passthrough partitions of that, to my vms. The reason i wanted the extra speed was not for any one vm but for times when using more than one vm at once i was hoping the extra nvme speed would help with multi vms running at once from the same drive. Quote Link to comment
matthope Posted September 21, 2016 Share Posted September 21, 2016 Hi all, I want to add a NVMe Intel 600P 512gb to my array, since my motherboard support 6 sata + 1 nvme, and I have some question: -With Unraid 6.2 stable, how hard it is to add a nvme cache device? -I want to remove my actual SSD and clone it to my new nvme SSD, is there a way to do that? Because i have VM & dockers on this drive. My build: -Pentium G4400 -Gigabyte B150N -Crucial 8G DDR4 2133 -4x 3TB WD RED -Kingston SSDnow 120gb -Seasonic SSR-350ST -Fractal Design Node 304 Quote Link to comment
saarg Posted September 21, 2016 Share Posted September 21, 2016 Hi all, I want to add a NVMe Intel 600P 512gb to my array, since my motherboard support 6 sata + 1 nvme, and I have some question: -With Unraid 6.2 stable, how hard it is to add a nvme cache device? -I want to remove my actual SSD and clone it to my new nvme SSD, is there a way to do that? Because i have VM & dockers on this drive. My build: -Pentium G4400 -Gigabyte B150N -Crucial 8G DDR4 2133 -4x 3TB WD RED -Kingston SSDnow 120gb -Seasonic SSR-350ST -Fractal Design Node 304 There should be no problem adding the nvme drive as a cache drive in 6.2 without doing anything special. As for cloning, I would add the nvme drive as the second cache disk and let unraid balance them (default raid 1), and then shutdown unraid and remove the old drive. Johnnie.black posted some info about it some time ago. Quote Link to comment
JorgeB Posted September 21, 2016 Share Posted September 21, 2016 Hi all, I want to add a NVMe Intel 600P 512gb to my array, since my motherboard support 6 sata + 1 nvme, and I have some question: -With Unraid 6.2 stable, how hard it is to add a nvme cache device? -I want to remove my actual SSD and clone it to my new nvme SSD, is there a way to do that? Because i have VM & dockers on this drive. My build: -Pentium G4400 -Gigabyte B150N -Crucial 8G DDR4 2133 -4x 3TB WD RED -Kingston SSDnow 120gb -Seasonic SSR-350ST -Fractal Design Node 304 There should be no problem adding the nvme drive as a cache drive in 6.2 without doing anything special. As for cloning, I would add the nvme drive as the second cache disk and let unraid balance them (default raid 1), and then shutdown unraid and remove the old drive. Johnnie.black posted some info about it some time ago. This works but only if current cache disk is BTRFS. Quote Link to comment
matthope Posted September 21, 2016 Share Posted September 21, 2016 Hi all, I want to add a NVMe Intel 600P 512gb to my array, since my motherboard support 6 sata + 1 nvme, and I have some question: -With Unraid 6.2 stable, how hard it is to add a nvme cache device? -I want to remove my actual SSD and clone it to my new nvme SSD, is there a way to do that? Because i have VM & dockers on this drive. My build: -Pentium G4400 -Gigabyte B150N -Crucial 8G DDR4 2133 -4x 3TB WD RED -Kingston SSDnow 120gb -Seasonic SSR-350ST -Fractal Design Node 304 There should be no problem adding the nvme drive as a cache drive in 6.2 without doing anything special. As for cloning, I would add the nvme drive as the second cache disk and let unraid balance them (default raid 1), and then shutdown unraid and remove the old drive. Johnnie.black posted some info about it some time ago. This works but only if current cache disk is BTRFS. Thanks for your fast replies! I've checked and my cache is BTRFS, I will try your solution. Thanks Quote Link to comment
jmbrnt Posted November 30, 2016 Share Posted November 30, 2016 Well, I finally got my new kit. This is an nvme P600 Intel drive, mounted and added to Win 10 directly (not part of the array). These numbers look suspicious to you (write speed seems very slow compared with others)? Quote Link to comment
jmbrnt Posted November 30, 2016 Share Posted November 30, 2016 Sorry to double post.. Oh boy, I changed from a mounted drive to a direct passthrough... Those write speeds are looking a bit more spicy now Quote Link to comment
SpaceInvaderOne Posted November 30, 2016 Share Posted November 30, 2016 Sorry to double post.. Oh boy, I changed from a mounted drive to a direct passthrough... Those write speeds are looking a bit more spicy now I hate to be the one to disappoint your happiness in getting those speeds. The disk speeds you are getting are not right. Programmes like crystal disk etc dont work measuring vm disk speed due to the caching. You could try using a programme like https://sourceforge.net/projects/iometer/ or http://www.iozone.org/ Manufacturer reports your drive specs as read 1775MB/Sec, Write Speed: 560MB/Sec Quote Link to comment
jmbrnt Posted December 1, 2016 Share Posted December 1, 2016 Sorry to double post.. Oh boy, I changed from a mounted drive to a direct passthrough... Those write speeds are looking a bit more spicy now I hate to be the one to disappoint your happiness in getting those speeds. The disk speeds you are getting are not right. Programmes like crystal disk etc dont work measuring vm disk speed due to the caching. You could try using a programme like https://sourceforge.net/projects/iometer/ or http://www.iozone.org/ Manufacturer reports your drive specs as read 1775MB/Sec, Write Speed: 560MB/Sec Haha, I am not quite so deluded to think they were real - in my heart I suspected they were synthetic numbers (I had in the back of my mind 2100MB/s read for the P600, but you are right). I have done some testing with iometer for work, good call, I will run it again at home. In fact, I was doing dd tests (writing to zero) to prove some raw IOPS speed on a RAID, only for both Dell and VMWare to suggest I use iometer instead. Even though my Crystal tests are bogus numbers, there is a definite difference in write speeds now it's passed directly to the VM (rather than passing a mounted partition)... The values aren't right, but the trend is upward - so I can take some happiness from that Quote Link to comment
SpaceInvaderOne Posted December 1, 2016 Share Posted December 1, 2016 Sorry to double post.. Oh boy, I changed from a mounted drive to a direct passthrough... Those write speeds are looking a bit more spicy now I hate to be the one to disappoint your happiness in getting those speeds. The disk speeds you are getting are not right. Programmes like crystal disk etc dont work measuring vm disk speed due to the caching. You could try using a programme like https://sourceforge.net/projects/iometer/ or http://www.iozone.org/ Manufacturer reports your drive specs as read 1775MB/Sec, Write Speed: 560MB/Sec Haha, I am not quite so deluded to think they were real - in my heart I suspected they were synthetic numbers (I had in the back of my mind 2100MB/s read for the P600, but you are right). I have done some testing with iometer for work, good call, I will run it again at home. In fact, I was doing dd tests (writing to zero) to prove some raw IOPS speed on a RAID, only for both Dell and VMWare to suggest I use iometer instead. Even though my Crystal tests are bogus numbers, there is a definite difference in write speeds now it's passed directly to the VM (rather than passing a mounted partition)... The values aren't right, but the trend is upward - so I can take some happiness from that Yes its going in the right direction Sorry if my post sounded rude. Reading it back now I think it sounds a bit that way and I didnt mean it that way at all. Was just late here in the UK when I wrote it so didnt really think Quote Link to comment
jmbrnt Posted December 1, 2016 Share Posted December 1, 2016 Yes its going in the right direction Sorry if my post sounded rude. Reading it back now I think it sounds a bit that way and I didnt mean it that way at all. Was just late here in the UK when I wrote it so didnt really think No, no, didn't come across rude at all - when dealing with mystical numbers it's good to have a dose of reality. I'm also in the UK, but was asleep from assembling my machine till 3am the night before. I'm in love with Unraid Quote Link to comment
hblockx Posted March 9, 2017 Share Posted March 9, 2017 Can it be that <alias name='virtio-disk0'/> is not set-able anymore?! Quote Link to comment
RifleJock Posted July 19, 2017 Share Posted July 19, 2017 I read through this whole post, and found some interesting stuff here. If you care to take a look at my build, I have found many errors with my Intel 750 Series 1.2TB 2.5" U.2 I have the Asus Rampage V Extreme 10 with a E5-2696v4 22c overclocked "base" 4 WD 1TB Velociraptors and 4 Seagate 8TB Archive Drives. Also attached is a GT 530 and GTX 1080ti, Used to also have my GTX 980 in there as well. 128GB of overclocked ram as well... Anyways on to some questions. I have been getting lots of errors, well spam of one error, which you can find in the attached zip below under logs>syslog.1.txt The error loops: Jul 17 09:34:56 DASH kernel: pcieport 0000:00:03.0: AER: Corrected error received: id=0018 Jul 17 09:34:56 DASH kernel: pcieport 0000:00:03.0: PCIe Bus Error: severity=Corrected, type=Data Link Layer, id=0018(Receiver ID) Jul 17 09:34:56 DASH kernel: pcieport 0000:00:03.0: device [8086:6f08] error status/mask=00000080/00002000 Jul 17 09:34:56 DASH kernel: pcieport 0000:00:03.0: [ 7] Bad DLLP I have done research only to find some band-aids for the problem. And thus far, my knowledge of Linux is limited, and I don't quite understand how to follow everyone's suggestions, or at least the ones I try to put together, the ones I find in the vast endless depths called the internet. But I learn a little more about Linux every day. I am curious to find out if that maybe this will be supported in the future, NVMe. To note: I use the drive in a U.2 port built into the mobo, though I also have a m.2 to u.2 asus adapter, and I still get the same errors when plugged in that way. Both M.2 and U.2 on my motherboard use the PCIe 3 lane. According to the internet, its common amoung many Linux distros to find this error when ever a x99 system comes into contact with m.2 or u.2 nvme drives. Other than my logs filling up, I notice High CPU load from the logger. Also note though, the drive still operates as part of my cache pool (though not quite as efficient as I have seen in windows), but I have to restart the machine every day and a half or the webgui will crash. For now it has been unplugged. I hope unRAID has a fix soon as I like expensive hardware and I want to make an incredible unRAID machine. Thanks to those who have read. -Rifle Note again: All mobo and nvme bios's have been updated at the time of this posting. dash-diagnostics-20170718-1049.zip Quote Link to comment
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.