geeksheikh

Gonna do some testing with the SAS connections. Perhaps I have a bad cable or rail or setup. Will double check. Here's a pretty telling story

great! Thanks for that clarification. That would have been a bad check box to miss. The rest of the process looks like best option overall given that I don't want to zero all the drives first?

Hi, I'm removing 3, 8TB drives from my array. I know two ways to do this and I'm planning to take the dangerous approach but am thinking I have a method to reduce the risk, looking for process confirmation or better options Why go the dangerous route? I don't want to wait 3+ days for my 3 8TB drives to zero out. I also understand that my docker and vm services should be offline during these zero operations which means 3+ days of down time for the server. I'm guessing that as long as I know no services would write to these disks during the zero, I wouldn't actually need to take the services down...I think.... I have 2 parity drives and 18 data disks totaling ~120TB in btrfs Shrink Process Move all data off of 3 data disks to other array disks Generate reports of all files on all data disks and save. In event of data loss, I'd at least know what was lost Offsite backups of all critical data Remove 3 empty data disks from array Unassign Parity 2 ?? This is my primary question -- if data disk fails during parity rebuild on parity1, can parity 2 be swapped back in with the original 3 data disks and new config with same config as before, check "parity is valid", rebuild failed data disk, get stable, restart process. Does this work? New Config Rebuild Parity only on Parity 1 disk Add parity 2 back, rebuild parity on parity 2 Remaining risks If parity1 disk fails during parity 2 rebuild, I would be left without a parity for a bit until parity 2 rebuild finishes Mitigation: Smart reports prior to rebuilds to minimize risk. Would an extended self test prior help or is SMART sufficient? Mitigation: Parity drives are NAS drives that should have a very low fail rate

Thanks. Yeah I just spun up 4 ssds in a zfs raid 0 and rsync’d a 175GB file. Average speed was 22.9MB/s. crazy. Something seems wrong. The ssds are connected via E-Sas using the card below. This goes to a sas controller in an external enclosure over 2ft esas cables Card https://a.co/d/b7Y8gwM External enclosure, inside of another box. https://a.co/d/fOXyRY4

Hi, I have several ssds that are to be added to an ssd pool. I did a perf test on them to see if I had any slow disks...and uhhh...it seems I have some VERY slow ssds. They start off writes around 500MB/s+ but then get down to 20MB/s or lower after a bit. I realize there is some slowdown expected but this is very slow and it's not across all disks. Some of my oldest, cheapest disks have no issue sustaining writes of over 100MB/s, any ideas? Attached the full output and the script used to test the devices, along with a screenshot of the device IDs and models so you can see which drives are so slow. Thanks for your thoughts. ssd_perf_test.sh perf_results.txt

I created a new vm and pointed to the new (and the old) qcow vdisks and both worked fine with a new VM. The vm edit seems to be causing the breakage. Not sure why.

In preparation for a cache pool change I used rsync to copy the vdisk to another drive. Edited the VM to point to the new vdisk, won't boot. Edited VM to point back to original vdisk, won't boot. It booted fine prior to editing the vm config. Any ideas? How can I fix this? Thanks rsync command -- not the issue because even the original vdisk is now not working either. rsync -avh --progress --partial --inplace --sparse /mnt/ssdpool/VMOS/Monica/ /mnt/cache/tmp/ssd_pool_snaps_202406/VMOS_snapshot_20240613/Monica/ monica_vm.xml

confirming issue on 6.12.10 as well. libvert wouldn't mount, read this thread so I stopped VM service, stopped docker service, started vm service and it worked fine. Started docker service back and docker loaded fine as well. No idea why this works or how the docker service is tied into vm service (maybe networking...?) but this is how I got it to work.

ok, so if I take this opportunity to rebuild the pool in the "recommended way" what is that recommended way if I want encryption? The data stored on this pool is not critical if there are failures as all the critical data is regularly backed up to the array...if there's a safe, recommended way to create an encrypted ssd drive pool I'd love to implement it.

that makes sense, honestly -- I forgot it was RAID5 until you asked the question. I'm trying to replace 2 drives with 1 larger drive. Those two vertex drives out and one 2tb ssd in. I get the replacing of one drive at a time but how to rebuild it after I permanently remove the second drive? Thanks again for your help.

It's in RAID5 config. I used btrfs because I wanted the contents encrypted.

Hello, how can I access data (mount?) on a single disk within a pool without removing them from the pool or stopping the array? I have an "ssdpool" which is a secondary pool to my primary. I have two old, nasty disks and I wanted to replaced them. I wanted to first mount them and move the contents off so that when I insert the replacement I can move the data back. I was hoping to be able to browse the data on these specific devices while the array was still on but I cannot figure out how to do this.

I have a high-iops pool formed by a bunch of heterogenous ssds (different brands, sizes, etc). This is an awesome idea but my write performance is only about 18MB/s (using fio test) but my read speed is > 680MB/s. I am wondering if I have a specific disk that is going bad or has some very slow speeds. How can I test read/write performance to each specific ssd? I'd prefer not to dismantle the pool. Is there some way to use the /dev/sdag or something to perform a read / write test? Thank you. ``` fio --name=randwrite --ioengine=libaio --iodepth=16 --rw=randwrite --bs=4k --direct=1 --size=512M --numjobs=4 --runtime=60 --group_reporting --directory=/mnt/ssdpool ```

yeah I already restored and it worked fine. This is more just about my curiosity at this point. I learned what I didn't know, hence the questions.

Yeah when the server crashed some of the files got corrupted / deleted. I didn't know super.dat had it, I think disk.cfg used to have it. I looked and looked for it. Is there any way to open super.dat?