Glenj48

ok that was genuinely better than I had thought. I had to do some wire cleanup in my lab area and by the time I was done with that I checked my email and boom I had a response and my server is now back up. Still I'm not sure why the limit as the old usb stick gets blacklisted so it's not like you can make 10 usb drives and have a bunch of servers that aren't paid for. Anyway thanks for the quick support response.

So my flashdrive died yet again, I kind of understand the whole making your key tied to a flashdrive to prevent piracy, I don't understand not letting you change your flashdrive automatically more than once a year. They certainly have more confidence in the manufacturers than they should. That being said how long will it take from the time I made a ticket - and hopefully I did that right - for me to get my server back?

I didn't know that store existed. I don't buy much from ebay though.

It is a startech card, here's the data on it. 8 Port SATA PCIe Card - PCI Express 6Gbps SATA Expansion Adapter Card with 4 Host Controllers - SATA PCIe Controller Card - PCI-e x4 Gen 2 to SATA III - SATA HDD/SSDSATA III (6Gbps) | PCI Express Gen 2 | PCIe x4 Slot | 2x Mini-SAS Connectors | 2x Mini-SAS to 4x-SATA Cables IncludedProduct ID: 8P6G-PCIE-SATA-CARD Amazon Rating: (57 Reviews) HIGH-PERFORMANCE: Add eight SATA III 6Gbps ports to your desktop computer or server using a PCIe x4 Gen 2 slot to connect SATA drives (HDD/SSD); 8-Port SATA PCIe Card features 2x Mini-SAS connections with Mini-SAS to 4x locking SATA adapter cables incl. QUAD CONTROLLERS: This PCI Express SATA 6 card features a PCIe MUX and four 2-Port SATA host controllers; This architecture allows for eight total SATA (serial ATA) 6Gbps ports through a single x4 PCIe slot, enabling combined total throughput up to 16Gbps COMPATIBILITY: Supports PCIe Gen 2 & SATA III Specs, and previous generations at lower performance (SATA I/II, PCIe Gen 1); Windows (XP+), macOS (10.6.8+), Linux (2.6.32+); Improved motherboard compatibility with no bifurcation required & no built-in RAID EASY INSTALL: Expansion card w/ plug-and-play setup in full or low-profile PCI-e slot (full-profile bracket installed, low-profile incl); Supports storage management systems such as Storage Spaces (Microsoft) , RAID Assistant (macOS), mdraid/mdadm (Linux) STARTECH.COM ADVANTAGE: IT professionals choice for over 30 years; This SATA PCIe controller card converter is backed for 2-years by StarTech.com, including free 24/5 North America based multi-lingual tech support I had ordered it and have it sitting at the house for when I get home next week. I have been fighting the in theory it should just work and am tired of it so I also ordered a 9207-8i 6Gbs SAS 2308 PCI-E 3.0 HBA IT Mode for ZFS FreeNAS unRAID 2*SFF-8087 SATA Host Bus Adapter from Amazon that will arrive by Monday and will put that one in instead and refund the startech card.

Is this a good choice for an 8 bay system? https://www.amazon.com/dp/B09FY9FBTN?ref_=ppx_hzsearch_conn_dt_b_fed_asin_title_1

I want to switch all disks from zfs to xfs. I currently don't have any free drive bays to accomplish moving data around. I now have 3 new drives coming that will be at the house when I get there next week. At this point my parity drives are invalid due to my previous mistake. This is what I want to do but I need to know if this will work the way I think it will. Can I format 1 of the new larger drives as xfs and manually mount it and one of the array drives and then copy the data onto it from the array drive. Then after that is done format the old drive from xfs to zfs and manually mount it and another of the array drives and repeat the process until all of the drives have been converted to xfs. Then set up the array with the other 2 larger drives as parity and the 6 freshly formatted and copied drives as data drives and reboot to have the new parity drives rebuild from the data drives. If that won't work I guess the next best way would be to start over and add just the 3 new drives to a fresh array and then 1 by 1 copy and add the drives into the new array.

when the system booted it did not autostart the array as the parity drives are considered invalid due to my previous mistakes. Thank you for your help I believe that next week when I get home I will begin the process of getting things back up and running while putting in the larger drives.

Ok, rebooted server. Since the parity drives are showing as needing to be rebuilt system started without starting array. In command line ran zpool import -o readonly=on disk2 and disk mounted read only just fine and pretty instantly. Here are my thoughts, tell me if you think this is a valid path forward. Once I get home next week I will install the 2 larger drives in place of the parity drives and make one of them parity and the other one data. This is the only way I will have a place to copy data from disk 2 I will then remove disk 2 from the drive assignments temporarily and start the array so it can rebuild the parity on the new drive and format the second new empty larger drive. Once that is done I can then mount the current disk 2 and copy it over to the new drive and begin the process of converting my drives to xfs one at a time.

The system is "hung" trying to mount disk 2 when starting the array. I can start the array in maintenance mode where the disks aren't mounted. Can this command run in maintenance mode? Also when I get home a week from today I will have 2 larger drives to put in the system in place of the current parity drives. What I would like to do is switch to a single parity drive - at least for now. I would then like to switch from zfs to xfs a disk at a time. Here is how I think I can go about it but let me run it by you just to be sure. I'm thinking I can add the 2 larger drives in and assign one of them as parity and the other as data. Then I'm thinking the path forward is to remove disk 2 from the drive assignments and start the array and let the new data disk format xfs and the parity drive rebuild from the other drives that are still assigned to the array. Once that is done I can then mount the current disk 2 as an unassigned device and move the data onto the new larger drive and then proceed to one at a time move data from disk to disk reformatting each one as I go. It seems all problems I have had so far has resulted in zfs so I really want to get away from it. If you think this is a valid path forward then I think I'll just leave things alone until next week when I can implement the changes that I want to do. I would really like to do this without having to recreate all of my dockers vms and shares if possible.

There are no options in the gui for disk 2. It is formatted zfs. Here is output from zpool status and zpool import root@Villastromoz:~# zpool status no pools available root@Villastromoz:~# zpool import pool: cache-protected id: 7165692313386357785 state: ONLINE status: Some supported features are not enabled on the pool. (Note that they may be intentionally disabled if the 'compatibility' property is set.) action: The pool can be imported using its name or numeric identifier, though some features will not be available without an explicit 'zpool upgrade'. config: cache-protected ONLINE mirror-0 ONLINE sdb1 ONLINE sdc1 ONLINE pool: disk6 id: 15280495122937660975 state: ONLINE status: Some supported features are not enabled on the pool. (Note that they may be intentionally disabled if the 'compatibility' property is set.) action: The pool can be imported using its name or numeric identifier, though some features will not be available without an explicit 'zpool upgrade'. config: disk6 ONLINE md6p1 ONLINE pool: disk4 id: 5159517906433113695 state: ONLINE status: Some supported features are not enabled on the pool. (Note that they may be intentionally disabled if the 'compatibility' property is set.) action: The pool can be imported using its name or numeric identifier, though some features will not be available without an explicit 'zpool upgrade'. config: disk4 ONLINE md4p1 ONLINE pool: disk3 id: 2051210034590522200 state: ONLINE status: Some supported features are not enabled on the pool. (Note that they may be intentionally disabled if the 'compatibility' property is set.) action: The pool can be imported using its name or numeric identifier, though some features will not be available without an explicit 'zpool upgrade'. config: disk3 ONLINE md3p1 ONLINE pool: disk2 id: 10557574453833155549 state: ONLINE status: Some supported features are not enabled on the pool. (Note that they may be intentionally disabled if the 'compatibility' property is set.) action: The pool can be imported using its name or numeric identifier, though some features will not be available without an explicit 'zpool upgrade'. config: disk2 ONLINE md2p1 ONLINE pool: disk1 id: 3506282359822825917 state: ONLINE status: Some supported features are not enabled on the pool. (Note that they may be intentionally disabled if the 'compatibility' property is set.) action: The pool can be imported using its name or numeric identifier, though some features will not be available without an explicit 'zpool upgrade'. config: disk1 ONLINE md1p1 ONLINE Here is the log files from before and after my messing up the flash drive. The one from yesterday is before, and the one from today is after I messed with the flash drive. villastromoz-diagnostics-20250629-1820.zipvillastromoz-diagnostics-20250628-1505.zip

Yesterday I noticed that some dockers weren't accessible, so I decided to reboot the server. It hung on mounting disk 2 all day. I ran some pool status commands and there appeared to be nothing wrong with the drive. In frustration I decided to roll back to version 6.12.15 as the array was much faster to start and stop before version 7 for some reason. This did nothing to change the system getting hung up on mounting disk 2. I then decided to go back to the latest version so I copied the files back into the root of the flash drive and tried to reboot. I started getting checksum errors from the boot files and couldn't boot so I had my son in law put the flash drive in the kvm that I have attached so I could rebuild it that way. This is where I took a very dumb wrong turn. When downgrading the system it overwrote some of the config and so I decided to use the latest flash backup I had available. Unfortunately I had a couple of drives that I changed out since that backup. After realizing my mistake I found the data recovery page that says don't use an old backup but it was too late. I decided that the best course of action was to delete the config/super.dat file and reassign the drives the way I had them before all this started. Things were looking like it would be ok as it showed that it would rebuild the 2 parity drives once the array started. Rebooted server and still getting stuck mounting disk 2 for some reason. At this point here is what I want to do and am looking for advice on how to proceed instead of just blindly running ahead and making things worse. My current setup is 8 12 tb drives set up as 2 parity and 6 data. I ordered 2 20 tb drives and want switch out 2 of the 12 tb drives with them. I am thinking 1 parity and 7 data drives. My thoughts at this point is that when I get home next week and can do the changes I will tell unraid to do a new configuration and only add the 2 20 tb drives to the array. I will then mount disk 1 and move the data from it to the 1st 20 tb disk. Once that is done add that disk into the array and repeat until I have the entire array back to where it should be. That's the only way I can think to get around disk 2 not mounting so the array can start and rebuild the parity drives.

That fixed it.

I can no longer update or install new containers from ghcr.io. Other containers show the version and update just fine. Any ideas what could be wrong?

One last question, I opted to use Krusader to copy the data to 2 usb drives. After I format drive should I use Krusader to move the data back to disk 5 directly or should I just move it into the shares and let the server decide where to put it?

Using unbalanced to scatter the data to other discs thank you.

root@Villastromoz:~# zpool status pool: cache-protected state: ONLINE status: Some supported and requested features are not enabled on the pool. The pool can still be used, but some features are unavailable. action: Enable all features using 'zpool upgrade'. Once this is done, the pool may no longer be accessible by software that does not support the features. See zpool-features(7) for details. scan: resilvered 46.3G in 02:16:09 with 0 errors on Fri Jan 10 22:48:18 2025 config: NAME STATE READ WRITE CKSUM cache-protected ONLINE 0 0 0 mirror-0 ONLINE 0 0 0 sdh1 ONLINE 0 0 0 sdi1 ONLINE 0 0 0 errors: No known data errors pool: disk1 state: ONLINE status: Some supported and requested features are not enabled on the pool. The pool can still be used, but some features are unavailable. action: Enable all features using 'zpool upgrade'. Once this is done, the pool may no longer be accessible by software that does not support the features. See zpool-features(7) for details. config: NAME STATE READ WRITE CKSUM disk1 ONLINE 0 0 0 md1p1 ONLINE 0 0 0 errors: No known data errors pool: disk2 state: ONLINE status: Some supported and requested features are not enabled on the pool. The pool can still be used, but some features are unavailable. action: Enable all features using 'zpool upgrade'. Once this is done, the pool may no longer be accessible by software that does not support the features. See zpool-features(7) for details. config: NAME STATE READ WRITE CKSUM disk2 ONLINE 0 0 0 md2p1 ONLINE 0 0 0 errors: No known data errors pool: disk3 state: ONLINE status: Some supported and requested features are not enabled on the pool. The pool can still be used, but some features are unavailable. action: Enable all features using 'zpool upgrade'. Once this is done, the pool may no longer be accessible by software that does not support the features. See zpool-features(7) for details. scan: scrub repaired 0B in 17:11:02 with 0 errors on Sat Jan 25 17:11:03 2025 config: NAME STATE READ WRITE CKSUM disk3 ONLINE 0 0 0 md3p1 ONLINE 0 0 0 errors: No known data errors pool: disk4 state: ONLINE status: Some supported and requested features are not enabled on the pool. The pool can still be used, but some features are unavailable. action: Enable all features using 'zpool upgrade'. Once this is done, the pool may no longer be accessible by software that does not support the features. See zpool-features(7) for details. scan: scrub repaired 0B in 13:29:09 with 0 errors on Sun Jan 26 13:29:11 2025 config: NAME STATE READ WRITE CKSUM disk4 ONLINE 0 0 0 md4p1 ONLINE 0 0 0 errors: No known data errors pool: disk5 state: ONLINE status: One or more devices has experienced an error resulting in data corruption. Applications may be affected. action: Restore the file in question if possible. Otherwise restore the entire pool from backup. see: https://openzfs.github.io/openzfs-docs/msg/ZFS-8000-8A config: NAME STATE READ WRITE CKSUM disk5 ONLINE 0 0 0 md5p1 ONLINE 0 0 16 errors: 2 data errors, use '-v' for a list pool: disk6 state: ONLINE status: Some supported and requested features are not enabled on the pool. The pool can still be used, but some features are unavailable. action: Enable all features using 'zpool upgrade'. Once this is done, the pool may no longer be accessible by software that does not support the features. See zpool-features(7) for details. config: NAME STATE READ WRITE CKSUM disk6 ONLINE 0 0 0 md6p1 ONLINE 0 0 0 errors: No known data errors root@Villastromoz:~# zpool status -v pool: cache-protected state: ONLINE status: Some supported and requested features are not enabled on the pool. The pool can still be used, but some features are unavailable. action: Enable all features using 'zpool upgrade'. Once this is done, the pool may no longer be accessible by software that does not support the features. See zpool-features(7) for details. scan: resilvered 46.3G in 02:16:09 with 0 errors on Fri Jan 10 22:48:18 2025 config: NAME STATE READ WRITE CKSUM cache-protected ONLINE 0 0 0 mirror-0 ONLINE 0 0 0 sdh1 ONLINE 0 0 0 sdi1 ONLINE 0 0 0 errors: No known data errors pool: disk1 state: ONLINE status: Some supported and requested features are not enabled on the pool. The pool can still be used, but some features are unavailable. action: Enable all features using 'zpool upgrade'. Once this is done, the pool may no longer be accessible by software that does not support the features. See zpool-features(7) for details. config: NAME STATE READ WRITE CKSUM disk1 ONLINE 0 0 0 md1p1 ONLINE 0 0 0 errors: No known data errors pool: disk2 state: ONLINE status: Some supported and requested features are not enabled on the pool. The pool can still be used, but some features are unavailable. action: Enable all features using 'zpool upgrade'. Once this is done, the pool may no longer be accessible by software that does not support the features. See zpool-features(7) for details. config: NAME STATE READ WRITE CKSUM disk2 ONLINE 0 0 0 md2p1 ONLINE 0 0 0 errors: No known data errors pool: disk3 state: ONLINE status: Some supported and requested features are not enabled on the pool. The pool can still be used, but some features are unavailable. action: Enable all features using 'zpool upgrade'. Once this is done, the pool may no longer be accessible by software that does not support the features. See zpool-features(7) for details. scan: scrub repaired 0B in 17:11:02 with 0 errors on Sat Jan 25 17:11:03 2025 config: NAME STATE READ WRITE CKSUM disk3 ONLINE 0 0 0 md3p1 ONLINE 0 0 0 errors: No known data errors pool: disk4 state: ONLINE status: Some supported and requested features are not enabled on the pool. The pool can still be used, but some features are unavailable. action: Enable all features using 'zpool upgrade'. Once this is done, the pool may no longer be accessible by software that does not support the features. See zpool-features(7) for details. scan: scrub repaired 0B in 13:29:09 with 0 errors on Sun Jan 26 13:29:11 2025 config: NAME STATE READ WRITE CKSUM disk4 ONLINE 0 0 0 md4p1 ONLINE 0 0 0 errors: No known data errors pool: disk5 state: ONLINE status: One or more devices has experienced an error resulting in data corruption. Applications may be affected. action: Restore the file in question if possible. Otherwise restore the entire pool from backup. see: https://openzfs.github.io/openzfs-docs/msg/ZFS-8000-8A config: NAME STATE READ WRITE CKSUM disk5 ONLINE 0 0 0 md5p1 ONLINE 0 0 16 errors: Permanent errors have been detected in the following files: disk5/data:<0x0> pool: disk6 state: ONLINE status: Some supported and requested features are not enabled on the pool. The pool can still be used, but some features are unavailable. action: Enable all features using 'zpool upgrade'. Once this is done, the pool may no longer be accessible by software that does not support the features. See zpool-features(7) for details. config: NAME STATE READ WRITE CKSUM disk6 ONLINE 0 0 0 md6p1 ONLINE 0 0 0 errors: No known data errors root@Villastromoz:~#

That command ran with no output. I ran it a second time as the disk still shows unmountable and got the following root@Villastromoz:~# zpool import -o readonly=on disk5 cannot import 'disk5': a pool with that name already exists use the form 'zpool import <pool | id> <newpool>' to give it a new name root@Villastromoz:~#

cannot import 'disk5': insufficient replicas Destroy and re-create the pool from a backup source.

pool: disk5 id: 5035721911779965809 state: ONLINE status: Some supported features are not enabled on the pool. (Note that they may be intentionally disabled if the 'compatibility' property is set.) action: The pool can be imported using its name or numeric identifier, though some features will not be available without an explicit 'zpool upgrade'. config: disk5 ONLINE md5p1 ONLINE

villastromoz-diagnostics-20250124-1637.zipHere it is

I have a zfs disk that is unmountable in my array, I can't find anything on how to recover from this. Is there any way back? I'm starting to regret using zfs file system on my array drives.