domidomi

Ugh, I'm not sure anymore which ports I've tried, which cables I've tried, but I started getting UDMA CRC issues again, so ultimately I removed the log file and I'm now trying to restart the process with this command: # ddrescue -n -v -f /dev/sdd /dev/sdb /boot/ddrescue.log Hopefully it will actually finish the process this time, and if not, at least I know what to keep track of and hopefully make this process more structured.

I tried it again (with no changes, no reboot), and the following happened: root@istanbul:/dev# ddrescue -f /dev/sdb /dev/sdc /boot/ddrescue.log GNU ddrescue 1.29.1 Press Ctrl-C to interrupt Initial status (read from mapfile) rescued: 2640 GB, tried: 45056 B, bad-sector: 0 B, bad areas: 0 Current status ipos: 2640 GB, non-trimmed: 45056 B, current rate: 0 B/s opos: 2640 GB, non-scraped: 0 B, average rate: 0 B/s non-tried: 15359 GB, bad-sector: 0 B, error rate: 0 B/s rescued: 2640 GB, bad areas: 0, run time: 0s pct rescued: 14.66%, read errors: 0, remaining time: n/a time since last successful read: n/a Copying non-tried blocks... Pass 1 (forwards) ddrescue: /dev/sdc: Write error: No space left on device (errno=28) Looking under /dev/disk/by-id, the disk doesn't even appear, not sure why the disk would drop offline like that, but I shut down the system, re-seated the disk with yet another new cable in a new SATA port, and now I'm running ddrescue in tmux again. It looks like it was able to continue where it left off. Is it "safer" (less chance of corruption) to restart ddrescue from zero again, or does ddrescue have safeguards against corruption when continuing from the log file?

The ddrescue command failed after a while, the replacement disk disappeared somehow. No idea why or what happened. /dev/sdc just disappeared. It doesn't show up in the terminal or in the UnRAID GUI. root@istanbul:~# ddrescue -f /dev/sdb /dev/sdc /boot/ddrescue.log 17:38:50 [162/162] GNU ddrescue 1.29.1 Press Ctrl-C to interrupt ipos: 2640 GB, non-trimmed: 45056 B, current rate: 122 MB/s opos: 2640 GB, non-scraped: 0 B, average rate: 265 MB/s non-tried: 15359 GB, bad-sector: 0 B, error rate: 0 B/s rescued: 2640 GB, bad areas: 0, run time: 2h 45m 55s pct rescued: 14.66%, read errors: 1, remaining time: 21h 27m time since last successful read: n/a Copying non-tried blocks... Pass 1 (forwards) ddrescue: /dev/sdc: Write error: Invalid argument (errno=22) root@istanbul:~# cat /boot/ddrescue.log # Mapfile. Created by GNU ddrescue version 1.29.1 # Command line: ddrescue -f /dev/sdb /dev/sdc /boot/ddrescue.log # Start time: 2025-09-18 09:43:52 # Current time: 2025-09-18 12:29:47 # Copying non-tried blocks... Pass 1 (forwards) # current_pos current_status current_pass 0x266CC6D0000 ? 1 # pos size status 0x00000000 0x46F3CD5000 + 0x46F3CD5000 0x0000B000 * 0x46F3CE0000 0x0ABB0000 ? 0x46FE890000 0x21FCDE40000 + 0x266CC6D0000 0xDF833930000 ? Attaching new diagnostics. istanbul-diagnostics-20250918-1740.zip

Thanks again for your continued support! So if I understand correctly: 1. Install a new disk 2. ddrescue -f /dev/failing /dev/replacement /boot/ddrescue.log 3. Set the replacement drive as the NEW PARITY disk 4. Set the OLD PARITY disk as the NEW DATA disk 5. Rebuild the NEW DATA disk (previously parity disk) from the NEW PARITY disk (replacement disk) Please let me know if this is correct?

It seems to me that contents does NOT look OK, in the sense that files that have been added to the array since August 26th do not appear on the parity disk when mounted using UD. Thanks for your help and please advise me on how to proceed from here? EDIT: I got unsure if I actually BOTH replaced the cables AND the ports as I reported in my initial post - I have now definitely replaced both and I'm running a read-check again. It will take about 18 hours. Still not sure what to do. EDIT 2: Only 2.3% into the read check there's already 77 read errors detected. Not sure what it means for me in this situation. I'm still unsure which of the two disks is actually failing.

Assuming that UD means the "Unassigned Devices" application, should I now stop the array, then unassign the parity disk, and try to mount it using UD? If so, should I start the array (with no parity) before doing anything with UD?

No worries. Sorry, but I don't understand your last sentence. What exactly is UD and how do I check if the parity disk mounts with UD?

Thanks. Just to be clear though, it seems that the data disk (disk 1) shows read errors, doesn't it? Should I still resync parity? Error 1 [0] occurred at disk power-on lifetime: 26868 hours (1119 days + 12 hours) When the command that caused the error occurred, the device was in standby mode. After command completion occurred, registers were: ER -- ST COUNT LBA_48 LH LM LL DV DC -- -- -- == -- == == == -- -- -- -- -- 40 -- 43 00 08 00 00 23 79 e6 a8 40 00 Error: UNC at LBA = 0x2379e6a8 = 595191464 Commands leading to the command that caused the error were: CR FEATR COUNT LBA_48 LH LM LL DV DC Powered_Up_Time Command/Feature_Name -- -- -- == == == -- -- -- -- -- --------------- -------------------- 60 05 40 00 68 00 00 23 79 fe 00 40 00 00:34:33.558 READ FPDMA QUEUED 60 00 80 00 60 00 00 23 79 fd 80 40 00 00:34:31.486 READ FPDMA QUEUED 60 05 40 00 58 00 00 23 79 f8 40 40 00 00:34:31.486 READ FPDMA QUEUED 60 05 40 00 50 00 00 23 79 f3 00 40 00 00:34:31.483 READ FPDMA QUEUED 60 05 40 00 48 00 00 23 79 ed c0 40 00 00:34:31.480 READ FPDMA QUEUED SMART Extended Self-test Log Version: 1 (1 sectors) Num Test_Description Status Remaining LifeTime(hours) LBA_of_first_error # 1 Extended offline Completed: read failure 10% 26952 595191464 # 2 Short offline Completed without error 00% 26868 - # 3 Short offline Completed without error 00% 26509 - # 4 Short offline Completed without error 00% 26429 - # 5 Short offline Completed without error 00% 16895 - # 6 Short offline Completed without error 00% 10394 - # 7 Short offline Completed without error 00% 5341 - # 8 Short offline Completed without error 00% 4876 - # 9 Short offline Interrupted (host reset) 50% 4876 - #10 Short offline Completed without error 00% 235 -

Attaching new diagnostics. Thanks! istanbul-diagnostics-20250917-1229.zip

This all started a week or two ago. I have two disks (one parity, one data), both 18 TB Toshiba drives. First the data drive was disabled and emulated. I switched SATA ports and SATA cables on both drives. Then I rebuilt the data. Now the parity drive is disabled. I'm not sure which disk is having issues, if any. Short SMART tests don't reveal any obvious issues. Attaching diagnostics. Not sure what to do from here? Any help appreciated.istanbul-diagnostics-20250911-1959.zip Unfortunately the diagnostics were created AFTER reboot, because the web GUI started acting strange, like it reloaded the list of disks every 1 second. I couldn't even reboot the system for some reason, I had to shut it off (from the web GUI).

Is this image supposed to work without registering? I never used it before and it just says the version is too old.

Thanks, I'm now using the "test" image tag. Not sure how to verify that it works to be honest!

Is there any chance we can get par2cmdline-turbo into the image? It would greatly improve repair performance. https://sabnzbd.org/wiki/installation/par2cmdline-turbo

I just found /root/keyfile with -rw-r--r-- permissions. Is this normal? Is this file part of the UnRAID system somehow? Should the permissions really be what they are? Seems risky to have a keyfile of some sort which is readable by anyone.

Much appreciated, thanks! Could someone please guide me to the "latest and greatest" regarding macOS + UnRAID SMB configuration? I have a hard time figuring out what the current consensus is.

In many attempts at (re-)configuring SMB for macOS since 6.10 was released, I'm lost in all my SMB settings. In /boot/config/ I now have two files: smb-extra.conf and smb-fruit.conf. Is it safe to just delete these files and then "samba restart"? Will that reset any SMB customizations that I have done?

What is port 9897 used for? I've looked everywhere, but I can't figure out what it's supposed to be for? It's not a default port in Sonarr.

Yes, many Docker containers are indeed accessing files on the array. I'm currently scheduling the Mover to run at a time when the services are unlikely to be significantly used, but I'm still searching for a solution which is less "greedy" for disk bandwidth, if at all possible.

I have a 1 TB cache drive which tends to fill up often. While the Mover is running, all my Docker services become unusably slow. Is this common for everyone else? Is there any way to customize the Mover so that it doesn't choke all other services? I don't imagine running the Mover without affecting the rest of the system at all. I just want it to be less greedy about disk bandwidth so that other services are at least usable. Any ideas? Many thanks!

The reason why that doesn't work is that you're essentially trying to evaluate a shell script twice. 1. The content of ~/.dircolors is a shell script 2. dircolors -b ~/.dircolors will evaluate the script and print the results, which is empty 3. eval "$(dircolors -b ...)" will evaluate the empty string At least that's how I think it is. Just copy /etc/DIR_COLORS to ~/dircolors and work with that instead. It's much easier to read and maintain, and it prevents misunderstandings.

I did some of investigation into xfs_fsr in order to determine if defragging XFS filesystems actually makes any significant difference. Below are my findings. Note that I am no expert on any of this, so take all of this with a grain of salt. My idea was to take the most fragmented file that I could find on a drive and see how long it would take to read the entire file. Then I was going to defragment the file and perform the same read test again and compare the difference in speed. I chose disk1 for this, but I guess it shouldn't matter much. Turns out that, by far, the most fragmented file is the Docker image file with a whopping 15,571 extents. My Docker image file is 20 GB in size. xfs_db gave me the inode number and then I just used "find" to figure out which file it corresponded to. Before defragmenting the file, reading the entire file took 98.6 seconds. After having defragmented the file, it took 82.96 seconds, which is an improvement by about 15%. I performed the experiment on the second-to-most fragmented file, on a different drive, which spanned 97 extents. The speed improvement there was around 10%. I don't know enough about HDD's to know if this could be a result of the drive head having been moved to a better position after the defragmentation or not. It doesn't seem unlikely to me. Some people in this forum have reported that defragmenting their drives improved performance tremendously, but based on what I've found here, I wouldn't put too much trust in those reports. Placebo is a hell of a drug! root@tower:~# xfs_db -r /dev/mapper/md1 -c "frag -v" | sort -k4n | tail -n 2 | head -n 1 inode 2147483777 actual 15571 ideal 1 root@tower:~# find /mnt/disk1/ -inum 2147483777 -printf "%p (%s bytes)\n" /mnt/disk1/system/docker/docker.img (21474836480 bytes) root@tower:~# dd if=/mnt/disk1/system/docker/docker.img of=/dev/null status=progress 21196114432 bytes (21 GB, 20 GiB) copied, 98 s, 216 MB/s 41943040+0 records in 41943040+0 records out 21474836480 bytes (21 GB, 20 GiB) copied, 98.6201 s, 218 MB/s root@tower:~# xfs_fsr -d -v /mnt/disk1/system/docker/docker.img /mnt/disk1/system/docker/docker.img /mnt/disk1/system/docker/docker.img extents=15571 can_save=15570 tmp=/mnt/disk1/system/docker/.fsr1141 DEBUG: fsize=21474836480 blsz_dio=16773120 d_min=512 d_max=2147483136 pgsz=4096 Temporary file has 3 extents (15571 in original) extents before:15571 after:3 /mnt/disk1/system/docker/docker.img root@tower:~# dd if=/mnt/disk1/system/docker/docker.img of=/dev/null status=progress 21222499840 bytes (21 GB, 20 GiB) copied, 82 s, 259 MB/s 41943040+0 records in 41943040+0 records out 21474836480 bytes (21 GB, 20 GiB) copied, 82.9575 s, 259 MB/s

Been a while since you asked, but "before:4709 after:1" means that before defragmenting, the inode spans across 4709 different extents, and afterwards it will occupy just one extent. Think of an extent like a contiguous range of segments on your disk, basically. So with before:4709, your OS would need to read across 4709 different areas to get all parts of that inode.

@mgutt Thanks for the help, much appreciated! If someone else reads this post in the future, I'll add that I balanced the disks so that they all have roughly 20% free space (1.5 TB free of 8 TB total), and it hasn't significantly changed the speed of the defragmentation in general. I will also add that I've tested the speed of all the disks and they all start at 250+ MB/s and end at around 130 MB/s, so that's not the issue either. I'll just be writing this up as caused by the parity drive and disk encryption. I still don't quite understand how the slowest of the disk extents can reach only around 13-15 MB/s, that doesn't really make much sense to me personally. Some extents do perform better, around 70-90 MB/s.

If you mean copying it from disk 1 to disk 1, then it's literally instant. Tried it with a 4.1 GB file. I have noticed that xfs_fsr is running faster on disk 5, which has about 3.5 TB free space. There I get around 40 MB/s write speed. Edit: A few minutes of observation shows that occasionally, on disk 5, it reaches 90 MB/s, then it'll jump down to 35-45 MB/s. It seems like it might depend on the which file it's currently working with?

They're five drives of the same model, Seagate IronWolf 8TB. This particular drive that is currently being defragged has 300+ GB free space.