ra_bliss

Do you think a professional service would be able to get everything modified in the last 6 months? Might not be worth it even if they could. I'll have to think if I have any super important things from last 6 months. Also, I am going to restart and try ddrescue one more time just in case, maybe do some more reading on ddrescue options first.

I need to try to rescue failed 4TB HDD and copy it onto an 8TB HDD. I ssh'ed into Unraid, triple-checked the /dev/sdX and /dev/sdY source and destination devices, and ran root@Tower:~# ddrescue -f /dev/sdf /dev/sdb /boot/ddrescue.log per Now 24hrs later, the "Fix Common Problems" CA is complaining: "/var/log is getting full (currently 100 % used)" and recommends posting diagnostics, which I will attach. Also , ps -aux does not show any ddrescue command process running, but "sudo ps -a" shows it. root@Tower:~# sudo ps -a PID TTY TIME CMD 8633 pts/0 00:00:00 sudo 8634 pts/0 00:00:00 ps 23977 pts/1 00:01:12 ddrescue ------------------------------------- root@Tower:~# sudo ps 23977 PID TTY STAT TIME COMMAND 23977 pts/1 D+ 1:12 ddrescue -f /dev/sdf /dev/sdb /boot/ddrescue.log ------------------------------------- root@Tower:~# ps gv PID TTY STAT TIME MAJFL TRS DRS RSS %MEM COMMAND ... 23977 pts/1 D+ 1:12 0 81 5970 1952 0.0 ddrescue -f /dev/sdf /dev/sdb /boot/ddrescue.log What's happening? Should I continue waiting? Clear var/log? Any recommendations? I really appreciate any help, ideas, or insight :-) Thanks ------------------------------------------------------------------------------ ddrescue through NerdTools PlugIn was installed and ran through ssh: ~:ra_bliss$ ssh unraid Linux 5.19.14-Unraid. root@Tower:~# ddrescue -f /dev/sdf /dev/sdb /boot/ddrescue.log GNU ddrescue 1.26 Press Ctrl-C to interrupt ipos: 0 B, non-trimmed: 0 B, current rate: 0 B/s opos: 0 B, non-scraped: 0 B, average rate: 0 B/s non-tried: 4000 GB, bad-sector: 0 B, error rate: 0 B/s ipos: 10211 MB, non-trimmed: 524288 B, current rate: 0 B/s opos: 10211 MB, non-scraped: 0 B, average rate: 0 B/s non-tried: 4000 GB, bad-sector: 0 B, error rate: 524 kB/s ipos: 521058 MB, non-trimmed: 968704 B, current rate: 0 B/s opos: 521058 MB, non-scraped: 0 B, average rate: 0 B/s non-tried: 4000 GB, bad-sector: 0 B, error rate: 444 kB/s ipos: 1081 GB, non-trimmed: 1284 kB, current rate: 0 B/s opos: 1081 GB, non-scraped: 0 B, average rate: 0 B/s non-tried: 4000 GB, bad-sector: 0 B, error rate: 315 kB/s ipos: 3961 GB, non-trimmed: 2928 kB, current rate: 0 B/s opos: 3961 GB, non-scraped: 0 B, average rate: 0 B/s non-tried: 4000 GB, bad-sector: 0 B, error rate: 315 kB/s rescued: 0 B, bad areas: 0, run time: 8s pct rescued: 0.00%, read errors: 109, remaining time: n/a time since last successful read: n/a Copying non-tried blocks... Pass 1 (forwards) ipos: 40042 kB, non-trimmed: 7823 kB, current rate: 0 B/s opos: 40042 kB, non-scraped: 0 B, average rate: 0 B/s non-tried: 4000 GB, bad-sector: 0 B, error rate: 602 kB/s rescued: 0 B, bad areas: 0, run time: 15s pct rescued: 0.00%, read errors: 218, remaining time: n/a time since last successful read: n/a Copying non-tried blocks... Pass 2 (backwards) ipos: 67112 MB, non-trimmed: 67118 MB, current rate: 0 B/s opos: 67112 MB, non-scraped: 0 B, average rate: 0 B/s non-tried: 3933 GB, bad-sector: 0 B, error rate: 917 kB/s rescued: 0 B, bad areas: 0, run time: 20h 30m 53s pct rescued: 0.00%, read errors: 1024248, remaining time: n/a time since last successful read: n/a Copying non-tried blocks... Pass 5 (forwards) and it has been like this for about 24 hrs, when I started the command. What do? ----------------- SMART report of failed drive. I go to Main Dev 1 -> Self-Test - > Download SMART Report and click Download and it gives me this txt below the "---": --- smartctl 7.3 2022-02-28 r5338 [x86_64-linux-5.19.14-Unraid] (local build) Copyright (C) 2002-22, Bruce Allen, Christian Franke, www.smartmontools.org === START OF INFORMATION SECTION === Device Model: ST4000DM005 Serial Number: ZDH0ZL81 LU WWN Device Id: 5 000c50 0a2390001 Firmware Version: 0001 User Capacity: 4,000,787,030,016 bytes [4.00 TB] Sector Size: 512 bytes logical/physical Rotation Rate: 7200 rpm Device is: Not in smartctl database 7.3/5405 ATA Version is: ATA8-ACS T13/1699-D revision 4 SATA Version is: SATA 3.0, 6.0 Gb/s Local Time is: Sat Dec 3 07:29:36 2022 PST SMART support is: Available - device has SMART capability. SMART support is: Enabled AAM feature is: Unavailable APM feature is: Unavailable Rd look-ahead is: Enabled Write cache is: Enabled DSN feature is: Unavailable ATA Security is: Disabled, NOT FROZEN [SEC1] Read SCT Status failed: scsi error badly formed scsi parameters Wt Cache Reorder: Unknown (SCT Feature Control command failed) Read SMART Data failed: scsi error badly formed scsi parameters === START OF READ SMART DATA SECTION === SMART Status command failed: scsi error badly formed scsi parameters SMART overall-health self-assessment test result: UNKNOWN! SMART Status, Attributes and Thresholds cannot be read. Read SMART Log Directory failed: scsi error badly formed scsi parameters ATA_READ_LOG_EXT (addr=0x00:0x00, page=0, n=1) failed: scsi error badly formed scsi parameters Read GP Log Directory failed SMART Extended Comprehensive Error Log (GP Log 0x03) not supported Read SMART Error Log failed: scsi error badly formed scsi parameters SMART Extended Self-test Log (GP Log 0x07) not supported Read SMART Self-test Log failed: scsi error badly formed scsi parameters Selective Self-tests/Logging not supported Read SCT Status failed: scsi error badly formed scsi parameters Read SCT Status failed: scsi error badly formed scsi parameters SCT (Get) Error Recovery Control command failed Device Statistics (GP/SMART Log 0x04) not supported Pending Defects log (GP Log 0x0c) not supported ATA_READ_LOG_EXT (addr=0x11:0x00, page=0, n=1) failed: scsi error badly formed scsi parameters Read SATA Phy Event Counters failed ------------- Fix Common Problems CA tower diagnostics tower-diagnostics-20221208-0734.zip

Still waiting on the Pre-Clear. But I remembered something that might be related, but also maybe not. I think this error window popped up the first time I plugged the failing drive back into my hackintosh and disconnecting it to work on the boot drive. Does anyone know what this means? Can anyone tell me if it's possible their related problems? Thanks " Legacy System Extension Existing software on your system loaded a system extension signed by "EDSS GLOBAL COMPANY LIMITED" which will be incompatible with a future version of macOS. Contact the developer for support. "

Ok great. I am pre-clearing the new 8tb first to see if it breaks, then I will just do a basic ddrescue -f /dev/sdX_*source* /dev/sdY_*dest* /boot/ddrescue.log Wish Me Luck ALSO Are there any checksums or data recovery tools that are especially good for APFS and HFS+(Journaled)? I want to maximize the data recovered. Are there any extra tricks or steps after, like giving it a way to be able to figure out bad sectors with checksums or anything?

Would ddrescue automatically format the destination drive the same FS as the source drive?

I'll give it a go because I got a replacement drive. I could copy it onto that. Might there be a way to determine precisely what file system the broken drive is? That way, I can format the new drive, so it's the same. Or should I always go for APFS if I am above only using macOS > v11. (Monterey or Ventura)? Thanks

I have had a 4TB on my main desktop Hackintosh for 5-10 yrs that has failed while I updated the macOS from Mojave to Monterey. I unplugged the drive until I got the boot drive working well and then tried to hook it back up. I got a message that was something like "Disk is not readable, unable to initialize." Luckily I mounted it on Unraid and backed it up a few months ago. Do you know if I can repair it or recover the data? --- I can't quite remember, but I think it is formatted as HFS+ (Journalled), or APFS. I am a noobie with Unraid, but I have looked at it in "scrutiny" and Tools->UnraidOS->Dianostics & System Devices In Main (there is no option to mount this disk, only an option to format). I have the feeling that there are a lot more tools, apps, and plugins that might be able to help. Is there any possibility to get the drive working again or to recover the data from these last few months? Thanks --- I go to Main Dev 1 -> Self-Test - > Download SMART Report and click Download and it gives me this txt below the "---": --- smartctl 7.3 2022-02-28 r5338 [x86_64-linux-5.19.14-Unraid] (local build) Copyright (C) 2002-22, Bruce Allen, Christian Franke, www.smartmontools.org === START OF INFORMATION SECTION === Device Model: ST4000DM005 Serial Number: ZDH0ZL81 LU WWN Device Id: 5 000c50 0a2390001 Firmware Version: 0001 User Capacity: 4,000,787,030,016 bytes [4.00 TB] Sector Size: 512 bytes logical/physical Rotation Rate: 7200 rpm Device is: Not in smartctl database 7.3/5405 ATA Version is: ATA8-ACS T13/1699-D revision 4 SATA Version is: SATA 3.0, 6.0 Gb/s Local Time is: Sat Dec 3 07:29:36 2022 PST SMART support is: Available - device has SMART capability. SMART support is: Enabled AAM feature is: Unavailable APM feature is: Unavailable Rd look-ahead is: Enabled Write cache is: Enabled DSN feature is: Unavailable ATA Security is: Disabled, NOT FROZEN [SEC1] Read SCT Status failed: scsi error badly formed scsi parameters Wt Cache Reorder: Unknown (SCT Feature Control command failed) Read SMART Data failed: scsi error badly formed scsi parameters === START OF READ SMART DATA SECTION === SMART Status command failed: scsi error badly formed scsi parameters SMART overall-health self-assessment test result: UNKNOWN! SMART Status, Attributes and Thresholds cannot be read. Read SMART Log Directory failed: scsi error badly formed scsi parameters ATA_READ_LOG_EXT (addr=0x00:0x00, page=0, n=1) failed: scsi error badly formed scsi parameters Read GP Log Directory failed SMART Extended Comprehensive Error Log (GP Log 0x03) not supported Read SMART Error Log failed: scsi error badly formed scsi parameters SMART Extended Self-test Log (GP Log 0x07) not supported Read SMART Self-test Log failed: scsi error badly formed scsi parameters Selective Self-tests/Logging not supported Read SCT Status failed: scsi error badly formed scsi parameters Read SCT Status failed: scsi error badly formed scsi parameters SCT (Get) Error Recovery Control command failed Device Statistics (GP/SMART Log 0x04) not supported Pending Defects log (GP Log 0x0c) not supported ATA_READ_LOG_EXT (addr=0x11:0x00, page=0, n=1) failed: scsi error badly formed scsi parameters Read SATA Phy Event Counters failed

To give you an idea of my experience level. I am fairly experienced with bash and have a BS in CS, but had never done anything with docker before trying out Unraid. I don't have any experience setting up networks, much less a docker network. I have been following the tutorial videos up until I ran into this problem I thought I could handle on my own. I got Nextcloud set up through Cloudflare with a DDNS with one of the tuts. Everything seemed to be working fine for a while, but then last week I decided to go into the Nextcloud WebGUI and click on the update button in the Nextcloud WebGUI, but then it ran in the problems. At one point I could see in the webGUI a checklist of how where it was in the process of updating, but then it got about half way down and said it had a problem and I don't remember exactly what it said, but it might have been about Cloudflare. Anyway, since then all I get when I try to open the gui is "Update in Process." in text on a blank page. Also if I go to the site (nextcloud.blahblahblah.com) I had it going through with Cloudflare I get the same "Update in Process.". I tried uninstalling the docker and then reinstalling it, but nothing changed. Help please? I What the next steps I should take to see what is going on?

So I ls one folder and compared it to the corresponding folder in the rsync error output. It seems that when i'm in the source drive and go to these vanished files, ls is saying it cannot access these files and that there is no such directory. But if it can't access them and there are no such files, why is it printing the "/bin/ls: cannot access" lines? I have checked every line and these two lists list the same files. 1 /bin/ls: cannot access '1087_The 4th dimension ??_HenWad_ffo0xw.mp4': No suc:h file or directory 2 /bin/ls: cannot access '1097_Some more of my doodles in my first doodlebook but fast paced in a train ??_elfinnfinite_dsxjov.mp4': No such file or directory 3 /bin/ls: cannot access '1137_Hammock, blanket, and some mushrooms. It’s a gorgeous day to die ??_ItsAGorgeouDayToDie_edr1os.jpg': No such file or directory 4 /bin/ls: cannot access '1147_??_Sacra_profanis_dppzwu.png': No such file or directory 5 /bin/ls: cannot access '1165_Leave a like or a comment! It helps me more than you know, love you all, stay strong ❤??_elfinnfinite_fqh3ac.jpg': No such file or directory 6 /bin/ls: cannot access '1168_currently tripping and I just saw this, good vibes everyone ??_the-heba_de9onw.jpg': No such file or directory 7 /bin/ls: cannot access '1232_Tripping with my boyfriend in our new house ??_rbigler22_d4dlj3.jpg': No such file or directory 8 /bin/ls: cannot access '1927_Found this on Facebook. It just spoke volumes ??_Trip_And_tell_hmu22m.jpg': No such file or directory 9 /bin/ls: cannot access '2096_Trip spot for the evening. 1.4grams cubensis and Handel’s Messiah with the San Francisco Symphony Orchestra. Feeling glorious reddit ❤️❤️???? ?? ?? ☮️_california_king_eau66k.jpg': No such file or directory 10 /bin/ls: cannot access '800_my grinder under my rainbow lights ??_DELETED_ejmqit.mp4': No such file or directory 11 /bin/ls: cannot access '814_Another one of these vids ?? And yes I know about the runs, it was from the way I twaxed it ??_chiliwilly001_izi1mw.mp4': No such file or directory 12 /bin/ls: cannot access '832_?? the cosmic sensation._DELETED_i9nxe7.jpg': No such file or directory 13 /bin/ls: cannot access '860_?? @luke_schroeder_art_BennyHG_jabfxj_1.jpg': No such file or directory 14 /bin/ls: cannot access '860_?? @luke_schroeder_art_BennyHG_jabfxj_2.jpg': No such file or directory 15 /bin/ls: cannot access '860_?? @luke_schroeder_art_BennyHG_jabfxj_3.jpg': No such file or directory 16 /bin/ls: cannot access '860_?? @luke_schroeder_art_BennyHG_jabfxj_4.jpg': No such file or directory 17 /bin/ls: cannot access '860_?? @luke_schroeder_art_BennyHG_jabfxj_5.jpg': No such file or directory 18 /bin/ls: cannot access '860_?? @luke_schroeder_art_BennyHG_jabfxj_6.jpg': No such file or directory 19 /bin/ls: cannot access '860_?? @luke_schroeder_art_BennyHG_jabfxj_7.jpg': No such file or directory 20 /bin/ls: cannot access '876_??_Nerihd522_m0ihn7.jpg': No such file or directory 21 /bin/ls: cannot access '928_Came up with an impromptu artwork during last night’s acid comedown. Thought I’d leave this here for those currently on a trip to enjoy. I’d appreciate any kind of feedback! Good vi bes and happy tripping! ??_josephleh92_fz2ax5.jpg': No such file or directory 22 /bin/ls: cannot access '957_Found this and had to share. Enjoy ??_jo-mc00_l3c3lh.mp4': No such file or directory 23 /bin/ls: cannot access '979_Latest doodle, hope you guys like it! Leave a comment or a like it helps me more than you know :) ❤️????_elfinnfinite_ey6j7h.jpg': No such file or directory 24 -------------------------------- below is rsync output selection of a folder, above is 23 files that I get when I ls on the source disk dir . 25 file has vanished: "/mnt/disks/Tetra/Trippy Pix Reddit Archive/Currentlytripping/1087_The 4th dimension ??_HenWad_ffo0xw.mp4" 26 file has vanished: "/mnt/disks/Tetra/Trippy Pix Reddit Archive/Currentlytripping/1097_Some more of my doodles in my first doodlebook but fast paced in a train ??_elfinnfinite_dsxjov.mp4" 27 file has vanished: "/mnt/disks/Tetra/Trippy Pix Reddit Archive/Currentlytripping/1137_Hammock, blanket, and some mushrooms. It’s a gorgeous day to die ??_ItsAGorgeouDayToDie_edr1os.jpg" 28 file has vanished: "/mnt/disks/Tetra/Trippy Pix Reddit Archive/Currentlytripping/1147_??_Sacra_profanis_dppzwu.png" 29 file has vanished: "/mnt/disks/Tetra/Trippy Pix Reddit Archive/Currentlytripping/1165_Leave a like or a comment! It helps me more than you know, love you all, stay strong ❤??_elfinnfinite_fqh3ac.jpg" 30 file has vanished: "/mnt/disks/Tetra/Trippy Pix Reddit Archive/Currentlytripping/1168_currently tripping and I just saw this, good vibes everyone ??_the-heba_de9onw.jpg" 31 file has vanished: "/mnt/disks/Tetra/Trippy Pix Reddit Archive/Currentlytripping/1232_Tripping with my boyfriend in our new house ??_rbigler22_d4dlj3.jpg" 32 file has vanished: "/mnt/disks/Tetra/Trippy Pix Reddit Archive/Currentlytripping/1927_Found this on Facebook. It just spoke volumes ??_Trip_And_tell_hmu22m.jpg" 33 file has vanished: "/mnt/disks/Tetra/Trippy Pix Reddit Archive/Currentlytripping/2096_Trip spot for the evening. 1.4grams cubensis and Handel’s Messiah with the San Francisco Symphony Orchestra. Feeling glorio us reddit ❤️❤️???? ?? ?? ☮️_california_king_eau66k.jpg" 34 file has vanished: "/mnt/disks/Tetra/Trippy Pix Reddit Archive/Currentlytripping/800_my grinder under my rainbow lights ??_DELETED_ejmqit.mp4" 35 file has vanished: "/mnt/disks/Tetra/Trippy Pix Reddit Archive/Currentlytripping/814_Another one of these vids ?? And yes I know about the runs, it was from the way I twaxed it ??_chiliwilly001_izi1mw.mp4" 36 file has vanished: "/mnt/disks/Tetra/Trippy Pix Reddit Archive/Currentlytripping/832_?? the cosmic sensation._DELETED_i9nxe7.jpg" 37 file has vanished: "/mnt/disks/Tetra/Trippy Pix Reddit Archive/Currentlytripping/860_?? @luke_schroeder_art_BennyHG_jabfxj_1.jpg" 38 file has vanished: "/mnt/disks/Tetra/Trippy Pix Reddit Archive/Currentlytripping/860_?? @luke_schroeder_art_BennyHG_jabfxj_2.jpg" 39 file has vanished: "/mnt/disks/Tetra/Trippy Pix Reddit Archive/Currentlytripping/860_?? @luke_schroeder_art_BennyHG_jabfxj_3.jpg" 40 file has vanished: "/mnt/disks/Tetra/Trippy Pix Reddit Archive/Currentlytripping/860_?? @luke_schroeder_art_BennyHG_jabfxj_4.jpg" 41 file has vanished: "/mnt/disks/Tetra/Trippy Pix Reddit Archive/Currentlytripping/860_?? @luke_schroeder_art_BennyHG_jabfxj_5.jpg" 42 file has vanished: "/mnt/disks/Tetra/Trippy Pix Reddit Archive/Currentlytripping/860_?? @luke_schroeder_art_BennyHG_jabfxj_6.jpg" 43 file has vanished: "/mnt/disks/Tetra/Trippy Pix Reddit Archive/Currentlytripping/860_?? @luke_schroeder_art_BennyHG_jabfxj_7.jpg" 44 file has vanished: "/mnt/disks/Tetra/Trippy Pix Reddit Archive/Currentlytripping/876_??_Nerihd522_m0ihn7.jpg" 45 file has vanished: "/mnt/disks/Tetra/Trippy Pix Reddit Archive/Currentlytripping/928_Came up with an impromptu artwork during last night’s acid comedown. Thought I’d leave this here for those currently on a tr ip to enjoy. I’d appreciate any kind of feedback! Good vibes and happy tripping! ??_josephleh92_fz2ax5.jpg" 46 file has vanished: "/mnt/disks/Tetra/Trippy Pix Reddit Archive/Currentlytripping/957_Found this and had to share. Enjoy ??_jo-mc00_l3c3lh.mp4" 47 file has vanished: "/mnt/disks/Tetra/Trippy Pix Reddit Archive/Currentlytripping/979_Latest doodle, hope you guys like it! Leave a comment or a like it helps me more than you know :) ❤️????_elfinnfinite_ey6j7h. jpg" Also what do you think about whether the "?" char in each of the filenames matters? Maybe I could batch rename the files on the source disk and remove all the question marks and see what that does. What do you think? Thanks

Yes I am seeing some warnings about filenames being too long root@Tower:~# rsync -azPhvn --stats /mnt/disks/Tetra/ /mnt/user/backups/Tetra --exclude '.Trashes' --exclude '.Spotlight-V100' --exclude '.fseventsd' --exclude '.DS_Store' sending incremental file list file has vanished: "/mnt/disks/Tetra/Back Ups/copy tfrom Terabyte time machine backup folder 2017-4-13/Backups.backupdb/Hackintosh/2017-04-13-120916/Mountain Lion/System/Library/User Template/th.lproj/Library/FontCollections/กำหนดความกว้าง.collection" rsync: [generator] recv_generator: failed to stat "/mnt/user/backups/Tetra/Excel Library.library/images/L0D1E3GWRTEL8.info/35_\#342\#200\------REDACTED 269 CHARS------thumbnail.png": File name too long (36) rsync: [generator] recv_generator: failed to stat "/mnt/user/backups/Tetra/Excel Library.library/images/L0D1E3M1DG6R9.info/------REDACTED 255 CHARS------thumbnail.png": File name too long (36) This excerpt also includes the file from the time machine you referenced. I guess I just don't understand it would fail to copy it over, but that seems like a possibility that it has something to do with the file systems. (from HFS+ to xfs-encrypted). Another thing that I noticed is that after the two "rsync: [generator] recv_generator: failed to stat ..." line, all of the file has vanished lines show the path/filename and all of the filenames contain the "?" char. Could that char being used in a filename be tripping rsync up? Is there any other ways you or anyone could recommend to make this process of duplicating one internal disk (hfs+) to another (xfs-encrypted)? Is there maybe an app or docker that could simplify the process without me needing to dive into the details of how rsync works? Apps like duplicati, duplicity, syncthing, do you think they would be able to accomplish the same thing if I just "rm -rf" the current xfs copy and redo it with one of those apps? By the way, thanks so much for your help :-)

Thanks for your reply. There is a large variety of files on it, it is about 3.9TB consisting of all kinds of text files, pdfs, images, audio files, video files. The filesystem is HFS+. I redacted the filenames because they are sensitive information and I thought it was unimportant to know what the specific files were. The redacted files that rsync says have vanished consist of various jpg, png, mp4, json files, with one exception being a system font collection file inside an old time machine backup. There is no way I can think of that any of these files could have been deleted from the src between the original rsync copy and the dry-run check. (I believe that is usually the reason rsync complains about files having vanished.)

I took a HDD from my main computer and put it in my unraid box so it would copy faster than over the network. I then researched and found rsync should be perfect for what I want to do, which is copy over every file (excluding a few I'll note later). I ssh into unraid and use this command rsync -azPhv --stats /mnt/disks/Tetra/ /mnt/user/backups/Tetra --exclude '.Trashes' --exclude '.Spotlight-V100' --exclude '.fseventsd' --exclude '.DS_Store' I left it overnight and it seemed to finish fine, but messed up by not copying the text that it said after finishing. Anyway, I want to verify to double check everything got copied over so I ssh in and run this rsync -azPhvn --stats /mnt/disks/Tetra/ /mnt/user/backups/Tetra --exclude '.Trashes' --exclude '.Spotlight-V100' --exclude '.fseventsd' --exclude '.DS_Store' (same thing but with the -n flag to make it be a dry run) Attached is a pic of what happens when I run that. I do not understand how any files could have vanished and I am new to using rsync. If y'all could help me figure this out I would really appreciate it.