falconexe

So here is the first script (attached). It is dynamic in that it prompts the user for a disk number. Note disks 1-9 are single digit, while 10+ are double digits. Once you enter the disk you want, the script will pull both a raw file listing with paths, and a metadata listing with folder structure to your desired location. Once completed, the script will automatically open the completed audits. This is coded in Windows batch so you will need to right click and select "edit" on the batch file. You will want to change out "MassEffect" with the name of your UNRAID server along with your server details. You will also need to change your destination path accordingly. Let me know if you run into any issues. If this works for you, I have 2 more scripts that do this same work for the entire array with ALL DISKS, and ALL SHARES. If you are interested in these, please let me know. Note: This script requires that each disk be shared on your array. I keep mine shared, but hidden so anyone else on my network would not see/know they are there. I also have my security per disk set to Read Only. UNRAID_Audit_Single_Disk.bat

Yeah I'm happy to share. They are just "simple" batch scripts that run on Windows. I'll sanitize my code and provide some templates soon. Please stand by...

I’m laying in bed super sick right now. Couldn’t sleep here in the USA, and saw your comment. I should have said started “as a kid” not “IT”. I’m an 80s baby. 😂 I started in IT professionally at 16. My first experience with a computer was in the fourth grade at 8 years old. I believe it was an Apple II or some such. We were quite poor growing up and someone gave us this super old Tandy. I messed with it for a few years. The first actual computer we could*nt afford was a Dell Dimension XPS B933 around 2000. My loving Dad took out a ridiculous $6K loan and bought it for us so we could be successful in school. I'm the one who ordered it on the phone and they laughed at me when I asked for these specs. They had to send me over the business division LOL. We also ended up getting some rediculous monitor and Altec Lansing 5.1 speakers. Oh, and a flight stick (My Dad Insisted). Pentium 3 933Mhz 512MB RAM 60GB Hard Drive (I believe that was the size...Yeah wasn’t cheap back then) And 1 Nvidia Hercules 3D Prophet II Ultra (AGP 4x, 64 MB) for playing Soldier of Fortune (The last thing that got my Dad to say Yes 🤣) Note: None of the pics below were my actual stuff. Just nostalgic references. It was like Christmas morning when those big boxes arrived. I set it up myself and my Dad was amazed. The next morning, I got a wool blanket 😳 and took the entire thing apart piece by piece to “study” the components and their locations. My Dad walks out and almost kills me. I told him not to worry, “I know what I’m doing!” I got the thing put back together and everything was perfect. Luckily no death by static. That was the beginning of my IT career. Thanks Dad. Love You!

Yep, I remember those. Zip drives. That sound! 😆 I started on the single layer floppies (1.2 MB if I recall) in the early 90s. I had a Tandy Corp TRS-80 5 1/4" "portable" 🤣 computer growing up and destroyed Oregon Trail on the daily. Thing was like 40 pounds!

Triple Parity Please 🙏 I ask this half serious, and I know the math behind this would be insane. I'd take this or multiple arrays on single hardware. I have nearly 30 disks in a 45 Drives Storinator with 206TB usable, including X5 14TB drives. As single drive sizes go up, I'm getting more and more nervous ha ha. I have backups of everything, on and off site, but still, recovering would be a PITA.

I have an IT friend in his late 70's who just retired. He said when they got 10MB drum (Yes DRUM) disks when he first started in the mid 60s it blew freakin people's minds. I always joke about 1PB thumb drives with him. I sent him a pic of my array at 206TB and he was floored. He is convinced in my lifetime I will see the PetaByte drives hanging on the shelves of stores some day. Some day people will laugh at this very post and they'll be storing crap on their .25 EB array. 😭 We have come so far is such a short time. Kind of scary and exciting to see what the next 50 years will bring!

I just can't wait to pick up my 1PB thumb-drive at the local Best Buy.

I had X2 1TB drives in a pool for a bit, but put one of them into my other UNRAID server. I'll add another one soon. Until then, I have all critical files being written directly to the array and larger (already backed up) files being written to the Cache. My mover runs every single night, so the risk is very low for me to lose something in cache, and if I did, I have backups.

I just Friggin hit 206TB usable. Approaching .25PB 😍

After adding the old Disk 04 back into the array with a clean bill of health, I just Friggin hit 206TB usable. Approaching .25PB 😍

If I recall, the PSU on the Storinator Q30 is a Corsair 700 Watt. Once I get some time, I can crack it open and confirm. But yeah, that is well over the 150W requirement at 5V X 30 Drives, with plenty of overhead for Dual CPUs, MB, RAM, etc... Also, that drive failed another round of HD Tune Pro Error Checks showing extreme DAMAGE to sectors all over the place. Lit up RED blocks all over like a Christmas tree. It's toast. I am eviscerating it with a 35 Pass Gutmann wipe as we speak just to take out some aggression. 😒 Well worth the electricity... I doubt any of the passes will actually finish with so many issues, but at least it makes me feel better. 😂

Ok thanks so much for getting back to me. I'll proceed with formatting the drive in XFS and remounting.

So the Old Disk 4 "sdad" (3 parity errors) that just passed the preclear process and is ready to be added back to the array. However, I was checking the SMART reports, and everything looks perfect except this bit under Error History. Should I be worried about any of this?

Funny enough, I am the hugest ME fan, and I never could get myself to play the Citadel DLC. Cause then it would truly be over. That was like 5 years ago. I am keeping a 4th full play through in my back pocket for nostalgia sake and will run through EVERYTHING one last time. Gotta finish The OuterWorlds first (which is fantastic BTW and I really don't like Fallout games per se). I played Andromeda for like 40 hours and my save got corrupted and I just could not get myself to start over. It was pretty bad. Those Faces Tho...

Agreed on MassEffect. My favorite video game franchise EVER. Bioware is in a sad state these days. Hoping Casey Hudson can deliver the magic again with ME4. 🙏 ...And Where the Crap is my ME Trilogy Remaster EA? Take my Money. Easiest Cash Grab in history and they won't do it. 🙄 Anyway, back to UNRAID talk ha ha.

That issue sounds insane with the tape. My temps are 23-26C during parity checks, so I don't think it was that, but good to know. Thanks. If you want to know about my specific build, go check out the below thread. Tons of pics. Not for the faint of heart on your wallet, but after my Norco 4224 case hitting 50+C, I LOVE the STORINATOR. Specs are insane. Around $6K not including disks ha ha. Not a fan of "Linus" LOL. And actually found 45 Drives through google after moving on from a great custom builder called Grean-Leaf http://greenleaf-technology.com/ He's out of Ohio, and was really helpful when I first got into UNRAID back in 2014. How could you miss this LMAO? It's a frackin RAINBOW 🌈 😂

So I have an update for everyone. Once the Old Disk 04 (sdad) finished preclearing successfully, I rebooted the server. Upon logging in I was greeted with this: And to take some good advice from @Squid and @trurl, you'll notice the serial numbers are not Photoshopped out. That was getting tedious anyway 🤣 This was the Old Disk 22 that ended a parity check in 2,000+ errors, passed a HD Tune Pro Full Error Scan, but Failed an UNRAID Preclear during the post-read process. So, it was having some major issues after-all. Can't understand why the SMART report was coming back clean until after this last reboot. For the last 6 years, EVERY SINGLE TIME, a disk would fail, I would see a failed SMART report first indicating pending sectors or reallocated, etc.. This drive was ghosting SMART health during 2 full health checks inside Windows and in UNRAID. I'm sure the disk power issues have a correlation as well. Anywho, this drive is TRASH and I'll be executing a 35 pass wipe with some "other" tools ha ha. We'll see if it can handle that type of onslaught. Then I will physically destroy the drive with a drill press for good measure. I guess we can mark this as SOLVED. By the way, since this is my first thread. Do I go change the title to "SOLVED" or does a MOD do that for me? Thanks everyone.

Thanks guys. It goes back to the SNs can be traced to purchase location and then tied to a business. Typically that kind of information is considered confidential/protected/proprietary and is not normally out in the wild due to various reasons. I get both sides... Thanks for responding. Finally got the @Squid in here ha ha. I feel special. Seen him and @trurl all over the place over the years. I've been a sleeper user reading forums for a very long time, but never felt the need to get involved until this recent mishap.

@trurl Will do. I might be on the more security conscience side. So my diagnostic anonymizing might be overkill for home users, but it would be really cool to see an extra check box that can incorporate some further sanitation for business purposes or "extremely paranoid" 🥺 people. Ha ha. 😂 One of my main concerns is suppressing network info, filepaths like found in "ps.txt", and drive serial numbers in all diagnostic files. I've been scripting changing drive SNs to something more generic like "DRIVE22". Maybe we are just paranoid, but with business, you can never be too careful. Where should I post such requests? And thanks for even entertaining this thought. I'm happy to sanitize to my needs manually. I just know for us, we will never post a full diag on the forum for the above and other various security reasons.

I'm not. It is a front...😉Na just kidding. We use it for a Media Production company.

After searching the forums, I noticed some others having issues with the November 2019 version of the PreClear plugin. I just updated to the latest version and will rerun both drives to see if I receive similar results. Even so, I still am at a loss as to why the disk unmounted and power cycled. More to come...

OK, so there is something really odd going on. I have never seen this before and have no idea what it means, or what is causing it. So as you know, I've been preclearing the previously "failed" data disks 04/22. Both passed HD Tune Pro Full Error scans. So I threw them back into the server and have been running Preclears on them. Both disks passed the PreRead Process. Both disks passed the Zeroing Process. However, immediately after the Zeroing finished on the old Disk 22 with ID "sdae" (the one that threw 2,000+ errors during the parity check), the preclear process failed and the disk literally disappeared from my array. It is nowhere to be found in the GUI. The error I received "Invalid unRAID's MBR signature" is below in RED (SNs have been Sanitized). The old Disk 4 with ID "sdad" (the one that thew only 3 errors during the parity check) is still going with the post-read process. Furthermore, something odd happened with the "sdad" disk where is now showing up as missing under Historical Devices? However, it is not missing and is shown in the preclear area above. (Perhaps it was always there and I forgot?) I am now thinking that the old disk 22 "sdae" is REALLY Messed Up For Some Reason. No issues with SMART, but something is going on. Can anyone tell me what that error means and what you think is going on with this Disk that disappeared? Also this disk is in a totally different slot than before (was 22, now is 27). So I am really thinking it is NOT my UNRAID Hardware and actually the disk itself. It appears to have powered down again after it failed the post-read. The log is below: Jan 13 03:43:16 MassEffect preclear_disk_OLDDISK22[11017]: Zeroing: progress - 100% zeroed Jan 13 03:43:16 MassEffect preclear_disk_OLDDISK22[11017]: Zeroing: dd - wrote 8001563222016 of 8001563222016. Jan 13 03:43:17 MassEffect preclear_disk_OLDDISK22[11017]: Zeroing: dd exit code - 0 Jan 13 03:43:20 MassEffect preclear_disk_OLDDISK22[11017]: Writing signature: 0 0 2 0 0 255 255 255 1 0 0 0 255 255 255 255 Jan 13 03:43:21 MassEffect preclear_disk_OLDDISK22[11017]: Failed test 1: MBR signature is not valid, byte 3 [00000] != [00170] Jan 13 03:43:21 MassEffect preclear_disk_OLDDISK22[11017]: array 'sectors' Jan 13 03:43:21 MassEffect preclear_disk_OLDDISK22[11017]: 0 -> 00000 Jan 13 03:43:21 MassEffect preclear_disk_OLDDISK22[11017]: 1 -> 00000 Jan 13 03:43:21 MassEffect preclear_disk_OLDDISK22[11017]: 2 -> 00000 Jan 13 03:43:21 MassEffect preclear_disk_OLDDISK22[11017]: 3 -> 00000 Jan 13 03:43:21 MassEffect preclear_disk_OLDDISK22[11017]: 4 -> 00000 Jan 13 03:43:21 MassEffect preclear_disk_OLDDISK22[11017]: 5 -> 00000 Jan 13 03:43:21 MassEffect preclear_disk_OLDDISK22[11017]: 6 -> 00000 Jan 13 03:43:21 MassEffect preclear_disk_OLDDISK22[11017]: 7 -> 00002 Jan 13 03:43:21 MassEffect preclear_disk_OLDDISK22[11017]: 8 -> 00000 Jan 13 03:43:21 MassEffect preclear_disk_OLDDISK22[11017]: 9 -> 00000 Jan 13 03:43:21 MassEffect preclear_disk_OLDDISK22[11017]: 10 -> 00255 Jan 13 03:43:21 MassEffect preclear_disk_OLDDISK22[11017]: 11 -> 00255 Jan 13 03:43:21 MassEffect preclear_disk_OLDDISK22[11017]: 12 -> 00255 Jan 13 03:43:21 MassEffect preclear_disk_OLDDISK22[11017]: 13 -> 00001 Jan 13 03:43:21 MassEffect preclear_disk_OLDDISK22[11017]: 14 -> 00000 Jan 13 03:43:21 MassEffect preclear_disk_OLDDISK22[11017]: 15 -> 00000 Jan 13 03:43:21 MassEffect preclear_disk_OLDDISK22[11017]: 16 -> 00000 Jan 13 03:43:21 MassEffect preclear_disk_OLDDISK22[11017]: 17 -> 00255 Jan 13 03:43:21 MassEffect preclear_disk_OLDDISK22[11017]: 18 -> 00255 Jan 13 03:43:21 MassEffect preclear_disk_OLDDISK22[11017]: 19 -> 00255 Jan 13 03:43:21 MassEffect preclear_disk_OLDDISK22[11017]: 20 -> 00255 Jan 13 03:43:24 MassEffect preclear_disk_OLDDISK22[11017]: error encountered, exiting... Jan 13 03:43:24 MassEffect preclear_disk_OLDDISK22[11017]: cat: 15853: No such file or directory Jan 13 03:50:17 MassEffect kernel: mpt3sas_cm1: log_info(0x31111000): originator(PL), code(0x11), sub_code(0x1000) Jan 13 03:50:19 MassEffect kernel: scsi 12:0:14:0: Direct-Access ATA ST8000DM004-2CX1 0001 PQ: 0 ANSI: 6 Jan 13 03:50:19 MassEffect kernel: scsi 12:0:14:0: SATA: handle(0x0026), sas_addr(0x300062b203fe85d7), phy(23), device_name(0x0000000000000000) Jan 13 03:50:19 MassEffect kernel: scsi 12:0:14:0: enclosure logical id (0x500062b203fe85c0), slot(13) Jan 13 03:50:19 MassEffect kernel: scsi 12:0:14:0: enclosure level(0x0000), connector name( ) Jan 13 03:50:19 MassEffect kernel: scsi 12:0:14:0: atapi(n), ncq(y), asyn_notify(n), smart(y), fua(y), sw_preserve(y) Jan 13 03:50:19 MassEffect kernel: sd 12:0:14:0: Power-on or device reset occurred Jan 13 03:50:19 MassEffect kernel: sd 12:0:14:0: Attached scsi generic sg30 type 0 Jan 13 03:50:19 MassEffect kernel: sd 12:0:14:0: [sdae] 15628053168 512-byte logical blocks: (8.00 TB/7.28 TiB) Jan 13 03:50:19 MassEffect kernel: sd 12:0:14:0: [sdae] 4096-byte physical blocks Jan 13 03:50:19 MassEffect kernel: sd 12:0:14:0: [sdae] Write Protect is off Jan 13 03:50:19 MassEffect kernel: sd 12:0:14:0: [sdae] Mode Sense: 9b 00 10 08 Jan 13 03:50:19 MassEffect kernel: sd 12:0:14:0: [sdae] Write cache: enabled, read cache: enabled, supports DPO and FUA Jan 13 03:50:20 MassEffect rc.diskinfo[15953]: SIGHUP received, forcing refresh of disks info. Jan 13 03:50:20 MassEffect kernel: sd 12:0:14:0: [sdae] Attached SCSI disk Jan 13 04:26:48 MassEffect rc.diskinfo[15953]: SIGHUP received, forcing refresh of disks info. Jan 13 04:26:48 MassEffect rc.diskinfo[15953]: SIGHUP ignored - already refreshing disk info.

Double Data Rebuild Completed SUCCESSFULLY! Took just over 43 Hours for 14TB Dual Parity Rebuilding Dual Data Disks. I'm super happy this all worked out as discussed. Now I am going to drop these old 8TB data disks back into the array in slots 26/27 and run some preclears. I'll post another update soon.

Also my HD Tune Pro Full Error Scans successfully completed on both of the "failed" data drives. 100% Healthy. Once the rebuilds finish on the new drives, I'm going to drop these back into the array and run preclears. If they pass, I get back 16TB. Or if I look at it the other way, I gained 28TB unnecessarily ha ha.

Double Data Rebuild Status: Made it past 50% with no issues thus far.