FreeMan Posted September 21, 2019 Share Posted September 21, 2019 3 hours ago, binhex said: what is the uptime for the container right now? Currently 15 hours. It was 12 when you asked. That would explain a lot... Not sure why it restarted - Server's been up and there wasn't an update. Quote Link to comment
binhex Posted September 22, 2019 Author Share Posted September 22, 2019 Currently 15 hours. It was 12 when you asked. That would explain a lot... Not sure why it restarted - Server's been up and there wasn't an update. [emoji20]Ok that's kinda good to confirm it was the container that restarted, but I'm clueless as to why that happened, even if the container stopped it wouldn't restart, and if the script crashed out it also wouldn't cause a restart, so unknown at this time, just check your server uptime incase your server restarted without you knowing.Sent from my CLT-L09 using Tapatalk Quote Link to comment
FreeMan Posted September 22, 2019 Share Posted September 22, 2019 (edited) hrm... this isn't a good sign. Container uptime 9 hours. Server uptime 2 days, 15 hours 43 minutes. The container shut down at 02:47 this morning. Here's the last line in the log from the start up on 9/21, followed by the shutdown early am on 9/22 and the startup 1 hour later on 9/22. Never mind... Quote Appdata Backup When a Backup is running (either manually or a scheduled backup), your docker applications will be stopped and then restarted at the conclusion of the backup Because: Disables CA Backup. Makes calendar entry to re-enable it when preclear is finished. Makes additional note to re-read Note in FAQ Answer #7. It may do well to highlight that note - it took both of us 2 days to realize what was causing the issue, so it's likely others will miss it as well. Actually - a better option exists in the CA Backup config: Leave backups enabled, just don't allow it to stop binhex-preclear. I don't know if there would be any open files in the docker that may not get backed up, but considering that there aren't really any options to set (defaults work just fine for me!) and even if you do customize ports or something else, you'll probably have multiple backups made before and after running an individual preclear, this seems reasonable. Edited September 22, 2019 by FreeMan Quote Link to comment
binhex Posted September 22, 2019 Author Share Posted September 22, 2019 hrm... this isn't a good sign. Container uptime 9 hours. Server uptime 2 days, 15 hours 43 minutes. The container shut down at 02:47 this morning. Here's the last line in the log from the start up on 9/21, followed by the shutdown early am on 9/22 and the startup 1 hour later on 9/22. Never mind... Appdata Backup When a Backup is running (either manually or a scheduled backup), your docker applications will be stopped and then restarted at the conclusion of the backup Because: Disables CA Backup. Makes calendar entry to re-enable it when preclear is finished. Makes additional note to re-read Note in FAQ Answer #7. It may do well to highlight that note - it took both of us 2 days to realize what was causing the issue, so it's likely others will miss it as well. Actually - a better option exists in the CA Backup config: Leave backups enabled, just don't allow it to stop binhex-preclear. I don't know if there would be any open files in the docker that may not get backed up, but considering that there aren't really any options to set (defaults work just fine for me!) and even if you do customize ports or something else, you'll probably have multiple backups made before and after running an individual preclear, this seems reasonable. Lol ok I had ruled that out as I assumed you had read the FAQ and just to confirm there is nothing to backup for this container, apart from final preclear reports which are written to /config [emoji16][emoji16] Sent from my CLT-L09 using Tapatalk Quote Link to comment
wgstarks Posted September 22, 2019 Share Posted September 22, 2019 48 minutes ago, binhex said: Actually - a better option exists in the CA Backup config: Where is this? I don’t see anything in CA Backup/Restore that resembles your screenshots at all. Quote Link to comment
Squid Posted September 22, 2019 Share Posted September 22, 2019 Where is this? I don’t see anything in CA Backup/Restore that resembles your screenshots at all.Advanced settings Sent from my NSA monitored device Quote Link to comment
wgstarks Posted September 22, 2019 Share Posted September 22, 2019 52 minutes ago, Squid said: Advanced settings Sent from my NSA monitored device Thanks. The purple text is really hard to see in the dark template and I was trying to find it in the top right corner. Totally missed it the first dozen times. 🙂 Quote Link to comment
Squid Posted September 22, 2019 Share Posted September 22, 2019 Thanks. The purple text is really hard to see in the dark template and I was trying to find it in the top right corner. Totally missed it the first dozen times. [emoji846]Submit a report on the thread I will forget all about thatSent from my NSA monitored device Quote Link to comment
FreeMan Posted September 23, 2019 Share Posted September 23, 2019 7 hours ago, wgstarks said: Where is this? I don’t see anything in CA Backup/Restore that resembles your screenshots at all. Well, there was a big arrow pointing at the text, but I suppose when it was totally out of context, it's hard to figure out where that was actually pointing. Especially since the list of dockers doesn't appear until after you click the "advanced" link. Quote Link to comment
FreeMan Posted September 23, 2019 Share Posted September 23, 2019 On 9/22/2019 at 12:47 PM, binhex said: Lol ok I had ruled that out as I assumed you had read the FAQ and just to confirm there is nothing to backup for this container, apart from final preclear reports which are written to /config Just because I had read it once did not mean that this little detail stuck with me several days later when the drive arrived and it was actually time to kick off the preclear. OK. The not-so-comedy of errors continues... It was nearly done - about 30% into the post-read. Honestly, this is the fastest I've ever had a disk preclear (in my memory). It was around 29 hours for an 8TB drive and it was doing the post read at about 200MB/s. I was trying to capture the whole text output to paste here to brag. Instead of hitting ctrl-shift-c to copy the selected text, it seems I hit ctrl-c and aborted the script. I did preclear_binhex.sh -t /dev/sdc to confirm that it was, in fact, precleared. At the end of the output, I get this: Quote Partition 1 does not start on physical sector boundary. ######################################################################## ========================================================================1.18 == == DISK /dev/sdc IS PRECLEARED with a GPT Protective MBR == ============================================================================ With the "Partition 1..." message highlighted in red. Obviously, the disk is cleared. Should I have any concern about the "Partition 1..." message? The original invocation was preclear_binhex.sh -f /dev/sdc exactly as specified in the instructions. I didn't set any other parameters or options. Quote Link to comment
binhex Posted September 24, 2019 Author Share Posted September 24, 2019 10 hours ago, FreeMan said: Just because I had read it once did not mean that this little detail stuck with me several days later when the drive arrived and it was actually time to kick off the preclear. its all cool, i wasnt having a go, honest, i appreciate the feedback, as you suggested i have put that 'note' regards ca backup at the top of the FAQ now, so hopefully it will be a bit more obvious. 10 hours ago, FreeMan said: Partition 1 does not start on physical sector boundary. ok so i believe this is due to the 4k alignment for disks that are larger than 2.2TB, from what i have read this can safely be ignored, i didn't see this in my testing as i was using a 500GB drive and thus the alignment would be 512KB, i will add this to the FAQ. i guess the question is, should you re-run the preclear again?, as its runs quite quickly on your system and you didn't see the final report where it confirms smart attributes havent changed during the preclear to confirm the disk is 'good' i would personally re-run it again if it was a disk i wanted to add to the array, alternatively you COULD just take a look at smartctl values to confirm it looks ok, the option is yours. Quote Link to comment
FreeMan Posted September 24, 2019 Share Posted September 24, 2019 Slightly more disturbing is the output of preclear_binhex.sh -f /dev/sdc Quote Pre-Clear unRAID Disk /dev/sdc ################################################################## 1.18 Model Family: Seagate Enterprise Capacity 3.5 HDD Device Model: ST8000NM0055-1RM112 Serial Number: ZA1FS9VW LU WWN Device Id: 5 000c50 0b6d764f6 Firmware Version: SN05 User Capacity: 8,001,563,222,016 bytes [8.00 TB] Disk /dev/sdc: 7.3 TiB, 8001563222016 bytes, 15628053168 sectors Disk model: ST8000NM0055-1RM Units: sectors of 1 * 512 = 512 bytes Sector size (logical/physical): 512 bytes / 4096 bytes I/O size (minimum/optimal): 4096 bytes / 4096 bytes Disklabel type: dos Disk identifier: 0x00000000 Device Boot Start End Sectors Size Id Type /dev/sdc1 1 4294967295 4294967295 2T 0 Empty Partition 1 does not start on physical sector boundary. ######################################################################## invoked as /usr/local/bin/preclear_binhex.sh -f /dev/sdc ######################################################################## (MBR 4k-aligned set. Partition will start on sector 64 for disks <= 2.2TB and sector 1 for disks > 2.2TB) Are you absolutely sure you want to clear this drive? (Answer Yes to continue. Capital 'Y', lower case 'es'): Why does it have one partition of 2TB? I've got 2.4TB of free space left in my data array, so I'm restarting the preclear again. This will be finished before I manage to fill that up. Quote Link to comment
binhex Posted September 24, 2019 Author Share Posted September 24, 2019 (edited) 54 minutes ago, FreeMan said: Why does it have one partition of 2TB? i think this is due to a limitation with fdisk, it cannot create (or list) partitions larger than 2TB, taken from an article:- Quote Frankly speaking, you cannot create a Linux partition larger than 2 TB using the fdisk command. The fdisk won’t create partitions larger than 2 TB. i think i will include 'parted' in the image and MAYBE (depends on how much work), switch to using this as it supports GPT partitions, or at least allow people to use parted to confirm. Edited September 24, 2019 by binhex 1 Quote Link to comment
binhex Posted September 24, 2019 Author Share Posted September 24, 2019 (edited) 1 hour ago, FreeMan said: Why does it have one partition of 2TB? if you havent already restarted a preclear then you could try issuing the following command, which should show the correct partition size (supports gpt):- sfdisk -l /dev/sdX edit - actually looks like sfdisk may have the same limitation as fdisk - the correct way is to use parted, so im going to include this in the next build. Edited September 24, 2019 by binhex Quote Link to comment
FreeMan Posted September 24, 2019 Share Posted September 24, 2019 3 minutes ago, binhex said: if you havent already restarted a preclear too late... Quote Link to comment
binhex Posted September 24, 2019 Author Share Posted September 24, 2019 too late... Ok no worries [emoji16]Sent from my CLT-L09 using Tapatalk Quote Link to comment
FreeMan Posted September 26, 2019 Share Posted September 26, 2019 (edited) Well, the docker updated on me last night in the middle of a preclear - when last I looked, it was 98% done with the preread. On the bright side, I got to run sfdisk -l /dev/sdX as you suggested, and this is what it gave me: [root@b377c9e81bea /]# sfdisk -l /dev/sdc Disk /dev/sdc: 7.3 TiB, 8001563222016 bytes, 15628053168 sectors Disk model: ST8000NM0055-1RM Units: sectors of 1 * 512 = 512 bytes Sector size (logical/physical): 512 bytes / 4096 bytes I/O size (minimum/optimal): 4096 bytes / 4096 bytes Disklabel type: dos Disk identifier: 0x00000000 Device Boot Start End Sectors Size Id Type /dev/sdc1 1 4294967295 4294967295 2T 0 Empty Partition 1 does not start on physical sector boundary. It still shows a 2TB partition. Starting the preclear for the... 3rd(?) time... Edited September 26, 2019 by FreeMan Quote Link to comment
binhex Posted September 26, 2019 Author Share Posted September 26, 2019 8 hours ago, FreeMan said: Well, the docker updated on me last night in the middle of a preclear hmm of course, ok i will add that an exclusion for auto updates needs to be done for this docker, im sure you have now done that right?. 8 hours ago, FreeMan said: On the bright side, I got to run sfdisk -l /dev/sdX yeah i guess you didnt see my edit :-), ok well lets just see what we get at the end of the process, i must say i am a little concerned that its showing as 2TB partition, but according to the forum posts i have read the preclear script should auto detect that the drive is over 2TB and thus use GPT partition type. I have historically always run preclear script with the -A flag, but this is more out of habit than anything, and again according to the forum posts and changelog the -A flag should actually be ignored (drive is over 2TB). so lets see what we get, if it does turn out that its created a 2TB partition then i guess one or more of these is true:- 1. the preclear script has a bug, where drives over 2TB are incorrectly getting detected as 2TB drives and thus a dos style partition is created - this MIGHT be worked around by using the -A flag which creates a partitions on sector 64 (without -a or -A it starts on sector 1) or 2. there is a change in the tool used to create the partition, and this is causing an issue. sadly i currently do not have a drive larger than 500GB to test with, otherwise i would run a preclear myself and see what happens :-(. for ref see the following in the changelog:- # 1.13 has the fixes and should work properly for GPT partitions and proper recognition of # default in the absence of "-A" or "-a" option. # Version 1.14 - Added text describing how -A and -a options are not used or needed on disks > 2.2TB. Quote Link to comment
wgstarks Posted September 26, 2019 Share Posted September 26, 2019 3 hours ago, binhex said: sadly i currently do not have a drive larger than 500GB to test with, otherwise i would run a preclear myself and see what happens :-(. I’ll have a new 8TB drive Saturday that I’ll be preclearing. I plan on running 3 passes. Any other flags you want me to try? Quote Link to comment
binhex Posted September 26, 2019 Author Share Posted September 26, 2019 (edited) 28 minutes ago, wgstarks said: Any other flags you want me to try? thanks for the offer, so depending on how @FreeMan gets on, if his disk preclears fine and the 2TB issue is purely fdisk limitation then you can run as per the FAQ, obviously in your case with the additional passes flag. if however freeman has an issue, where the drive preclears but is only showing as a 2TB drive (when in fact its a 8TB drive) then you could try the following, perhaps a single pass to see how it goes first, no point doing 3 passes if its not working as expected:- preclear_binhex.sh -A -f /dev/sdX # where X is your drive letter if you can also run the following command at the end of the preclear that would be handy too, this should show the size of the partition and partition table, it should be obviously the size of your disk and Partition Table of 'gpt' parted -l Edited September 26, 2019 by binhex Quote Link to comment
wgstarks Posted September 27, 2019 Share Posted September 27, 2019 Can't get this to work. Sorry: /dev/sdo does not exist as a block device Clearing will NOT be performed [root@bef0414753a7 /]# /dev/sdo is seen as an unassigned disk. I tried formatting and mounted/unmounted. Same error every time. The drive is installed in the server, not external. Any ideas? Quote Link to comment
binhex Posted September 27, 2019 Author Share Posted September 27, 2019 Can't get this to work.Sorry: /dev/sdo does not exist as a block deviceClearing will NOT be performed[root@bef0414753a7 /]# /dev/sdo is seen as an unassigned disk. I tried formatting and mounted/unmounted. Same error every time. The drive is installed in the server, not external. Any ideas?Try restarting the containerSent from my CLT-L09 using Tapatalk Quote Link to comment
wgstarks Posted September 27, 2019 Share Posted September 27, 2019 3 minutes ago, binhex said: Try restarting the container Sent from my CLT-L09 using Tapatalk worked thanks 1 Quote Link to comment
wgstarks Posted September 27, 2019 Share Posted September 27, 2019 Trying to figure out the mail options. Would it be something like this? preclear_binhex.sh -A -f -M 4 [email protected] /dev/sdX And not sure what I need to do for this? Quote The -m, -M options requires that a valid working mail command is installed. Quote Link to comment
FreeMan Posted September 28, 2019 Share Posted September 28, 2019 40 hours 37 minutes 3 seconds later... ========================================================================1.18 == invoked as: /usr/local/bin/preclear_binhex.sh -f /dev/sdc == ST8000NM0055-1RM112 ZA1FS9VW == Disk /dev/sdc has been successfully precleared == with a starting sector of 1 == Ran 1 cycle == == Using :Read block size = 1000448 Bytes == Last Cycle's Pre Read Time : 14:00:01 (158 MB/s) == Last Cycle's Zeroing time : 12:18:43 (180 MB/s) == Last Cycle's Post Read Time : 14:17:20 (155 MB/s) == Last Cycle's Total Time : 40:37:03 == == Total Elapsed Time 40:37:03 == == Disk Start Temperature: 31C == == Current Disk Temperature: 32C, == ============================================================================ ** Changed attributes in files: /tmp/smart_start_sdc /tmp/smart_finish_sdc ATTRIBUTE NEW_VAL OLD_VAL FAILURE_THRESHOLD STATUS RAW_VALUE Raw_Read_Error_Rate = 83 82 44 ok 217697608 Seek_Error_Rate = 77 76 45 ok 55541351 Spin_Retry_Count = 100 100 97 near_thresh 0 End-to-End_Error = 100 100 99 near_thresh 0 Airflow_Temperature_Cel = 68 69 40 ok 32 G-Sense_Error_Rate = 99 100 0 ok 2432 Temperature_Celsius = 32 31 0 ok 32 Hardware_ECC_Recovered = 83 82 0 ok 217697608 No SMART attributes are FAILING_NOW 0 sectors were pending re-allocation before the start of the preclear. 0 sectors were pending re-allocation after pre-read in cycle 1 of 1. 0 sectors were pending re-allocation after zero of disk in cycle 1 of 1. 0 sectors are pending re-allocation at the end of the preclear, the number of sectors pending re-allocation did not change. 0 sectors had been re-allocated before the start of the preclear. 0 sectors are re-allocated at the end of the preclear, the number of sectors re-allocated did not change. ============================================================================ Also: parted -l Model: ATA ST8000NM0055-1RM (scsi) Disk /dev/sdc: 8002GB Sector size (logical/physical): 512B/4096B Partition Table: msdos Disk Flags: Number Start End Size Type File system Flags So it does recognize that it's an 8TB drive. The partition table of "msdos" is a little disconcerting, the others are showing gpt. The others also show partitions under the table listing at the bottom and a file system of "xfs". I would presume at this point that these differences are due to the fact that the disk hasn't been formatted yet. Any thoughts, comments or concerns, or should I just shove it in the array now with a variety of lessons learned? Quote Link to comment
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.