Jump to content

FreeMan

Members
  • Content Count

    1059
  • Joined

  • Last visited

Community Reputation

39 Good

About FreeMan

  • Rank
    Advanced Member

Converted

  • Gender
    Undisclosed
  • Location
    Indiana

Recent Profile Visitors

1087 profile views
  1. Thanks for the write up @tr0910! Thanks also to @ken-ji and @Hoopster for your trouble shooting and help. It took some head scratching for me to get my head wrapped around your source & destination designations and where you were actually running the rsync command. Once I did, I was able to alter it to push from my main server to my backup server (instead of pulling main from backup). I did the whole thing with the backup server already at the remote location. I'd been running Duplicati with the backup server sitting right next to the main server and finally got the backup shipped off with the son in college (half-way across the state should be a safe distance, right? ). Once the server was there, Duplicati refused to launch the GUI, so my hand was forced to make a change. I've only got about 2.5TB of data left to backup, so I may have the kid bring the backup server home with him when he returns for Christmas break to make this initial backup see run just a bit faster... I'm running ZeroTier (as noted earlier by Hoopster). The kids are using it to access the Emby server on my main machine, and it's working great for having the 2 servers talk to each other. I'm not sure of the impact on backup speeds yet, but we'll see what happens. It's also handy with the ZT client on my phone - I can use ControlR to check on the servers any time and from anywhere. I haven't yet attempted a reboot of either machine to see if the ssh keys survive. I'm going to let the first phase of backups finish before doing that. I put the keys I created in the /root/config/ssh directory which was already on the flash drive. I've modified my go file to copy just the 3 specific files created for this process. Are there any (known) issues to either this process or unRAID in general by having the extra files in that directory?
  2. Thanks, @TechMed. I've looked through that thread a couple of times. I've wanted the "Oh, carp. I just realized I made changes and now I need to go back to an older version" version of backup. But, in the last 10 years or so, I don't think I've ever gone back to recover an old version of a file, so maybe it's time to give rsync a 2nd look.
  3. I set up a Backup server sitting right next to my Primary server. From Primary, I used UD to SMB mount a share on Backup, and I'm using Duplicati to run the backups from Primary to Backup. Backup had been assigned a static IP address and all was good. In preparation for moving Backup off-site I switched it from static to DHCP (I'm not sure of networking at its new home-to-be) and now Primary cannot access it via the UD mount point. This makes some sense to me since its IP address changed (though I'd specified the UD mount by name, not IP) - I believe this is because the DNS cache has not yet updated to recognize Backup at its new IP address. Having done this yesterday evening, this morning (about 12 hours later) it still wasn't properly recognizing the server at its new IP. (in UD, I clicked the mount point to browse and all it showed me was "parent folder"). This morning I unmounted and remounted the share (it took quite a while to unmount, but the remount went very quickly), and now I can properly browse the directory structure via UD's mount point share. A couple of questions: 1) Is it normally necessary to unmount/remount when an IP address changes or should I normally just have to wait out the DNS cache update? 2) Was it something else totally that was the issue here? 3) Once the server goes to its remote location, it will be on a totally different subnet as I will be using the great ZeroTier plug in to get the two of them to talk to each other, I presume that going from a 192.168.* address to a 172.29.* address will not have any impact on UD's ability to map this, correct? 4) Is SMB the best option for having the 2 servers talk to each other, or would I be better off using an NFS mount? (If so, I'd probably keep the SMB mount in order to be able to browse via Windows, just because I can.) Thanks (again, I hope) for picking this up and continuing to run with it!
  4. Hmm... maybe this isn't completely solved. I'd started the extended SMART test on Disk2 after noticing that the last one had been aborted. Now this: The test was aborted by host. I'm 100% certain that the server didn't go down (current uptime > 25 days). The drive is not spinning, but I don't know when it spun down. I'd think that an extended SMART self-test would be enough to prevent UNRAID from spinning down the drive, wouldn't it? Why is the extended test aborting? Why are there now 3 extended tests listed including a new one (that wasn't in the previous screen shot) at only 6533 hours when the drive's been powered for 72074 hours and head flying hours are 55496? Is there anything else in the SMART log that I should be worried about? Updated SMART log and Diagnostics attached. backup-diagnostics-20191015-1849.zip backup-smart-20191015-1439.zip
  5. Winner winner chicken dinner! I have one error on each drive: 199 UDMA CRC error count 0x0032 200 200 000 Old age Always Never 1 Thanks! And I never even thought to click the orange thumb down to notice that there were options there.
  6. The dashboard on my Backup server (UnRAID Plus 6.7.2) is showing 2 drives with SMART errors: However, extended SMART tests on both drives show no errors. (OK, just realized that the Disk 2 test was aborted - I think the UPS shut the server down when the power was out for a while. I'm starting a full extended test again now. However, short tests have been run since then with no sign of error. I will return to post results of the new Extended SMART test on Disk2 as soon as it's completed.) It's been like this for a couple of weeks (yes, I've been slow to get back here to ask...) and it's survived a reboot or two with this mismatched display, so I don't believe that it's just a quick quirk. Also, I've noticed that these two drives do not spin down now - I have a status sent via Pushover 3 times per day and they're always spinning. Before the errors appeared on the dashboard, they would spin down like the rest. The server is being written to by Duplicati with 3 different backups running to it each day, but those backups only run about 15 minutes as they check files, backup the few new changes and prune the oldest backups - there should be plenty of time for all drives to spin down. There isn't anything else touching this server that I'm aware of, nothing new (plugins or dockers) has been installed and I don't have any torrents, movies, TV shows, etc that would be read from the machine - it's purely backup. These are, obviously, older drives - they've done several years service in my main server and have been migrated to the backup server where they should see much lower usage and live out their convalescence in relative peace and ease. 1) What would cause this mismatch between what's being reported by the disk and what's being reported on the dashboard? 2) Is the mismatch something to be concerned about? 3) Is there anything to be concerned about on either drive? (It is odd to note that the sort order of SMART tests for the 2 drives is totally different, and that they're really not sorted, especially for Disk2. But that's a very minor quibble and probably down to the way the vendor reports it.) backup-smart-20191015-0904 (Parity).zip backup-smart-20191015-0900 (Disk2).zip backup-diagnostics-20191015-1310.zip
  7. Thanks for digging into it - this seems like a good option. To the uninformed and/or uninitiated, "start on sector boundary 64" instead of "start on sector boundary 1" screams "OhMerGersh I'm not getting full use of my disk!!!". Considering it's an 8TB drive, even if that is true, it's a minor loss at worst... This sounds really handy and I'll be sure to use it for my next preclear! I know you're not after bells and whistles, so let's just call this "a random glitter that worked its way in, cause that's what glitter does".
  8. Awesome! Thanks for the links and the reassurances. I'm going to say that the new docker is a success! The only thing you may want to recommend is using the -A flag on larger (i.e. currently normal sized) drives.
  9. Even before the post-read completed, Unassigned Devices is reporting: I'd take that as a positive sign! It seems that the `-A` parameter made the difference. This time it was 40:44:43. Sooooooo much slower. ========================================================================1.18 == invoked as: /usr/local/bin/preclear_binhex.sh -f -A /dev/sdc == ST8000NM0055-1RM112 ZA1FS9VW == Disk /dev/sdc has been successfully precleared == with a starting sector of 1 == Ran 1 cycle == == Using :Read block size = 1000448 Bytes == Last Cycle's Pre Read Time : 14:02:00 (158 MB/s) == Last Cycle's Zeroing time : 12:17:22 (180 MB/s) == Last Cycle's Post Read Time : 14:24:22 (154 MB/s) == Last Cycle's Total Time : 40:44:43 == == Total Elapsed Time 40:44:43 == == Disk Start Temperature: 31C == == Current Disk Temperature: 32C, == ============================================================================ ** Changed attributes in files: /tmp/smart_start_sdc /tmp/smart_finish_sdc ATTRIBUTE NEW_VAL OLD_VAL FAILURE_THRESHOLD STATUS RAW_VALUE Raw_Read_Error_Rate = 66 83 44 near_thresh 3626020 Seek_Error_Rate = 79 78 45 ok 74098720 Spin_Retry_Count = 100 100 97 near_thresh 0 End-to-End_Error = 100 100 99 near_thresh 0 Airflow_Temperature_Cel = 68 69 40 ok 32 Temperature_Celsius = 32 31 0 ok 32 Hardware_ECC_Recovered = 66 83 0 ok 3626020 No SMART attributes are FAILING_NOW 0 sectors were pending re-allocation before the start of the preclear. 0 sectors were pending re-allocation after pre-read in cycle 1 of 1. 0 sectors were pending re-allocation after zero of disk in cycle 1 of 1. 0 sectors are pending re-allocation at the end of the preclear, the number of sectors pending re-allocation did not change. 0 sectors had been re-allocated before the start of the preclear. 0 sectors are re-allocated at the end of the preclear, the number of sectors re-allocated did not change. ============================================================================ All looks good to me, but what does "near_thresh" mean: Raw_Read_Error_Rate = 66 83 44 near_thresh 3626020 Spin_Retry_Count = 100 100 97 near_thresh 0 End-to-End_Error = 100 100 99 near_thresh 0 And out would these counts be near threshold this early in the disk's life? Is that something to be concerned about? They're the exact same numbers as in the report from the 1st preclear. (Well, the first one to complete...) Actually, I just realized that the "Raw_Read_Error_Rate" was OK after the first run and is now "near_thresh" after this 2nd run. I've added it to the call-out. Is that something to be concerned about?
  10. We'll see what happens... The drive does not show up in Unassigned Devices - is that to be expected? I was going to look there to see what it thought of the drive, but no love.
  11. 40 hours 37 minutes 3 seconds later... ========================================================================1.18 == invoked as: /usr/local/bin/preclear_binhex.sh -f /dev/sdc == ST8000NM0055-1RM112 ZA1FS9VW == Disk /dev/sdc has been successfully precleared == with a starting sector of 1 == Ran 1 cycle == == Using :Read block size = 1000448 Bytes == Last Cycle's Pre Read Time : 14:00:01 (158 MB/s) == Last Cycle's Zeroing time : 12:18:43 (180 MB/s) == Last Cycle's Post Read Time : 14:17:20 (155 MB/s) == Last Cycle's Total Time : 40:37:03 == == Total Elapsed Time 40:37:03 == == Disk Start Temperature: 31C == == Current Disk Temperature: 32C, == ============================================================================ ** Changed attributes in files: /tmp/smart_start_sdc /tmp/smart_finish_sdc ATTRIBUTE NEW_VAL OLD_VAL FAILURE_THRESHOLD STATUS RAW_VALUE Raw_Read_Error_Rate = 83 82 44 ok 217697608 Seek_Error_Rate = 77 76 45 ok 55541351 Spin_Retry_Count = 100 100 97 near_thresh 0 End-to-End_Error = 100 100 99 near_thresh 0 Airflow_Temperature_Cel = 68 69 40 ok 32 G-Sense_Error_Rate = 99 100 0 ok 2432 Temperature_Celsius = 32 31 0 ok 32 Hardware_ECC_Recovered = 83 82 0 ok 217697608 No SMART attributes are FAILING_NOW 0 sectors were pending re-allocation before the start of the preclear. 0 sectors were pending re-allocation after pre-read in cycle 1 of 1. 0 sectors were pending re-allocation after zero of disk in cycle 1 of 1. 0 sectors are pending re-allocation at the end of the preclear, the number of sectors pending re-allocation did not change. 0 sectors had been re-allocated before the start of the preclear. 0 sectors are re-allocated at the end of the preclear, the number of sectors re-allocated did not change. ============================================================================ Also: parted -l Model: ATA ST8000NM0055-1RM (scsi) Disk /dev/sdc: 8002GB Sector size (logical/physical): 512B/4096B Partition Table: msdos Disk Flags: Number Start End Size Type File system Flags So it does recognize that it's an 8TB drive. The partition table of "msdos" is a little disconcerting, the others are showing gpt. The others also show partitions under the table listing at the bottom and a file system of "xfs". I would presume at this point that these differences are due to the fact that the disk hasn't been formatted yet. Any thoughts, comments or concerns, or should I just shove it in the array now with a variety of lessons learned?
  12. Well, the docker updated on me last night in the middle of a preclear - when last I looked, it was 98% done with the preread. On the bright side, I got to run sfdisk -l /dev/sdX as you suggested, and this is what it gave me: [root@b377c9e81bea /]# sfdisk -l /dev/sdc Disk /dev/sdc: 7.3 TiB, 8001563222016 bytes, 15628053168 sectors Disk model: ST8000NM0055-1RM Units: sectors of 1 * 512 = 512 bytes Sector size (logical/physical): 512 bytes / 4096 bytes I/O size (minimum/optimal): 4096 bytes / 4096 bytes Disklabel type: dos Disk identifier: 0x00000000 Device Boot Start End Sectors Size Id Type /dev/sdc1 1 4294967295 4294967295 2T 0 Empty Partition 1 does not start on physical sector boundary. It still shows a 2TB partition. Starting the preclear for the... 3rd(?) time...
  13. Slightly more disturbing is the output of preclear_binhex.sh -f /dev/sdc Why does it have one partition of 2TB? I've got 2.4TB of free space left in my data array, so I'm restarting the preclear again. This will be finished before I manage to fill that up.
  14. Just because I had read it once did not mean that this little detail stuck with me several days later when the drive arrived and it was actually time to kick off the preclear. OK. The not-so-comedy of errors continues... It was nearly done - about 30% into the post-read. Honestly, this is the fastest I've ever had a disk preclear (in my memory). It was around 29 hours for an 8TB drive and it was doing the post read at about 200MB/s. I was trying to capture the whole text output to paste here to brag. Instead of hitting ctrl-shift-c to copy the selected text, it seems I hit ctrl-c and aborted the script. I did preclear_binhex.sh -t /dev/sdc to confirm that it was, in fact, precleared. At the end of the output, I get this: With the "Partition 1..." message highlighted in red. Obviously, the disk is cleared. Should I have any concern about the "Partition 1..." message? The original invocation was preclear_binhex.sh -f /dev/sdc exactly as specified in the instructions. I didn't set any other parameters or options.