bsim

Members
  • Posts

    191
  • Joined

  • Last visited

Everything posted by bsim

  1. But will a drive be flagged/emulated by unraid (Red X) because of too many uncorrectables? Or does there have to be something else wrong with the drive?
  2. Scratching my head on this one...unraid fluke? Dual parity 12TB, 6.12.6, all XFS drives, monthly checks, 0 errors on last parity check. 1. I precleared a 5TB that I've had as a hot spare for my array, 0 errors, shows in unassigned devices. 2. Removed Smart Failed/But Still Good 4TB from array (10 Uncorrectables) 3. Replaced With Pre-cleared 5TB 4. Started array with Rebuild, 0 Errors on rebuild 5. Red X a different 5TB drive!? ****The drive does have several uncorrectables**** Will a drive red X and emulate if it has too many uncorrectables? And not show any errors in the rebuild?
  3. I've read extensively on the forums and on google, but haven't found someone with my exact, relatively simple interests... I have multiple large external hard drives (22TBx) and I have extensive scripts (using rsync) that do automatic backups when powered on (unassigned devices script) every month or so. My question is, what drive format should I use for the external drives? ***I would love to turn my current backup (based-on-changed-files) to verify integrity of backed up files based on FS checksums that would refresh files if bitrot occurs. I'm not extremely worried about accessing the drives from outside machines (windows/ntfs), and definitely don't have any computational limitations. I'm thinking BTRFS and ZFS would be my best bet, but: Can rsync backups use the FS checksumming built into these file systems to determine differences? (better option than rsync?) Does FS checksumming in BTRFS/ZFS happen automatically in the background without complications (outside of the unassigned devices gui)? Is scriptable command line FS checking easy for BTRFS/ZFS? (currently using xfs_repair for XFS)
  4. I see the point for being careful with automatic parity corrections, but with how stable my system is hardware wise, it's worked for years flawlessly. Just every once in a while i get a burst of sync errors on a 140TB array, the number of errors don't seem like a major issue vs the potential problems automatic parity correction would save me from. I considered installing some type of indexing/checksum software to watch for any type of bit rot or actual corruption...just haven't got around to it. It would be awesome if there was a way to translate the location of incorrect bits to at least a controller/drive/file...would help greatly in my case. I don't see why the main unraid driver wouldn't be able to spit out the details of the parity issue when doing corrections, seems like it would be a great diagnostic tool.
  5. Are corrected parity sync errors truly corrected or will there be some sort of hidden corruption? The errors are not recurring and I often go several months/checks with no errors detected. The hardware has been stable/unchanged for years now. If I can't determine the issue using smart, obvious unraid errors or any log files then why wouldn't a correcting parity check just save me time?
  6. Running the latest unraid pro 6.10.3 with dual parity using mirrored SSD's (240GB, mover nightly) as cache drives... I run a large array and run parity checks automatically every month. Most times I get no parity errors. But sometimes get a few thousand corrected parity errors. I have a ups that does a graceful shutdown (but i guess it's possible that the shutdown process takes longer than the ups power could hold out for waiting for unraid to shutdown the array). I do have power outages, but the UPS that can stand 20 minutes of run time before it tells unraid to issue a shut down. No drives have red balls or have any issues with SMART 5, 187, 188, 197 or 198 (Backblaze recommended) The physical server has not been moved/opened in several months. Two questions: 1.) What files in the diagnostics download (saved immediately after sync errors) would show me what files/drives reported the sync errors? What am I looking for in the files that would be able to tell me the details? 2.) Do corrected sync parity errors (with dual parity) mean that the data was corrected and no corruption has occurred?
  7. Is there a place for feature requests? Specifically, 1.) Creating a button to easily archive or delete log files 2.) A button to enable autoscroll when viewing the log...Really handy when clicking on the refresh button to watch the scripts progress. 3.) Easy way to specify log file location (storing on the flash drive is handy)
  8. 6.9.2 is the version I found it starting as well...From the research it seems like a common bug in SMBD...can anyone confirm this or if it's due to client requests vs server?
  9. While reviewing syslog, I found thousands of these repeating lines for several months now...no real problems with the server, but I can't find any type of explanation/others having the same problem anywhere to see if this is a client connection issue or a server side issue. Nov 1 03:23:11 UNRAID smbd[17025]: [2021/11/01 03:23:11.874565, 0] ../../source3/smbd/smbXsrv_client.c:663(smbXsrv_client_connection_pass_loop) Nov 1 03:23:11 UNRAID smbd[17025]: smbXsrv_client_connection_pass_loop: got connection sockfd[847] Anyone have any ideas?
  10. I get it, but I can't figure out why the data would be dumped to a file when it started on an actual device that was connected...shouldn't it just error out and disappear completely from the dev folder?
  11. I would think that if the drive wasn't part of the array (abruptly disconnected), it would have just errored out...If I would have not started from basics to figure out why the drive was being wonky, I could have sworn it was a very specific hard drive defect. The fact that the utility was writing data directly to the /dev folder and it was being written as a file rather than a device is really screwy. All three drives are now pre-clearing without issue.
  12. Nevermind...It has to be a bug in the preclear plugin. by chance ssh'ed (WinSCP) to unraid, and found that the /dev/sdac was actually a 30GB file. Deleted the file, unplugged and replugged sata connection for that drive and VOILLA! My problem must be with a bug or unhandled case in the preclear script, previously, when I was preclearing the drives, I had unplugged one of them in mid preclear...and even after attempting to stop the preclear on that drive, it must have choked on something. After wiping out that /dev file, this came up in the preclear!
  13. Racking my brain on this one... 3 new 12TB drives (identical shucked drives) (they do not have disable pin) verified smart ok seperate windows machine BEFORE shuck (HD Tune Pro), ALL THREE OK verified smart ok seperate windows machine AFTER shuck (HD Tune Pro), ALL THREE OK wiped all partitions, ran read and write tests for all three, no problems attached 3 drives to server, preclear plugin identified 2 fine, 3rd drive does not show "identity", only "_" (Attached pics) switched sata connection with one of the working drives, 3rd drive still does not identify switched power connection, 3rd drive still does not identify from command line, attempted to manually pull smart (smartctl -H /dev/sdac) 2 drives results passed, possible bad drive "INQUIRY failed" Is this just an oddly bad drive or is there some sort of cached drive data going wonky?
  14. If a docker app stores it's data in the appdata location, what else would be stored in the docker image?
  15. NM...batting a thousand this afternoon...settings for storage locations right in the docker settings...oops.
  16. Thank you for the help...The df works, but I used "sudo du /mnt/user/system..." for each of the folders on the cache and found the full size docker image being recreated, it looks like the the docker image is not counted in the file size listing via smb/sftp, and somehow (me or automatic) the docker image initial size was set to 100GB (which explains the reboot, but not before the reboot). I disabled docker, deleted the image and everything is empty again. does docker.img have different permissions causing smb shares not to list it? For the future, is there an easy way to store the docker image/each docker data to the array instead of the cache drive? Thank you for your help!
  17. I'm running the latest 6.8.1 (problem existed in 6.8.0) and have a btrfs cache pool (mirrored identical 120GB SSDs) that initially showed 75GB used, I confirmed they are almost EMPTY through the SMB cache share and through SFTP...I first ran the mover script a few times, still no change, then thinking it could just be a glitch, I rebooted unraid, and it now shows 111GB USED...verifying through the SMB cache share, still only a few GB are used. Is this a BTRFS issue or an unraid issue? Are there files on the share that possibly would not show up in an SMB listing or an SFTP? listing?
  18. The main reason I had to build all of the variables manually was because the unraid system can't pull in the compose file that the docker(s) came with (using the gui)...the compose file actually has all of the seperate dockers configured in one file...What differences are there between what unraid needs and what a typical compose file looks like? Has anyone considered making a plugin to translate standard compose files into what unraid requires? Or even the ability of unraid to export a compose file for a configured docker so that users can add apps to the community apps easier for others?
  19. Is it possible to create a compose file for each of the dockers that I have if they didn't come from compose files originally?
  20. will docker compose pull all of the settings (variables, ports...) that ive setup into something that unraid would be able to reproduce easily for others?
  21. So, I've setup Ambar from dockerhub using Community apps...But it has several very specific dependency dockers that are required to run it that are also housed in the same Ambar docker repository as part of the project. After getting the several dockers setup (with all of their seperate settings), Is there a way to create an overall unraid template for pulling all of the seperate ambar dockers together and with all of their seperate settings that I scoured to find?
  22. So...I've finally got Ambar docker (https://ambar.cloud/docs/installation/) up and running with a few bugs yet to work out...but during the celebration of completing the massive frustration of configuration options of all of the different dockers necessary, I had a heart drop for a moment. The user shares all disappeared (but disk shares and drive space didn't change), so remembering that user shares are dynamically created, and figuring something with unraid just crashed in the background, I did to clean reboot (after waiting for a long while for attempt to unmount user shares). Everything came backup up ok afterwards, but I'm thinking that my docker mappings had something to do with the crash. I was relating docker location(s) IE "/usr/share/elasticsearch/data" directly to "/mnt/user/appdata/ambar" via docker paths. At some point as I was restarting several dockers at once a few times over, the crash occurred. (I pulled a diag before restarting just in case anyone wants to fish out the source issue). Is there something wrong with directly relating the above folders that would cause the user shares to crash like that? Is there a better/safer way to refer docker data to the cache drive shares ?
  23. So...I've finally got Ambar docker up and running with a few bugs yet to work out...but during the celebration of completing the massive frustration of configuration options of all of the different dockers, I had a heart drop for a moment. The user shares all disappeared (but disk shares and drive space didn't change)...so remembering that user shares are dynamically created, and figuring something with unraid just crashed in the background, I attempted to clean reboot (after waiting for a long while for attempt to unmount user shares). Everything came backup up ok afterwards, but I'm thinking that my docker mappings had something to do with the crash. I was relating docker location(s) IE "/usr/share/elasticsearch/data" directly to "/mnt/user/appdata/ambar" via docker config paths. At some point as I was restarting several dockers at once a few times over, the crash occurred. (I pulled a diag beforehand just in case anyone wants to fish out the source issue). Is there something wrong with directly relating the above folders that would cause the user shares to crash like that? Is there a better way to refer docker data to a cache drive (that doesn't move certain folders to the array)?
  24. Holy crap huge difference! So.... So with only a few standoffs different (a few movable, a few permanent onces electrical taped well), my 4U Supermicro 24 bay case works well with the supermicro x9dr3-ln4f+ motherboard (was worried about the standoffs and the size difference), now running dual Intel E5-2637V2 's...parity check now runs at the drive speeds, and using the array during parity is even just as fast, but does slow down the parity check, but just slightly. I don't notice any server lag when using the array during parity checks either. I wanted to share my adventure with the supermicro x9dr3-ln4f+ motherboard upgrade...Ordered from ebay for 180$ (included face and processor sinks), received my processors (120$ for the pair), installed everything...wasn't getting any post or beeps. pulled all but one processor...nothing....swapped processors...nothing... Previous research had shown that unless the motherboard bios was at least 3.0, it would not be compatible with V2 processors, so it was most likely my bios version....(great...bios upgrade without a processor? i thought)... Luckily, the BMC on X9 boards and above has a feature that allows remote bios updates without any processors or memory being installed! The one catch was that there was a required "OOB license" that was necessary in order to use either the direct web interface of the BMC or use the command line "sum" from supermicro. Luckily, there is a Perl script on the web from a frustrated user (that had reversed the BMC license code) that allows you to generate your own OOB license key. It works on all boards as of recent. So, I was able to update the BMC firmware easily through the interface, then update the motherboard BIOS directly with the new BMC interface. After that it worked like a charm! So where does it leave me with figuring out what my issue was? I'm still left with either the motherboard didn't have enough lanes to support the hard throughput, a possible bug in the bios that caused issues, or unraid wasn't quite supporting the motherboard somehow (driver bugs?). Thanks guys for your help! Time to ebay!