Dissones4U

Hopefully moving from USB3 to USB2 fixes it (also remember that the quality of the flash is important, see the recommendations here). If you continue to have issues, I'm still learning how to read these logs but could the snippet below indicate trouble starting smb shares and would that restart the server? Looking at the next snippet it is clear that the server just snaps into reboot vs the hang originally described so I suspect that tailing the log would not help in this case.

@dgomel From lssci.txt The reviews I've found say that's not a great Flash Drive, have you ruled out the Flash as the problem? From syslog.text From lspci.text This is the only USB controller listed in the lscpi.txt, make sure your flash is not using USB3 Hopefully this leads you in the right direction, if not always include diagnostics. At this point I'd say you should also include the syslog from the syslog server folder.

I assume the original "disk disabled" problem is resolved and now your only issue is docker connectivity? So is this just cosmetic? My guess is that the above errors are generated due to a lack of internet access and are directly related to the docker updates showing as not available. With that in mind, I'd start with some basic network troubleshooting. Try to rule out one hardware component at a time such as the ethernet cable, NIC, (maybe SSD that houses Docker but I doubt it). Then move to software settings such as unRaid network settings, docker configuration, CA auto update applications settings etc. If you haven't already, you should enable syslog server just in case and always include your latest diagnostics... On a final note, if your original issue of 'disk disabled' is resolved you may want to either change the title to better reflect the current problem or start a new thread. A disabled disk is pretty basic whereas docker connectivity is a whole different specialty.

@luizmont I may be over thinking this but I wouldn't post my external IP and various ports on any forum, it's unnecessary. Although it may be less of an issue than I think I'm going to bring @johnnie.black in, just in case the image should be removed.

I'm by no means an expert but I suspect the SAS controller is either bad or loose because you have so many disks with read errors in the log. Before you do anything, I'd suggest that you setup your syslog server so that it persists after reboot, a link to instructions is in my signature. Then I'd re-seat the card and the cables and finish the server setup. Assign a parity disk, setup shares, and write some data. I think that 200-300 GB should suffice, then run a non-correcting parity check. If that shows errors stop and come back with info requested below. If parity is good then run extended smart on several (if not all) disks and come back with the info requested below. Once finished: Come back with new diagnostics (before rebooting) Attach the syslog from the server folder that you just setup Provide a screen shot of the main tab of the unraid UI showing any errors Honestly, I'm not sure whether I can help you or not but, I suspect that these steps will provide better diagnostics of the situation. This forum has a great group of guys so I'm certain that you'll get squared away, just be patient.

Diagnostics does not grab the syslog from the syslog server file folder it would only be since last reboot. You'd need to upload the file from the syslog folder. If you need to anonymize (I don't think you do) open the text file hit CTRL+F and search for the term(s) you want to delete or replace and do so. Might as well include diagnostics too.... Finally, looking at JB's link above did you read all of this about Global C-state Control (it's a link within the first link)

I think you mean create a list of files on the disk? If so use the webterminal command.... This (↑) will provide a list of top level directories (below I add the recursive option) but the output was too long and so I had to break the command down by folders then cut and paste into a text document. There may be an easier way but I don't know it

@Greeno237 At first blush I've compared your first diagnostic SMART with the second and there is something going on with parity Serial #ZCT1Y56S, the following attributes have gone up significantly in only a few weeks (346 hours). I'm not sure if this could cause your reboots or not. I'll keep looking... 195 Hardware_ECC_Recovered went from 0 ==> 69,487,100 7 Seek_Error_Rate went from 6,068,679 ==> 40,377,770 I just looked at your latest diagnostic, specifically the lsscsi.text, maybe someone can correct me but it looks like you're still on USB3, hopefully I'm misunderstanding that though (maybe this just means the drive is USB 3 capable?) I'm not seeing anything obvious, maybe someone with more experience will find something. Generally when I troubleshoot, the first thing I'd do is return the server to the most basic state, no VMs, No dockers etc. especially if you've made recent changes right before the problems started. Then after running for some time if the trouble is gone, re-enable one docker or VM at a time. You can also see jonp's post here and try tailing the log as described, in short he says that this method may be useful because

@itlists Assuming you followed the shrink array procedure keep in mind (from the wiki): Make sure that the drive or drives you are removing have been removed from any inclusions or exclusions for all shares, including in the global share settings. Shares should be changed from the default of "All" to "Include". This include list should contain only the drives that will be retained.

From here Consider keeping a backup folder outside of the array with periodic flash / diagnostics backups (including your key file which is in the tower-flash-backup.zip ==> config folder ==> Basic/Plus/Pro.key file ). Here is what mine looks like, I'll clean it up once a year or so....

I guess I should have realized that, but since I live in the land of hopes and dreams I assumed that the diagnostics pulled the log from the syslog server folder once it was setup. Good to know....

@RStock I don't still have this from when I upgraded in November so I tried to download from Dell without success. After Googling the file name PET410-011400C.exe I found this site, I downloaded the file and scanned with MalwareBytes and then verified the SHA256, the file checks out: From Dell: SHA-256: B777EB0CCF5133B2A6F775FDD268135550905A0AE9A3AF97863743007B40A632 From downloaded file using 7z: b777eb0ccf5133b2a6f775fdd268135550905a0ae9a3af97863743007b40a632 PET410-011400C.exe

Looking at the smart data for the disabled disk above, it looks fine BUT the disk was disabled for a reason, if you overwrite the existing disk (re-enable) then you'll have limited recovery options. Best practice dictates that you rebuild to a new disk, read here. I don't think you have FS corruption on the disabled disk as I'd expect to see an error like: "XFS (md3) Metadata corruption detected" or something similar. even so, do not proceed with any type of rebuild, format etc until you understand the potential for data loss. I can't tell for sure if it is or not but make sure your flash is on USB 2 not USB 3, otherwise it won't work and could cause various errors including disabling disk due to dropped connectivity. Is the disabled disk on USB 3 as an unassigned device? USB 3 may affect that too but someone else will need to confirm... Some of these may indicate a network issue but I think they are irrelevant to your disabled disk. Squid's Fix Common Problems is really good about telling you what's wrong, if you can't figure these errors out then consider posting in the CA thread for support.

definitely yes, it has way more info such as SMART data etc (this also contains syslog) I'm no expert but the post Iinked to above suggests that even when GUI freezes direct connection with monitor and keyboard (mine boots non GUI mode) may provide a response that is unattainable otherwise?

yes Honestly I can't tell for certain but it looks like your flash may be using the xhci_hcd driver. Once memtest is complete, make sure your USB is not in the 3.0 port, use of USB 2 is preferred. Someone better at reading the log output above may be able to confirm or deny.... If this is happening all of the time you should familiarize yourself with the unclean shutdown thread as well as the FAQ, specifically the maintenance and troubleshooting as you may well end up with corruption in the file system after repeated unclean shutdowns. If you should get a disabled / emulated disk then make sure you come here and get help before rebooting.... Of course your first post should always include the diagnostics zip file by going to the Tools tab, clicking on the Diagnostics icon, and then clicking on the Download button and uploading the zip here. If you can't access the GUI then see here for instructions on using the command line.

This is a little off topic but worth mentioning, you should feel totally comfortable experimenting with your setup because your array should never be your only copy of irreplaceable data. Unraid provides data redundancy which in and of itself is not backup, in fact, data redundancy is only meant to ensure high availability. Unraid will never protect you from all types of data loss (think file system corruption) and in no way constitutes a backup if it's your only copy. To take it a step further I'd suggest that redundancy is far less important for most home users than a legitimate second copy of critical data. Just an FYI...

This definitely adds some clarity too, thank you JB. I've been trying to understand file system corruption better because I've noticed that there have been several users who have encountered it lately and most of them seem to mishandle the recovery. I think that part of my confusion was not understanding emulated disk corruption vs physical disk corruption, but I think I'm starting to get it. The key being emulation has to be verified before anything is done and then knowing that the physical disk is probably bad but the smart data will help to determine whether or not it can be salvaged. I think I'll prepare a new disk that is specifically for rebuilding disabled / emulated disks as soon as I can afford it. Then, best case scenario, the old disk can be reused once it's been thoroughly tested. All of you guys were great in clarifying this, thank you.

Just so that I have a clear understanding of this scenario, it sounds like when JB said: "the best option here would likely have been to re-enable the disk, instead of rebuilding, in some cases when a disk gets dropped filesystem might get some corruption on the emulated disk." that he did not mean to use the "parity is already valid" function (as I first thought) but rather the FS repair should have been done first, and then the re-enable and rebuild procedure should have followed. Assuming of course that the preferred option referred to by johnathanm, to rebuild to a different disk, wasn't on the table. Hopefully I have this right now, it does seem to make more sense...

I think he is referring to the option that makes unraid trust that "parity is already valid" this avoids a rebuild. See here... I've not used this procedure yet so if you have questions make sure to ask someone before making changes.

@trurl I'm trying to learn how to read these logs better, what line should be telling me to think memtest here?

Can you do a graceful shutdown, if not then wait for one of the other guys to help, you don't want to repeatedly force shutdown. My guess is that this has something to do with the Flash device, is it on a USB 3? If so use USB 2.0 [11:0:0:0] disk SanDisk Cruzer Fit 1.00 /dev/sds /dev/sg0 state=running queue_depth=1 scsi_level=7 type=0 device_blocked=0 timeout=30 dir: /sys/bus/scsi/devices/11:0:0:0 [/sys/devices/pci0000:00/0000:00:14.0/usb3/3-11/3-11:1.0/host11/target11:0:0/11:0:0:0] Also your 4TB Seagate with serial # Z1Z23XWV00009411RSUV seems to be disabled, I can't tell why Finally, implement syslog server, this log is only since last 17 minutes... Feb 28 15:07:02 Predator kernel: usb 3-11: USB disconnect, device number 4 Feb 28 15:07:02 Predator kernel: md: unRAID driver removed Feb 28 15:07:02 Predator emhttpd: shcmd (70): modprobe md-mod super=/boot/config/super.dat Feb 28 15:07:02 Predator kernel: FAT-fs (sda1): Directory bread(block 29976) failed Feb 28 15:07:02 Predator kernel: FAT-fs (sda1): Directory bread(block 29977) failed Feb 28 15:07:02 Predator kernel: FAT-fs (sda1): Directory bread(block 29978) failed Feb 28 15:07:02 Predator kernel: FAT-fs (sda1): Directory bread(block 29979) failed Feb 28 15:07:02 Predator kernel: FAT-fs (sda1): Directory bread(block 29980) failed Feb 28 15:07:02 Predator kernel: FAT-fs (sda1): Directory bread(block 29981) failed Feb 28 15:07:02 Predator kernel: FAT-fs (sda1): Directory bread(block 29982) failed Feb 28 15:07:02 Predator kernel: FAT-fs (sda1): Directory bread(block 29983) failed Feb 28 15:07:02 Predator kernel: FAT-fs (sda1): Directory bread(block 29984) failed Feb 28 15:07:02 Predator kernel: FAT-fs (sda1): Directory bread(block 29985) failed Feb 28 15:07:02 Predator kernel: md: unRAID driver 2.9.13 installed Feb 28 15:07:02 Predator kernel: read_file: error 2 opening /boot/config/super.dat Feb 28 15:07:02 Predator kernel: md: could not read superblock from /boot/config/super.dat Feb 28 15:07:02 Predator kernel: md: initializing superblock Feb 28 15:07:02 Predator emhttpd: Unregistered - flash device error (ENOFLASH3) Feb 28 15:07:04 Predator unassigned.devices: Error: Cannot mount device '/dev/sds1' with label 'UNRAID'.

@chris_netsmart It may not say "used server pull" that is simply saying that it is used and someone tested it coming from a working server. If buying from eBay read carefully, some sellers may be selling new units from china. I've purchased from these guys before. This guy says "pulled from working unit" This guy seems to be thorough, talks about unraid in description and mentions that he weeds out fakes.

I think this can be done but I'm not 100% sure and it's moderately complex. Please do your due diligence before making any changes as I'm no expert, also carefully document your current state prior to changes for easy reversion. If any of the networking experts out there want to chime in please feel free as I'm making a best guess without equipment in hand. First, both gateways need to be on the same address range so for example: Gateway 1 is currently 192.168.6.69 for ease I'd change this to 192.168.1.1 <--- should be faster of the two, it will provide DHCP and house most of your devices Set the DHCP address range for Gateway 1 (192.168.1.1) to go from 168.1.3 - 168.1.150 (this is up to you, just make sure you don't allow the same address elsewhere) Gateway 2 is currently 172.30.1.254 for ease I'd change this to 192.168.1.2 <-- this will not provide DHCP after the two routers are connected Set the address range for Gateway 2 (192.168.1.2) to go from 168.1.151 - 168.1.200 (DHCP for this device will be disabled just before you connect them) Next, make sure that all works as expected for each router individually and configure your unRaid so that it works as it does now Once you are confident that each Gateway is now on the same address range and working correctly independent of one another as it does now, you can disable DHCP in Gateway 2 (192.168.1.2) and connect the two routers via ethernet cable to the LAN port. Once the Gateways are connected Manually configure any device that you want to use Gateway 2 Do this by configuring the device IP to fall within the original 168.1.151 - 200 range and assigning the gateway as 192.168.1.2. It is critical not to use an IP that is within the DHCP range of Gateway 1. As I've said I'm no expert and I don't have the equipment here to do a lab test or I would as it seems like an interesting project so hopefully this will at least get you going in the right direction. (Just an FYI an easier fix may be to make a second profile on your machine that logs into the second network and just switch profiles?)

I think this is mostly personal preference and is dependent on your file structure, I mean you don't want to spin up 5 disks to binge watch 5 episodes right. Other than that, well, the best laid plans of mice and men as they say! Here is an example of my TV share, it is set to split the top two directories; Top most share (TV - splits as needed) ---> Second Level (Series Names - splits as needed) ---> Third Level (seasons - folders stay completely intact) Here is an example of my Movies share, it 's also set to split the top two directories; Top most share (Movies - splits as needed) ---> Second Level (Genre - splits as needed) ---> Third Level (Actual movie - folders stay completely intact) Hope this helps...

This may not be possible, I've tried for the same reason and unRaid insists that you have a sending address in there...