August 11, 201411 yr very strange...all of a sudden, my user share isn't showing any content (0 files), and when i click on the unRAID SMB share in the Finder, it will not allow me to change the "Connected As:" username. i have rebooted the server twice already, without any changes to this behavior, and don't know what to do now...it all worked fine until yesterday afternoon, but started to act up in the evening (dropped my share a couple of times)...i did upgrade a few plugins of the Dynamix UI today and rebooted the server afterwards, but don't see how that would have anything to do with this behavior, since the Samba component is independent of this, right? i attached the log contents that i get when i click on the "log" button in the Web GUI, since i can't reach the USB Stick with the full system Log right now (am at a remote location), and i see that there are a bunch of permission problems in there and some other errors that kinda make my pulse go up a bit...hope nothing went seriously wrong here! i should probably add that i had a disk go bad on me a few days ago and had to rebuild parity, which went without a hitch (as it always has)...or so it said. now what? log_2014-08-11_1156AM.txt
August 11, 201411 yr Maybe the disk13 (md13) is the one that went bad few days ago. Looks like there were many read errors pertaining to this drive. When unraid has trouble writing to a drive, it puts the entire system in read only mode. In this case, it appears the drive 13 had many I/O errors. I would login to the shell and try to recover the data from this drive ASAP. Another alternative is to use the cache drive and reconstruct the data .. would solicit recommendations from more experienced users here.
August 11, 201411 yr Author yes, it is drive 13 that went bad, and i replaced it with a brand new WD 3TB Red drive...the Parity check that followed went through without a hitch, but the log file appears to say otherwise. what is the best way to proceed from here? i need to leave town tomorrow, so if it requires physical interaction with the array, i only have a few hours left to do so. could anyone give me a "play-by-play" of how i can safely get the array back into read-write mode? curiously enough, i *can* access the individual drives through my Slink (Bonjour) connected mounts...unRAID appears in my local Finder (thanks to the magic of Bonjour), and i can see all of the drives (disk1-disk14 plus flash and the user share "unRAID") and access their contents, including those on drive 13...although the unRAID user share doesn't show any contents at all...why would *only* the user share not show the directories contained therein (which are the 5 main directories i have all of my data in)?
August 11, 201411 yr When unraid has trouble writing to a drive, it puts the entire system in read only mode.No. When unraid tries to write to a drive and it fails, it red balls the drive and life goes on pretty much uninterrupted for the user. File system corruption is the only mechanism I know of that sets a drive to read only, and even then it's limited to the drive(s) that have the corruption, it's never applied system wide unless all drives are corrupt.
August 11, 201411 yr Shutdown safely, check all SATA and power connections, reboot in SAFE MODE, then post a complete syslog.
August 11, 201411 yr Author since i am not at the location where my unRAID server is located, i can not check the connections, but since it hasn't been touched in months and has worked fine until just yesterday, i'd like to assume that the connections are still fine. sorry for the question, but how do i reboot in safe mode? i did attach the most recent full system log, since i just saw that i *can* access all of the individual shares via Bonjour. your advice is appreciated! syslog-20140811-114857.txt.zip
August 11, 201411 yr since it hasn't been touched in months and has worked fine until just yesterday, i'd like to assume that the connections are still fine.Not a good assumption. Heat cycling and vibration can cause connection issues. Not saying it is an issue here, just that a long time operating without issue doesn't rule out connection problems.sorry for the question, but how do i reboot in safe mode?It's in the console boot menu, assuming you updated to the most recent version when you updated the core unraid files. If you absolutely can't operate with a local keyboard and mouse, you should be able to edit the menu files on the flash share to make safe mode the default temporarily. I haven't personally done that, so I can't advise further.
August 11, 201411 yr Author i am running version 5.0.4 of the unRAID server, and i had access to the console boot menu, i know i could do so, but i only have remote access to the server until i leave to Germany for many months. does anyone have instructions as to how i can boot the server in safe-mode by modifying the files on the flash drive? my current go script is as simple as it gets: #!/bin/bash # Start the Management Utility /usr/local/sbin/emhttp & but since i have plugins loaded under the Dynamix tab, i suppose something there *could* be the culprit. having said that, i did just rename the plugins folders, so that none of them can load, and rebooted the server...the unRAID user share still shows as empty, and all of the other disk shares still show up as usual and are fully accessible...what could be resulting in *just* the user share not behaving properly...why is it empty? most recent log file is attached. syslog-20140811-145130.txt.zip
August 11, 201411 yr Author don't have one...but thanks for your suggestions and feedback, jonathanm! i just don't get why *only* the user share won't show any content. also, now that i am back to the standard vanilla Web GUI (without any plugins), the log button there brings up a log that doesn't have *any* of the errors that the log in the Dynamix Web GUI showed...maybe they are still there, and only in the full log (which i also uploaded last), but why would the vanilla Web GUI and the Dynamix Web GUI show different logs? al the vanilla Web GUI shows is this (is this a different log than in the Dynamix one?): /usr/bin/tail -f /var/log/syslog Aug 11 16:02:36 unRAID kernel: mdcmd (62): spindown 4 Aug 11 16:02:37 unRAID kernel: mdcmd (63): spindown 6 Aug 11 16:02:37 unRAID kernel: mdcmd (64): spindown 7 Aug 11 16:02:38 unRAID kernel: mdcmd (65): spindown 9 Aug 11 16:02:38 unRAID kernel: mdcmd (66): spindown 10 Aug 11 16:02:39 unRAID kernel: mdcmd (67): spindown 11 Aug 11 16:02:39 unRAID kernel: mdcmd (68): spindown 12 Aug 11 16:02:40 unRAID kernel: mdcmd (69): spindown 13 Aug 11 16:02:50 unRAID kernel: mdcmd (70): spindown 14 Aug 11 16:03:11 unRAID kernel: mdcmd (71): spindown 1
August 12, 201411 yr The latest syslog you attached shows that you are still loading plugins, and you have dynamix installed incorrectly. The main dynamix webgui file must be in /boot/plugins so it gets installed before the other dynamix plugins in /boot/config/plugins. Doubt that this is the actual cause of your problems, though. Are you saying that you have a user share that is actually named unRAID? And this user share named unRAID is empty? Do any of your drives have a top level folder named unRAID (or whatever the actual name of the user share in question is)? Any top level folder of cache or an array drive is considered a user share by unRAID, and conversely, a user share must have a top level folder on cache and/or array disk(s).
August 12, 201411 yr Author yes, trurl, due to habits formed over 25 years of digital production work, i created an unRAID folder on every drive, and within it, i have my Media, Design, Production and Data folders that contain all of my work of the past 25 years...so while i do see the unRAID user share, the folder itself doesn't show any of the aforementioned folders...i do see them spread out over all of the individual drive shares, though, so i know they are there...the system just doesn't seem to serve out the user share correctly, for some reason. so you're saying that the dynamix.xxx.plg plugins folder needs to be in the root of the flash drive, and the plugins folder that is now in the root inside of the config folder? i wonder how i could have screwed it up that badly? and where do they get loaded if my go script only says (and has only been, for months): #!/bin/bash # Start the Management Utility /usr/local/sbin/emhttp &
August 12, 201411 yr Looking at that syslog again, it looks like the only plugin that is out of place is dynamix.plugin.control. That plugin should be in /boot/config/plugins (user plugins), but it is in /boot/plugins (system plugins). This causes it to try (and fail) to install before the main dynamix plugin. unRAID automatically installs some things for you when it boots, in this order: 1)Linux packages in /boot/extra 2)System plugins in /boot/plugins (in alphanumeric order. This is why plugin.control is trying to install before webGui). 3)User plugins in /boot/config/plugins (in alphanumeric order) 4)Go script is executed Most likely you installed dynamix.webGUI and then dynamix.plugin.control manually, but both in the /boot/plugins folder. Then you used plugin.control to install the other dynamix plugins, and it put those in the right place, /boot/config/plugins. Back to your user share problem. Are you saying that the individual drives have /unRAID/Media, /unRAID/Design, etc.? And you can see these folders on the individual disks? But the user share named unRAID does not have Media, Design, etc. folders in it? When you look at the individual drive contents, does it look like your files are probably all there? Drive 13 is still showing problems. Not clear that should cause your user share problems either, but it is probably the main thing to be concerned about. Is drive 13 showing green in the unRAID webGUI?
August 12, 201411 yr Author very good comprehension, trurl! regarding the user share vs disk share issue you've got it exactly right...i have unRAID/Design, unRAID/Media, etc. folders on all of my individual disks, and all of them have all of the data on there and accessible, as it should be, but the actual unRAID user share itself shows 0 folders and 0 files. and yes, drive 13 is green and passed the parity check with 0 errors...i can even access it as a disk share and see all of my data on it, so this is a mystery to me. as for the dynamic plugins, can i move them into their proper order manually, and will they work correctly then, or do i have to uninstall and re-install them again? if i can move them manually, would i just move dynamix.plugin.control to /boot/config/plugins and leave everything else where it is? i may have messed up on that install in the first place, because when i look at the files in their folders, there is a *folder* called dynamix.plugin.control inside of /boot/plugins, which only has a read me file in it, and on the same level as the folder is a file called dynamix.plugin.control-2.0.3-noarch-bergware.plg and a file called dynamix.webgui-2.2.5-i486-1.txz and dynamix.webgui-2.0.5-noarch-bergware.plg...so maybe i didn't extract something properly or didn't clean up after myself? do you know what to make of that and what is to be trashed or what is to be extracted and how? sorry for all of these rather naive questions, but haven't had to look at any of this in over a year and have completely lost track of what goes where and why. but if i understood you correctly, i need to have a file called "dynamix.plugin.control" inside of /boot/plugins...and since i only have a folder by that name with a lone read me file in it, i must have done something wrong, right? thanks so much for your patience with my rather shoddy situation and administration of it.
August 12, 201411 yr Just move dynamix.plugin.control-2.0.3-noarch-bergware.plg from /boot/plugins to /boot/config/plugins. That will allow dynamix.webgui-2.0.5-noarch-bergware.plg to install first like it is supposed to. If you look at the Shares page, do you see the user share named unRAID? And if you click on it, it should take you to the settings for that user share. Are those settings correct?
August 12, 201411 yr Author funny you'd mention it...i was just about to ask you about that share settings panel...both fields ("Included disks" and "Excluded disks") are empty...does an empty field imply *all* disks? but if that is so, then wouldn't both fields include and exclude all disks and hence the problem? i didn't touch this panel, ever, so i don't see why it would have changed from when it was all working correctly. i'll move the plugin file before doing a reboot...thanks!
August 12, 201411 yr Included means only include disks in the list. Excluded means don't include disks in the list. If both are empty then all disks are included. If you have never set a share's settings, then unRAID gives it default settings. The part I'm interested in is the Security settings for the share. Default would be Public. Are you using SMB, AFP, NFS? Also, what does ls -al /mnt/user show?
August 12, 201411 yr Author i drove all the way across town to the studio where the unRAID server is located, just to re-seat drive 13 and check other connections...not sure whether it did anything, but i attached the newest log. i hope the dynamic plugins loaded correctly too now. how would i see the Security settings for the share? i am using SMB and AFP is enabled, although it only seems to work as a TimeMachine share and nothing else, if i remember correctly. and yes, Share settings have User Share enabled. i'm gonna have to drive back across town now...i know it's probably late where you are, and i've gotta fly out to Austin at the crack of dawn, but will be up until midnight (California time), so if you've got some energy and advice left, i can keep going for as long as you can...thanks so much for all of your time so far, trurl! oh, and since you asked: root@unRAID:~# ls -al /mnt/user total 24 drwxrwxrwx 1 nobody users 184 2014-08-11 19:36 ./ drwxr-xr-x 17 root root 0 2014-08-11 20:17 ../ -rw-rw---- 1 nobody users 6148 2014-08-11 18:09 .DS_Store drwxrwx--- 1 till users 112 2012-09-30 21:43 .TemporaryItems/ drwxrwx--- 1 nobody users 72 2011-09-18 02:06 .Trashes/ -rw-rw-rw- 1 till users 4096 2014-08-11 19:36 ._.DS_Store -rw-rw---- 1 nobody users 4096 2010-09-11 05:55 ._.TemporaryItems -rw-rw---- 1 till users 4096 2012-09-30 21:43 ._.apdisk -rw-rw---- 1 nobody users 250 2011-07-13 02:06 .apdisk drwxrwxrwx 1 nobody users 72 2014-01-23 10:45 TimeMachine-rMBP/ drwxrwx--- 1 nobody users 184 2014-08-11 20:04 unRAID/ syslog-20140811-145130.txt.zip
August 12, 201411 yr Also, what does ls -al /mnt/user show? How about ls -al /mnt/user/unRAID as well.
August 12, 201411 yr Author root@unRAID:~# ls -al /mnt/user/unRAID /bin/ls: reading directory /mnt/user/unRAID: Permission denied total 0 so, there do appear to be some permission problems that somehow crept into this setup...not sure how, and how to fix them, but the above is telling, no?
August 12, 201411 yr Author now that i've got the Dynamix Web GUI up and running (i hope it's loading correctly now, trurl!), i took another look at its "log" output, and what i see does not look good...it really seems like that brand new 3TB WD Red drive has problems...not sure why it passed parity without errors, but something's seriously wrong! here's what the log says: tail -n 40 -f /var/log/syslog Aug 11 22:21:46 unRAID shfs/user: shfs_readdir: fstatat: _Downloads (13) Permission denied Aug 11 22:21:46 unRAID shfs/user: shfs_readdir: readdir_r: /mnt/disk13/unRAID (13) Permission denied Aug 11 22:21:47 unRAID kernel: REISERFS warning: reiserfs-5090 is_tree_node: node level 16439 does not match to the expected one 1 Aug 11 22:21:47 unRAID kernel: REISERFS error (device md13): vs-5150 search_by_key: invalid format found in block 461930501. Fsck? Aug 11 22:21:47 unRAID kernel: REISERFS error (device md13): vs-13070 reiserfs_read_locked_inode: i/o failure occurred trying to find stat data of [447288 683085 0x0 SD] Aug 11 22:21:47 unRAID shfs/user: shfs_readdir: fstatat: _Downloads (13) Permission denied Aug 11 22:21:47 unRAID shfs/user: shfs_readdir: readdir_r: /mnt/disk13/unRAID (13) Permission denied Aug 11 22:21:47 unRAID kernel: REISERFS warning: reiserfs-5090 is_tree_node: node level 16439 does not match to the expected one 1 Aug 11 22:21:47 unRAID kernel: REISERFS error (device md13): vs-5150 search_by_key: invalid format found in block 461930501. Fsck? Aug 11 22:21:47 unRAID kernel: REISERFS error (device md13): vs-13070 reiserfs_read_locked_inode: i/o failure occurred trying to find stat data of [447288 683085 0x0 SD] Aug 11 22:21:47 unRAID shfs/user: shfs_readdir: fstatat: _Downloads (13) Permission denied Aug 11 22:21:47 unRAID shfs/user: shfs_readdir: readdir_r: /mnt/disk13/unRAID (13) Permission denied Aug 11 22:21:47 unRAID kernel: REISERFS warning: reiserfs-5090 is_tree_node: node level 16439 does not match to the expected one 1 Aug 11 22:21:47 unRAID kernel: REISERFS error (device md13): vs-5150 search_by_key: invalid format found in block 461930501. Fsck? Aug 11 22:21:47 unRAID kernel: REISERFS error (device md13): vs-13070 reiserfs_read_locked_inode: i/o failure occurred trying to find stat data of [447288 683085 0x0 SD] Aug 11 22:21:47 unRAID shfs/user: shfs_readdir: fstatat: _Downloads (13) Permission denied Aug 11 22:21:47 unRAID shfs/user: shfs_readdir: readdir_r: /mnt/disk13/unRAID (13) Permission denied Aug 11 22:21:47 unRAID kernel: REISERFS warning: reiserfs-5090 is_tree_node: node level 16439 does not match to the expected one 1 Aug 11 22:21:47 unRAID kernel: REISERFS error (device md13): vs-5150 search_by_key: invalid format found in block 461930501. Fsck? Aug 11 22:21:47 unRAID kernel: REISERFS error (device md13): vs-13070 reiserfs_read_locked_inode: i/o failure occurred trying to find stat data of [447288 683085 0x0 SD] Aug 11 22:21:47 unRAID shfs/user: shfs_readdir: fstatat: _Downloads (13) Permission denied Aug 11 22:21:47 unRAID shfs/user: shfs_readdir: readdir_r: /mnt/disk13/unRAID (13) Permission denied Aug 11 22:21:47 unRAID kernel: REISERFS warning: reiserfs-5090 is_tree_node: node level 16439 does not match to the expected one 1 Aug 11 22:21:47 unRAID kernel: REISERFS error (device md13): vs-5150 search_by_key: invalid format found in block 461930501. Fsck? Aug 11 22:21:47 unRAID kernel: REISERFS error (device md13): vs-13070 reiserfs_read_locked_inode: i/o failure occurred trying to find stat data of [447288 683085 0x0 SD] Aug 11 22:21:57 unRAID shfs/user: shfs_readdir: fstatat: _Downloads (13) Permission denied Aug 11 22:21:57 unRAID shfs/user: shfs_readdir: readdir_r: /mnt/disk13/unRAID (13) Permission denied Aug 11 22:21:57 unRAID kernel: REISERFS warning: reiserfs-5090 is_tree_node: node level 16439 does not match to the expected one 1 Aug 11 22:21:57 unRAID kernel: REISERFS error (device md13): vs-5150 search_by_key: invalid format found in block 461930501. Fsck? Aug 11 22:21:57 unRAID kernel: REISERFS error (device md13): vs-13070 reiserfs_read_locked_inode: i/o failure occurred trying to find stat data of [447288 683085 0x0 SD] Aug 11 22:21:59 unRAID shfs/user: shfs_readdir: fstatat: _Downloads (13) Permission denied Aug 11 22:21:59 unRAID shfs/user: shfs_readdir: readdir_r: /mnt/disk13/unRAID (13) Permission denied Aug 11 22:21:59 unRAID kernel: REISERFS warning: reiserfs-5090 is_tree_node: node level 16439 does not match to the expected one 1 Aug 11 22:21:59 unRAID kernel: REISERFS error (device md13): vs-5150 search_by_key: invalid format found in block 461930501. Fsck? Aug 11 22:21:59 unRAID kernel: REISERFS error (device md13): vs-13070 reiserfs_read_locked_inode: i/o failure occurred trying to find stat data of [447288 683085 0x0 SD] Aug 11 22:21:59 unRAID shfs/user: shfs_readdir: fstatat: _Downloads (13) Permission denied Aug 11 22:21:59 unRAID shfs/user: shfs_readdir: readdir_r: /mnt/disk13/unRAID (13) Permission denied Aug 11 22:21:59 unRAID kernel: REISERFS warning: reiserfs-5090 is_tree_node: node level 16439 does not match to the expected one 1 Aug 11 22:21:59 unRAID kernel: REISERFS error (device md13): vs-5150 search_by_key: invalid format found in block 461930501. Fsck? Aug 11 22:21:59 unRAID kernel: REISERFS error (device md13): vs-13070 reiserfs_read_locked_inode: i/o failure occurred trying to find stat data of [447288 683085 0x0 SD] i've gotta get up for a flight to Austin in 4hrs, so i better get some ZZZzzzsss before i get back to this from Austin...looks like i'll have to get WD to drop-ship a replacement and then hope to get a friend of mine to swap drives in my absence. if anyone has any other ideas, i'd love to hear them, but the fact that drive 13 is throwing up errors that produce permission errors *might* be the root of my unRAID user share also not showing up correctly, right?
August 12, 201411 yr That syslog does not necessarily show that disk 13 has problems that will justify a RMA. Instead it looks as though there is file-system corruption at the reiserfs level. This could be from when the previous disk had problems rather than the new disk (as a rebuild of a failed disk onto a new disk does not fix such issues). Instead they need to be fixed by running reiserfsck against the disk
August 12, 201411 yr Author i feel more than a little silly for not having tried this before you mentioned it...it even says so in the log file...but since i am so far from being a command prompt jockey, i guess that i try to avoid going there as much as i can...but a quick search in Google took me right back to the appropriate page on the lime-technology wiki, from where i've been able to follow the instructions...here is where i am so far: root@unRAID:~# cd root@unRAID:~# samba stop root@unRAID:~# unmount /dev/md13 -bash: unmount: command not found root@unRAID:~# umount /dev/md13 root@unRAID:~# reiserfsck --check /dev/md13 reiserfsck 3.6.21 (2009 www.namesys.com) ************************************************************* ** If you are using the latest reiserfsprogs and it fails ** ** please email bug reports to [email protected], ** ** providing as much information as possible -- your ** ** hardware, kernel, patches, settings, all reiserfsck ** ** messages (including version), the reiserfsck logfile, ** ** check the syslog file for any related information. ** ** If you would like advice on using this program, support ** ** is available for $25 at www.namesys.com/support.html. ** ************************************************************* Will read-only check consistency of the filesystem on /dev/md13 Will put log info to 'stdout' Do you want to run this program?[N/Yes] (note need to type Yes if you do):Yes ########### reiserfsck --check started at Tue Aug 12 03:23:22 2014 ########### Replaying journal: Trans replayed: mountid 253, transid 119556, desc 3655, len 1, commit 3657, next trans offset 3640 Trans replayed: mountid 253, transid 119557, desc 3658, len 1, commit 3660, next trans offset 3643 Replaying journal: Done. Reiserfs journal '/dev/md13' in blocks [18..8211]: 2 transactions replayed Checking internal tree.. \/ 1 (of 24|/ 34 (of 102\/162 (of 167/block 470941697: The level of the node (7237) is not correct, (1) expected the problem in the internal node occured (470941697), whole subtree is skipped / 5 (of 24\/ 38 (of 86|/118 (of 118-block 473563147: The level of the node (47657) is not correct, (1) expected the problem in the internal node occured (473563147), whole subtree is skipped / 6 (of 24|/ 28 (of 111\/154 (of 170/ and it's going...it's 3:28am and i will have to finish my packing and get ready to head to the airport...i'll keep an eye on it before i leave, but might have to finish this when i am in Austin. thanks itimpi, for that pointer, and everyone else to getting me this far...i will report back with reiserfsck's findings, and am crossing my fingers that an RMA will *not* be necessary...hope always dies last. back in a bit!
August 12, 201411 yr Author and: / 90 (of 129\block 460718081: The level of the node (58461) is not correct, (1) expected the problem in the internal node occured (460718081), whole subtree is skipped / 15 (of 24|/ 75 (of 170\/ 18 (of 105/block 461930501: The level of the node (16439) is not correct, (1) expected the problem in the internal node occured (461930501), whole subtree is skipped / 19 (of 24|/ 56 (of 118// 92 (of 170/block 468156417: The level of the node (45571) is not correct, (1) expected the problem in the internal node occured (468156417), whole subtree is skipped / 57 (of 118-block 468238996: The level of the node (58588) is not correct, (2) expected the problem in the internal node occured (468238996), whole subtree is skipped / 20 (of 24\block 456214135: The level of the node (47511) is not correct, (3) expected the problem in the internal node occured (456214135), whole subtree is skipped finished Comparing bitmaps..vpf-10640: The on-disk and the correct bitmaps differs. Bad nodes were found, Semantic pass skipped 20 found corruptions can be fixed only when running with --rebuild-tree ########### reiserfsck finished at Tue Aug 12 03:42:06 2014 ########### root@unRAID:~# reiserfsck --rebuild-tree /dev/md13 reiserfsck 3.6.21 (2009 www.namesys.com) ************************************************************* ** Do not run the program with --rebuild-tree unless ** ** something is broken and MAKE A BACKUP before using it. ** ** If you have bad sectors on a drive it is usually a bad ** ** idea to continue using it. Then you probably should get ** ** a working hard drive, copy the file system from the bad ** ** drive to the good one -- dd_rescue is a good tool for ** ** that -- and only then run this program. ** ** If you are using the latest reiserfsprogs and it fails ** ** please email bug reports to [email protected], ** ** providing as much information as possible -- your ** ** hardware, kernel, patches, settings, all reiserfsck ** ** messages (including version), the reiserfsck logfile, ** ** check the syslog file for any related information. ** ** If you would like advice on using this program, support ** ** is available for $25 at www.namesys.com/support.html. ** ************************************************************* Will rebuild the filesystem (/dev/md13) tree Will put log info to 'stdout' Do you want to run this program?[N/Yes] (note need to type Yes if you do):Yes Replaying journal: Done. Reiserfs journal '/dev/md13' in blocks [18..8211]: 0 transactions replayed rebuilding tree as i type this...
Archived
This topic is now archived and is closed to further replies.