September 4, 200916 yr Established unRaid user/Linux noob needs help. I'm experiencing strange behavior. unRaid bombes on a file copy only to a specific directory on disk 1. It will act as though it is copying the file, but the copy never actually completes, and the file is inaccessible from disk 1. This occurs whether I copy to the user share or directly to the disk - and only to that specific directory on disk 1. Everything else about the array seems to function normally. I'm on 4.5 beta 6 with Joe's cache_dirs and BJP's myMain running...8 1TB disks (7 data 1 parity). Client system is WinXP Pro SP3 via Windows explorer drag/drop file copy. After manually stopping the failed copy attempt in Win explorer, I can access the array normally, except if I try to access that specific directory. This 'freezes' the server and it becomes unresponsive to web interface as well as Telnet; it eventually disappears from the network completely. The only fix is to hard shutdown via power button (causing parity check on next reboot). I've searched through the forums and the WIKI looking for similar problem and can't seem to find a solution. I should add that disk SMART reports show no unusual errors (pending/reallocated sectors, etc.) I can duplicate this behavior (and provide a full syslog) if necessary for troubleshooting purposes. Here is a clip from myMain/system info/open files: Open Files (from /usr/bin/lsof /dev/md*) COMMAND PID USER FD TYPE DEVICE SIZE NODE NAME shfs 2036 root 4u REG 9,1 165207832 253705 /mnt/disk1/[Problem Directory]/[Problem File.wmv] And here is a clip of the end of my syslog (at the point of the file copy attempt): Sep 5 02:25:12 Tower kernel: kernel BUG at fs/reiserfs/bitmap.c:1222! Sep 5 02:25:12 Tower kernel: invalid opcode: 0000 [#1] SMP Sep 5 02:25:12 Tower kernel: last sysfs file: /sys/devices/pci0000:00/0000:00:1c.3/0000:03:00.0/host1/target1:0:0/1:0:0:0/block/sdb/stat Sep 5 02:25:12 Tower kernel: Modules linked in: md_mod ata_piix r8169 pata_jmicron jmicron ide_core ahci libata Sep 5 02:25:12 Tower kernel: Sep 5 02:25:12 Tower kernel: Pid: 26175, comm: shfs Not tainted (2.6.29.1-unRAID #2) EP35-DS3R Sep 5 02:25:12 Tower kernel: EIP: 0060:[] EFLAGS: 00210246 CPU: 0 Sep 5 02:25:12 Tower kernel: EIP is at reiserfs_cache_bitmap_metadata+0x14/0x53 Sep 5 02:25:12 Tower kernel: EAX: cc582000 EBX: f82d1060 ECX: f82d1060 EDX: 00001000 Sep 5 02:25:12 Tower kernel: ESI: ca922300 EDI: c2f1aa00 EBP: c2d53a7c ESP: c2d53a74 Sep 5 02:25:12 Tower kernel: DS: 007b ES: 007b FS: 00d8 GS: 0033 SS: 0068 Sep 5 02:25:12 Tower kernel: Process shfs (pid: 26175, ti=c2d52000 task=c28c4340 task.ti=c2d52000) Sep 5 02:25:12 Tower kernel: Stack: Sep 5 02:25:12 Tower kernel: ca922300 000c0000 c2d53a98 c0197a03 00000018 f82d1000 00000018 c2d53b70 Sep 5 02:25:12 Tower kernel: 0000000a c2d53ac4 c0197b39 00000018 ee031880 c2f1aa00 f82d1060 00000000 Sep 5 02:25:12 Tower kernel: 0003d039 00000018 c2d53c48 0000000a c2d53b80 c019887a 00008000 00000001 Sep 5 02:25:12 Tower kernel: Call Trace: Sep 5 02:25:12 Tower kernel: [] ? reiserfs_read_bitmap_block+0xb8/0xc2 Sep 5 02:25:12 Tower kernel: [] ? scan_bitmap_block+0x61/0x1e5 Sep 5 02:25:12 Tower kernel: [] ? reiserfs_allocate_blocknrs+0x890/0xa54 Sep 5 02:25:12 Tower kernel: [] ? search_for_position_by_key+0x37/0x2e8 Sep 5 02:25:12 Tower kernel: [] ? reiserfs_get_block+0x436/0x1150 Sep 5 02:25:12 Tower kernel: [] ? __getblk+0x1a/0x238 Sep 5 02:25:12 Tower kernel: [] ? ll_rw_block+0xb8/0xd2 Sep 5 02:25:12 Tower kernel: [] ? __getblk+0x1a/0x238 Sep 5 02:25:12 Tower kernel: [] ? __block_prepare_write+0x14d/0x30c Sep 5 02:25:12 Tower kernel: [] ? add_to_page_cache_locked+0x63/0x7f Sep 5 02:25:12 Tower kernel: [] ? block_write_begin+0x75/0xce Sep 5 02:25:12 Tower kernel: [] ? reiserfs_get_block+0x0/0x1150 Sep 5 02:25:12 Tower kernel: [] ? reiserfs_write_begin+0x118/0x18f Sep 5 02:25:12 Tower kernel: [] ? reiserfs_get_block+0x0/0x1150 Sep 5 02:25:12 Tower kernel: [] ? generic_file_buffered_write+0xc3/0x220 Sep 5 02:25:12 Tower kernel: [] ? reiserfs_dirty_inode+0x74/0x7d Sep 5 02:25:12 Tower kernel: [] ? __generic_file_aio_write_nolock+0x3d0/0x40a Sep 5 02:25:12 Tower kernel: [] ? generic_file_aio_write+0x57/0xb4 Sep 5 02:25:12 Tower kernel: [] ? do_sync_write+0xbb/0xf9 Sep 5 02:25:12 Tower kernel: [] ? complete_signal+0x171/0x178 Sep 5 02:25:12 Tower kernel: [] ? autoremove_wake_function+0x0/0x30 Sep 5 02:25:12 Tower kernel: [] ? reiserfs_file_write+0x6b/0x74 Sep 5 02:25:12 Tower kernel: [] ? reiserfs_file_write+0x0/0x74 Sep 5 02:25:12 Tower kernel: [] ? vfs_write+0x8c/0x108 Sep 5 02:25:12 Tower kernel: [] ? sys_pwrite64+0x44/0x5d Sep 5 02:25:12 Tower kernel: [] ? syscall_call+0x7/0xb Sep 5 02:25:12 Tower kernel: Code: ea 6a fd ff 8b 47 10 83 c0 78 e8 b4 da 1a 00 5b 5e 5f 5d c3 90 90 90 55 89 e5 56 89 d6 53 8b 42 18 89 cb 8b 52 14 f6 00 01 75 04 <0f> 0b eb fe 8d 0c 10 c7 03 00 00 00 00 eb 24 8b 01 85 c0 75 05 Sep 5 02:25:12 Tower kernel: EIP: [] reiserfs_cache_bitmap_metadata+0x14/0x53 SS:ESP 0068:c2d53a74 Sep 5 02:25:12 Tower kernel: ---[ end trace 97eecf16c43da8e8 ]--- Sep 5 02:25:12 Tower kernel: ------------[ cut here ]------------ Sep 5 02:25:12 Tower kernel: WARNING: at kernel/exit.c:1013 do_exit+0x2d/0x611() Sep 5 02:25:12 Tower kernel: Hardware name: EP35-DS3R Sep 5 02:25:12 Tower kernel: Modules linked in: md_mod ata_piix r8169 pata_jmicron jmicron ide_core ahci libata Sep 5 02:25:12 Tower kernel: Pid: 26175, comm: shfs Tainted: G D 2.6.29.1-unRAID #2 Sep 5 02:25:12 Tower kernel: Call Trace: Sep 5 02:25:12 Tower kernel: [] warn_slowpath+0x74/0x8a Sep 5 02:25:12 Tower kernel: [] ? notify_update+0x22/0x24 Sep 5 02:25:12 Tower kernel: [] ? vt_console_print+0x224/0x232 Sep 5 02:25:12 Tower kernel: [] ? vt_console_print+0x0/0x232 Sep 5 02:25:12 Tower kernel: [] ? __call_console_drivers+0x58/0x65 Sep 5 02:25:12 Tower kernel: [] ? up+0x2b/0x2f Sep 5 02:25:12 Tower kernel: [] ? release_console_sem+0x170/0x185 Sep 5 02:25:12 Tower kernel: [] ? common_interrupt+0x27/0x2c Sep 5 02:25:12 Tower kernel: [] ? vgacon_set_cursor_size+0xe4/0xeb Sep 5 02:25:12 Tower kernel: [] do_exit+0x2d/0x611 Sep 5 02:25:12 Tower kernel: [] ? printk+0x15/0x17 Sep 5 02:25:12 Tower kernel: [] ? print_oops_end_marker+0x1e/0x23 Sep 5 02:25:12 Tower kernel: [] oops_end+0x75/0x7c Sep 5 02:25:12 Tower kernel: [] die+0x53/0x59 Sep 5 02:25:12 Tower kernel: [] do_trap+0x89/0xa2 Sep 5 02:25:12 Tower kernel: [] ? do_invalid_op+0x0/0x7b Sep 5 02:25:12 Tower kernel: [] do_invalid_op+0x71/0x7b Sep 5 02:25:12 Tower kernel: [] ? reiserfs_cache_bitmap_metadata+0x14/0x53 Sep 5 02:25:12 Tower kernel: [] ? generic_make_request+0x1e1/0x223 Sep 5 02:25:12 Tower kernel: [] ? io_schedule+0x1e/0x28 Sep 5 02:25:12 Tower kernel: [] ? __wait_on_bit+0x54/0x5d Sep 5 02:25:12 Tower kernel: [] ? sync_buffer+0x0/0x37 Sep 5 02:25:12 Tower kernel: [] ? out_of_line_wait_on_bit+0x58/0x60 Sep 5 02:25:12 Tower kernel: [] error_code+0x72/0x78 Sep 5 02:25:12 Tower kernel: [] ? reiserfs_cache_bitmap_metadata+0x14/0x53 Sep 5 02:25:12 Tower kernel: [] reiserfs_read_bitmap_block+0xb8/0xc2 Sep 5 02:25:12 Tower kernel: [] scan_bitmap_block+0x61/0x1e5 Sep 5 02:25:12 Tower kernel: [] reiserfs_allocate_blocknrs+0x890/0xa54 Sep 5 02:25:12 Tower kernel: [] ? search_for_position_by_key+0x37/0x2e8 Sep 5 02:25:12 Tower kernel: [] reiserfs_get_block+0x436/0x1150 Sep 5 02:25:12 Tower kernel: [] ? __getblk+0x1a/0x238 Sep 5 02:25:12 Tower kernel: [] ? ll_rw_block+0xb8/0xd2 Sep 5 02:25:12 Tower kernel: [] ? __getblk+0x1a/0x238 Sep 5 02:25:12 Tower kernel: [] __block_prepare_write+0x14d/0x30c Sep 5 02:25:12 Tower kernel: [] ? add_to_page_cache_locked+0x63/0x7f Sep 5 02:25:12 Tower kernel: [] block_write_begin+0x75/0xce Sep 5 02:25:12 Tower kernel: [] ? reiserfs_get_block+0x0/0x1150 Sep 5 02:25:12 Tower kernel: [] reiserfs_write_begin+0x118/0x18f Sep 5 02:25:12 Tower kernel: [] ? reiserfs_get_block+0x0/0x1150 Sep 5 02:25:12 Tower kernel: [] generic_file_buffered_write+0xc3/0x220 Sep 5 02:25:12 Tower kernel: [] ? reiserfs_dirty_inode+0x74/0x7d Sep 5 02:25:12 Tower kernel: [] __generic_file_aio_write_nolock+0x3d0/0x40a Sep 5 02:25:12 Tower kernel: [] generic_file_aio_write+0x57/0xb4 Sep 5 02:25:12 Tower kernel: [] do_sync_write+0xbb/0xf9 Sep 5 02:25:12 Tower kernel: [] ? complete_signal+0x171/0x178 Sep 5 02:25:12 Tower kernel: [] ? autoremove_wake_function+0x0/0x30 Sep 5 02:25:12 Tower kernel: [] reiserfs_file_write+0x6b/0x74 Sep 5 02:25:12 Tower kernel: [] ? reiserfs_file_write+0x0/0x74 Sep 5 02:25:12 Tower kernel: [] vfs_write+0x8c/0x108 Sep 5 02:25:12 Tower kernel: [] sys_pwrite64+0x44/0x5d Sep 5 02:25:12 Tower kernel: [] syscall_call+0x7/0xb Sep 5 02:25:12 Tower kernel: ---[ end trace 97eecf16c43da8e9 ]--- And here is a snip of the syslog at attempt of stopping the array from standard Limetech GUI page: Sep 5 03:20:33 Tower emhttp: shcmd (75): /etc/rc.d/rc.samba stop | logger Sep 5 03:20:33 Tower emhttp: shcmd (76): /etc/rc.d/rc.nfsd stop | logger Sep 5 03:20:34 Tower emhttp: Spinning up all drives... Sep 5 03:20:53 Tower emhttp: shcmd (77): sync Sep 5 03:23:07 Tower kernel: mdcmd (125): stop Sep 5 03:23:07 Tower kernel: md: 14 devices still in use. Any help or advice would be greatly appreciated.
September 4, 200916 yr The reiser file system on that disk is very likely corrupted. Since you have unMENU running, you can use the "Filesystem Check" button on the Disk Management page to check it. That button may not exist on earlier versions of the disk-management page, but a newer version is here http://lime-technology.com/forum/index.php?topic=2595.msg30513#msg30513 and it will make the repair of the file-system easy. The "Check Filesystem" button will turn into a "Fix Filesystem" button if the check suggests a fix. Unzip the disk-management plugin in the directory with your other unMENU files and then re-start unmenu. (The original disk-management was not a plug-in, the new is and will override the built-in page with improved capabilities) If you do not wish to use the button in unMENU, you can follow the wiki instructions for a reiserfsck here: http://lime-technology.com/wiki/index.php/Check_Disk_Filesystems Joe L.
September 4, 200916 yr As far as the warning that your disks are in use when you attempted to stop the array. It is on purpose. The cache_dirs program was written to keep your disks busy and prevent them from showing as "Unformatted" when they are un-mounted. A second press of the "Stop" button, within a minute of the first, should stop the array. Make sure you have not "cd'd" to any of the disks..... so log off from telnet sessions and stop any other programs you might have running before attempting to stop the array. To repeat, if cache_dirs is running it will take two presses of the "Stop" button, within a minute or so of each other, to stop the array. The "Fuser" button in unMENU should show you the disks that are busy, and why. When cache_dirs is normally running there is a "sleep" and a "current directory" on each data disk. When you first press "Stop" it should detect the "devices still in use" line in the syslog, and terminate all those processes. if you press "Stop" a second time within a minute of the first, the array should stop normally if you have nothing else keeping a drive "busy" Of course, you can use cache_dirs -q to quit it and then stop the array. Joe L.
September 4, 200916 yr Author Thanks, Joe. I had the old version, so I updated the plug-in and ran the check on disk 1. It returned with 2 errors saying to run with --fix-fixable option. Another button appeared to the right of the first button says fix, but pressing it had no affect. So I ran reiserfsck again - this time from a telnet prompt.... Tower login: root Password: Linux 2.6.29.1-unRAID. root@Tower:~# cd root@Tower:~# samba stop root@Tower:~# umount /dev/sdd umount: /dev/sdd: not mounted root@Tower:~# reiserfsck --fix-fixable /dev/sdd reiserfsck 3.6.19 (2003 www.namesys.com) ************************************************************* ** If you are using the latest reiserfsprogs and it fails ** ** please email bug reports to [email protected], ** ** providing as much information as possible -- your ** ** hardware, kernel, patches, settings, all reiserfsck ** ** messages (including version), the reiserfsck logfile, ** ** check the syslog file for any related information. ** ** If you would like advice on using this program, support ** ** is available for $25 at www.namesys.com/support.html. ** ************************************************************* Will check consistency of the filesystem on /dev/sdd and will fix what can be fixed without --rebuild-tree Will put log info to 'stdout' Do you want to run this program?[N/Yes] (note need to type Yes if you do):Yes reiserfs_open: the reiserfs superblock cannot be found on /dev/sdd. Failed to open the filesystem. If the partition table has not been changed, and the partition is valid and it really contains a reiserfs partition, then the superblock is corrupted and you need to run this utility with --rebuild-sb. Yes, I'm a little slow at command-line programming, so bear with me.
September 4, 200916 yr You need to run reiserfsck on /dev/md1 not /dev/sdd (assuming it was disk1) in the same way you need to un-mount /dev/md1 /dev/sdd does not have a file-system on it, exactly as reported to you.
September 4, 200916 yr Author Got it. reiserfsck --fix-fixable started at Sun Sep 6 00:47:49 2009 ########### Replaying journal.. Reiserfs journal '/dev/md1' in blocks [18..8211]: 0 transactions replayed Checking internal tree..finished Comparing bitmaps..vpf-10630: The on-disk and the correct bitmaps differs. Will be fixed later. Checking Semantic tree: finished No corruptions found There are on the filesystem: Leaves 129437 Internal nodes 823 Directories 10263 Other files 223393 Data block pointers 113601123 (0 of them are zero) Safe links 0 ########### reiserfsck finished at Sun Sep 6 01:05:31 2009 ########### root@Tower:~# Again, thanks so much for your great help here, Joe. Let me know where to send the check
Archived
This topic is now archived and is closed to further replies.