Jump to content

Help, unRaid frozen after a "rsync" that segfaulted...


Polopollo

Recommended Posts

Hello,

I am new to unRaid, still on trial version for another 15days, version 6.3.5.

I plugged an external MacOS HDDs in USB3 to my tower and was emptying it to unRaid.

  1. I mounted it (I think with the assigned device plugin).

  2. Then I ssh-ed as root into my tower through the "ssh [email protected]", started to "rsync -avhE" all my files to unRaid, which worked very very very well:

    root@Tower:/mnt/user0/Media/testmedias# rsync -avhE --remove-source-files /mnt/disks/All_of_2015/medias/ .

     

After several of these rsync commands that were lightspeed fast, this last one above did a segfault... I didn't think too much of it and relaunched the command. And now I cannot cancel it:

root@Tower:/mnt/user0/Media/testmedias# rsync -avhE --remove-source-files /mnt/disks/All_of_2015/medias/ .
sending incremental file list
^C^C^C^C^C^C^C^C^C

I opened a second ssh connection and got:

 
root@Tower:~# ps aux | grep -i rsync
root     15963  0.0  0.0   9640  1832 pts/1    S+   07:53   0:00 grep -i rsync
root     31265  0.0  0.0  14200  2428 pts/0    D+   07:49   0:00 rsync -avhE --remove-source-files /mnt/disks/All_of_2015/medias/ .
root     31266  0.0  0.0      0     0 pts/0    Z+   07:49   0:00 [rsync] <defunct>

Sure enough, I tried to "pkill rsync" but the "D+" of the process prevents me to do so.

I went to the WebUI and tried to stop the array to reboot the tower, but now the whole WebUI is frozen. (I can still SSH).

What is the procedure in this case? Help... :(

 

------------

I did check my /var/log/syslog (saw an old few years old post here about reading it), and here are the last lines of it:

(We can see I mounted the drive, I ssh-ed into, I tried to remove a "test" directory from my Media share (that had a .DS_Store file into), then I rsync stuff several time till it broke.

 


 

Jan 20 07:18:08 Tower kernel: usb 2-3: new SuperSpeed USB device number 2 using xhci_hcd
Jan 20 07:18:08 Tower kernel: usb-storage 2-3:1.0: USB Mass Storage device detected
Jan 20 07:18:08 Tower kernel: scsi host8: usb-storage 2-3:1.0
Jan 20 07:18:09 Tower kernel: scsi 8:0:0:0: Direct-Access     Seagate  Backup+  Desk    0511 PQ: 0 ANSI: 6
Jan 20 07:18:09 Tower kernel: sd 8:0:0:0: Attached scsi generic sg7 type 0
Jan 20 07:18:26 Tower kernel: sd 8:0:0:0: [sdh] Very big device. Trying to use READ CAPACITY(16).
Jan 20 07:18:26 Tower kernel: sd 8:0:0:0: [sdh] 15628053167 512-byte logical blocks: (8.00 TB/7.28 TiB)
Jan 20 07:18:26 Tower kernel: sd 8:0:0:0: [sdh] 4096-byte physical blocks
Jan 20 07:18:26 Tower kernel: sd 8:0:0:0: [sdh] Write Protect is off
Jan 20 07:18:26 Tower kernel: sd 8:0:0:0: [sdh] Mode Sense: 47 00 00 08
Jan 20 07:18:26 Tower kernel: sd 8:0:0:0: [sdh] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
Jan 20 07:18:26 Tower kernel: sd 8:0:0:0: [sdh] Very big device. Trying to use READ CAPACITY(16).
Jan 20 07:18:26 Tower kernel: sdh: sdh1 sdh2
Jan 20 07:18:26 Tower kernel: sd 8:0:0:0: [sdh] Very big device. Trying to use READ CAPACITY(16).
Jan 20 07:18:26 Tower kernel: sd 8:0:0:0: [sdh] Attached SCSI disk
Jan 20 07:18:26 Tower rc.diskinfo[8997]: PHP Warning: Missing argument 2 for force_reload() in /etc/rc.d/rc.diskinfo on line 691
Jan 20 07:18:26 Tower rc.diskinfo[8997]: SIGHUP received, forcing refresh of disks info.
Jan 20 07:18:27 Tower unassigned.devices: Disk with serial 'ST8000AS0002-1NA17Z_Z84096DN', mountpoint 'EFI' is not set to auto mount and will not be mounted...
Jan 20 07:18:27 Tower unassigned.devices: Disk with serial 'ST8000AS0002-1NA17Z_Z84096DN', mountpoint 'All_of_2015' is not set to auto mount and will not be mounted...
Jan 20 07:21:32 Tower unassigned.devices: Adding disk '/dev/sdh1'...
Jan 20 07:21:32 Tower unassigned.devices: Mount drive command: /sbin/mount -t vfat -o auto,async,noatime,nodiratime,nodev,nosuid,umask=000 '/dev/sdh1' '/mnt/disks/EFI'
Jan 20 07:21:34 Tower unassigned.devices: Successfully mounted '/dev/sdh1' on '/mnt/disks/EFI'.
Jan 20 07:21:34 Tower unassigned.devices: Disk with serial 'ST8000AS0002-1NA17Z_Z84096DN', mountpoint 'EFI' is not set as sharable and will not be shared...
Jan 20 07:21:35 Tower unassigned.devices: Device '/dev/sdh1' script file not found.  'ADD' script not executed.
Jan 20 07:21:35 Tower unassigned.devices: Adding disk '/dev/sdh2'...
Jan 20 07:21:35 Tower unassigned.devices: Mount drive command: /sbin/mount -t hfsplus -o force,rw,users,async,umask=000 '/dev/sdh2' '/mnt/disks/All_of_2015'
Jan 20 07:21:38 Tower unassigned.devices: Successfully mounted '/dev/sdh2' on '/mnt/disks/All_of_2015'.
Jan 20 07:21:38 Tower unassigned.devices: Disk with serial 'ST8000AS0002-1NA17Z_Z84096DN', mountpoint 'All_of_2015' is not set as sharable and will not be shared...
Jan 20 07:21:38 Tower unassigned.devices: Device '/dev/sdh2' script file not found.  'ADD' script not executed.
Jan 20 07:22:15 Tower sshd[1310]: Accepted password for root from 192.168.0.103 port 59775 ssh2
Jan 20 07:26:13 Tower emhttp: shcmd (28328): chmod 0777 '/mnt/user/Docus'
Jan 20 07:26:13 Tower emhttp: shcmd (28329): chown 'nobody':'users' '/mnt/user/Docus'
Jan 20 07:29:11 Tower afpd[26317]: afp_disconnect: primary reconnect failed
Jan 20 07:33:01 Tower emhttp: shcmd (28498): chmod 0777 '/mnt/user/Docus'
Jan 20 07:33:01 Tower emhttp: shcmd (28499): chown 'nobody':'users' '/mnt/user/Docus'
Jan 20 07:34:22 Tower shfs/user0: err: shfs_rmdir: rmdir: /mnt/disk1/Media/test (39) Directory not empty
Jan 20 07:34:26 Tower shfs/user0: err: shfs_rmdir: rmdir: /mnt/disk1/Media/test (39) Directory not empty
Jan 20 07:34:28 Tower shfs/user0: err: shfs_rmdir: rmdir: /mnt/disk1/Media/test (39) Directory not empty
Jan 20 07:49:06 Tower kernel: hfsplus: trying to free free bnode 0(1)
Jan 20 07:49:06 Tower kernel: general protection fault: 0000 [#1] PREEMPT SMP
Jan 20 07:49:06 Tower kernel: Modules linked in: ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_conntrack_ipv4 nf_nat_ipv4 iptable_filter ip_tables nf_nat md_mod mxm_wmi x86_pkg_temp_thermal coretemp kvm_intel kvm e1000e mpt3sas i2c_i801 ptp i2c_smbus i2c_core raid_class ahci scsi_transport_sas pps_core video libahci wmi backlight
Jan 20 07:49:06 Tower kernel: CPU: 1 PID: 16669 Comm: rsync Not tainted 4.9.30-unRAID #1
Jan 20 07:49:06 Tower kernel: Hardware name: Micro-Star International Co., Ltd. MS-7B49/Z370 PC PRO (MS-7B49), BIOS 1.10 10/31/2017
Jan 20 07:49:06 Tower kernel: task: ffff88000da0c240 task.stack: ffffc9000973c000
Jan 20 07:49:06 Tower kernel: RIP: 0010:[<ffffffff811f0ed4>]  [<ffffffff811f0ed4>] hfsplus_bnode_write+0x65/0xbf
Jan 20 07:49:06 Tower kernel: RSP: 0018:ffffc9000973fc18  EFLAGS: 00010286
Jan 20 07:49:06 Tower kernel: RAX: ffe7090000000000 RBX: 0000000000000002 RCX: 0000000000000002
Jan 20 07:49:06 Tower kernel: RDX: 0000000000000000 RSI: ffffc9000973fc56 RDI: ffe7090000000000
Jan 20 07:49:06 Tower kernel: RBP: ffffc9000973fc40 R08: ffffc9000973fc56 R09: 0000000000000000
Jan 20 07:49:06 Tower kernel: R10: ffff88042ff3aa78 R11: ffff880000000000 R12: ffffc9000973fc56
Jan 20 07:49:06 Tower kernel: R13: 0000000000000002 R14: ffff88042ff3aa80 R15: 0000160000000000
Jan 20 07:49:06 Tower kernel: FS:  00002b0df07b0e80(0000) GS:ffff88046ec40000(0000) knlGS:0000000000000000
Jan 20 07:49:06 Tower kernel: CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
Jan 20 07:49:06 Tower kernel: CR2: 0000000000713ff0 CR3: 000000042ffc4000 CR4: 00000000003406e0
Jan 20 07:49:06 Tower kernel: Stack:
Jan 20 07:49:06 Tower kernel: ffff88042ff3aa20 ffffc9000973fce0 0000000000001ff8 0000000000001ffc
Jan 20 07:49:06 Tower kernel: 000000000000005a ffffc9000973fc58 ffffffff811f0f49 b4ff00000000000e
Jan 20 07:49:06 Tower kernel: ffffc9000973fc98 ffffffff811f2a4f 0000000e00000000 ffffc9000973fce0
Jan 20 07:49:06 Tower kernel: Call Trace:
Jan 20 07:49:06 Tower kernel: [<ffffffff811f0f49>] hfsplus_bnode_write_u16+0x1b/0x1d
Jan 20 07:49:06 Tower kernel: [<ffffffff811f2a4f>] hfsplus_brec_remove+0x108/0x156
Jan 20 07:49:06 Tower kernel: [<ffffffff811f480f>] __hfsplus_delete_attr+0x90/0xb7
Jan 20 07:49:06 Tower kernel: [<ffffffff811f4c7c>] hfsplus_delete_all_attrs+0x82/0x9e
Jan 20 07:49:06 Tower kernel: [<ffffffff811ef3a7>] hfsplus_delete_cat+0x2a4/0x2c7
Jan 20 07:49:06 Tower kernel: [<ffffffff811efafc>] hfsplus_unlink+0xc3/0x173
Jan 20 07:49:06 Tower kernel: [<ffffffff8112a058>] ? __inode_permission+0x73/0xa7
Jan 20 07:49:06 Tower kernel: [<ffffffff8112a0c7>] ? inode_permission+0x3b/0x3d
Jan 20 07:49:06 Tower kernel: [<ffffffff8112a892>] vfs_unlink+0xc1/0x15c
Jan 20 07:49:06 Tower kernel: [<ffffffff8112d42e>] do_unlinkat+0xf9/0x201
Jan 20 07:49:06 Tower kernel: [<ffffffff8112e8ab>] SyS_unlink+0x11/0x13
Jan 20 07:49:06 Tower kernel: [<ffffffff8167f537>] entry_SYSCALL_64_fastpath+0x1a/0xa9
Jan 20 07:49:06 Tower kernel: Code: 8d 74 c7 50 4c 89 f8 0f 4f d9 49 03 06 48 b9 00 00 00 00 00 88 ff ff 48 c1 f8 06 48 c1 e0 0c 48 01 c8 48 63 cb 48 01 d0 48 89 c7 <f3> a4 49 8b 3e e8 a6 d3 ed ff 41 29 dd 74 40 48 ba 00 00 00 00
Jan 20 07:49:06 Tower kernel: RIP  [<ffffffff811f0ed4>] hfsplus_bnode_write+0x65/0xbf
Jan 20 07:49:06 Tower kernel: RSP <ffffc9000973fc18>
Jan 20 07:49:06 Tower kernel: ---[ end trace d312bc4f77e643b0 ]---
Jan 20 07:51:26 Tower sshd[6661]: Accepted password for root from 192.168.0.103 port 56254 ssh2
Jan 20 07:55:12 Tower kernel: mdcmd (40): nocheck
Jan 20 07:55:12 Tower kernel: md: nocheck_array: check not active
Jan 20 07:55:12 Tower kernel: mdcmd (41): spinup 0
Jan 20 07:55:12 Tower emhttp: Spinning up all drives...
Jan 20 07:55:12 Tower emhttp: shcmd (29086): /usr/sbin/hdparm -S0 /dev/sdb &> /dev/null
Jan 20 07:55:12 Tower kernel: mdcmd (42): spinup 1
Jan 20 07:55:12 Tower kernel: mdcmd (43): spinup 2
Jan 20 07:55:12 Tower kernel: mdcmd (44): spinup 29
Jan 20 07:55:12 Tower emhttp: shcmd (29087): /usr/sbin/hdparm -S0 /dev/sdf &> /dev/null
Jan 20 07:55:13 Tower emhttp: Stopping services...
Jan 20 07:55:13 Tower emhttp: shcmd (29090): /etc/rc.d/rc.docker stop |& logger
Jan 20 07:55:13 Tower root: stopping docker ...
Jan 20 07:55:14 Tower root: waiting for docker to die...
Jan 20 07:55:15 Tower avahi-daemon[8760]: Interface docker0.IPv4 no longer relevant for mDNS.
Jan 20 07:55:15 Tower avahi-daemon[8760]: Leaving mDNS multicast group on interface docker0.IPv4 with address 172.17.0.1.
Jan 20 07:55:15 Tower avahi-daemon[8760]: Withdrawing address record for 172.17.0.1 on docker0.
Jan 20 07:55:15 Tower emhttp: shcmd (29091): umount /var/lib/docker |& logger
Jan 20 07:55:15 Tower cache_dirs: Stopping cache_dirs process 12751
Jan 20 07:55:15 Tower unassigned.devices: Unmounting All Devices...
Jan 20 07:55:15 Tower unassigned.devices: Device '/dev/sdh1' script file not found.  'REMOVE' script not executed.
Jan 20 07:55:15 Tower emhttp:
Jan 20 07:55:15 Tower emhttp: Warning: preg_grep() expects parameter 2 to be array, boolean given in /usr/local/emhttp/plugins/unassigned.devices/include/lib.php on line 174
Jan 20 07:55:15 Tower unassigned.devices: Unmounting disk 'EFI'...
Jan 20 07:55:15 Tower unassigned.devices: Unmounting '/dev/sdh1'...
Jan 20 07:55:38 Tower sshd[6661]: Received disconnect from 192.168.0.103 port 56254:11: disconnected by user
Jan 20 07:55:38 Tower sshd[6661]: Disconnected from 192.168.0.103 port 56254
Jan 20 08:45:23 Tower sshd[7878]: Accepted password for root from 192.168.0.103 port 56444 ssh2

 

Link to comment

Crash appears related to the hfs+ filesystem:

 

Jan 20 07:49:06 Tower kernel: hfsplus: trying to free free bnode 0(1)

 

Reboot the server, check filesystem on that disk and try again, you can try typing reboot on the console, if it doesn't work after 10 or 20 minutes you'll need to do a hard reboot.

Link to comment

Archived

This topic is now archived and is closed to further replies.

×
×
  • Create New...