Mover Causing Crashes


Recommended Posts

I'm running 5.0 rc8a, and when mover runs, I always get lots of errors, sometimes they turn into full out system crashes (nothing responsive: telnet, GUI, unMenu, etc).

 

I've tried reiserfsck for all the data disks with no corruption found, have swapped ports around.  I've tried different sticks of memory (including one at a time), and have tested one (4GB Crucial) overnight for 7 hours with memtest with no errors.

 

I can manually perform mover functions (manual copying of cache disk contents to an array drive share) without any resulting errors, but mover creates problems.

 

Here is a tail of syslog:

Oct 13 01:00:01 fileserver logger: mover started

Oct 13 01:00:01 fileserver logger: moving Downloads/

Oct 13 01:00:01 fileserver logger: ./Downloads/

Oct 13 01:00:01 fileserver logger: .d..t...... ./

Oct 13 01:00:01 fileserver logger: .d..t.....x Downloads/

Oct 13 01:00:02 fileserver logger: moving TV Shows/

Oct 13 01:00:02 fileserver logger: ./TV Shows/Strike Back/Season 03/Strike Back - 3x10 - Vengeance, Episode 10.mkv

Oct 13 01:00:02 fileserver logger: .d..t...... ./

Oct 13 01:00:02 fileserver logger: rsync: get_xattr_names: llistxattr("TV Shows",1024) failed: Input/output error (5)

Oct 13 01:00:02 fileserver logger: .d..t.....x TV Shows/

Oct 13 01:00:02 fileserver logger: .d..t...... TV Shows/Strike Back/

Oct 13 01:00:02 fileserver logger: .d..t...... TV Shows/Strike Back/Season 03/

Oct 13 01:00:02 fileserver logger: >f+++++++++ TV Shows/Strike Back/Season 03/Strike Back - 3x10 - Vengeance, Episode 10.mkv

Oct 13 01:00:07 fileserver kernel: BUG: unable to handle kernel NULL pointer dereference at 00000036

Oct 13 01:00:07 fileserver kernel: IP: [<c108224d>] __kmalloc+0xb1/0xf0

Oct 13 01:00:07 fileserver kernel: *pdpt = 00000000377d8001 *pde = 0000000000000000

Oct 13 01:00:07 fileserver kernel: Oops: 0000 [#1] SMP

Oct 13 01:00:07 fileserver kernel: Modules linked in: md_mod xor i2c_i801 sg i2c_core sata_promise coretemp ahci libahci hwmon sata_mv r8168(O) sata_sil24 mperf

Oct 13 01:00:07 fileserver kernel:

Oct 13 01:00:07 fileserver kernel: Pid: 23431, comm: sh Tainted: G          O 3.4.11-unRAID #1 Gigabyte Technology Co., Ltd. Z68A-D3H-B3/Z68A-D3H-B3

Oct 13 01:00:07 fileserver kernel: EIP: 0060:[<c108224d>] EFLAGS: 00010206 CPU: 0

Oct 13 01:00:07 fileserver kernel: EIP is at __kmalloc+0xb1/0xf0

Oct 13 01:00:07 fileserver kernel: EAX: 00000000 EBX: 00000400 ECX: 00000036 EDX: 000019c1

Oct 13 01:00:07 fileserver kernel: ESI: f2002580 EDI: c14aafc8 EBP: f21b3ec4 ESP: f21b3eac

 

Message from syslogd@fileserver at Sat Oct 13 01:00:07 2012 ...

fileserver kernel: Process sh (pid: 23431, ti=f21b2000 task=f206b2a0 task.ti=f21b2000)

Oct 13 01:00:07 fileserver kernel:  DS: 007b ES: 007b FS: 00d8 GS: 0033 SS: 0068

Oct 13 01:00:07 fileserver kernel: CR0: 8005003b CR2: 00000036 CR3: 375b0000 CR4: 000407f0

Oct 13 01:00:07 fileserver kernel: DR0: 00000000 DR1: 00000000 DR2: 00000000 DR3: 00000000

Oct 13 01:00:07 fileserver kernel: DR6: ffff0ff0 DR7: 00000400

Oct 13 01:00:07 fileserver kernel: Process sh (pid: 23431, ti=f21b2000 task=f206b2a0 task.ti=f21b2000)

Oct 13 01:00:07 fileserver kernel: Stack:

Oct 13 01:00:07 fileserver kernel:  c10971d6 000002d0 00000036 00000400 d9e14020 d9e14020 f21b3ed0 c10971d6

Oct 13 01:00:07 fileserver kernel:  00000100 f21b3ee4 c109723c f2329508 00000100 f21b3f3c f21b3f18 c1097370

Oct 13 01:00:07 fileserver kernel:  f2329004 f2329040 f2329040 d9e14f80 f2329508 f2329000 f206fc94 f2329500

Oct 13 01:00:07 fileserver kernel: Call Trace:

Oct 13 01:00:07 fileserver kernel:  [<c10971d6>] ? alloc_fdmem+0x17/0x25

Oct 13 01:00:07 fileserver kernel:  [<c10971d6>] alloc_fdmem+0x17/0x25

Oct 13 01:00:07 fileserver kernel:  [<c109723c>] alloc_fdtable+0x58/0xa5

Oct 13 01:00:07 fileserver kernel:  [<c1097370>] dup_fd+0xe7/0x23b

Oct 13 01:00:07 fileserver kernel:  [<c1021da8>] copy_process+0x322/0xa4d

Oct 13 01:00:07 fileserver kernel:  [<c102258f>] do_fork+0xbc/0x21f

Oct 13 01:00:07 fileserver kernel:  [<c102c383>] ? set_current_blocked+0x27/0x39

Oct 13 01:00:07 fileserver kernel:  [<c102c572>] ? sigprocmask+0x7e/0x89

Oct 13 01:00:07 fileserver kernel:  [<c100850e>] sys_clone+0x1b/0x20

Oct 13 01:00:07 fileserver kernel:  [<c132049d>] ptregs_clone+0x15/0x38

Oct 13 01:00:07 fileserver kernel:  [<c131fbb5>] ? syscall_call+0x7/0xb

Oct 13 01:00:07 fileserver kernel: Code: 76 4a c1 8b 50 04 8b 08 85 c9 89 4d f0 75 14 8b 4d e8 8b 55 ec 50 89 f0 e8 2c fd ff ff 59 89 45 f0 eb 25 8b 46 14 8b 4d f0 8b 3e <8b> 1c 01 8d 4a 01 8b 45 f0 64 0f c7 0f 0f 94 c0 88 c1 fe c9 75

Oct 13 01:00:07 fileserver kernel: EIP: [<c108224d>] __kmalloc+0xb1/0xf0 SS:ESP 0068:f21b3eac

Oct 13 01:00:07 fileserver kernel: CR2: 0000000000000036

Oct 13 01:00:07 fileserver kernel: ---[ end trace 0d19ac68d3260816 ]---

 

Message from syslogd@fileserver at Sat Oct 13 01:00:07 2012 ...

fileserver kernel: Code: 76 4a c1 8b 50 04 8b 08 85 c9 89 4d f0 75 14 8b 4d e8 8b 55 ec 50 89 f0 e8 2c fd ff ff 59 89 45 f0 eb 25 8b 46 14 8b 4d f0 8b 3e <8b> 1c 01 8d 4a 01 8b 45 f0 64 0f c7 0f 0f 94 c0 88 c1 fe c9 75

 

Message from syslogd@fileserver at Sat Oct 13 01:00:07 2012 ...

 

Message from syslogd@fileserver at Sat Oct 13 01:00:07 2012 ...

fileserver kernel: Call Trace:

fileserver kernel: EIP: [<c108224d>] __kmalloc+0xb1/0xf0 SS:ESP 0068:f21b3eac

 

Message from syslogd@fileserver at Sat Oct 13 01:00:07 2012 ...

fileserver kernel: Stack:

Oct 13 01:00:12 fileserver crond[1323]: exit status 137 from user root /usr/local/sbin/overtemp_shutdown.sh 1>/dev/null 2>&1

Oct 13 01:02:29 fileserver logger: rsync: get_xattr_names: llistxattr("TV Shows",1024) failed: Input/output error (5)

Oct 13 01:02:29 fileserver logger: rsync error: some files/attrs were not transferred (see previous errors) (code 23) at main.c(1042) [sender=3.0.7]

Oct 13 01:02:29 fileserver logger: ./TV Shows/Strike Back/Season 03/Strike Back - 3x10 - Vengeance, Episode 10.nfo

Oct 13 01:02:29 fileserver logger: rsync: get_xattr_names: llistxattr("TV Shows",1024) failed: Input/output error (5)

Oct 13 01:02:29 fileserver logger: .d........x TV Shows/

Oct 13 01:02:29 fileserver logger: .d..t...... TV Shows/Strike Back/Season 03/

Oct 13 01:02:29 fileserver logger: >f+++++++++ TV Shows/Strike Back/Season 03/Strike Back - 3x10 - Vengeance, Episode 10.nfo

Oct 13 01:02:29 fileserver logger: rsync: get_xattr_names: llistxattr("TV Shows",1024) failed: Input/output error (5)

Oct 13 01:02:29 fileserver logger: rsync error: some files/attrs were not transferred (see previous errors) (code 23) at main.c(1042) [sender=3.0.7]

Oct 13 01:02:29 fileserver logger: ./TV Shows/Strike Back/Season 03/Strike Back - 3x10 - Vengeance, Episode 10.sfv

Oct 13 01:02:29 fileserver logger: rsync: get_xattr_names: llistxattr("TV Shows",1024) failed: Input/output error (5)

Oct 13 01:02:29 fileserver logger: .d........x TV Shows/

Oct 13 01:02:29 fileserver logger: .d..t...... TV Shows/Strike Back/Season 03/

Oct 13 01:02:29 fileserver logger: >f+++++++++ TV Shows/Strike Back/Season 03/Strike Back - 3x10 - Vengeance, Episode 10.sfv

Oct 13 01:02:29 fileserver logger: rsync: get_xattr_names: llistxattr("TV Shows",1024) failed: Input/output error (5)

Oct 13 01:02:29 fileserver logger: rsync error: some files/attrs were not transferred (see previous errors) (code 23) at main.c(1042) [sender=3.0.7]

Oct 13 01:02:29 fileserver logger: ./TV Shows/Strike Back/Season 03/Strike Back - 3x10 - Vengeance, Episode 10.srr

Oct 13 01:02:29 fileserver logger: rsync: get_xattr_names: llistxattr("TV Shows",1024) failed: Input/output error (5)

Oct 13 01:02:29 fileserver logger: .d........x TV Shows/

Oct 13 01:02:29 fileserver logger: .d..t...... TV Shows/Strike Back/Season 03/

Oct 13 01:02:29 fileserver logger: >f+++++++++ TV Shows/Strike Back/Season 03/Strike Back - 3x10 - Vengeance, Episode 10.srr

Oct 13 01:02:29 fileserver logger: rsync: get_xattr_names: llistxattr("TV Shows",1024) failed: Input/output error (5)

Oct 13 01:02:29 fileserver logger: rsync error: some files/attrs were not transferred (see previous errors) (code 23) at main.c(1042) [sender=3.0.7]

Oct 13 01:02:29 fileserver logger: ./TV Shows/Strike Back/Season 03/Strike Back - 3x10 - Vengeance, Episode 10.nfo-orig

Oct 13 01:02:29 fileserver logger: rsync: get_xattr_names: llistxattr("TV Shows",1024) failed: Input/output error (5)

Oct 13 01:02:29 fileserver logger: .d........x TV Shows/

Oct 13 01:02:29 fileserver logger: .d..t...... TV Shows/Strike Back/Season 03/

Oct 13 01:02:29 fileserver logger: >f+++++++++ TV Shows/Strike Back/Season 03/Strike Back - 3x10 - Vengeance, Episode 10.nfo-orig

Oct 13 01:02:29 fileserver logger: rsync: get_xattr_names: llistxattr("TV Shows",1024) failed: Input/output error (5)

Oct 13 01:02:29 fileserver logger: rsync error: some files/attrs were not transferred (see previous errors) (code 23) at main.c(1042) [sender=3.0.7]

Oct 13 01:02:29 fileserver logger: mover finished

Oct 13 01:03:05 fileserver kernel: BUG: unable to handle kernel NULL pointer dereference at 00000036

Oct 13 01:03:05 fileserver kernel: IP: [<c108224d>] __kmalloc+0xb1/0xf0

Oct 13 01:03:05 fileserver kernel: *pdpt = 0000000031a70001 *pde = 0000000000000000

Oct 13 01:03:05 fileserver kernel: Oops: 0000 [#2] SMP

Oct 13 01:03:05 fileserver kernel: Modules linked in: md_mod xor i2c_i801 sg i2c_core sata_promise coretemp ahci libahci hwmon sata_mv r8168(O) sata_sil24 mperf

Oct 13 01:03:05 fileserver kernel:

Oct 13 01:03:05 fileserver kernel: Pid: 24409, comm: mdcmd Tainted: G      D    O 3.4.11-unRAID #1 Gigabyte Technology Co., Ltd. Z68A-D3H-B3/Z68A-D3H-B3

Oct 13 01:03:05 fileserver kernel: EIP: 0060:[<c108224d>] EFLAGS: 00010206 CPU: 0

Oct 13 01:03:05 fileserver kernel: EIP is at __kmalloc+0xb1/0xf0

Oct 13 01:03:05 fileserver kernel: EAX: 00000000 EBX: 00000400 ECX: 00000036 EDX: 00001a0d

Oct 13 01:03:05 fileserver kernel: ESI: f2002580 EDI: c14aafc8 EBP: d9e03ec4 ESP: d9e03eac

Oct 13 01:03:05 fileserver kernel:  DS: 007b ES: 007b FS: 00d8 GS: 0033 SS: 0068

Oct 13 01:03:05 fileserver kernel: CR0: 8005003b CR2: 00000036 CR3: 19a5a000 CR4: 000407f0

Oct 13 01:03:05 fileserver kernel: DR0: 00000000 DR1: 00000000 DR2: 00000000 DR3: 00000000

Oct 13 01:03:05 fileserver kernel: DR6: ffff0ff0 DR7: 00000400

Oct 13 01:03:05 fileserver kernel: Process mdcmd (pid: 24409, ti=d9e02000 task=f735ca40 task.ti=d9e02000)

Oct 13 01:03:05 fileserver kernel: Stack:

Oct 13 01:03:05 fileserver kernel:  c10971d6 000002d0 00000036 00000400 d9e14d80 d9e14d80 d9e03ed0 c10971d6

Oct 13 01:03:05 fileserver kernel:  00000100 d9e03ee4 c109723c f2138108 00000100 d9e03f3c d9e03f18 c1097370

Oct 13 01:03:05 fileserver kernel:  f2070e04 f2070e40 f2070e40 d9e14740 f2138108 f2070e00 f72f5ad4 f2138100

Oct 13 01:03:05 fileserver kernel: Call Trace:

Oct 13 01:03:05 fileserver kernel:  [<c10971d6>] ? alloc_fdmem+0x17/0x25

Oct 13 01:03:05 fileserver kernel:  [<c10971d6>] alloc_fdmem+0x17/0x25

Oct 13 01:03:05 fileserver kernel:  [<c109723c>] alloc_fdtable+0x58/0xa5

Oct 13 01:03:05 fileserver kernel:  [<c1097370>] dup_fd+0xe7/0x23b

Oct 13 01:03:05 fileserver kernel:  [<c1021da8>] copy_process+0x322/0xa4d

Oct 13 01:03:05 fileserver kernel:  [<c102258f>] do_fork+0xbc/0x21f

Oct 13 01:03:05 fileserver kernel:  [<c102c383>] ? set_current_blocked+0x27/0x39

Oct 13 01:03:05 fileserver kernel:  [<c102c572>] ? sigprocmask+0x7e/0x89

Oct 13 01:03:05 fileserver kernel:  [<c100850e>] sys_clone+0x1b/0x20

Oct 13 01:03:05 fileserver kernel:  [<c132049d>] ptregs_clone+0x15/0x38

Oct 13 01:03:05 fileserver kernel:  [<c131fbb5>] ? syscall_call+0x7/0xb

Oct 13 01:03:05 fileserver kernel: Code: 76 4a c1 8b 50 04 8b 08 85 c9 89 4d f0 75 14 8b 4d e8 8b 55 ec 50 89 f0 e8 2c fd ff ff 59 89 45 f0 eb 25 8b 46 14 8b 4d f0 8b 3e <8b> 1c 01 8d 4a 01 8b 45 f0 64 0f c7 0f 0f 94 c0 88 c1 fe c9 75

Oct 13 01:03:05 fileserver kernel: EIP: [<c108224d>] __kmalloc+0xb1/0xf0 SS:ESP 0068:d9e03eac

Oct 13 01:03:05 fileserver kernel: CR2: 0000000000000036

Oct 13 01:03:05 fileserver kernel: ---[ end trace 0d19ac68d3260817 ]---

syslog-2012-10-11.txt

Link to comment

And I forgot to add that all of the shares are set to a min free size of 30000000, and same with mover settings.  No drives are listed under included/excluded, and I've even tried setting the split level between  4-10 for all user shares to see if that was the limiting factor.

Link to comment
  • 5 months later...

Make a new share and test the mover.

 

Ok so to update this thread, I am now running RC12a and am still having the same problems (I just stopped trying to fix cache/mover functions).  I tried adding a new share, and it the mover works fine for that share.  I get these errors still for other files going to other pre-existing shares:

 

rsync: get_xattr_names: llistxattr("[b][i]<share name>[/i][/b]",1024) failed: Input/output error (5)

 

I've also run the new permissions script and that didn't help.  Any ideas on how to get mover working without crashing?

 

 

Link to comment

See Check Disk File systems in my sig

 

It took a while, but I ran it all on my drives (except parity).  The output for all the drives were 0 transactions replayed and No corruptions found (I'm assuming that's the main part that's important).

 

Here's the output from my cache drive (all other drives were the same other than the filesystem stats):

 

Do you want to run this program?[N/Yes] (note need to type Yes if you do):Yes
###########
reiserfsck --check started at Fri Apr  5 14:42:12 2013
###########
Replaying journal: Done.
Reiserfs journal '/dev/sdh1' in blocks [18..8211]: 0 transactions replayed
Checking internal tree.. finished
Comparing bitmaps..finished
Checking Semantic tree:
finished
No corruptions found
There are on the filesystem:
        Leaves 3809
        Internal nodes 26
        Directories 62
        Other files 173
        Data block pointers 3810444 (186274 of them are zero)
        Safe links 0
###########
reiserfsck finished at Fri Apr  5 14:42:13 2013
###########
root@fileserver:~#

Link to comment

It looks like "./TV Shows/Dexter/Season 07/Dexter - 7x03 - Buck the System" is causing the problem. Move Season 07 to the new share and try running the mover again.

 

Thanks, but it didn't work.  I'm finding it (xattr error) happens for some shares only: Downloads, TV Shows and Movies.  But a new share I created works fine with mover.

Link to comment

Would a good next move to try be to make new shares ('TV Shows 2' vs 'TV Shows') and move everything to the new shares?  Would I be able to create a new share and just rename the folder within each disk share to the name of the new share?

 

I think actually copying the files rather than renaming them may fix the xattrs. This may also do nothing...

Link to comment

Would a good next move to try be to make new shares ('TV Shows 2' vs 'TV Shows') and move everything to the new shares?  Would I be able to create a new share and just rename the folder within each disk share to the name of the new share?

 

I think actually copying the files rather than renaming them may fix the xattrs. This may also do nothing...

 

I'm assuming I use rsync to copy from a share to another? ('TV Shows' to 'TV Shows 2')

Then I can delete the 'TV Shows' share, and rename 'TV Shows 2' back to 'TV Shows'?

 

What would the rsync command be? (sorry I'm not familiar with linux)

Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.