Alexandro Posted November 10, 2015 Share Posted November 10, 2015 Hi guys, I beg your help please. Every 10 days or so, the file system on my cache drive goes to read only. Stopping/ starting the array usually solves the problem, but I do not think that is normal. Here is an excerpt from my log file showing certain BTRFS problems: Nov 10 16:16:24 UNRAID kernel: mpt2sas0: log_info(0x31110d00): originator(PL), code(0x11), sub_code(0x0d00) (Drive related) Nov 10 16:16:24 UNRAID kernel: blk_update_request: I/O error, dev sde, sector 0 (Errors) Nov 10 16:16:24 UNRAID kernel: sd 1:0:2:0: [sde] UNKNOWN(0x2003) Result: hostbyte=0x00 driverbyte=0x08 (System) Nov 10 16:16:24 UNRAID kernel: sd 1:0:2:0: [sde] Sense Key : 0x2 [current] (Drive related) Nov 10 16:16:24 UNRAID kernel: sd 1:0:2:0: [sde] ASC=0x4 ASCQ=0x0 (Drive related) Nov 10 16:16:24 UNRAID kernel: sd 1:0:2:0: [sde] UNKNOWN(0x2003) Result: hostbyte=0x00 driverbyte=0x08 (System) Nov 10 16:16:24 UNRAID kernel: sd 1:0:2:0: [sde] CDB: opcode=0x2a 2a 00 0d f4 b6 a8 00 00 08 00 (Drive related) Nov 10 16:16:24 UNRAID kernel: blk_update_request: I/O error, dev sde, sector 234141352 (Errors) Nov 10 16:16:24 UNRAID kernel: Buffer I/O error on dev sde1, logical block 29267661, lost async page write (Errors) Nov 10 16:16:24 UNRAID kernel: sd 1:0:2:0: [sde] Sense Key : 0x2 [current] (Drive related) Nov 10 16:16:24 UNRAID kernel: sd 1:0:2:0: [sde] ASC=0x4 ASCQ=0x0 (Drive related) Nov 10 16:16:24 UNRAID kernel: sd 1:0:2:0: [sde] CDB: opcode=0x2a 2a 00 0d f7 a5 d8 00 00 08 00 (Drive related) Nov 10 16:16:24 UNRAID kernel: blk_update_request: I/O error, dev sde, sector 234333656 (Errors) Nov 10 16:16:24 UNRAID kernel: Buffer I/O error on dev sde1, logical block 29291699, lost async page write (Errors) Nov 10 16:16:24 UNRAID kernel: blk_update_request: I/O error, dev loop0, sector 128 (Errors) Nov 10 16:16:24 UNRAID kernel: BTRFS: lost page write due to I/O error on /dev/loop0 (Errors) Nov 10 16:16:24 UNRAID kernel: BTRFS: bdev /dev/loop0 errs: wr 1, rd 0, flush 0, corrupt 0, gen 0 Nov 10 16:16:39 UNRAID kernel: sd 1:0:2:0: [sde] UNKNOWN(0x2003) Result: hostbyte=0x00 driverbyte=0x08 (System) Nov 10 16:16:39 UNRAID kernel: sd 1:0:2:0: [sde] Sense Key : 0x2 [current] (Drive related) Nov 10 16:16:39 UNRAID kernel: sd 1:0:2:0: [sde] ASC=0x4 ASCQ=0x0 (Drive related) Nov 10 16:16:39 UNRAID kernel: sd 1:0:2:0: [sde] CDB: opcode=0x2a 2a 00 00 00 00 c0 00 00 08 00 (Drive related) Nov 10 16:16:39 UNRAID kernel: blk_update_request: I/O error, dev sde, sector 192 (Errors) Nov 10 16:16:39 UNRAID kernel: Buffer I/O error on dev sde1, logical block 16, lost async page write (Errors) Nov 10 16:16:45 UNRAID kernel: sd 1:0:2:0: attempting task abort! scmd(ffff88001cc83c80) Nov 10 16:16:45 UNRAID kernel: sd 1:0:2:0: [sde] CDB: opcode=0x12 12 00 00 00 24 00 (Drive related) Nov 10 16:16:45 UNRAID kernel: scsi target1:0:2: handle(0x000b), sas_address(0x4433221103000000), phy(3) (Drive related) Nov 10 16:16:45 UNRAID kernel: scsi target1:0:2: enclosure_logical_id(0x590b11c036116a00), slot(0) (Drive related) Nov 10 16:16:49 UNRAID kernel: sd 1:0:2:0: task abort: SUCCESS scmd(ffff88001cc83c80) Nov 10 16:16:49 UNRAID kernel: program smartctl is using a deprecated SCSI ioctl, please convert it to SG_IO Nov 10 16:16:49 UNRAID kernel: sd 1:0:2:0: [sde] UNKNOWN(0x2003) Result: hostbyte=0x00 driverbyte=0x08 (System) Nov 10 16:16:49 UNRAID kernel: sd 1:0:2:0: [sde] Sense Key : 0x2 [current] (Drive related) Nov 10 16:16:49 UNRAID kernel: sd 1:0:2:0: [sde] ASC=0x4 ASCQ=0x0 (Drive related) Nov 10 16:16:49 UNRAID kernel: sd 1:0:2:0: [sde] CDB: opcode=0x2a 2a 00 00 00 3c c0 00 00 08 00 (Drive related) Nov 10 16:16:49 UNRAID kernel: blk_update_request: I/O error, dev sde, sector 15552 (Errors) Nov 10 16:16:49 UNRAID kernel: Buffer I/O error on dev sde1, logical block 1936, lost sync page write (Errors) Nov 10 16:16:49 UNRAID kernel: sd 1:0:2:0: [sde] UNKNOWN(0x2003) Result: hostbyte=0x00 driverbyte=0x08 (System) Nov 10 16:16:49 UNRAID kernel: sd 1:0:2:0: [sde] Sense Key : 0x2 [current] (Drive related) Nov 10 16:16:49 UNRAID kernel: sd 1:0:2:0: [sde] ASC=0x4 ASCQ=0x0 (Drive related) Nov 10 16:16:49 UNRAID kernel: sd 1:0:2:0: [sde] CDB: opcode=0x2a 2a 00 00 00 3c c8 00 00 08 00 (Drive related) Nov 10 16:16:49 UNRAID kernel: blk_update_request: I/O error, dev sde, sector 15560 (Errors) Nov 10 16:16:49 UNRAID kernel: Buffer I/O error on dev sde1, logical block 1937, lost sync page write (Errors) Nov 10 16:16:49 UNRAID kernel: REISERFS abort (device sde1): Journal write error in flush_commit_list (Errors) Nov 10 16:16:54 UNRAID kernel: sd 1:0:2:0: [sde] UNKNOWN(0x2003) Result: hostbyte=0x00 driverbyte=0x08 (System) Nov 10 16:16:54 UNRAID kernel: sd 1:0:2:0: [sde] Sense Key : 0x2 [current] (Drive related) Nov 10 16:16:54 UNRAID kernel: sd 1:0:2:0: [sde] ASC=0x4 ASCQ=0x0 (Drive related) Nov 10 16:16:54 UNRAID kernel: sd 1:0:2:0: [sde] CDB: opcode=0x2a 2a 00 0d 0b d0 d8 00 00 08 00 (Drive related) Nov 10 16:16:54 UNRAID kernel: blk_update_request: I/O error, dev sde, sector 218878168 (Errors) Nov 10 16:16:54 UNRAID kernel: Buffer I/O error on dev sde1, logical block 27359763, lost async page write (Errors) Nov 10 16:16:54 UNRAID kernel: sd 1:0:2:0: [sde] UNKNOWN(0x2003) Result: hostbyte=0x00 driverbyte=0x08 (System) Nov 10 16:16:54 UNRAID kernel: sd 1:0:2:0: [sde] Sense Key : 0x2 [current] (Drive related) Nov 10 16:16:54 UNRAID kernel: sd 1:0:2:0: [sde] ASC=0x4 ASCQ=0x0 (Drive related) Nov 10 16:16:54 UNRAID kernel: sd 1:0:2:0: [sde] CDB: opcode=0x2a 2a 00 0d 21 b1 08 00 00 08 00 (Drive related) Nov 10 16:16:54 UNRAID kernel: blk_update_request: I/O error, dev sde, sector 220311816 (Errors) Nov 10 16:16:54 UNRAID kernel: Buffer I/O error on dev sde1, logical block 27538969, lost async page write (Errors) Nov 10 16:17:01 UNRAID kernel: program smartctl is using a deprecated SCSI ioctl, please convert it to SG_IO Nov 10 16:17:01 UNRAID kernel: program smartctl is using a deprecated SCSI ioctl, please convert it to SG_IO Nov 10 16:17:01 UNRAID kernel: program smartctl is using a deprecated SCSI ioctl, please convert it to SG_IO Nov 10 16:17:01 UNRAID kernel: program smartctl is using a deprecated SCSI ioctl, please convert it to SG_IO Nov 10 16:17:01 UNRAID kernel: program smartctl is using a deprecated SCSI ioctl, please convert it to SG_IO Nov 10 16:17:19 UNRAID kernel: blk_update_request: I/O error, dev loop0, sector 0 (Errors) Nov 10 16:17:19 UNRAID kernel: BTRFS: bdev /dev/loop0 errs: wr 1, rd 0, flush 1, corrupt 0, gen 0 Nov 10 16:17:19 UNRAID kernel: BTRFS: error (device loop0) in write_all_supers:3454: errno=-5 IO failure (errors while submitting device barriers.) (Errors) Nov 10 16:17:19 UNRAID kernel: BTRFS info (device loop0): forced readonly Nov 10 16:17:19 UNRAID kernel: BTRFS warning (device loop0): Skipping commit of aborted transaction. (Minor Issues) Nov 10 16:17:19 UNRAID kernel: ------------[ cut here ]------------ Nov 10 16:17:19 UNRAID kernel: WARNING: CPU: 0 PID: 30212 at fs/btrfs/super.c:260 __btrfs_abort_transaction+0x4d/0x10e() (Minor Issues) Nov 10 16:17:19 UNRAID kernel: BTRFS: Transaction aborted (error -5) (Errors) Nov 10 16:17:19 UNRAID kernel: Modules linked in: veth xt_nat kvm_intel kvm vhost_net vhost macvtap macvlan md_mod xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 ipt_REJECT nf_reject_ipv4 ebtable_filter ebtables iptable_filter ip_tables tun nct6775 hwmon_vid jc42 coretemp mvsas libsas e1000e i2c_i801 ptp ftdi_sio ahci mpt2sas raid_class libahci usbserial pps_core scsi_transport_sas [last unloaded: md_mod] (Drive related) Nov 10 16:17:19 UNRAID kernel: CPU: 0 PID: 30212 Comm: btrfs-transacti Tainted: G W 4.0.4-unRAID #4 (Errors) Nov 10 16:17:19 UNRAID kernel: Hardware name: Supermicro X9SCI/X9SCA/X9SCI/X9SCA, BIOS 2.10 01/09/2014 Nov 10 16:17:19 UNRAID kernel: 0000000000000009 ffff88021626fc58 ffffffff815ff789 0000000000000000 Nov 10 16:17:19 UNRAID kernel: ffff88021626fca8 ffff88021626fc98 ffffffff810443a3 ffff88021626fce8 Nov 10 16:17:19 UNRAID kernel: ffffffff812a0739 00000000fffffffb ffff8802163ea800 ffff88020ecfa840 Nov 10 16:17:19 UNRAID kernel: Call Trace: (Errors) Nov 10 16:17:19 UNRAID kernel: [<ffffffff815ff789>] dump_stack+0x4c/0x6e Nov 10 16:17:19 UNRAID kernel: [<ffffffff810443a3>] warn_slowpath_common+0x97/0xb1 Nov 10 16:17:19 UNRAID kernel: [<ffffffff812a0739>] ? __btrfs_abort_transaction+0x4d/0x10e Nov 10 16:17:19 UNRAID kernel: [<ffffffff810443fe>] warn_slowpath_fmt+0x41/0x43 Nov 10 16:17:19 UNRAID kernel: [<ffffffff812a0739>] __btrfs_abort_transaction+0x4d/0x10e Nov 10 16:17:19 UNRAID kernel: [<ffffffff812c532d>] cleanup_transaction+0x80/0x21d Nov 10 16:17:19 UNRAID kernel: [<ffffffff8106dbeb>] ? wait_woken+0x7d/0x7d Nov 10 16:17:19 UNRAID kernel: [<ffffffff812c650d>] btrfs_commit_transaction+0x9e3/0x9f8 Nov 10 16:17:19 UNRAID kernel: [<ffffffff812c22a7>] transaction_kthread+0xfa/0x1cb Nov 10 16:17:19 UNRAID kernel: [<ffffffff812c21ad>] ? btrfs_cleanup_transaction+0x461/0x461 Nov 10 16:17:19 UNRAID kernel: [<ffffffff81058dfe>] kthread+0xd6/0xde Nov 10 16:17:19 UNRAID kernel: [<ffffffff81058d28>] ? kthread_create_on_node+0x172/0x172 Nov 10 16:17:19 UNRAID kernel: [<ffffffff81604d58>] ret_from_fork+0x58/0x90 Nov 10 16:17:19 UNRAID kernel: [<ffffffff81058d28>] ? kthread_create_on_node+0x172/0x172 Nov 10 16:17:19 UNRAID kernel: ---[ end trace a358015f718f37a3 ]--- Nov 10 16:17:19 UNRAID kernel: BTRFS: error (device loop0) in cleanup_transaction:1686: errno=-5 IO failure (Errors) Nov 10 16:17:19 UNRAID kernel: BTRFS info (device loop0): delayed_refs has NO entry Nov 10 18:29:13 UNRAID kernel: REISERFS warning (device sde1): clm-6006 reiserfs_dirty_inode: writing inode 370966 on readonly FS (Minor Issues) The cache drive is fine, showing no SMART errors (both short and long tests have been performed). I am experiencing this situation since I converted all my plugins to dockers. Please also see the full syslog attached. syslog.txt.zip Quote Link to comment
Squid Posted November 10, 2015 Share Posted November 10, 2015 Odds on, your docker.img file is filling up. You've probably got one or more of your containers (one of the downloaders?) storing data within the docker.img which is causing this Quote Link to comment
Alexandro Posted November 10, 2015 Author Share Posted November 10, 2015 Thanks for your comment Squid, The only downloader I have installed is transmission, which is storing the downloaded files outside of the docker.img...to the root of the cache drive. Quote Link to comment
Squid Posted November 10, 2015 Share Posted November 10, 2015 Thanks for your comment Squid, The only downloader I have installed is transmission, which is storing the downloaded files outside of the docker.img...to the root of the cache drive. What about the unfinished downloads? Quote Link to comment
Alexandro Posted November 11, 2015 Author Share Posted November 11, 2015 What about the unfinished downloads? Usually it happens when there are no downloads at all on the queue. Quote Link to comment
Squid Posted November 11, 2015 Share Posted November 11, 2015 What about the unfinished downloads? Usually it happens when there are no downloads at all on the queue. Next time it happens, post a screen shot of the docker settings page (the middle part shows the usage) Quote Link to comment
Alexandro Posted November 11, 2015 Author Share Posted November 11, 2015 Will do it. Thanks again for your participation in the discussion. Quote Link to comment
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.