October 13, 20178 yr Had this crop up in my system today. Oct 13 16:06:44 Tower kernel: loop: Write error at byte offset 715571200, length 4096. Oct 13 16:06:44 Tower kernel: blk_update_request: I/O error, dev loop0, sector 1397600 Oct 13 16:06:44 Tower kernel: BTRFS error (device loop0): bdev /dev/loop0 errs: wr 2, rd 0, flush 0, corrupt 0, gen 0 Oct 13 16:06:44 Tower kernel: loop: Write error at byte offset 715591680, length 4096. Oct 13 16:06:44 Tower kernel: blk_update_request: I/O error, dev loop0, sector 1397632 Oct 13 16:06:44 Tower kernel: BTRFS error (device loop0): bdev /dev/loop0 errs: wr 3, rd 0, flush 0, corrupt 0, gen 0 Oct 13 16:06:44 Tower kernel: loop: Write error at byte offset 1789317120, length 4096. Oct 13 16:06:44 Tower kernel: blk_update_request: I/O error, dev loop0, sector 3494752 Oct 13 16:06:44 Tower kernel: BTRFS error (device loop0): bdev /dev/loop0 errs: wr 4, rd 0, flush 0, corrupt 0, gen 0 Oct 13 16:06:44 Tower kernel: loop: Write error at byte offset 1789333504, length 4096. Oct 13 16:06:44 Tower kernel: blk_update_request: I/O error, dev loop0, sector 3494784 Oct 13 16:06:44 Tower kernel: BTRFS error (device loop0): bdev /dev/loop0 errs: wr 5, rd 0, flush 0, corrupt 0, gen 0 Oct 13 16:06:44 Tower kernel: loop: Write error at byte offset 2411786240, length 4096. Oct 13 16:06:44 Tower kernel: blk_update_request: I/O error, dev loop0, sector 4710512 Oct 13 16:06:44 Tower kernel: loop: Write error at byte offset 2411913216, length 4096. Oct 13 16:06:44 Tower kernel: blk_update_request: I/O error, dev loop0, sector 4710767 Oct 13 16:06:44 Tower kernel: BTRFS error (device loop0): bdev /dev/loop0 errs: wr 6, rd 0, flush 0, corrupt 0, gen 0 Oct 13 16:06:44 Tower kernel: loop: Write error at byte offset 2520440832, length 4096. Oct 13 16:06:44 Tower kernel: blk_update_request: I/O error, dev loop0, sector 4922736 Oct 13 16:06:44 Tower kernel: BTRFS error (device loop0): bdev /dev/loop0 errs: wr 7, rd 0, flush 0, corrupt 0, gen 0 Oct 13 16:06:44 Tower kernel: loop: Write error at byte offset 2556149760, length 4096. Oct 13 16:06:44 Tower kernel: blk_update_request: I/O error, dev loop0, sector 4992480 Oct 13 16:06:44 Tower kernel: loop: Write error at byte offset 2556280832, length 4096. Oct 13 16:06:44 Tower kernel: blk_update_request: I/O error, dev loop0, sector 4992735 Oct 13 16:06:44 Tower kernel: BTRFS error (device loop0): bdev /dev/loop0 errs: wr 8, rd 0, flush 0, corrupt 0, gen 0 Oct 13 16:06:44 Tower kernel: loop: Write error at byte offset 2411782144, length 4096. Oct 13 16:06:44 Tower kernel: blk_update_request: I/O error, dev loop0, sector 4710512 Oct 13 16:06:44 Tower kernel: BTRFS error (device loop0): bdev /dev/loop0 errs: wr 9, rd 0, flush 0, corrupt 0, gen 0 Oct 13 16:06:45 Tower kernel: BTRFS error (device loop0): bdev /dev/loop0 errs: wr 10, rd 0, flush 0, corrupt 0, gen 0 Oct 13 16:06:45 Tower kernel: BTRFS error (device loop0): bdev /dev/loop0 errs: wr 11, rd 0, flush 0, corrupt 0, gen 0 Oct 13 16:06:48 Tower kernel: BTRFS: error (device loop0) in btrfs_commit_transaction:2227: errno=-5 IO failure (Error while writing out transaction) Oct 13 16:06:48 Tower kernel: BTRFS info (device loop0): forced readonly Oct 13 16:06:48 Tower kernel: BTRFS warning (device loop0): Skipping commit of aborted transaction. Oct 13 16:06:48 Tower kernel: ------------[ cut here ]------------ Oct 13 16:06:48 Tower kernel: WARNING: CPU: 11 PID: 21502 at fs/btrfs/transaction.c:1850 cleanup_transaction+0x8c/0x238 Oct 13 16:06:48 Tower kernel: BTRFS: Transaction aborted (error -5) Oct 13 16:06:48 Tower kernel: Modules linked in: md_mod iptable_mangle ip_tables bonding ixgbe ptp pps_core mdio x86_pkg_temp_thermal coretemp kvm_intel mpt3sas i2c_i801 ahci i2c_smbus kvm raid_class i2c_core libahci megaraid_sas scsi_transport_sas wmi ipmi_si [last unloaded: pps_core] Oct 13 16:06:48 Tower kernel: CPU: 11 PID: 21502 Comm: dockerd Not tainted 4.9.30-unRAID #1 Oct 13 16:06:48 Tower kernel: Hardware name: Supermicro X9DRH-7TF/7F/iTF/iF/X9DRH-7TF/7F/iTF/iF, BIOS 3.2a 06/18/2016 Oct 13 16:06:48 Tower kernel: ffffc9001a7a3c78 ffffffff813a4a1b ffffc9001a7a3cc8 ffffffff8196b262 Oct 13 16:06:48 Tower kernel: ffffc9001a7a3cb8 ffffffff8104d0d9 0000073a1a7a3d30 ffff883fc58adef0 Oct 13 16:06:48 Tower kernel: ffff881fefbf6800 ffff883fd63d6aa0 00000000fffffffb 0000000000000000 Oct 13 16:06:48 Tower kernel: Call Trace: Oct 13 16:06:48 Tower kernel: [<ffffffff813a4a1b>] dump_stack+0x61/0x7e Oct 13 16:06:48 Tower kernel: [<ffffffff8104d0d9>] __warn+0xb8/0xd3 Oct 13 16:06:48 Tower kernel: [<ffffffff8104d13a>] warn_slowpath_fmt+0x46/0x4e Oct 13 16:06:48 Tower kernel: [<ffffffff812eeff1>] cleanup_transaction+0x8c/0x238 Oct 13 16:06:48 Tower kernel: [<ffffffff8107c0fd>] ? wake_up_bit+0x25/0x25 Oct 13 16:06:48 Tower kernel: [<ffffffff812f0d59>] btrfs_commit_transaction.part.11+0x912/0x927 Oct 13 16:06:48 Tower kernel: [<ffffffff812f0db4>] btrfs_commit_transaction+0x46/0x4d Oct 13 16:06:48 Tower kernel: [<ffffffff81304934>] btrfs_sync_file+0x2ca/0x2f5 Oct 13 16:06:48 Tower kernel: [<ffffffff81146f6e>] vfs_fsync_range+0x87/0x99 Oct 13 16:06:48 Tower kernel: [<ffffffff81146f97>] vfs_fsync+0x17/0x19 Oct 13 16:06:48 Tower kernel: [<ffffffff81146fc5>] do_fsync+0x2c/0x45 Oct 13 16:06:48 Tower kernel: [<ffffffff811471d7>] SyS_fdatasync+0xe/0x12 Oct 13 16:06:48 Tower kernel: [<ffffffff8167f537>] entry_SYSCALL_64_fastpath+0x1a/0xa9 Oct 13 16:06:48 Tower kernel: ---[ end trace 54a3fd7b9446e8d1 ]--- Oct 13 16:06:48 Tower kernel: BTRFS: error (device loop0) in cleanup_transaction:1850: errno=-5 IO failure Oct 13 16:06:48 Tower kernel: BTRFS info (device loop0): delayed_refs has NO entry Oct 13 16:06:48 Tower emhttp: shcmd (111): umount /var/lib/docker |& logger Oct 13 16:06:48 Tower kernel: BTRFS error (device loop0): cleaner transaction attach returned -30 Oct 13 16:06:48 Tower emhttp: nothing to sync Oct 13 16:06:48 Tower emhttp: shcmd (123): set -o pipefail ; /usr/local/sbin/mount_image '/mnt/user/system/docker/docker.img' /var/lib/docker 60 |& logger Oct 13 16:06:48 Tower root: truncate: failed to truncate '/mnt/cache/system/docker/docker.img' at 64424509440 bytes: No space left on device Oct 13 16:06:48 Tower kernel: BTRFS info (device loop0): disk space caching is enabled Oct 13 16:06:48 Tower kernel: BTRFS info (device loop0): has skinny extents Oct 13 16:06:48 Tower kernel: BTRFS info (device loop0): bdev /dev/loop0 errs: wr 1, rd 0, flush 0, corrupt 0, gen 0 Oct 13 16:06:48 Tower root: Resize '/var/lib/docker' of 'max' Oct 13 16:06:48 Tower kernel: BTRFS info (device loop0): new size for /dev/loop0 is 64424509440 Oct 13 16:06:48 Tower emhttp: shcmd (124): /etc/rc.d/rc.docker start |& logger Oct 13 16:06:48 Tower root: starting docker ... Oct 13 16:06:49 Tower kernel: lo_write_bvec: 201 callbacks suppressed Oct 13 16:06:49 Tower kernel: loop: Write error at byte offset 1146310656, length 4096. Oct 13 16:06:49 Tower kernel: blk_update_request: 201 callbacks suppressed Oct 13 16:06:49 Tower kernel: blk_update_request: I/O error, dev loop0, sector 2238880 Oct 13 16:06:49 Tower kernel: btrfs_dev_stat_print_on_error: 191 callbacks suppressed Oct 13 16:06:49 Tower kernel: BTRFS error (device loop0): bdev /dev/loop0 errs: wr 22, rd 0, flush 0, corrupt 0, gen 0 Oct 13 16:06:49 Tower kernel: loop: Write error at byte offset 1441157120, length 4096. Oct 13 16:06:49 Tower kernel: blk_update_request: I/O error, dev loop0, sector 2814752 Oct 13 16:06:49 Tower kernel: BTRFS error (device loop0): bdev /dev/loop0 errs: wr 23, rd 0, flush 0, corrupt 0, gen 0 Oct 13 16:06:49 Tower kernel: loop: Write error at byte offset 1492357120, length 4096. Oct 13 16:06:49 Tower kernel: blk_update_request: I/O error, dev loop0, sector 2914752 Oct 13 16:06:49 Tower kernel: BTRFS error (device loop0): bdev /dev/loop0 errs: wr 24, rd 0, flush 0, corrupt 0, gen 0 Oct 13 16:06:49 Tower kernel: loop: Write error at byte offset 1510035456, length 4096. Oct 13 16:06:49 Tower kernel: blk_update_request: I/O error, dev loop0, sector 2949280 Oct 13 16:06:49 Tower kernel: BTRFS error (device loop0): bdev /dev/loop0 errs: wr 25, rd 0, flush 0, corrupt 0, gen 0 Oct 13 16:06:49 Tower kernel: loop: Write error at byte offset 1611321344, length 4096. Oct 13 16:06:49 Tower kernel: blk_update_request: I/O error, dev loop0, sector 3147104 Oct 13 16:06:49 Tower kernel: BTRFS error (device loop0): bdev /dev/loop0 errs: wr 26, rd 0, flush 0, corrupt 0, gen 0 Oct 13 16:06:49 Tower kernel: loop: Write error at byte offset 1618743296, length 4096. Oct 13 16:06:49 Tower kernel: blk_update_request: I/O error, dev loop0, sector 3161600 Oct 13 16:06:49 Tower kernel: BTRFS error (device loop0): bdev /dev/loop0 errs: wr 27, rd 0, flush 0, corrupt 0, gen 0 Oct 13 16:06:49 Tower kernel: loop: Write error at byte offset 1629851648, length 4096. Oct 13 16:06:49 Tower kernel: blk_update_request: I/O error, dev loop0, sector 3183296 Oct 13 16:06:49 Tower kernel: BTRFS error (device loop0): bdev /dev/loop0 errs: wr 28, rd 0, flush 0, corrupt 0, gen 0 Oct 13 16:06:49 Tower kernel: loop: Write error at byte offset 573804544, length 4096. Oct 13 16:06:49 Tower kernel: blk_update_request: I/O error, dev loop0, sector 1120704 Oct 13 16:06:49 Tower kernel: BTRFS error (device loop0): bdev /dev/loop0 errs: wr 29, rd 0, flush 0, corrupt 0, gen 0 Oct 13 16:06:50 Tower kernel: loop: Write error at byte offset 622366720, length 4096. Oct 13 16:06:50 Tower kernel: blk_update_request: I/O error, dev loop0, sector 1215552 Oct 13 16:06:50 Tower kernel: BTRFS error (device loop0): bdev /dev/loop0 errs: wr 30, rd 0, flush 0, corrupt 0, gen 0 Oct 13 16:06:50 Tower kernel: loop: Write error at byte offset 622448640, length 4096. Oct 13 16:06:50 Tower kernel: blk_update_request: I/O error, dev loop0, sector 1215712 Oct 13 16:06:50 Tower kernel: BTRFS error (device loop0): bdev /dev/loop0 errs: wr 31, rd 0, flush 0, corrupt 0, gen 0 Oct 13 16:06:52 Tower kernel: BTRFS: error (device loop0) in btrfs_commit_transaction:2227: errno=-5 IO failure (Error while writing out transaction) Oct 13 16:06:52 Tower kernel: BTRFS info (device loop0): forced readonly Oct 13 16:06:52 Tower kernel: BTRFS warning (device loop0): Skipping commit of aborted transaction. Oct 13 16:06:52 Tower kernel: ------------[ cut here ]------------ Oct 13 16:06:52 Tower kernel: WARNING: CPU: 30 PID: 21982 at fs/btrfs/transaction.c:1850 cleanup_transaction+0x8c/0x238 Oct 13 16:06:52 Tower kernel: BTRFS: Transaction aborted (error -5) Oct 13 16:06:52 Tower kernel: Modules linked in: md_mod iptable_mangle ip_tables bonding ixgbe ptp pps_core mdio x86_pkg_temp_thermal coretemp kvm_intel mpt3sas i2c_i801 ahci i2c_smbus kvm raid_class i2c_core libahci megaraid_sas scsi_transport_sas wmi ipmi_si [last unloaded: pps_core] Oct 13 16:06:52 Tower kernel: CPU: 30 PID: 21982 Comm: dockerd Tainted: G W 4.9.30-unRAID #1 Oct 13 16:06:52 Tower kernel: Hardware name: Supermicro X9DRH-7TF/7F/iTF/iF/X9DRH-7TF/7F/iTF/iF, BIOS 3.2a 06/18/2016 Oct 13 16:06:52 Tower kernel: ffffc9001b757c78 ffffffff813a4a1b ffffc9001b757cc8 ffffffff8196b262 Oct 13 16:06:52 Tower kernel: ffffc9001b757cb8 ffffffff8104d0d9 0000073a1b757d30 ffff883fccc85658 Oct 13 16:06:52 Tower kernel: ffff881fdb092800 ffff883fcce84d20 00000000fffffffb 0000000000000000 Oct 13 16:06:52 Tower kernel: Call Trace: Oct 13 16:06:52 Tower kernel: [<ffffffff813a4a1b>] dump_stack+0x61/0x7e Oct 13 16:06:52 Tower kernel: [<ffffffff8104d0d9>] __warn+0xb8/0xd3 Oct 13 16:06:52 Tower kernel: [<ffffffff8104d13a>] warn_slowpath_fmt+0x46/0x4e Oct 13 16:06:52 Tower kernel: [<ffffffff812eeff1>] cleanup_transaction+0x8c/0x238 Oct 13 16:06:52 Tower kernel: [<ffffffff8107c0fd>] ? wake_up_bit+0x25/0x25 Oct 13 16:06:52 Tower kernel: [<ffffffff812f0d59>] btrfs_commit_transaction.part.11+0x912/0x927 Oct 13 16:06:52 Tower kernel: [<ffffffff812f0db4>] btrfs_commit_transaction+0x46/0x4d Oct 13 16:06:52 Tower kernel: [<ffffffff81304934>] btrfs_sync_file+0x2ca/0x2f5 Oct 13 16:06:52 Tower kernel: [<ffffffff81146f6e>] vfs_fsync_range+0x87/0x99 Oct 13 16:06:52 Tower kernel: [<ffffffff81146f97>] vfs_fsync+0x17/0x19 Oct 13 16:06:52 Tower kernel: [<ffffffff81146fc5>] do_fsync+0x2c/0x45 Oct 13 16:06:52 Tower kernel: [<ffffffff811471d7>] SyS_fdatasync+0xe/0x12 Oct 13 16:06:52 Tower kernel: [<ffffffff8167f537>] entry_SYSCALL_64_fastpath+0x1a/0xa9 Oct 13 16:06:52 Tower kernel: ---[ end trace 54a3fd7b9446e8d2 ]--- Oct 13 16:06:52 Tower kernel: BTRFS: error (device loop0) in cleanup_transaction:1850: errno=-5 IO failure Oct 13 16:06:52 Tower kernel: BTRFS info (device loop0): delayed_refs has NO entry Oct 13 16:06:52 Tower emhttp: shcmd (126): umount /var/lib/docker |& logger Oct 13 16:06:52 Tower kernel: BTRFS error (device loop0): cleaner transaction attach returned -30 Oct 13 16:06:52 Tower emhttp: shcmd (138): set -o pipefail ; /usr/local/sbin/mount_image '/mnt/user/system/docker/docker.img' /var/lib/docker 60 |& logger Oct 13 16:06:52 Tower root: truncate: failed to truncate '/mnt/cache/system/docker/docker.img' at 64424509440 bytes: No space left on device Oct 13 16:06:52 Tower kernel: BTRFS info (device loop0): disk space caching is enabled Oct 13 16:06:52 Tower kernel: BTRFS info (device loop0): has skinny extents Oct 13 16:06:52 Tower kernel: BTRFS info (device loop0): bdev /dev/loop0 errs: wr 1, rd 0, flush 0, corrupt 0, gen 0 Oct 13 16:06:52 Tower root: Resize '/var/lib/docker' of 'max' Oct 13 16:06:52 Tower emhttp: shcmd (139): /etc/rc.d/rc.docker start |& logger Oct 13 16:06:52 Tower kernel: BTRFS info (device loop0): new size for /dev/loop0 is 64424509440 Have already deleted the docker img and keep getting the error. Logs are attached from before I tried deleting the docker img and after tower-diagnostics-20171013-1554.zip tower-diagnostics-20171013-1653.zip Edited October 13, 20178 yr by tooviral
October 14, 20178 yr Author Seems like this was the problem Had to delete some files, and ran the commands @johnnie.black mentioned. Haven't seen an error yet. Any way to prevent this from happening. This has been the second time I've had to do a balance. Edited October 14, 20178 yr by tooviral
October 17, 20178 yr Author On 10/14/2017 at 11:33 AM, tooviral said: Any way to prevent this from happening. This has been the second time I've had to do a balance. Any ideas?
October 17, 20178 yr Community Expert 12 minutes ago, tooviral said: Any ideas? Run this balance once a week, you can use the user scripts plugin: btrfs balance start -dusage=75 /mnt/cache Once unRAID gets to kernel 4.14 this issue should get fixed or at least improve.
Archived
This topic is now archived and is closed to further replies.