Mover has not completed since last Saturday. I tried to manually invoke it and got an error in the log. Nothing moved. I rebooted earlier in the week for a different issue but didn't notice cache pool was filling up until today.

 

Jan 12 08:36:24 Tower root: HDIO_DRIVE_CMD(setidle) failed: Inappropriate ioctl for device
Jan 12 08:36:24 Tower root:
Jan 12 08:36:24 Tower root: /dev/nvme0n1:
Jan 12 08:36:24 Tower root: setting standby to 0 (off)
Jan 12 08:36:24 Tower emhttpd: shcmd (574): exit status: 25
Jan 12 08:36:24 Tower emhttpd: shcmd (575): /usr/sbin/hdparm -S0 /dev/nvme1n1
Jan 12 08:36:24 Tower root: HDIO_DRIVE_CMD(setidle) failed: Inappropriate ioctl for device
Jan 12 08:36:24 Tower root:
Jan 12 08:36:24 Tower root: /dev/nvme1n1:
Jan 12 08:36:24 Tower root: setting standby to 0 (off)
Jan 12 08:36:24 Tower emhttpd: shcmd (575): exit status: 25
Jan 12 08:36:51 Tower emhttpd: req (3): cmdStartMover=Move+now&csrf_token=****************
Jan 12 08:36:51 Tower emhttpd: shcmd (576): /usr/local/sbin/mover &> /dev/null &
Jan 12 08:36:51 Tower root: Manually starting mover

 

diags attached.

tower-diagnostics-20190112-0848.zip

Edited January 12, 20197 yr by interwebtech

Check if mover logging is enable, it might show some more info.

Set to log Mover and right away I see the problem. It thinks there is no space on the array? Lines like this over and over:

 

Jan 12 13:25:09 Tower root: Manually starting mover
Jan 12 13:25:09 Tower move: move: file /mnt/cache/BACKUPS/Dell-7570/2441165996B22D2E-32-32.mrimg
Jan 12 13:25:16 Tower move: move_object: /mnt/cache/BACKUPS/Dell-7570/2441165996B22D2E-32-32.mrimg No space left on device

I did fiddle with my Allocation settings a while back as I saw it filling disk to just MBs left free when there are essentially empty 8TB drives. Attached screenshot of settings for the BACKUPS share.

Split level over-rides Allocation Method when selecting a disk.    This can result in a disk being selected even when it does not have much free space.    You probably need to either relax your Split Level Settings, or move files off disk1 to free up space.

Set all shares back to High-Water. It's stuck on trying to write to disk 1 that is full (tries, fails, moves to the next file, repeat) . It has less than 200 MB free. Why is stuck trying to use that drive?

6 minutes ago, itimpi said:

Split level over-rides Allocation Method when selecting a disk.    This can result in a disk being selected even when it does not have much free space.    You probably need to either relax your Split Level Settings, or move files off disk1 to free up space.

That appears to be the issue. I set it to split only the top two and it appears to be moving files again.