(SOLVED) Root filesystem filled up when preclearing a disk, all gui elements blank

[Ouze]

Hello all,

 

I have kind of an odd issue.  I was preclearing a disk, and had the status window open in the background. I opened the window to check if zeroing was finished, and there was an error about the log being full, which I thought was odd but OK. I closed it.

 

I opened the gui via my browser, and it comes up, but literally everything in it is blank - no parity disk, no array, nothing. I knew there wasn't anything seriously wrong since I was just watching a movie. I checked the shares via Explorer, it all looks good there. 

 

I tried going back a directory, which usually displays the flash share - that, I couldn't see, just the main array share.

 

I logged in via putty. Came up OK.

 

I did a df -h and am seeing that the root filesystem is full, which I assume caused everything to break:

 

root@Shiva:~# df -h
Filesystem      Size  Used Avail Use% Mounted on
rootfs          1.9G  1.9G     0 100% /
tmpfs           2.0G  288K  2.0G   1% /run
devtmpfs        1.9G     0  1.9G   0% /dev
cgroup_root     2.0G     0  2.0G   0% /sys/fs/cgroup
tmpfs           128M   32M   97M  25% /var/log
/dev/sda1       1.9G  602M  1.3G  32% /boot
/dev/md1        1.9T  1.7T  143G  93% /mnt/disk1
/dev/md2        932G  851G   81G  92% /mnt/disk2
/dev/md3        1.9T  1.7T  157G  92% /mnt/disk3
/dev/md4        1.9T  1.8T   97G  95% /mnt/disk4
/dev/md5        1.9T  1.8T   93G  96% /mnt/disk5
/dev/md6        1.9T  1.7T  140G  93% /mnt/disk6
/dev/md7        1.9T  1.7T  174G  91% /mnt/disk7
/dev/md8        1.9T  1.8T  100G  95% /mnt/disk8
/dev/md9        2.8T  2.4T  348G  88% /mnt/disk9
/dev/md10       2.8T  2.2T  589G  79% /mnt/disk10
/dev/sdf1       233G   16G  217G   7% /mnt/cache
shfs             20T   18T  1.9T  91% /mnt/user0
shfs             20T   18T  2.1T  90% /mnt/user
/dev/loop0       10G  900M  7.3G  11% /var/lib/docker
shm              64M     0   64M   0% /var/lib/docker/containers/8672950ae21eb605d57614b0338eac7de53477e27e979d28230733b879257d5a/shm
shm              64M  8.0K   64M   1% /var/lib/docker/containers/92474a28b452f53f58e43394b4716c015b4e95645e3e596ccb346d335374162f/shm
 

 

 

I'm not sure where to go from here.  

 

A.) How do I clean up that filesystem - rebooting?

B.) How do I find out how it filled up? I was running a preclear, but I don't know enough to say "that's what did it". I'm just mentioning it. 

 

Thank you!

 

 

 

 

[Ouze]

I found a similar thread on the forums. I ran this command:

 

root@Shiva:~# cp /var/log/syslog /boot/syslog.txt

 

and then rebooted. My rootFS is now at 22%.

 

My log is here: https://www.dropbox.com/s/vffbouij7j6kg7z/syslog.txt?dl=0

 

and I've got some of this action going upon reboot:

 

 

Should I be starting the preclear again? 

 

edit:  OK, I think the drive is bad. So I suppose this is what happened: 

 

Started the preclear, ran for 10 hours, then the drive broke. The script or (whatever) kept trying, and logging failures, until the root fs filled up. Is that right?

 

Smart log for the drive is here: https://www.dropbox.com/s/p2yi7ht6b8d057i/ST3000DM008-2DM166_Z504RTGE-20170802-1829.txt?dl=0

 

Is that more or less accurate?

 

[Squid]

Looks like this drive ST3000DM008-2DM166_Z504RTGE dropped offline (probably power cable and/or sata cable loose or terrible quality)

 

Reseat the cabling, reboot,  and assuming that's the drive you were trying to preclear, you'll probably have to start over.

[Ouze]

Thanks - thats the drive.

 

So those failures in the smart log don't mean a bad drive, then? 

[JorgeB]

That disk has pending sectors and needs to be replaced, I'm guessing Squid was distracted

[Ouze]

Nah, we were posting nearly simultaneously - I think he read the post while I was still editing and attaching that smart log.

 

Thanks for confirming it's bad. If a drive is gonna fail, I at least appreciate when it's polite enough to do so within 10 hours of arriving and before any data has been written to it.  

[Squid]

11 hours ago, johnnie.black said:

I'm guessing Squid was distracted

What me?!?!