Is my thumb drive failing ? Shares down, "no license", reboot everything works for a day or so

[Troy144]

Is my thumb drive failing ? About once a day I notice I can no longer access my share drives. When I go to the unraid machine, main tab, none of the shares are showing.  I click  "Unraid OS Basic" in the upper right corner and it will say something about no license key or corrupt key. I redownloaded the key and placed on the thumb drive but that has not alleviated the issue. The next day it will do the same thing. Rebooting the machine fixes it.

 

In the short term, is there a way I can tell it to recheck the USB key and work so I don't have to reboot? 

 

What is the long term fix? Is my thumb drive going out?

anubarak-diagnostics-20230126-1817.zip

[trurl]

Are you booting from USB2 port (recommended)?

[Troy144]

When I was troubleshooting I tried different ports, so maybe not every time (I was having to restart nearly everyday) - but I just doublechecked and yes it is on a USB 2 port currently and I'll keep that in mind going forward. 

 

And sure enough it is down again, so there is something else going on.

Edited January 27, 2023 by Troy144

[Frank1940]

Setup the syslog server.  Use these instructions. 

 

https://forums.unraid.net/topic/46802-faq-for-unraid-v6/page/2/?tab=comments#comment-781601

 

Since the problem happens rather quickly, use the  Mirror syslog to flash  method.  After the next time it happens-- pull the flash drive, look for the syslog file in the   /logs  folder/directory.  Upload it in a NEW post. 

Edited January 27, 2023 by Frank1940

[Troy144]

ok will do

[Troy144]

Followup from my previous post I turned on syslog to the flash drive, sure enough it crashes in about a day. I don't think any drives are full though - it just stops recognizing the thumb drive / license key about every 24 hrs.

 

The server has been working non stop about a year and a half before these last few weeks - I don't know what's going on. a fresh reboot it will all work as expected for about a day then crash again.

syslog

[JorgeB]

10 hours ago, Troy144 said:

it just stops recognizing the thumb drive / license key about every 24 hrs.

Don't see that in the log but it suggests the flash drive is dropping offline, make sure it's using a USB 2.0 port and if issues persist try a different flash drive.

[Squid]

And are you using Active Directory?

[Troy144]

I'm not using Active Directory, and yes I am using USB 2.0 port

[Troy144]

When it crashes, this is what it looks like. I reboot it and everything works for about a day.

[Troy144]

On 1/27/2023 at 4:36 PM, Frank1940 said:

Setup the syslog server.  Use these instructions. 

 

https://forums.unraid.net/topic/46802-faq-for-unraid-v6/page/2/?tab=comments#comment-781601

 

Since the problem happens rather quickly, use the  Mirror syslog to flash  method.  After the next time it happens-- pull the flash drive, look for the syslog file in the   /logs  folder/directory.  Upload it in a NEW post. 

As requested: 

 

 

[trurl]

On 1/27/2023 at 5:36 PM, Frank1940 said:

Upload it in a NEW post. 

 

17 minutes ago, Troy144 said:

As requested: 

We didn't mean for you to start a new thread, just make a new post in this thread.

 

I have merged the thread.

[Troy144]

Sorry misunderstood, here is the syslog from the latest crash

syslog

[trurl]

Have you rebooted since then?

[trurl]

I suspect you are filling rootfs.

 

What do you get from command line with this?

df -h /

 

[Troy144]

23 hours ago, trurl said:

I suspect you are filling rootfs.

 

What do you get from command line with this?

df -h /

 

Looks like you're correct 

root@Anubarak:~# df -h /
Filesystem      Size  Used Avail Use% Mounted on
rootfs          7.7G  7.7G     0 100% /

 

 

 

Once I reboot it reads:
root@Anubarak:~# df -h /
Filesystem      Size  Used Avail Use% Mounted on
rootfs          7.7G  764M  7.0G  10% /

 

What do I do to fix that and stop from happening ?

 

Edited January 31, 2023 by Troy144

[trurl]

You will have to reboot and figure out what you are doing to make it happen.

 

The usual cause of filling rootfs is something writing to a path that isn't actual storage. Only subfolders of /mnt are actual storage. All other folders are in the OS space, rootfs. If you fill rootfs, the OS has no room to work in.

 

Check your docker mappings

[Troy144]

I may have found it... rclone log. 

 

rclone backs up google drive nightly. well, someone in our group put a shortcut to another folder in a folder and a shortcut in that folder back to the original (so convenient). Rclone doesn't appear smart enough to know a shortcut from a folder and is making infinite nesting of folder 1/folder 2/folder 1/folder 2  ... Eventually the filename gets too long and it starts writing that to the log, infinitely. The default log is in rootfs. 

 

I changed the rclone script to use a /mnt point in the future and deleted the "shortcuts" on google drive. Will inform the team to don't do that. Will wait a couple days to make sure this is the true culprit , I'm 99% sure this is it though.

 

Yep 2 days no issues - that was it, thank you!

Edited February 2, 2023 by Troy144

[trurl]

On 1/30/2023 at 8:50 PM, Troy144 said:

use a /mnt point

What exactly? You can't just invent a new subfolder of /mnt, it has to be a subfolder of /mnt already created by Unraid.