File Activity Plugin - How can I figure out what keeps spinning up my disks?


dlandon

Recommended Posts

Jul 13 19:13:45 Tower root: plugin: installing: /boot/config/plugins/file.activity.plg
Jul 13 19:13:45 Tower root: plugin: skipping: /boot/config/plugins/file.activity/file.activity-2021.06.08.tgz already exists
Jul 13 19:13:45 Tower root: plugin: running: anonymous
### [PREVIOUS LINE REPEATED 1 TIMES] ###
Jul 13 19:13:45 Tower root: plugin: skipping: /boot/config/plugins/file.activity/file.activity.cfg already exists
Jul 13 19:13:45 Tower root: plugin: running: anonymous
Jul 13 19:13:45 Tower root: 
Jul 13 19:13:45 Tower root: -----------------------------------------------------------
Jul 13 19:13:45 Tower root:  file.activity is installed.
Jul 13 19:13:45 Tower root:  Copyright 2017-2021, dlandon
Jul 13 19:13:45 Tower root:  Version: 2021.06.08
Jul 13 19:13:45 Tower root: -----------------------------------------------------------
Jul 13 19:13:45 Tower root: 
Jul 13 19:13:45 Tower root: plugin: file.activity.plg installed

 

Have you tried removing and reinstalling it?

Link to comment

The most common problem with file activity not starting is not enough inotify watches.  Look in your log and see if there is a log about not having enough.

 

Several suggestions:

  • Cut down on what you are monitoring.  Don't include Unassigned Devices or Cache and Pool devices.
  • Install the Tips and Tweaks plugin and increase the 'Max Watches'.
Edited by dlandon
Link to comment
  • 2 months later...
On 2/21/2017 at 3:46 AM, dlandon said:

You turned off the cache and that is probably where your files are located.  Turin it on and try again.  Be sure the file activity shows as running.

Here is my system, just 4 discs right now, two 8 TB and 2 1TB NVME for raid 0 cache which i might upgrade later. it seems to be aparant that {I cannot spin all the drives down when I am not acessing them. One drive always stays spinning no matter what I try, to disable everything in dockers and alll the plug in. read SMART /dev/sdb occurs in the logs right after I spin down my discs, meaning disc one the only one that always spins.

 

Disc 1 is always spun on and if i spin it down it spins back up again,
disc 2 is always spun down. It annoyed me that disc 1 was always spun up even
when i wasnt acessing it. SOLUTION:_____? or is it normal for one disc to always stay spun up even with every cache configuration taken into consideration, such as appdata set as only cache and all the system shares same, only cache nothing made any difference so I'm beginning to believe this is how its intended to run, not all the drives will spin down and one must stay spun up to run applications and the system. the rest stay spun down unless accessed. Im probably chasing my own tail trying to spin them all down, there is no set guidline I could find on how its supposed to be. I have nextcloud installed, jellyfin for my media and its set to scan new files manually. and I have binhex-krusader all running. if one disc wears out faster than the others then mabye the only solution is to have a parity drive for it. and the others don't worry about.  Somone with more knowledge tell me, I'm still in my trial period and I spent two full days investigating and trying different settings to make the discs all go into standby after 15 min and it just doesn't seem possible.

Link to comment
15 minutes ago, RYZEN5000 said:

Here is my system, just 4 discs right now, two 8 TB and 2 1TB NVME for raid 0 cache which i might upgrade later. it seems to be aparant that {I cannot spin all the drives down when I am not acessing them. One drive always stays spinning no matter what I try, to disable everything in dockers and alll the plug in. read SMART /dev/sdb occurs in the logs right after I spin down my discs, meaning disc one the only one that always spins.

 

Disc 1 is always spun on and if i spin it down it spins back up again,
disc 2 is always spun down. It annoyed me that disc 1 was always spun up even
when i wasnt acessing it. SOLUTION:_____? or is it normal for one disc to always stay spun up even with every cache configuration taken into consideration, such as appdata set as only cache and all the system shares same, only cache nothing made any difference so I'm beginning to believe this is how its intended to run, not all the drives will spin down and one must stay spun up to run applications and the system. the rest stay spun down unless accessed. Im probably chasing my own tail trying to spin them all down, there is no set guidline I could find on how its supposed to be. I have nextcloud installed, jellyfin for my media and its set to scan new files manually. and I have binhex-krusader all running. if one disc wears out faster than the others then mabye the only solution is to have a parity drive for it. and the others don't worry about.  Somone with more knowledge tell me, I'm still in my trial period and I spent two full days investigating and trying different settings to make the discs all go into standby after 15 min and it just doesn't seem possible.

Post your diagnostics.

Link to comment
20 hours ago, RYZEN5000 said:

Here is my system, just 4 discs right now, two 8 TB and 2 1TB NVME for raid 0 cache which i might upgrade later. it seems to be aparant that {I cannot spin all the drives down when I am not acessing them. One drive always stays spinning no matter what I try, to disable everything in dockers and alll the plug in. read SMART /dev/sdb occurs in the logs right after I spin down my discs, meaning disc one the only one that always spins.

 

Disc 1 is always spun on and if i spin it down it spins back up again,
disc 2 is always spun down. It annoyed me that disc 1 was always spun up even
when i wasnt acessing it. SOLUTION:_____? or is it normal for one disc to always stay spun up even with every cache configuration taken into consideration, such as appdata set as only cache and all the system shares same, only cache nothing made any difference so I'm beginning to believe this is how its intended to run, not all the drives will spin down and one must stay spun up to run applications and the system. the rest stay spun down unless accessed. Im probably chasing my own tail trying to spin them all down, there is no set guidline I could find on how its supposed to be. I have nextcloud installed, jellyfin for my media and its set to scan new files manually. and I have binhex-krusader all running. if one disc wears out faster than the others then mabye the only solution is to have a parity drive for it. and the others don't worry about.  Somone with more knowledge tell me, I'm still in my trial period and I spent two full days investigating and trying different settings to make the discs all go into standby after 15 min and it just doesn't seem possible.

 

New update - SOLUTION FOUND TO WHY IS UNRAID ALWAYS READING AND WRITING TO DISC 1, ALWAYS ON DISC 1. 

Changed alot of settings, spent many hours investigating and it turned out to be something really simple.

Docker.img was located on disc 1 and the docker directory was all disc 1. All I had to do was "settings""docker"

"disable" then I was able to change it to a new directory on the cash. I chose appdata and set my share to cache only and immediately after the disc1 stopped reading and writing all the time, I litterally jumped up and down for joy that I finally found a solution because Im a newbie to Unraid and I like it but I also don't want my discs spinning when I'm not acessing them because my server spends 90% of the day at idle not being used. 100% verified, non of my discs stay spinning all the time now they all sit in standby mode untill acessed. The DOCKER is on the cache now. Was very difficult for me to learn how this all works but now I am supper happy with my unraid system and I dont install shit to discs anymore, the default directories the programs give may not always be ideal for all of us.

Link to comment
Just now, RYZEN5000 said:

 

New update - SOLUTION FOUND TO WHY IS UNRAID ALWAYS READING AND WRITING TO DISC 1, ALWAYS ON DISC 1. 

Changed alot of settings, spent many hours investigating and it turned out to be something really simple.

Docker.img was located on disc 1 and the docker directory was all disc 1. All I had to do was "settings""docker"

"disable" then I was able to change it to a new directory on the cash. I chose appdata and set my share to cache only and immediately after the disc1 stopped reading and writing all the time, I litterally jumped up and down for joy that I finally found a solution because Im a newbie to Unraid and I like it but I also don't want my discs spinning when I'm not accessing them because my server spends 90% of the day at idle not being used. 100% verified, non of my discs stay spinning all the time now they all sit in standby mode untill acessed. The DOCKER is on the cache now. Was very difficult for me to learn how this all works but now I am supper happy with my unraid system and I dont install shit to discs anymore, the default directories the programs give may not always be ideal for all of us.

 

Link to comment
9 hours ago, RYZEN5000 said:

 

Unless you know your docker is installed on the cache dont assume it is, My discs are all on standby still and they work perfectly since I changed the location of the docker to appdata. Perfect perfect, I am too happy now that I finally got all the things worked out in my system, I cant complain about anything really. I've tried every OS there is and Unraid is the one for me. Its the only one that my 2.5 G motherboard is instantly compatible with regard to the ethernet port, I don't need an external ethernet card to connect it uses my original port so I just ordered a 2.5 switch and adapter for my laptop and i hope to achieve greater than 114 mbs transfers and I will update you all when I test the new hardware out. Whether it worked or not. I must say, size matters when it comes to system cpu and ram. you don't need two lan cables with a server, just one. my system is as follows: large PC case glass side panel with 3 200mm fans 2 front and 1 top and 1  140mm back. 8 quick snap 3.5 drive bays and can mount total 10 drives.

 

This is my medium expensive brand new system starter kit, I will add more drives later and I might also swap return my cpu power supply for a 750 watt just because I feel 600 is low for the amout of fans I have and drives I plan on adding if they all apin up at once i dont want to be low on power, also I wish I had gotten 2X 2TB cache ssd to begin with instead of 1 TB X2. Its so easy to use 500 MB accumulated in minutes while transferring and downloading from two laptops at once. I might return those or exchange them.

Thermaltake Core V71 Tempered Glass Edition E-ATX Full Tower Tt LCS Certified Gaming Computer Case CA-1B6-00F1WN-04

EVGA 600 BR, 80+ Bronze 600W, 3 Year Warranty, Power Supply 100- BR-0600-K1

ASUS ROG Strix B550-F Gaming AMD AM4 (3rd Gen Ryzen™) ATX gaming motherboard (PCIe 4.0, 2.5Gb LAN, BIOS FlashBack, HDMI 2.1, Addressable Gen 2 RGB header and AURA Sync)  

XPG Gammix D10 DDR4 32GB (2x16GB) 3200MHz PC4 25600 288-Pin CL16-20-20 Memory Module Kit (Black)

AMD Ryzen 5 5600G 6-Core 12-Thread Desktop Processor with Radeon Graphics

LINESO 6 Pack Straight SATA III Cable 6.0 Gbps 18 Inches (red)

 

Arctic MX-4 Thermal Compound Paste, Carbon Based High Performance, Heatsink Paste, CPU for All Coolers, Interface Material, 4 Grams with Tool

Ethernet Cable CAT8, 10ft 2Pack, High Speed Flat CAT8 Network Internet LAN Cable, Durable Patch Cord with Gold Plated RJ45 Connector, Outdoor & Indoor, Compatible with CAT7 CAT6 CAT5

TRENDnet 5-Port Unmanaged 2.5G Switch, 5 x 2.5GBASE-T Ports, 25Gbps Switching Capacity, Backwards Compatible with 10-100-1000Mbps Devices, Fanless, Wall Mountable, Black, TEG-S350

Cable Matters USB to 2.5G Ethernet Adapter Supporting 2.5 Gigabit Ethernet Network - USB C Adapter Included for USB-C and Thunderbolt 3

Lexar JumpDrive S47 32GB USB 3.1 Flash Drive, Black

8 Ports PCI-E to SATA 3.0 Card,PCI Express PCI-e Expansion Card,PCI-E to SATA3.0 Controller Expansion Board,for Windows 7/8/XP/10,with Driver Disc,Semi-High Baffle

2 x WD Red Pro 8TB NAS Internal Hard Drive - 7200 RPM Class, SATA 6 Gb/s, CMR, 256 MB Cache, 3.5" - this is just a start

2 cache drives

WD Blue SN550 1TB NVMe Internal SSD - Gen3 x4 PCIe 8Gb/s, M.2 2280, 3D NAND, Up to 2,400 MB/s - WDS100T2B0C - two of these.

Link to comment
  • 1 month later...
  • 1 month later...

i tried to use this plugin for my specific problem but to no avail. i only get cache drive logs (because of docker / vm stuff), no disk activity logs. so this plugin does not report any file activity on my disks. however, my disks do not stay spun down, as you can see from the log (spin downs are manual):

 

Jan 14 23:54:11 hp emhttpd: spinning down /dev/sdb
Jan 14 23:54:11 hp emhttpd: spinning down /dev/sdf
Jan 14 23:54:13 hp emhttpd: read SMART /dev/sdb
Jan 14 23:54:17 hp emhttpd: spinning up /dev/sdf
Jan 14 23:54:27 hp emhttpd: read SMART /dev/sdf
Jan 14 23:54:31 hp emhttpd: spinning down /dev/sdb
Jan 14 23:54:31 hp emhttpd: spinning down /dev/sdf
Jan 14 23:54:34 hp emhttpd: read SMART /dev/sdb
Jan 14 23:56:39 hp emhttpd: read SMART /dev/sdc
Jan 14 23:56:52 hp emhttpd: read SMART /dev/sdf

 

any idea what else i can try to find the culprit?

 

i have docker and vms completely on cache (libvir, docker.img, appdata...) and all docker containers shut down just to be sure.

 

still, something spins up my disks and at the rate it is happening, i might be better of not spinning them down at all.

 

any hint would be appreciated

Link to comment

im sorry, please find it attached.

 

also, here is the output from tonight from your plugin and the tail of syslog:

 

** Disk 1 **
Jan 15 00:53:10 OPEN => /mnt/disk1/Serien/The Simpsons/S07/The Simpsons - S07E11.avi
Jan 15 00:53:10 OPEN => /mnt/disk1/Serien/The Simpsons/S07/The Simpsons - S07E11.avi
Jan 15 00:53:10 OPEN => /mnt/disk1/Serien/The Simpsons/S07/The Simpsons - S07E11.avi
Jan 15 00:55:53 OPEN => /mnt/disk1/Serien/The Simpsons/S07/The Simpsons - S07E11.avi
Jan 15 01:15:11 OPEN => /mnt/disk1/Serien/The Simpsons/S07/The Simpsons - S07E12.avi
Jan 15 01:15:12 OPEN => /mnt/disk1/Serien/The Simpsons/S07/The Simpsons - S07E12.avi
Jan 15 01:15:12 OPEN => /mnt/disk1/Serien/The Simpsons/S07/The Simpsons - S07E12.avi

 

Jan 15 03:11:29 hp emhttpd: spinning down /dev/sdc
Jan 15 03:14:42 hp emhttpd: read SMART /dev/sdc
Jan 15 03:22:30 hp emhttpd: spinning down /dev/sdf
Jan 15 03:25:42 hp emhttpd: read SMART /dev/sdf
Jan 15 03:44:30 hp emhttpd: spinning down /dev/sdc
Jan 15 03:47:42 hp emhttpd: read SMART /dev/sdc
Jan 15 03:55:31 hp emhttpd: spinning down /dev/sdf
Jan 15 03:58:43 hp emhttpd: read SMART /dev/sdf
Jan 15 04:17:30 hp emhttpd: spinning down /dev/sdc
Jan 15 04:20:43 hp emhttpd: read SMART /dev/sdc
Jan 15 04:28:31 hp emhttpd: spinning down /dev/sdf
Jan 15 04:31:43 hp emhttpd: read SMART /dev/sdf
Jan 15 04:50:31 hp emhttpd: spinning down /dev/sdc
Jan 15 04:53:43 hp emhttpd: read SMART /dev/sdc
Jan 15 05:01:32 hp emhttpd: spinning down /dev/sdf
Jan 15 05:04:44 hp emhttpd: read SMART /dev/sdf
Jan 15 05:23:31 hp emhttpd: spinning down /dev/sdc
Jan 15 05:26:44 hp emhttpd: read SMART /dev/sdc
Jan 15 05:34:32 hp emhttpd: spinning down /dev/sdf
Jan 15 05:37:44 hp emhttpd: read SMART /dev/sdf
Jan 15 05:56:32 hp emhttpd: spinning down /dev/sdc
Jan 15 05:59:44 hp emhttpd: read SMART /dev/sdc
Jan 15 06:07:33 hp emhttpd: spinning down /dev/sdf
Jan 15 06:10:45 hp emhttpd: read SMART /dev/sdf
Jan 15 06:29:32 hp emhttpd: spinning down /dev/sdc
Jan 15 06:32:45 hp emhttpd: read SMART /dev/sdc
Jan 15 06:40:33 hp emhttpd: spinning down /dev/sdf
Jan 15 06:43:45 hp emhttpd: read SMART /dev/sdf
Jan 15 07:02:33 hp emhttpd: spinning down /dev/sdc
Jan 15 07:05:45 hp emhttpd: read SMART /dev/sdc
Jan 15 07:13:33 hp emhttpd: spinning down /dev/sdf
Jan 15 07:16:46 hp emhttpd: read SMART /dev/sdf
Jan 15 07:26:59 hp emhttpd: read SMART /dev/sdd

 

please focus only on sdd, sdc, scf. the others are not part of the array - i guess you would find that out from diagnostics anyway, but just saying...

 

while creating the diagnostics file i found parity also spinning up, which was also not written into your plugin. so my guess is that its a system job that keeps spinning up the disks, not a file access.

 

hp-diagnostics-20220115-0726.zip

Edited by Michael H
Link to comment

I woudl start by instaling the "Cache Dirs" plugin and set to include only array shares on spinning disks.  Don't let cache dirs cache everything.  I gets pretty aggressive.  You can use the default settings for everything else.

 

What may be happening is that directories may be being accessed that File Activity doesn't pick up.  It's not meant to.  If it did the log would be huge and unmanageable.

 

As for the file activity on disk1, any idea why that filw would be opening so often?  IYou are using a Plex docker container, try stopping it for a while and see if the disk access stops.

Link to comment

Thanks for the tip with the cache dirs, i will do that and see if that helps. you said your plugin is not made to log that, as it would log too much... do you have a command line tool or tip for me how i could create such a log anyway? i dont mind greping through a few thousand lines, even manually searching would be fine at this point. literally nothing runs and still the disks spin up - i have to find the reason, even if its the last thing i do.

 

at the time of logging i had literally stopped every docker and vm (except pfsense in vm, running from cache only) and the only computer that was on at the time was the one where i was accessing the gui from, while i made sure that all smb connections from that machine were down.

 

plex container was alsso down at the time of logging. also i have made sure that its appdata is on cache, the automatic jobs are disabled and its not accessible via the internet - so no background file access should take place i guess. especially not when the docker is not started :D

 

as for the opening of one file in rapid succession... i dont know, seems vlc on mac specific. that would not bother me though, as i wanted that file played, so the disk has to spin up. i only included it in the log to prove to myself that i had the logging set up correctly and it was in fact logging file access.

 

as software seems to be a dead end for now - because if nothing runs, nothing can do anything, in my mind at least and your tool confirms that - i think maybe its the hp hardware p840 raid controller (running in hba mode) that wants to get smart data... that thing is very fuzzy, unraid cant read the smart values from it right - maybe its this combination. i would have no idea what to replace it with though, im not a server hardware compatibility expert...

 

 

 

 

Link to comment

i did that now, plex docker was also offline for good measure. disks came back from spindown twice BUT: read and write counter did not increase. so now i suspect that my wd red drives do not spin down correctly or at lease report their status wrong...

 

 

that would also fit to my observation that parity does not spin up on its own. coicidentally, this is an ironwolf drive, not a wd red... i wish i had more ironwolfs for testing this theory...

grafik.thumb.png.580747b50e4a909c6ca37ae3b4cbc817.png

 

what i would like to try next is disabling smart. i know that smart should skip spun down disks, but maybe the problem is there. if it was "real" disc access, i would assume the read counter would increase.

 

unfortunately i have no idea how i would do that... do you know by any chance? i know its not a good thing to do and in the long run id rather have spun up disks and smart reporting, but i would sleep easier when i know what causes the spin up.

 

anyways, diagnostics is attached. thanks for your investment in this issue!

 

 

hp-diagnostics-20220116-1739.zip

Link to comment

You have other docker contaoiners installed.  Turn them all off, turn off your VMs and see if the disks stop spinning up.  If they don't spin up, start dockers and VMs one at a time until you find which one is causing the spin ups.  My guess is a misconfigured docker container.

 

6 hours ago, Michael H said:

read and write counter did not increase

Not every disk acces will show in the read or write counters.

Link to comment
15 hours ago, dlandon said:

You have other docker contaoiners installed.  Turn them all off, turn off your VMs and see if the disks stop spinning up.  If they don't spin up, start dockers and VMs one at a time until you find which one is causing the spin ups.  My guess is a misconfigured docker container.

 

That was my first idea as well and i double and triple checked every single container and vm.

 

anyways, just so that i quadruple-check again i stopped vm and docker service like you said, spun down disks and like clockwork they spin up again even before i start to miss my internet access (because pfsense is down):

 

Jan 17 15:35:27 hp avahi-daemon[4341]: Withdrawing address record for 192.168.122.1 on virbr0.
Jan 17 15:35:27 hp root: Network a67b7b9b-bef4-488b-b243-c9e5f391a3b1 destroyed
Jan 17 15:35:27 hp root:
Jan 17 15:35:30 hp root: Stopping virtlogd...
Jan 17 15:35:31 hp root: Stopping virtlockd...
Jan 17 15:35:32 hp emhttpd: shcmd (78933): umount /etc/libvirt
Jan 17 15:35:45 hp emhttpd: spinning down /dev/sdc
Jan 17 15:35:47 hp emhttpd: spinning down /dev/sdf
Jan 17 15:36:12 hp nmbd[1511]: [2022/01/17 15:36:12.357874, 0] ../../source3/libsmb/nmblib.c:922(send_udp)
Jan 17 15:36:12 hp nmbd[1511]: Packet send failed to 192.168.122.255(138) ERRNO=Network is unreachable
Jan 17 15:41:49 hp emhttpd: read SMART /dev/sdc
Jan 17 15:42:02 hp emhttpd: read SMART /dev/sdf

 

and its always sdc first and half a minute later sdf (disk 1 & 2 of the array)...

Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.