Project: Disk status daemon

[NAS]

As pointed out by Joe and hinted at been needed in many threads this post is to discuss the creation of a disk status daemon.

 

This should actually be that difficult.

 

The daemon could:

 

Using /proc/diskstats check per minute and log usage stats. Particularly important would be some mechanism to log when the disk is last used and usage per hour.

Log disk temperatures

Finer spin down control based on all available timing periods per disk.

Disk spin down based on temperature.

Logs should be based on disk model number and not mount point or dev number. This way logs will be useful permanently and not obsolete should disk order change.

Logs should be optimised for size, we don't want a log explosion.

Logs should be plain text.

 

One age old debate that should be laid to rest with this damon is that its funtions should not cause a disk to stay spun up.

 

This is just a start but once we have this done all sorts of tools could utilise this daemon or its stats. For instance we could chart bandwidth usage vs. time vs. disk. We could also use the stats to help predict a good time for putting the server into standby or even time based spin down of all disks.

 

Using the model number for the logs also would allow us to pool our logs as a community so that we could look at temp stats for say all WD or samsung drives etc

 

 

Opinions

[WeeboTech]

Logs should...

Logs should...

Logs should...

 

I would just as well write log messages to syslog and let a user application choose how to extract that information.

In my case, I forward all syslog messages to a central logging host which is capable of processing these messages in any way.

I would not want to add more logs to unRAID when it's possible to capture messages from syslog.txt or redirect them via a log facility.

 

As far as log messages based on disk model number, this could be a standard part of the message.

 

[NAS]

I think you have taken what i have written the wrong way.

 

e.g. Logs should be optimised for size, we don't want a log explosion.

 

we could write "The current temperature of disk 1 is 45 degrees Celsius" or we could write "T1:54"

 

Where they are written to is pretty much irrelevant and to be honest i never considered them being anywhere else other than syslog.

 

Where i wrote "logs" read "log entries".

 

This does mean that any apps that will use this data will for efficiency sake need to monitor syslog real time as you cant safely assume syslogs will always be a efficient size for processing.

 

 

 

 

[WeeboTech]

I think you have taken what i have written the wrong way.

Where they are written to is pretty much irrelevant and to be honest i never considered them being anywhere else other than syslog.

 

It's not clear, so I made a comment.

 

This does mean that any apps that will use this data will for efficiency sake need to monitor syslog real time as you cant safely assume syslogs will always be a efficient size for processing.

 

It depends, what is this application? When is it going to look at this data and do something with it?

Of course we cannot assume syslogs will be an efficient size, but then again, what is it you are trying to retrieve and what is it you are trying to do with it?

Maybe there needs to be a single DB structure or callable interface that has all of the most current/relevant information so it can be slurped up.

 

I guess I'm curious as to the uses of this data.

I see what looks to be, logging and usage statistics over a period of time.

 

Then there is the hint of something accessing this information from the logs in real time?

[Joe L.]

Perhaps a fuser process with a pseudo fuser file system.  The "files" could each represent a disk, and hold parameters as needed to be monitored by anything needing to act.    I don't really like the idea of having to look at the tail of the syslog for data. 

 

If you want to take a snapshot of data over time for graphing purposes, you can periodically look at the fuser file system and make a record on a physical disk.

 

Joe L.

[WeeboTech]

Would it be easier to write a tcp/ip interface to connect and dump a structure?

Then you are not depending on a local filesystem to access data.

Heck, maybe a gawk script with a http interface ??