Syslog color-coder from MyMain made into a stand alone program

[Rajahal]

I'm trying to learn more about reading and understanding syslogs.  This is something that I believe is completely normal, yet it has always confused me.

 

Nov 25 18:03:17 Tower kernel: ata4: SATA link up 3.0 Gbps (SStatus 123 SControl 300)

Nov 25 18:03:17 Tower kernel: ata4.00: ATA-8: WDC WD20EARS-00MVWB0, 51.0AB51, max UDMA/133

Nov 25 18:03:17 Tower kernel: ata4.00: 3907029168 sectors, multi 0: LBA48 NCQ (depth 31/32), AA

Nov 25 18:03:17 Tower kernel: ata4.00: configured for UDMA/133

 

In the above snippet, this drive is first recognized as SATAII (this is good).  However, then on the next few lines it is referred to as 'max UDMA/133' and 'configured for UDMA/133'.  Doesn't that mean it is running at IDE speeds?

 

If you need to see the full syslog, I grabbed it from this post.

[Joe L.]

One post I found doing a google search said:

Justin Piszcz wrote:

> Do some chipsets (SiI 3132 vs. Intel ICH9) run certain drives at

> UDMA/100 vs. UDMA/133?

 

On native SATA drives and controllers, the UDMA speed really has no

effect. At the low level there are just frames going back and forth on

the link at either 1.5 or 3.0 Gbps so the UDMA speed doesn't really

control anything. If there is a PATA bridge involved (either internal or

external to the drive) then this can affect the speed on the PATA bus,

but that won't be the case here (NCQ support wouldn't be possible in

that case).

 

 

[Rajahal]

So it is safe to just ignore that line?  I'm finding that the bulk of the art of syslog interpretation is knowing what can be safely ignored

 

Thanks Joe.

 

By the way, considering the huge amount of work that you and BJP999 have been doing with MyMain, I was wondering if the new color-coded syslog viewer could be wrapped up in a stand-alone executable?  I figure something like that would be a great help to those of us who help others interpret their syslogs.  I'm sure both of us spend far more time looking at others' syslogs in Notepad then we do looking at our own in MyMain...

[SSD]

The myMain syslog viewer allows you to easily select a different syslog for viewing. Is that the feature you are looking for.

 

Btw, if you Re studying syslogs, you might consider keeping a list of items that should be highlighted as errors, warnings, or other syslog viewer categories. These can be used to improve the highlighting accuracy and usefulness.

[Rajahal]

Excellent, thanks.  However, I actually spend most of my time reading syslogs during my downtime at work, where I have no access to my server at home.  Therefore, a stand-alone executable would suit me personally.  However, I understand that there may not be a big need for it, so no worries if you don't feel like making it.  If I knew anything about programing I would make it myself.

[Joe L.]

Excellent, thanks.  However, I actually spend most of my time reading syslogs during my downtime at work, where I have no access to my server at home.  Therefore, a stand-alone executable would suit me personally.  However, I understand that there may not be a big need for it, so no worries if you don't feel like making it.  If I knew anything about programing I would make it myself.

You don't need access to your server...  Just ANY pc you can boot on a flash drive with a copy of unRAID that has unMENU added to it.

 

Then, boot on the flash drive, DO NOT ASSIGN ANY DISKS, and use unMENU's enhanced syslog viewer. 

 

unplug the flash drive, reboot, and the work PC is exactly as it was, with no changes.

[prostuff1]

Or you could try getting the Virtual Box image up and running.  Update it to the latest version of unRAID and you should be able to get it working.

[Rajahal]

Thanks for the suggestions, but none of that would work for me.  As these are gov't computers, all USB ports are locked down, so I can't boot from anything.  I also doubt that I could run Virtual Box without getting in trouble.

 

I have basic admin rights to my computer (which I'm not supposed to have), so I can generally install programs and run .exe's.  I feel like Virtual Box would be pushing my luck, though.  I have to disguise my FireFox install as something else to hide it from IT

 

Again, I completely understand that if I'm the only person who would use this then it isn't worth pursuing.  I just thought it might be useful to some of the other hero members as well.

[Joe L.]

Thanks for the suggestions, but none of that would work for me.  As these are gov't computers, all USB ports are locked down, so I can't boot from anything.  I also doubt that I could run Virtual Box without getting in trouble.

That does limit your options, doesn't it.

 

Don't want the gov't getting mad at you. Or you  might get "locked down" like the USB ports. 

[Rajahal]

Very true.  I suppose the solution is for me to stop spending time on these forums while at work

[prostuff1]

Very true.  I suppose the solution is for me to stop spending time on these forums while at work

 

gasp... blasphemy... gasp

 

An app like what you mention would not be overly complicated to do in Java (or any other language for that matter), but I have enough stuff to think about currently.  If I get some free time I might throw something together.

[Joe L.]

Very true.  I suppose the solution is for me to stop spending time on these forums while at work

 

gasp... blasphemy... gasp

 

An app like what you mention would not be overly complicated to do in Java (or any other language for that matter), but I have enough stuff to think about currently.  If I get some free time I might throw something together.

Perhaps a firefox extension??? Javascript supports regular-expressions, and that would certainly do it.

[prostuff1]

Perhaps a firefox extension??? Javascript supports regular-expressions, and that would certainly do it.

 

I have not done any firefox extension work, but would guess that it could be done that way also.

 

Will have to look into it if/when I find the time.

[Rajahal]

I could definitely work with a FF extension.  Thanks guys, and no rush.

[Spectrum]

ooh this sounds like a perfect application for a greasemonkey script..... Of course I don't know javascript so I couldn't do it

The other way I thought about doing it would be using a Perl script to match regexes and spit out HTML; that I could whip up in no time flat, ok a few minutes, but I would need a list of things to match against.  I can scan a syslog and have pretty good feel for what is going on, but I don't have a comprehensive list of what should be what color.

[prostuff1]

ooh this sounds like a perfect application for a greasemonkey script..... Of course I don't know javascript so I couldn't do it

The other way I thought about doing it would be using a Perl script to match regexes and spit out HTML; that I could whip up in no time flat, ok a few minutes, but I would need a list of things to match against.  I can scan a syslog and have pretty good feel for what is going on, but I don't have a comprehensive list of what should be what color.

 

Look at the current syslog viewer in unMenu for an idea of what needs to be what color.  There is a file in the unmenu folder on the flash drive called syslog_match.conf. It contains all the things that the syslog viewer is matching against.

[Joe L.]

ooh this sounds like a perfect application for a greasemonkey script..... Of course I don't know javascript so I couldn't do it

The other way I thought about doing it would be using a Perl script to match regexes and spit out HTML; that I could whip up in no time flat, ok a few minutes, but I would need a list of things to match against.  I can scan a syslog and have pretty good feel for what is going on, but I don't have a comprehensive list of what should be what color.

the regular expressions, the phrases, and the colors are all in syslog_match.conf.  The first expression matched sets the color.  This allows a few exceptions to the more general patterns to go at the top of the set of patterns to match.    The first term on each line in syslog_match.conf indicates if a case-insensitive match should be made (any_case), or to match the exact case (match_case).  Lines in the file with a leading "#" are comments. The terms on each line are separated by "||"

 

The lines in syslog_match.conf look like this:

any_case||"DHCP_ACK received from"||gray

any_case||": connect from [0-9.]*"||gray

#any_case||"ROOT LOGIN  on "||gray

any_case||"kernel:[ \t]*$"||gray

 

# Match these first, to prevent from false hits in other colors.

match_case||" ACPI Error "||orange

match_case||" read_file: error 2 opening /boot/config/super.dat"||green

match_case||" ntpd.*Listening on interface"||purple

match_case||" checksumming"||blue

match_case||"HPA detected: current [0-9]*168,"||black

 

# Other user-defined syslog entries

# Match these first, to prevent from false hits in other colors.

match_case||"preclear_disk"||brown

 

any_case  ||"error"||red

match_case||"Emask "||red

match_case||"arity incorrect"||red

 

 

[Spectrum]

Hah you two were on top of that real quick!  Here's v0.001, don't blame me if it goes haywire

 

It's written in perl so you can take a look inside and see what's going on. Nothing fancy, just matching regexes that Joe and Prostuff have already figured out   It doesn't stop processing on a hit like the syslog page on the webserver does so that may create a few false hits or something getting the incorrect color.

 

It takes the filename of the syslog as an argument and outputs HTML version to stdout.  To save to a file use redirection:

slc syslog > syslog.htm

then open in your webrowser of choice.

 

If it's useful let me know!

Bah forum won't take an extensionless file.  Just save without an extension or use a .pl extension in Windows and you should be good.

slc.txt

[Rajahal]

Thanks Spectrum!  I renamed it to slc.pl (using WinXP at work), but it still wants to open with Notepad.  Opening it with FF also just shows the code.  How do I run it?

[Spectrum]

Need to have a Perl interpreter and Windows doesn't have that by default.

Strawberry is what I use on my Windows boxes.  If you can't install something to the root drive, you can grab this and give it a try.  It looks like the host that was hosting the portable edition of Strawberry isn't hosting it anymore

 

If you don't have the .pl extension registered as an executable type, you can put the perl interpreter in your path and call the script with

perl scl.pl syslog >> syslog.html

or just specify the direct path to the perl.exe

c:\myhiddenperl\perl.exe scl.pl syslog >> syslog.html

 

Wish Windows had better scripting by default.  Sadly it's just batch files, vbscript, jscript, and wsh on Windows XP.

[Spectrum]

Flash of insight after I posted that last one.  I installed par and packed that into an exe so you don't actually need to grab a perl interpreter.  Seems to work fine but is a little shy of 1 MB zipped so I can't post it here

Also I updated it to 0.002 for more graceful exits if command args are incorrect.

Anyone that wants the exe feel free to PM me for a link.

slc.txt

[Rajahal]

Incredible, Spectrum, this works great.

[Spectrum]

If I get a burr up my butt I may update the perl script to read in the conf files from the unMENU script.  That way it's more universal and easy to update.  Easy to update as in I don't have to do it, Joe and/or prostuff can take care of any changes in that conf file

If (when! gotta think positive) I do I'll shoot you a copy of the exe and I'll post the perl script in this thread.

[Spectrum]

Here is  the updated version, v0.003.  It uses the syslog_match.conf file from unMenu to get regexes and colors.  PM me if you need the compiled binary.

slc.txt