make the diagnostics files anonymous

[Fireball3]

Collecting info about a server is OK for troubleshooting but since we are supposed to

post them on a public forum, I would expect/recommend to respect privacy and therefore

remove (probably more reasonable to alter) all user specific customization.

e.g. user names, share names, server name etc.

[John_M]

I thought about this, when I uploaded a syslog and diagnostics just a few minutes ago. Should I edit them in order to obfuscate information such as serial numbers? In the end, I decided that it was more important not to tamper with the files than to suppress what is only trivially sensitive information.

 

User names and share names: why would I have any concern over them? My username is the same as my username here and I only have two shares on each server: Public and Private.

 

Server names: no concern, whatsoever. In fact, part of the joy of a new build it to give it a meaningful name: my three unRAID servers are called Lapulapu, Mandaue and Northolt and they are named for places around the world that have a personal significance to me. You'll notice they progress alphabetically as they represent my 12th, 13th and 14th builds. The 15th will be called Opao. Forgive me that indulgence; some people have elaborate avatars.

 

Passwords: they are not included, so no problem there.

 

Serial numbers: this is the one that I thought about, though I can't say I agonised over it. Serial numbers are a little bit sensitive but I always register my disks and other serial numbered hardware with the manufacturer for potential RMA purposes so I have little worry about anyone trying to steal the serial numbers and register them in their own name. unRAID uses disk serial numbers to identify them so redacting the syslog would actually make it difficult to follow.

 

I'm cautious, but not paranoid 

 

[Squid]

Disk serial numbers are only useful if you try and return a drive.  Beyond that they are meaningless.  Do you worry about the fact that your car's VIN is visible to the world?

[Fireball3]

User names and share names: why would I have any concern over them? My username is the same as my username here and I only have two shares on each server: Public and Private.

 

Server names: no concern, whatsoever. In fact, part of the joy of a new build it to give it a meaningful name: my three unRAID servers are called Lapulapu, Mandaue and Northolt and they are named for places around the world that have a personal significance to me. You'll notice they progress alphabetically as they represent my 12th, 13th and 14th builds. The 15th will be called Opao. Forgive me that indulgence; some people have elaborate avatars.

 

Passwords: they are not included, so no problem there.

 

Yes, I don't care about hardware related information!

 

With respect to share names and user names etc., there may be more obvious and meaningful setups

like yours (no offense) where you can deduce things from.

[jonp]

I'm not sure why share names or user names need to be scrubbed as they are non-identifiable information and may be relevant to how a problem is described and resolved.  How would you tell me that X share is having problems or isn't writing to the cache and how would correlate that inside your log?

[trurl]

Another thing I think some would like scrubbed is syslog entries from mover. I don't cache user share writes myself, but there can be things in file and folder names that some might not like revealed to the world, such as your latest PRON download.

 

Of course there are times when mover entries are needed for troubleshooting, so there should be some way to get them when needed. Maybe only scrub them for the diagnostics zip but leave them in when getting syslog separately.

[gundamguy]

I'm not sure why share names or user names need to be scrubbed as they are non-identifiable information and may be relevant to how a problem is described and resolved.  How would you tell me that X share is having problems or isn't writing to the cache and how would correlate that inside your log?

 

From the prospective of actual security you are 100% correct.

 

From the prospective of embarrassment / legality issues it's a bit more complicated.

 

I mean I don't know about you but like I don't really need you guys to know that I pirate romcoms... (I don't... it's just an example )

 

But more importantly there is some things that you can posses in some countries that you can't in others, and a log that contains information that points to that can be used against you. Like they wouldn't have enough information to know who you are / where you are based on only the log but combined with other information they can piece together on the internet it becomes a pretty clearer picture.

 

In the case of mover, couldn't it just suppress logging of individual files moved and just report success or errors I.E.

Mover Starts 11:59
Mover Finished: 11 Files Moved successfully 1 file not moved? Or Mover Failed Error 2?

 

Yeah I'm not sure this can actually be done, but I can see why some people might appreciate not having to air there dirty laundry every time there is an error with there system.

 

I can see how this might take a bit of work, to change.

[jonp]

I'm not sure why share names or user names need to be scrubbed as they are non-identifiable information and may be relevant to how a problem is described and resolved.  How would you tell me that X share is having problems or isn't writing to the cache and how would correlate that inside your log?

 

From the prospective of actual security you are 100% correct.

 

From the prospective of embarrassment / legality issues it's a bit more complicated.

 

I mean I don't know about you but like I don't really need you guys to know that I pirate romcoms... (I don't... it's just an example )

 

But more importantly there is some things that you can posses in some countries that you can't in others, and a log that contains information that points to that can be used against you. Like they wouldn't have enough information to know who you are / where you are based on only the log but combined with other information they can piece together on the internet it becomes a pretty clearer picture.

 

In the case of mover, couldn't it just suppress logging of individual files moved and just report success or errors I.E.

Mover Starts 11:59
Mover Finished: 11 Files Moved successfully 1 file not moved? Or Mover Failed Error 2?

 

Yeah I'm not sure this can actually be done, but I can see why some people might appreciate not having to air there dirty laundry every time there is an error with there system.

 

I can see how this might take a bit of work, to change.

Yeah, I see your point. Not sure how to deal with that. I see apps like sickbeard, couch, and sab all logging detailed information like that in their respective apps as well, so even if we did scrub that info out of our logs, what's the solution for the app logs?  It seems to me if we can't scrub it all, there's no point.  And then you have the issue of confusion when trying to work with someone to resolve their specific issue(s).

 

This seems like an undecidable problem to solve.

[trurl]

Yeah, I see your point. Not sure how to deal with that. I see apps like sickbeard, couch, and sab all logging detailed information like that in their respective apps as well, so even if we did scrub that info out of our logs, what's the solution for the app logs?  It seems to me if we can't scrub it all, there's no point.  And then you have the issue of confusion when trying to work with someone to resolve their specific issue(s).

 

This seems like an undecidable problem to solve.

If the apps are running in dockers, even if they log to unRAID storage those apps logs are seldom requested on this forum but the diagnostics and/or syslog are often posted. The bottom of the page for Tools - Diagnostics even includes this disclaimer

No personal information such as user names, passwords, or any other file contents not specified above is included by unRAID Server OS; however, your server name, IP address, and user share names will be included.

Note that 3rd-party plugins may or may not store personal information in plugin-specific configuration files and/or output to the system log.

According to that it seems some things are already being scrubbed, and anything logged by apps are not unRAIDs responsibility.

 

It seems like some things like mover logs are clearly unRAIDs responsibility, are easily recognized and could be scrubbed.

[BRiT]

The problem is simple. Just never get yourself in to a situation where you need community help.

[kizer]

As long as serious security issues are not in my log files I don't think anybody should have anything to hide. If your looking at Porn and have server issues I think he/she should more or less man/woman up and take the embarrassment if necessary to get your machine 100%.

 

More information is better than redacted information that might miss critical review.

[trurl]

I don't have anything moving and anything my dockers are doing don't appear in the syslog.  But I have tried to help others in the past that were hesitant to post their syslog and instead gave me a link by PM. Then nobody but me can take on the problem which is not the way I think things should work around here, two (or more) heads are better than one, keeping things in the forum allows others to learn from someone else's experience, etc. In fact, I often get PMs for help and I redirect it back to the forum for those reasons.

[RobJ]

I don't have anything moving and anything my dockers are doing don't appear in the syslog.  But I have tried to help others in the past that were hesitant to post their syslog and instead gave me a link by PM. Then nobody but me can take on the problem which is not the way I think things should work around here, two (or more) heads are better than one, keeping things in the forum allows others to learn from someone else's experience, etc. In fact, I often get PMs for help and I redirect it back to the forum for those reasons.

 

I agree with this.

 

It would be nice to have an obfuscation script tied to a button on the Diagnostics download page.  JoeL, when he was less busy, used to be great at creating small sed scripts, that could do just about anything.  In this case, it would search the syslog for lines between "logger: mover started" and "logger: mover finished", then locate the file names, strings after the last '/', keep the extension and punctuation and digits but replace all letters with 'a' or 'x'.  The remaining structure of the file name and path might be enough for the user to recognize, if needed.

[bonienl]

Perhaps an option to anonymize the diagnostics, for public or private viewing ?

 

[trurl]

Perhaps an option to anonymize the diagnostics, for public or private viewing ?

+1!

[BRiT]

Perhaps an option to anonymize the diagnostics, for public or private viewing ?

 

It would be hilarious if the anonymizer replaces all legit file names and share names and mover logs with random porn titles.

[bonienl]

It would be hilarious if the anonymizer replaces all legit file names and share names and mover logs with random porn titles.

 

I like that idea 

[trurl]

It would be hilarious if the anonymizer replaces all legit file names and share names and mover logs with random porn titles.

 

I like that idea 

Or we could just pretend it does thus giving the users plausible deniability.

[gundamguy]

Perhaps an option to anonymize the diagnostics, for public or private viewing ?

 

Yeah that is awesome. 

 

 

[Fireball3]

I'm not sure why share names or user names need to be scrubbed as they are non-identifiable information and may be relevant to how a problem is described and resolved.

 

Good point. Some issues might be burried within the name itself e.g. special characters etc.

Therefore I suggest to export both versions.

The anonymous one for posting in the forum and the uncensored one for a further investigation if necessary.

The latter would be PM'ed to the respective problem solver if necessary.

[John_M]

I'm sorry, but I really don't like this feature. It's a major hindrance to troubleshooting. See here:http://lime-technology.com/forum/index.php?topic=45889.msg438788#msg438788

 

Yes, passwords, email addresses and possibly IP addresses (though I should think few people use publicly accessible IP addresses on a LAN these days) should be obscured, but share names and file names is a step too far, IMHO.

 

The trouble, of course, is that while I may choose not to obfuscate my own diagnostics, I have no control over its use by people asking for help. It just makes extra work for people trying to offer help.

[RobJ]

I'm sorry, but I really don't like this feature. It's a major hindrance to troubleshooting. See here:http://lime-technology.com/forum/index.php?topic=45889.msg438788#msg438788

 

Yes, passwords, email addresses and possibly IP addresses (though I should think few people use publicly accessible IP addresses on a LAN these days) should be obscured, but share names and file names is a step too far, IMHO.

 

The trouble, of course, is that while I may choose not to obfuscate my own diagnostics, I have no control over its use by people asking for help. It just makes extra work for people trying to offer help.

I heartily sympathize!  Useful info was obfuscated, making it harder to recognize the true problem source.  But I'm afraid we may have to live with this, going forward.  Perhaps some day, we'll have a minimal anonymization option - obfuscating only passwords and email addresses.

[JonathanM]

Perhaps some day, we'll have a minimal anonymization option - obfuscating only passwords and email addresses.

Maybe there could be an option to show the user what is going to be changed, and let the user decide what they want scrubbed.  Show a side by side text window where only the differences are shown and only stuff that the user finds objectionable would be changed. Repeat instances would only have to be shown once, so the list shouldn't be too long.

[BRiT]

Maybe dont replace non-alphabetic characters in share names or filenames?

[John_M]

I still don't see the need to obscure share names. What are people afraid of? They tend to be called Media, Music, Movies, Backups, etc. Ok, so maybe my collection of Barry Manilow MP3s is mildly embarrassing, but I'll get over it! Are some people storing even more incriminating stuff than that? Perhaps the default should be for Anonymize diagnostics to be unchecked?