[SOLVED] Bit Rot Mitigation - Should I care?

[jeffreywhunter]

Does anyone have a strategy to combat Bit Rot?  Especially for libraries that hang around forever...i.e. Movies, Pictures, etc.  Or is it something that's not worth worrying about?

[JonathanM]

Checksum verification (Dynamix File Integrity Plugin is one way, Corz is another) and backups from which to restore a verified clean copy.

 

In general, bitrot is pretty low on the scale of what causes you to need to restore from a backup. It's rare. Very rare. But backups are needed anyway, so it's nice to keep checksums around to verify content.

[garycase]

Worth worrying about ??    Certainly not worth losing sleep over.

 

Simple fact is if a bit changes you're going to know it on the next parity check ... although you won't know WHICH bit changed.

 

So it's a VERY good idea to maintain checksums.

 

e.g. when I add a movie to my server I (a) create a checksum for the movie (using Corz);  then copy the movie folder to the server and to my backup.  So at any given time I can validate a movie with a simple right-click, "Verify checksums"; and if I find an error I can repeat this check on my backup (just to be CERTAIN it's good) ... and then just replace the corrupted movie with a good copy from the backups.

 

[Frank1940]

The reason why bit rot is not a major problem is that all of the drive manufacturers know that it occurs and they have error detecting and correcting algorithms that are suppose to check and correct it.  If you look at the raw data read error rates, you would quickly realize that large hard drives are not practical EXCEPT for these algorithms...

 

Are there conditions when they don't catch and correct an error?  Absolutely, but since we don't have access to exactly what scheme is being used, it is impossible for us to know what conditions will allow an error to slip through.  These failures are probably what most people call 'bit rot'.  Obviously, if they detect but can't correct an error, the drive will report a read failure.  (The manufacturer hopes that latter is the default and not the former for his algorithm!)

 

As an aside, I can remember being exposed to extended error detection and correction schemes in Grad School almost fifty years ago.  I never used it after that brief exposure and the state of the art is probably many times more advanced than it was back then...

[jeffreywhunter]

Checksum verification (Dynamix File Integrity Plugin is one way, Corz is another) and backups from which to restore a verified clean copy.

 

In general, bitrot is pretty low on the scale of what causes you to need to restore from a backup. It's rare. Very rare. But backups are needed anyway, so it's nice to keep checksums around to verify content.

 

Love the concept of the file integrity plugin from Dynamix.  I installed it and its running, but there is a whole section called "disk verification tasks" with rows and columns of check boxes, but no instructions on what should be done, how to use it or how to define a 'task'?  Is there documentation or a sub-forum somewhere?

[JorgeB]

Checksum verification (Dynamix File Integrity Plugin is one way, Corz is another) and backups from which to restore a verified clean copy.

 

In general, bitrot is pretty low on the scale of what causes you to need to restore from a backup. It's rare. Very rare. But backups are needed anyway, so it's nice to keep checksums around to verify content.

 

Love the concept of the file integrity plugin from Dynamix.  I installed it and its running, but there is a whole section called "disk verification tasks" with rows and columns of check boxes, but no instructions on what should be done, how to use it or how to define a 'task'?  Is there documentation or a sub-forum somewhere?

 

Turn the HELP on.

[John_M]

Checksum verification (Dynamix File Integrity Plugin is one way, Corz is another) and backups from which to restore a verified clean copy.

 

In general, bitrot is pretty low on the scale of what causes you to need to restore from a backup. It's rare. Very rare. But backups are needed anyway, so it's nice to keep checksums around to verify content.

 

Love the concept of the file integrity plugin from Dynamix.  I installed it and its running, but there is a whole section called "disk verification tasks" with rows and columns of check boxes, but no instructions on what should be done, how to use it or how to define a 'task'?  Is there documentation or a sub-forum somewhere?

 

I found the plugin extremely confusing until I realised it combines two separate functions, namely (1) automatic checksum generation as files are written to the array, with the checksums being stored in the files' extended attributes, combined with a scheduled check to verify the files agains these checksums; and (2) the ability to export those checksums as a separate file for each disk, for use with external verification applications.

 

It became less confusing when the two functions were split into two separate places - the plugin now has an icon under Settings for the first, automatic function, and another icon under Tools for the second, manual function. I use the former but currently not the latter. The automatic function just works, now that I have excluded the Macintosh-specific files. The manual function was a pain because it was always complaining that it needed manual intervention to keep the exported files up to date, so I stopped using that part.

 

There is a sub-forum here: http://lime-technology.com/forum/index.php?topic=36543.0

 

[BobPhoenix]

The reason I don't worry about bit rot is I have backups of important files.  The others are media files and it is unlikely I would notice a single bit change when playing it back.  At most it might give a tiny glitch and my recorded media has more glitches than would EVER be caused by bit rot when they are brand new.  If I start with pristine media files like DVD or BluRay rips where errors are even less likely - I can just re-rip the media again if it would become unplayable or a glitch in playback becomes noticeable in them.

[Sn3akyP3t3]

On 7/6/2016 at 10:40 AM, JorgeB said:

 

Turn the HELP on.

I saw no response to this and just wanted to point out many years later that there is no Help provided for the Verification tasks section.