[TriLUG] Bit rot detection without ZFS/btrfs?

[Francois Dion]

You'd have to write a tool to scrub your FS in the background, calculate a
checksum and compare it against a checksum that is on a different area of
the disk. That still doesn't give you end to end in flight data integrity,
but that would be a good start.

Francois


On Mon, Jul 1, 2013 at 2:41 PM, Steve Litt <slitt at troubleshooters.com>wrote:

> On Sun, 30 Jun 2013 20:42:22 -0400
> "Randy Barlow" <randy at electronsweatshop.com> wrote:
>
> > Hello fellow Linux people,
> >
> > I've been wearing my tin foil hat a little bit too much lately, and
> > I've started to become worried about all the files sitting on my
> > various hard drives, bit rotting away.
> >
> > I am aware that ZFS and btrfs are designed to help with this problem,
> > but ZFS has its licensing issues, and btrfs isn't yet "blessed" by
> > the Linux elders.
> >
> > Do any of you use anything to detect file corruption on your disks?
> > I'm mostly interested in detection at this point, as I think I can
> > pretty well use my backups or off site backups to recover, but I need
> > something to tell me when I need to do that.
> >
> > I've considered writing something homegrown to do this, as it's not
> > terribly complicated. I could store a checksum on the FS extended
> > attributes, or maybe just in a database of some kind.
> >
> > If anyone knows of a distro package that can do this, I'd love to
> > hear about it. There are some interesting challenges to get past
> > (lots of opportunities for false positives, for example when
> > checksumming a file that is currently being written to by another
> > process.)
>
> I wonder if there's a way to do this with rsync. You can make rsync
> write to a log. On the destination machine, if a file changed but its
> date didn't, that's pretty good evidence of bit rot.
>
> This has the added benefit of acting as a backup server, and if you use
> cp -al each time, you keep older versions so you can restore
> non-bit-rotted files after changing out your hard disk.
>
> Thanks,
>
> SteveT
>
> Steve Litt                *  http://www.troubleshooters.com/
> Troubleshooting Training  *  Human Performance
> --
> This message was sent to: Francois Dion <francois.dion at gmail.com>
> To unsubscribe, send a blank message to trilug-leave at trilug.org from that
> address.
> TriLUG mailing list : http://www.trilug.org/mailman/listinfo/trilug
> Unsubscribe or edit options on the web  :
> http://www.trilug.org/mailman/options/trilug/francois.dion%40gmail.com
> Welcome to TriLUG: http://trilug.org/welcome
>