[TriLUG] Using Btrfs

[Paul Boyle]

On Wed, 19 Nov 2014 14:18:33 -0500
Aaron Joyner <aaron at joyner.ws> wrote:

> A couple clarifying comments (or nitpicks, depending on your
> perspective)... which hopefully may help you clarify your question,
> or at least inform your thinking.

Thanks for your comments.  Responses below your text.

> 
> - You assert the workstations have "a single hard drive (about
> 500-750GB in size)", yet you express concerns about "the default RAID
> configuration". RAID <http://en.wikipedia.org/wiki/RAID> is short for
> Redundant Array of Inexpensive Disks, it does not apply (at least not
> in any reasonable fashion) when you talk about machines with single
> disks.

I guess another way to frame the question with regard to btrfs' RAID
capabilities would be:  Does it makes sense to use btrfs on single disk
systems?

This page:
http://www.howtoforge.com/a-beginners-guide-to-btrfs

mentions that mkfs.btrfs defaults to the "raid1" option for the
filesystem's metadata but not for the data.  Does this imply a
multi-disk system?  If not, how is the metadata mirrored (and where)?  



> - Snapshots in btrfs are "copy on write".  They start by taking
> approximately zero room, and then begin to consume additional space
> as you make more changes to disk.  They're typical use is to allow
> for flexible local "backups", which insure against modest user error
> (deleting an important file), 

This is exactly what I think would be most important about moving to
btrfs.  Sometimes students hose an important file and it would be really
nice to be able to "go back to before the accident".  I was thinking to
make /home a btrfs files system with each user having their own
subvolume.  I would leave the other paritions as ext4 partitions.

> but not against hardware failure (bad
> disk) or colossal administrator error (deleting the partition
> containing the filesystem). How much space your snapshots take up is
> a function of how far back in time you want to be able to go, and
> with what granularity (eg. you might take one snapshot every hour,
> and keep the last day's worth, the one from the previous Monday, and
> the first of last month).

I think it might take a while to get a feel as to what the space
tradeoff versus granularity for typical usage on my systems.  I think
different users would mostly likely have need/use different granularity
(e.g. the student who is the lab everyday versus the student who only
occasionally processes data).

> - You don't configure btrfs to store snapshots; you can choose to
> automate taking a snapshot with a script.  Thus, you control the
> frequency and number of snapshots by how often you run $(btrfs
> subvolume snapshot ...) and $(btrfs subvolume delete ...).

I understand that part on further reading.

Thanks for the tip on not using btrfs on USB external backup drives.
Makes sense.

Thanks,

Paul


> 
> Aaron S. Joyner
> 
> PS - More on snapshots: https://lwn.net/Articles/579009/ (see the
> section "Snapshots")
> PPS - Don't use btrfs on the external backup drives.  ;-)
> 
> My general advice on filesystem adoption is that people are, in
> general, much too averse to new adopting new filesystems.  This is
> because good backup and restore practices are rare, so silent
> background data corruption is a very scary thing, and it takes a lot
> of people using a filesystem for a long time to have confidence that
> you've sussed out all the edge cases where that can happen.  For
> shared workstations, which typically don't have a lot of data that
> couldn't be easily reproduced, and for which you have good external
> backups ("an external USB drive to store backups"), I wouldn't
> hesitate to try out btrfs.
> 
> 
> 
> On Wed, Nov 19, 2014 at 12:42 PM, Paul Boyle <pboyle at uwo.ca> wrote:
> 
> > Hi,
> >
> > I would like to know the advantages/disadvantages of using btrfs on
> > Linux machines which are primarily used as scientific workstations.
> > The workstations typically have staff or student user accounts which
> > are used for X-ray crystallographic calculations (i.e. floating
> > point intensive).  The workstations typically contain a single hard
> > drive (about 500-750GB in size) and an external USB drive to store
> > backups and almost all workstations can interchange data via NFS.
> >
> > I've been doing some reading on btrfs (mostly because the Linux
> > distro I use has made btrfs the default filesystem for its newly
> > released current version).  In addition, it sounds like the ext4
> > filesystem is probably the end of the line for the development of
> > ext series of filesystems.
> >
> > I can see a number of advantages of using btrfs which are outlined,
> > among other places, in this article:
> >
> >
> > https://www.linux.com/learn/tutorials/533112-weekend-project-get-to-know-btrfs
> >
> > I guess my main concern is with the default RAID configuration is
> > regarding how much of my disk gets dedicated to redundant data and
> > metadata?  Will I need to go and buy second hard drives for all of
> > my workstations?  I hope these questions don't sound too naive for
> > the IT professionals who inhabit this list.  My overall goal: I
> > would like to maximize usable disk space without risking losing
> > data (e.g. I am considering keeping the metadeta redundant, but not
> > the data).
> >
> > How much room do snapshots take up?
> >
> > Can the filesystem be configured to store a certain number (or
> > amount of data in) snapshots?  If so, how does one determine what
> > is a reasonable value?
> >
> > I would appreciate people sharing their experience of any
> > advantages or pitfalls they have experiences in using btrfs.  (In
> > particular, if you use btrfs on individual workstations).
> >
> > If btrfs does seem to be a viable option for me, I would consider
> > migrating my existing systems to btrfs rather than keeping my my
> > current multi-partition ext4 formatted hard drives.
> >
> > Thanks for any input, advice, guidance, etc.
> >
> > Paul
> >
> >
> > --
> > Paul D. Boyle, Ph. D.
> > Manager, X-ray Facility
> > Department of Chemistry
> > Western University
> > London, ON N6A 5B7
> > Canada
> > GPG Fingerprint: 8ECE 516D 9046 FE83 4A46  7E8E D720 555D 8CC3 EC6B
> > --
> > This message was sent to: Aaron S. Joyner <aaron at joyner.ws>
> > To unsubscribe, send a blank message to trilug-leave at trilug.org
> > from that address.
> > TriLUG mailing list : http://www.trilug.org/mailman/listinfo/trilug
> > Unsubscribe or edit options on the web  :
> > http://www.trilug.org/mailman/options/trilug/aaron%40joyner.ws
> > Welcome to TriLUG: http://trilug.org/welcome
> >



-- 
Paul D. Boyle, Ph. D.
Manager, X-ray Facility
Department of Chemistry
Western University
London, ON N6A 5B7
Canada
GPG Fingerprint: 8ECE 516D 9046 FE83 4A46  7E8E D720 555D 8CC3 EC6B