[TriLUG] Recommendations for a systemd-less Linux distribution

[Steve Litt via TriLUG]

On Mon, 13 Jul 2015 13:27:37 -0400
Paul Boyle via TriLUG <trilug at trilug.org> wrote:

> Hi,
> 
> The short version:  
> I am looking for recommendations (based on your experience) for
> systemd-less Linux distributions. Ideally, it would be an rpm based
> distro.  

Paul, I'll tell you right now as somebody who has studied this
situation a great deal. People with very deep pockets have been very
successful making sans-systemd difficult. I'd recommend that if you
want to go sans-systemd, you make it a priority.

And when people attack you for your stance, as they surely will, ignore
them.

> However, if no other options are available which fit these
> two criteria, I would consider another non-rpm based distro. The
> important thing is that systemd and all of its infections of other
> software must not be present. 

Complete systemd eradication is *very* difficult. I just installed
Gentoo on a Qemu VM this weekend, and even though it inits with
sysvinit->OpenRC, the smell of systemd is all over it, including this
FreeDesktop ridiculousity:

http://www.freedesktop.org/wiki/Software/systemd/PredictableNetworkInterfaceNames/

In fact, Gentoo has not stated catagorically that it will not migrate
to systemd later on. Funtoo has, but I've had some problems making
a bootable Funtoo.

The Devuan project is making a plug-compatible no-systemd udev
substitute, called vdev. This will go a looooong way to giving you the
kind of purity you express in your previous paragraph. Devuan isn't
stable yet, but it will be worth waiting for: No-system from the ground
up.

Manjaro, OpenRC edition, is one I'd recommend. It has udev, but it's
pretty easy to run udev the same way as it ran pre-systemd. I'm very
impressed with Manjaro.


> I am familiar with the
> http://without-systemd.org website. I've read through some of Steve
> Litt's writings on his efforts in getting a systemd free system.  I
> don't know if I could do the same with opensuse (my current
> distribution).

I wouldn't try. OpenSuSE's priority, like Ubuntu and Redhat, is "let's
make it easy for the guy who's never used Linux before, even if we need
to replace all Linuxisms with our own cluttered layer of stuff." Or at
least that was what I thought for the 4 months I used it on a laptop.
Don't get me wrong, I liked OpenSuSE, but I doubt they'll make it easy
for you to alt-init.

> 
> So far, I am considering either VectorLinux or PCLinuxOS.  The only
> problem is that they don't appear to be rpm based distros.

If that's the only problem, use them. Believe me, when you prioritize
going systemd-free, you'll need to do a lot of compromise on some of
your other priorities. I find it worth it: There's almost nothing on my
computer I can't go in and fix, and I don't just mean using systemctl
and journalctl.

> 
> In any case, thanks in advance for any guidance or suggestions.
> 
> Paul
> 
> 
> #######
> Longer version: (a little bit of a rant, I guess -- N.B. I know that
> the Linux community has fought the systemd wars, I'm not looking to
> restart one, but I feel the need to vent, given my frustration level)
> 
> I'm not an IT professional like most of the people one this list, but
> I've been using Linux for scientific computing since around 1992. I've
> also used it for my personal computing since that time as well. I went
> to Linux because I wanted a low cost computing environment which was
> more stable than and more capable than DOS and Windows. For most of
> the last 23 years I've been pretty happy.  Linux provided a faster
> (less bloated), more stable environment for scientific computing. Up
> until recently, I've gotten this from Linux distros I've used (SLS,
> slackware, redhat, suse, opensuse).
> 
> I've noticed since opensuse started using systemd as its default boot
> system, my workstations have gotten less stable, more sluggish, if not
> downright unresponsive in certain situations and needed a reboot. In
> the days before systemd, I've had instrument control Linux boxes up
> running continuously for over 300 days.  As far as I can tell,
> systemd  (or maybe systemd in conjunction with recent versions of
> KDE)

KDE is an absolute pig. I banned all KDE libraries from my boxes years
ago. It has much of the same problem as systemd: monolithic
entanglement. Some people do monolithic entanglement better than
others, but eventually, we all pay the piper for entangled software.


>  has put an end to Linux as a stable computing platform. From
> what I've read, systemd just hasn't replaced init, but has infected
> more and more subsystems used in Linux.  

Yes. If systemd contented itself with simply being PID1 and supervising
processes, I'd never have bothered learning all the other inits. This
udev situation is an atrocity. 

> In fact, using opensuse's
> YaST tool, I looked at the packages which were dependent somehow on
> systemd.  There were well over a hundred. (under opensuse, apache is
> dependent on systemd, WTF?) 

Systemd starts daemons quicker and more reliably if the daemons report
to systemd that they're now ready to work. Hey, that's a great idea. But
it didn't have to be done by forcing "systemd compliant" (I'm sure
you'll be hearing that phrase within the next year or two) daemons
compile in systemd libraries. There are much simpler mechanisms:
Mechanisms that would have worked with *all* the init systems. Makes
you wonder about Red Hat/FreeDesktop's motivations.

> It has gotten to the point where the
> level of instability is no longer acceptable to me, and I need to
> explore alternatives.

It's not absolutely certain systemd is causing these instabilities.
What *is* certain is that systemd takes over so much of your computer
that you're troubleshooting in a huge black box, using only the tools
systemd gives you (systemctl, journalctl, etc). Well, sometimes you
need to swap, and they've made it harder and harder to swap. For
instance, as a first step in alt-initting Lubuntu 15.04, I put
"init=/bin/bash" on the kernel line. Up until now, this has been pretty
much guaranteed to bring up some sort of system, with bash and all the
programs in /bin. With Ubuntu, it brought me up in the middle of
initramfs, without the ability to read ext4 or do a switch_root.
Whaaaaat? This is the true problem with systemd.

> 
> From what I've read systemd's initial virtue seemed to be that the
> system booted faster.  

This is true, but keep in mind that typical systemd systems postpone a
lot of stuff until the GUI is run. But yeah, it's faster, there's
little doubt. I'm not sure what I gain with a 1 second VM boot instead
of a 7 second VM boot, but whatever it is, systemd gives it to me.

> I guess that's OK for people who are used to
> rebooting their machines a lot, but traditionally, that's not
> something I've had to do with Linux and it's not that is important to
> me.  I want snappy response times and stability. systemd systems, in
> my experience, do not deliver in this regard.
> 
> In my experience, systemd systems don't necessarily boot faster
> anyway. As an example, in my lab, there is a student data
> processing workstation which is both an NFS client and NFS server.
> After upgrading to opensuse's systemd based distribution the system
> never boots smoothly and I need to wait for some stuff to timeout
> before it get to a login prompt. Then I have to go in kill a bunch of
> hung 'mount' processes, start the nfs client and nfs server services
> (using systemctl) and then do a 'mount -a'.  

I bet I could fix all of that for you, using systemd. Trouble is, that
kind of work is so complicated and so black-boxy, that you'd need to
pay big bucks for it. Today's average admin can't do it well.

> As far as I can tell the
> hang comes that systemd starts crond before nfs. In my configuration,
> the nfs client services need to be started first. 

That's actually pretty easy to change in systemd. You just mess with
the "Run_after" or whatever they're called.

> I've tried to
> figure this out some, but the relevant systemd configuration files
> are hard to find (whatever happened to putting config files
> under /etc?). 

See this presentation to learn what you need to know
to fix these kinds of problems in systemd:

https://rhsummit.files.wordpress.com/2014/04/summit_demystifying_systemd1.pdf

> I've gotten to the point, where even if
> I could figure out the problem, I'm not sure I want to. 

NOW you're talking my language. Yes, if I wanted to, I'm sure I could
get systemd to walk and talk. I could probably make some pretty good
coin doing so, and might still do it. But I **NEVER** want that
entangled albatross on *my* machine. I learned very early the cost of
complexity, and want none of it. Hey, U want to see how simple init
could really be without Red Hat/FreeDesktop's constant further
entanglements and interface changes, look at this:

http://troubleshooters.com/linux/diy/suckless_init_on_plop.htm

Yes, it really *could* be *that* simple, after you back out all the
systemdisms that have been welded into every part of your OS. An 83
line (or 16 line if you don't need to respond to signals) PID1 passing
off to a very simple process supervisor: That's all there is. Yeah, it
takes 8 or ten seconds to boot a VM because it runs daemons
sequentially, so it's not for every use case, but it's sure good for
the majority, as you mentioned earlier. It's not like we're all running
superservers.

>  I don't want
> to do anything that would encourage more systemd use. I don't want to
> silently accept the systemd crap which is getting shoved down my
> throat now by major distro providers. Systemd seems to be a project
> designed to Microsoft-ize Linux. This not a good thing.
> 
> After telling graduate students and faculty about how rock stable
> Linux is, it has been a little bit of a personal and professional
> embarrassment to have these boot time hangs, as well as sluggish or
> crashing software, and sluggish desktops.  I went to Linux to get away
> from this behaviour.
> 
> If I could, I would probably migrate my lab to some *BSD flavour.
> However, the vendor who supplies the (expensive) scientific
> instruments in my lab has done a Linux port.  I would rather use
> their Linux port than their Windows version.

I have a very good impression of PC-BSD. And I'm pretty sure PC-BSD
offers qemu. Perhaps you could use PC-BSD, and just for your scientific
instruments, use Linux VMs.

Anyway, if you're willing to accept some moderate inconveniences you
would have considered untenable before systemd, you have many, many
choices.

SteveT

Steve Litt 
July 2015 featured book: Rapid Learning for the 21st Century
http://www.troubleshooters.com/rl21