Systemd, software design and the future of Linux

I’m a conservative person. I usually know what I like and what I don’t like. I was triggered to write about Systemd by this thread:

So here are my thoughts about Systemd, software design and the future.

Systemd vs more traditional init systems

I have a feeling that Systemd is equally liked and disliked. It makes it easier for sysadmins if Systemd is everywhere. For home users I’m skeptical. It’s the Windows mentality that moved to Linux (do all things and do nothing well). :slight_smile: Worst of all is that it isn’t a drop-in init replacement, but a very aggressive piece of software that completely overtakes the distro.

I can understand Ubuntu’s choice. They want maximum compatibility with Debian and Gnome/Red Hat. If nothing else it’s cheaper that way.

From a home user perspective I notice the following: With System V or Upstart a distro boots without “race conditions” and shuts down in a very predicable manner.

On my low-end 1,6 GHz dual-core machine with HDD shut-down with Upstart takes 5-10 seconds. It’s five lines that are written on the screen with OK at the end of each line. The shut-down sequence is always the same and always takes the same amount of time. There is no waiting in the shut down process. The computer is actively shut down during those seconds.

Software design

System V and Upstart are very boring because they are predicable and that’s exactly how it should be. Maybe Systemd developers will fix “the bugs” in Systemd, but I wouldn’t hold my breath. I think it’s a design problem. Systemd is not only designed for physical hardware, but virtual machines, IoT and God knows what. Just like Pulseaudio suffers from latency because it was partially designed with low power consumption in mind (smartphones).

I think developers should design whatever system they feel like. It doesn’t have to follow the Unix-way. Problem starts when your project is actively killing other projects. People wouldn’t dislike Systemd if they didn’t have to use it. I think those who like Systemd don’t realize how much power now lies in very few hands. Only Systemd developers can fix the Systemd “shut down bug”. I don’t think Canonical developers will patch Systemd like they patched GTK 3.

Because Systemd is so pervasive, I still think we underestimate the number of bugs that are directly or indirectly tied to Systemd. Pulseaudio has become better over the years. Hopefully Systemd follows the same path.

System initialization is just one of the tasks Systemd is responsible for. How much correlation is there between Systemd and Network Manager? I expect quite a lot. The problem for Systemd developers is that they created something that can be blamed for many things just because it’s everywhere. It’s not confined to a specific task or place in the system. It’s Windows design plain and simple.

I think Unix-design: “Do one thing and do it well” will serve Linux better in the long run. Developers are only human and if your garden is small it’s easier to keep it tidy.

The future

I would like to fully embrace Ubuntu MATE, because Ubuntu has the most packages, high quality packages and Wimpy and lah7 are doing an excellent job with Ubuntu MATE.

However, Systemd stands in the way. It just rubs me the wrong way. I’m not saying I never will use it. Everything in life is a trade-off. I used to feel the same way about Pulseaudio. Now I use Pulseaudio.

One thing I don’t think I will use is Wayland, but who knows? If the alternative is Windows 10 then pretty much anything is usable. I’m pretty pessimistic about the future of Linux. If Wayland replaces standalone X-server then many of the freedoms we take for granted are gone: GUI apps as root (Synaptic, Gparted), different window-managers and many theming possibilities will disappear. Will we be allowed to take screenshots?

My best possible future scenario: I get to keep as much of what I have today as possible. My window animations are smooth. Moving windows over the screen is a fluid experience. I’m not saying Wayland is a performance downgrade, because I’ve never tried it. From a freedom perspective the price seems Way too high. It’s a Land I don’t want to go to.

2 Likes

The only drop-in replacement that I am aware being used with some success is runit being used on Void. Another popular alternative to systemd is OpenRC, but that isn’t a drop in replacement neither. And on one thing all three agree: SysV init needed replacing.

I’m sorry, but that is just not true. You are falling to confirmation bias. You know, as well as anyone else that a shutdown sequence is dependent on numerous factors. What applications/daemons are still in memory, what is being cached, is for some reason an application or daemon going to fail to respond to the SIGTERM signal? Are operations still being performed when I request the shutdown, etc etc etc.

The better your init system is at managing the whole process dependency tree on your system, the more fluctuations you are going to see during normal shutdowns. Some will take just a couple of seconds, others may take close to a minute. And the fact SysV and Upstart show you such regularity during a shutdown are just one part of the reason everyone abandoned them.

Although I can assure you SysV is not that regular as you claim. And never was. Contrary to Upstart, it was a good and reliable init system that served the Linux community for many years. But it was at the end of its life. Linux had evolved sufficiently for SysV to no longer be able to serve it.

Parallel execution of services is not a bug. It is a much needed feature in modern linux computing, and the reason Upstart, systemd, runit or OpenRC where created.

But what you don’t seem to realize is that systemd (like all the others) does not run by itself. Any racing conditions, if they exist, are entirely the responsibility of the person that created the booting rules. Systemd has very powerful dependency definition capabilities and a much better and evolved Targets feature to replace the archaic SysV runlevels. An administrator needs to learn to make use of these (and they do!) in order to create a stable boot process that doesn’t run into amateurish race conditions because they paired the wrong services to launch at the same time.

Let me state this again: any racing conditions, as you call it, are entirely the fault of the administrator.

There is no shutdown bug. Can you now understand why?
The entire behavior of shutdown is being defined both in configuration files and on the unit files that configure each and every service, socket, target, path, etc on the system. And these were written either by distro maintainers or system administrators.

I’ll be blunt here and I hope you apologize me for it. I won’t be patting you on the back, but instead slap the back of your head and say that an even better way to serve Linux is to learn about the things we don’t like, not just the things we like. At least you’ll know what you are talking about. I suggest you start here: https://n0where.net/understanding-systemd/

So you are essentially saying that systemd is badly configured in Ubuntu?

No problem. You need to understand that I’m criticizing systemd from a user perspective. If distro maintainers are unable to tweak “race conditions” and “shut down bugs” then the user is left with a “malfunctioning” system that is unable to perform on the same level as a traditional init.

It doesn’t matter that systemd is technically superior if the user experience sucks and users choose something that “works” instead. Maybe systemd is awesome if you are a technical wizard, but most users aren’t.

3 Likes

Am I? Read the Shutdown thread more slowly. What do you see?

In any case, that thread suggests that the default settings are simply not working for two users. Something is holding their shutdown process hostage and that situation eventually takes the whole 90 seconds of the timeout period. Everything in that thread implies that another thing (a service or daemon) is halting. Do you recognize a parallel here with countless similar problems with SysV init? Did you use to blame SysV back then because your system hanged during boot or shutdown, or instead did you recognize that the problem was elsewhere on some daemon or service that was not shutting down properly for one reason or another?

So, no. I am not saying that systemd is badly configured on Ubuntu.

Aside: I’ll be posting in there shortly though, since you called my attention to that thread which I hadn’t realize it existed. I however cannot reproduce their problem, despite being myself a big user of VMs for linux testing. Although I suspect why is that.

I don’t know if that is entirely fair Marfig.

To give you an example of a race condition I encounter regularly;

For example, Tilda is set to load at login. However it is very often the case that if a compositor is also running at login, Tilda will not pick up the transparency effects unless it is prefixed by a sleep command.

This is just one race condition. There are many others, often quite unpredictable until they happen. At which point, they can be adjusted with sleep prefixes.

Now, don’t get me wrong. All of the above is a tiny thing and it is of no consequence to me to have to make these adjustments. So, I am not actually complaining here at all. What I am saying, however, is that I am no administrator and my technical knowledge is limited. But, then, that probably makes me pretty typical of the vast majority of Linux users.

In other words, the vast majority of Linux users are never going to know anything about “very powerful dependency definition capabilities and a much better and evolved Targets feature to replace the archaic SysV runlevels”.

Again, though, I don’t wish to give the impression that I see all of the above as a negative. I am quite happy with having to make the adjustments - often by stumbling around in the dark until I can get things to work. I do understand that the only way to make such issues go away out of the box for ignorant numptys like me is to have a closed down system like MS Windows where there is much less room for personalization of a system. So, in the end, I am more than happy to have a wide open adjustable system like Linux.

A lot of users are less curious or prepared to get their hands dirty like me, though, and just want to be able to install programs and have their system boot up and work. For them, unpredictable race conditions at login and unpredictable, prolonged periods of hanging around at logoff may not be tolerable.

I don’t know what the answer is for them.

But, I will end with this positive note:

I have been on Linux since 2006. In that time, despite all of the bumps in the road, the usability and reliability of Linux has increased dramatically. Back in 2006 it took a lot of determination to get a good Linux system up and running. These days that has all changed and it is usually possible to install and everything just works out of the box.

2 Likes

Naturally, they won’t. But you didn’t quite answer my argument, either. Consider the following:

The Tilda behavior you mentioned, is largely outside the scope of the init system. You will experience it regardless of what system you use, including a purely sequential system like SysV. You will always have to include a sleep statement to ensure it works as expected.

Now, Tilda doesn’t fit well in the service/daemon concept of an init system. It’s something I would rather have started from a post-boot startup applications system as that which is offered by the MATE desktop or from within a startup script being called from .bashrc. But for argument sake, let us generalize Tilda to any other service that had a similar problem.

This isn’t your problem as an user. If I decide to include the Tilda package on my distro, independently of whether this being a default package that ships with the distro, or something the user can download later, I am required to configure it so you, the user, doesn’t experience problems running it. So, on the case of systemd, I would include on the Tilda package a tilda.service file properly configured to account for the delay required (and be it SysV init, runit, Upstart, systemd, runit, or Openrc, I have facilities just for that).

So, systemd – any type of init system for that matter – doesn’t impose itself on the user. And this is why it is the responsibility of the distro maintainer to make sure things work as expected.

But if instead the tilda package is not meant to be configured by the init system (which is the case), then my responsibilities as a distro maintainer are partially lifted. But then, we are no longer speaking of systemd and so the point becomes moot.

You see what I mean, when I say any so-called race conditions on an init system are entirely the responsibility of the distro maintainer?

The good thing though is that problems rarely express themselves to users at the low level of the init system. Instead you’ll find that problems at that level are opened as bugs by the distro developers/maintainers and they know it is something THEY need to fix.

So, there is really no argument to be made that systemd (or any other init system with or without parallel execution features) forces users into understanding its inner workings.

1 Like