I was pro-systemd at the time of the controversial discussions. I still think it's a net positive relative to what was there before. But personally speaking, it's only the core of the software (service management) that improved things for me. The other things (timers, journald) I either ignore or don't like, but perhaps they're useful for distributions.
I've been quite happy with -boot, -networkd, -resolved. -logind too, what a powerful upshift in routing input.
I forget what but timesync is missing a lot of nice to haves. But ditching grub for something much much much more manageable & a joy to use. Networkd and resolved have both had a lot of nice clear straightforward options, and I love how systemd unit files snap together for these networking concerns.
Journald has been ok for me, and I use it a lot, but not intensively. Having adequate performance is definitely a huge concern.
I run, at home and at work, vpns and having a good way to configure split-horizon dns is incredibly useful.
At a previous company we were able to ship a binary that would automatically configure a bunch of company settings on developers' linux laptops and got VPN configurations working with split-horizon dns very easily.
prior to this, doing split-horizon dns on linux basically meant running dnsmasq (!!!) on every linux laptop
Yes, systemd's weirdly bundled system services largely suck. The ones that I'm sure about: journald is more annoying to use than syslog (it takes forever - like a minute on a top of the line CPU - to scan two months worth of logs) and resolvd has or used to have basic bugs that other systems don't have.
I've worked as a sysadmin/devops for a double digit number of years.
Across companies, jobs, and hobbyist collaborators - I've never met someone who didn't at least like systemd, if not sing its praises. It made so many incredibly painful things about Linux administration disappear.
E.g. logging, now that there is journald, you have to pay attention to another hop in your logging chain and take care of journald in addition to rsyslogd.
Boot and shutdown has previously been deterministic, now things just randomly hang. You do the "windows solution", reboot again, now it magically works through the power of race conditions that are inherent in systemd's mode of operations.
Network device naming is also a fun one. Remember how systemd was supposed to do away with the inconsistency and mess that is "eth0, eth1, eth2, eth..."? Well, now you get either eth0, or ens123, or enp17s7f9, or enx8b220b34. Each distro does it differently, so put in a live-CD to debug something and your network devices are suddenly weird. Reinstall to another version, suddenly weird again. Btw, the issue that names could change depending on boot order was already fixed by udev before, consistent-net.rules nailed it to a mac address on first boot. So systemd took a situation that was already fixed and better than before, and made it far far worse.
Oh, and all the fun that is systemd-dbus, policykit, systemd-logind and permissions on those. Because you always wanted a Javascript interpreter and libxml to tell who could be root on your system... And tons of confused deputy exploits on that cloud of confused systemd-subdaemons.
I'll grant that service files were a nice idea. But beyond that, it has been a very mixed bag, and in places a desaster.
The grand-parent post was about the general consensus: sysadmins generally like systemd. So I'm not sure answering individual concerns is useful, but I'll take the bait...
> now that there is journald, you have to pay attention to another hop in your logging chain and take care of journald in addition to rsyslogd.
No, you don't have to, because a syslog daemon is not required any more. BTW, rsyslogd is not the only syslog service, I've had to work with alternatives like rsyslog-ng.
And the daily usage is now much simpler: `journalctl -b -1 -p err` or `journalctl -u mariadb --since "24 hour ago"` were painful queries with (rotated and compressed) syslog files.
> Boot and shutdown has previously been deterministic, now things just randomly hang.
That was not my experience. IIRC, with sysv-init boot dependencies were declared with comments inside shell scripts and parsed by insserv. That was a mess, and race conditions did occur. I have no experience with other boot systems, but I remember that upstart was not immune either.
> Network device naming is also a fun one.
The naming policy is vaguely relevant to systemd. `net.ifnames=1` is the default configuration of the Linux kernel. I think it's still possible to configure the kernel, udev and systemd to work with the old naming, but if it's just because "shorter is better", I don't think it's worth it.
> No, you don't have to, because a syslog daemon is not required any more.
That is "not required" for the single-computer-sysadmin. If all you know is your laptop, then journald might be fine. If your computers are cattle and have numbers, you don't want to read your logs on each individual computer, if possible you don't want to read logs at all, you want some automation to do it for you, aggregate it, warn you about things, etc. journald just makes that harder, and going through some syslog protocol implementation (of which there are a few, you are right there) is the only relevant game in town.
> IIRC, with sysv-init boot dependencies were declared with comments inside shell scripts and parsed by insserv. That was a mess, and race conditions did occur. I have no experience with other boot systems, but I remember that upstart was not immune either.
Dependencies and insserv were a later addition. Basically you just defined a boot order by numbering your /etc/rc[0-6].d/S* links in ascending order, either manually or in a dependency-based fashion through insserv. This means that you can have a suboptimal ordering if those dependencies are wrong or you choose your ascending order poorly. But it also means that once you have a working boot order, it can be repeated, and it will be repeated. That is what I mean by determinism. Once you have a working boot, it will keep working. Solaris SMF did something more modern, you would define dependencies in service definitions, and it would compute a static deterministic ordering from that, without those /etc/rc[0-6].d/S* symlinks. Deterministic, but dependency-based. If it has worked once, it will work again. And they even had a rollback: if after a change, the next boot wouldn't work, you could go back to the previous boot order and fix things from there. And neither had race conditions, because you usually wouldn't do things in parallel, one service would wait for the previous one in the boot order (parallel was possible but rarely used, because of the inherent fragility and possible race conditions).
Systemd does it dynamically on every boot, no precomputation, just looking at all the files and starting everything that could possibly be started at the same time, in parallel. The reason for this is that this makes Lennarts laptop boot faster, which is true in the ideal case. But it is neither deterministic, nor debuggable. And other than the previous methods, it can lead to race conditions.
Note that there is one more race condition at boot, which is races against kernel devices and systemd-udevd. That neither got worse nor better with systemd, it just lead to the kernel people giving up and handling more things like firmware loading in kernel.
> The naming policy is vaguely relevant to systemd. `net.ifnames=1` is the default configuration of the Linux kernel.
The Linux kernel does not have a net.ifnames parameter, and the kernel interface naming scheme is still eth[0-9]+. When you stick that parameter in the kernel commandline, the kernel ignores it and only systemd and systemd-udevd interpret it later on.
Please, if you want to sing the praises of software or deflect blame from broken crap, make sure that you know what you are talking about...
It's a little weird to reply to someone complaining about administrative problems caused by systemd with "generally sysadmins seem to like it."
That kind of response is exactly why everyone gets emotional in these discussions. Technical problems are one thing but when you gaslight people experiencing and reporting legitimate problems you should not be surprised when they get mad and rip your solution out.
GP is being far kinder to the systemd devs than they deserve, I gave up on the project years ago for exactly this reason.
> consistent-net.rules nailed it to a mac address on first boot
fails even harder for
> put in a live-CD to debug something and your network devices are suddenly weird
I don't know that there is really any good answer here. Just methods that break in ways you're used to, and methods that break in ways you aren't. I appreciate the attempt at static naming based on hardware connection, even if the realities of how these things are presented by the bios make the results an inscrutable mess.
Maybe pure enx would have been better? But that looks like crap for even the simple cases, and fails if you swap out a piece of hardware or start doing weird things with MAC addresses.
The only consistent naming scheme I've ever seen working was Solaris on Sun hardware. But that only worked because they controlled both the hardware and the software side of things.
And yes, consistent-net.rules doesn't solve LiveCDs and reinstallations. But as I've said, systemd doesn't solve those either, and in most cases actively makes them worse.
My experience has not been that the systemd device names are "actively worse". The biggest problem I've had is changing around VMs where subtle shifts in PCIe IDs (specifically subdevices) end up changing changing names. Meanwhile, consistent-net.rules seems like a hack, always in the back of your mind as an arbitrary stateful part of the system. Why should I think of my primary network device as eth2 if there is no reason for it to be eth2 other than merely being the order on the first install?
But if you prefer consistent-net.rules, then more power to you. I'm guessing most of your frustrations could have been addressed by systemd having some shame about changing functionality out from under users, and providing an up-front option to switch between the different methods.
I used to be fine with it, until I started writing my own software tools, then looking at how to actually do stuff with systemd and I'm like, nope nope nope nope. At some point when I have space for a new project I'm going to migrate from Arch to Obarun, which uses a modern update on the excellent daemontools to provide init. When systemd works it's mostly fine, when it breaks, path of least resistance is often to reinstall the OS. Which is fine if you're just working on it. Painful if it's your personal machine, and those are what's most likely to break systemd.
> when it breaks, path of least resistance is often to reinstall the OS
Can you elaborate how do you even break this?
Adding a systemd unit cannot really break much on systemd itself. Unless you’re messing with internal systemd stuff, I am not sure what is even there to break.
At the moment, on this machine I'm using, audio just doesn't work. Dunno why. Tried a few different things to fix it, no dice. Trying to troubleshoot is made enormously more difficult by the complexity of systemd. If I had a simpler system I could run it all the way back to the source, but really, I can't be bothered, I don't really need audio on this machine atm.
i have the same sentiment, and for me systemd was getting so slow and hard to tshoot (the tools to analyze boot graphs and sequence showed service sequence breakage, some times there is no problem, sometimes sequence changed and problem happened) and i just changed for another distro without systemd and boom, startup in seconds no problem at all.
This doesn't match my experience at all. For the life of me, I cannot imagine what you are talking about, what any of your issues are. Did you file any tickets, or report any issues?
What about "actually doing stuff with systems" was "nope nope nope nope"? What situations did systemd break in? What did you try before giving up to reinstall the OS?
You probably use a desktop environment like GNOME or KDE. Maybe do some light coding in an IDE. I do not. I run window manager only, i3, with a really stripped down toolset. I like the idea of knowing my system in and out. My current project is writing my own shell. It can switch between command mode and ruby REPL with ctrl-z.
You mention filing tickets / reporting issues, sorry, that means contributing to a project. I'm not wholly opposed to doing that, but I have to like the project first. And there's nothing to like about systemd.
You go do that if you like it so much. My guess is you simply won't, and will just keep using other people's software without a care in the world, thinking this kind of online warriorship is actually helping anything.
Bro, I've been running wmii since before i3 forked off from it. Don't come at me with this dripping disdainful attitude, pretending like you are some high and mighty elite system whisperer.
It's so unfortunate that there is such a widespread persistence of hater-dom. It's so toxic and awful, and so rarely does it have even the faintest shred of anything to stand up on. I struggle so much to see these attractors into the negative as so compelling, so ensnaring of the human souls as they have been lately.
Systemd is amazing, provides such a rich world versus piecemeal poorly managed systems of yore. Yes you could understand init scripts with very little effort but what a sad virtue, to dare to go no further than what an afternoon of exploration might get you.
I don't fucking hate systemd. I'm marginally glad it exists. I just don't want to live under it's despotic regime. You might like it because you think trains running on time is more important than freedom.
You're the one coming at me with fucking attitude, like a 16 year old who just discovered IntelliJ. I have done the exploration, then looked online for why everything is so damn complicated now. It's because systemd colonized the world.
It's not just an init system, it's everything below the userland. And a lot of those components are very poorly designed. And it's this big ball of mush where if you use one component, thy must use them all. It's so bad Arch does not support replacing systemd, like at all. You're totally on your own.
Go ahead, try replacing systemd components in an Arch system. I'll wait. And lose the fucking attitude.
I don't hate journald because it's not plaintext, I hate it because it's worse than plaintext.
Somehow journald manages to provide a database which is 40x slower to query than running grep on a compressed text file.
I'm all in favour of storing logs in an indexed structured format but journald ain't it.
Journald is an odd one. I don't think it being a binary log/database makes sense. If you have a tiny operation, with a single server, then the binary database doesn't really make sense, having plain text is just easier and faster. If you're a bigger operation, you'll have a central logging solution, in which case you need journald to store the longs as plain text as well, before you can do log shipping.
The only use case where the binary format might make sense is if you ship journald logs to another central journald instance. That's just very much an edge case.
Doesn't that still involve a conversion? I believe that rsyslog can read the journald database, but you're typically not querying syslog data directly, so there's a conversion between rsyslog and logstash, Splunk, Datalog, whatever.
Same, that was my first experience as well. It was a breath of fresh air in many ways, and after my initial learning of the new syntax, I quickly became a convert.
That said, I have had some concerns over the years about the growing scope of systemd. I think history has shown my concerns were overblown, though I think it is definitely possible and even likely that people were affected by those arguments and maybe moderated the ambitions a little bit. That could be for better or worse, I don't know, but I do think it was a factor
I think the biggest problem with systemd is that they named the main init service manager systemd and also named the organization systemd! It'd be like if GNU named their initrd "GNU". Their other tools just have systemd prepended, e.g. systemd-journald, much like GNU Make or GNU libc (glibc) or similar. The "scope creep" issue doesn't get complained about for GNU because it's clear they're an organization making a bunch of tools for different purposes. Systemd-the-organization is the same, but the naming makes it unclear that many of these tools are not depended on by systemd-the-init-system.
Does anyone actually use journald? The last time I tried(2 years ago?) it didn't even work with any log management software (like cloudwatch for example).
You had to either use some (often abandoned) third party tool or defeat the purpose by just reconfiguring everything to dump a text log to a file.
I use journald whenever I feel my blood pressure getting too low.
It's slow, truncates lines, and doesn't work well at all with less. It's almost like Pottering created it so that pulseaudio wouldn't be his worst program anymore.
I architected and built our entire log-ingestion pipeline for intrusion detection on it, at Square.
I built a small Ruby wrapper around the C API. Then I used that to slurp all the logs, periodically writing the current log ID to disk. Those logs went out onto a pubsub queue, where they were ingested into both BigQuery for long-term storage / querying, and into our alerting pipeline for real-time detection.
Thanks to journald, all the logs were structured and we were able to keep a bunch of trusted metadata like timestamp, PID, UID, the binary responsible, etc. (basically anything with an underscore prefix) separate from the log message all the way to BigQuery. No parsing, and you get free isolation of the trusted bits of metadata never intermingling with user-controlled attributes.
Compared to trying to durably tail a bunch of syslog files, or having a few SPOF syslog servers that everyone forwarded to, or implementing syslog plugins, this was basically the Promised Land for us. I think we went from idea to execution in maybe a month or two (I say “we” but really this was “me”) and rolled it out as a local daemon to the entire fleet of thousands. It has received—I think—one patch release in its six+ year lifetime, and still sits there quietly collecting everything to be shipped off-host.
The only issue we’ve ever really ran into that I never figured out is a handful of times per year (across a fleet of thousands) the journald database corrupted and you couldn’t resume collecting from the saved message ID. But we were also on an absolutely ancient version of RHEL, and I suspect anything newer probably fixed that bug. We just caught the error and restarted from an earlier timestamp. We built the whole thing around at-least-once delivery so having duplicates enter the pipeline didn’t really matter.
Damn, honestly at this point I’m wishing I’d pushed to open source it.
Ironically, actually, I did write a syslog server that also forwarded into this pipeline since we had network hardware we couldn’t install custom services onto but you could point them at syslog. I also wrote this in Ruby, using the new (at the time) Fibers (“real” concurrency) feature. The main thread fired up four background threads for listening (UDP, UDP/DTLS, TCP, TCP/TLS), and each of those would hand off clients to a dedicated per-connection worker thread for message parsing. Once parsed they went onto one more background thread for collecting and sending to PubSub. Even in Ruby it could handle gazillions of messages without breaking a sweat. Fun times.
Since I’m rambling, we also made cool use of zstd’s pre-trained dictionary feature. Log messages are small and very uniform so they were perfect for it. By pre-sharing a dictionary optimized for our specific data with both ends of the pubsub queue, we got something like 90%–95% compression rates. Given the many terabytes of logs we were schlepping from our datacenters to GCP, this was a pretty nice bit of savings.
Well, everyone and noone uses journald. The usual way to log is journald -> (r)syslogd -> remote log destination(s). I've never actually seen any other way.
The preferred journald way of "fetch your logs periodically" wouldn't pass any audit and doesn't work with any kind of log processing software that people use.
You're being downvoted but you're absolutely right. The fact that Cloudwatch doesn't support journald is a major, major fail on AWS' part. It's not like this is new or obscure software.
I wouldn’t call it a success it all. Almost managed to do was moving or forcing other software to get locked in. GDM, Podman, Pipewire…just to name a few.
I've been a fan of systemd even if just for how much simpler creating services is. I just need to write a simple config file instead of a complex init script.
+1 - there was a misconception that initscripts were easy or simple - if you wanted _correct, bug free_ initscript you were probably looking at around 100 lines of shell script.
Or if the service didn't support pam_limits because it was legacy trash, you had to hack something into the initscript like `ulimit -n XYZ` and restart it. Now things like this are trivial and easy to solve. Using systemd makes large scale Linux systems administration much easier.
Now it has gone a bit overboard. Some of the stuff like the dns resolver or the nspawn capability seem a bit over the top, but overall, it has massively improved all Linux distributions it is used in.
Never again will I worry about trash buggy init scripts not actually stopping a service due to a stale pid file. Now it puts the service into a control group and can kill all things in the control group even if the service is bad code.
Hi, author here. The title is intentionally a little attention-grabbing hyperbole but I've appreciated the discussion and feedback about the post. Thanks for reading!
This is a nit-pick, but if you picked the title that way to BE hyperbolic, I am not sure if that is the best thing to do for a technical discussion. It sets an internal opinion before even clicking on it. It subconsciously makes people go into the article with an already preconceived notion.
You can have S66 with AntiX too. Just use the 'Init Diversity Edition' (Respin), and pick it from several other options. AFAIK the author of S66 directly collaborated with Antix, to make it perfect.
I personally love the init/service/unit-file portion of systemd and have few complaints. It has a lot of powerful features for security, cron, etc that are very simple to use. I also really like journald.
The parts I don’t really like are the half baked “ancillary” managers like resolvd, which are frankly baffling to use and tend to make simple things needlessly difficult.
And those ancillary managers are usually half-baked or outright broken and dangerous, in that they only implement half of the stuff they are trying to replace. E.g. https://github.com/systemd/systemd/issues/25676
Hard disagree. Name resolution feels like a solved problem, so why fix it if it isn't broken?
But that's just like the old init & logging systems. They mostly worked, and yes it was annoying to relearn a new way of doing things. But the new way is better.
I've got software that works with resolvconf, NetworkManager and resolved. I definitely prefer resolved.
Yes, the cost of switching likely exceeds the benefits for you, and likely for many greybeards. But for those in complicated situations, and for the next generation of sysadmins who never have to learn the old way, resolved is nice.
The older init systems were incredibly frail and often did not work. The reason is because Bash, as much as we all love it, is actually super mega ass. It's very bug-prone and actually one of the least portable languages ever invented, in practice. Because you can't, or shouldn't, call to any other binary from within Bash, otherwise you risk your portability. Now your script won't work the same on other platforms, or even the same platform with slightly different configuration.
But um... yeah... calling to other binaries is the entire ethos of Bash. It's a shell, that's supposed to be all it does. Even getting a "true" value should be a call to another binary.
Unit files, despite being a bespoke format for one singular program, are actually more portable. Like, WAY more portable.
I think even for new people the systemd configs are much more complex (and the APIs are certainly much more complex) than the libc ones which worked well (often better.)
The problem is that almost nothing in Systemd shares theory with the rest of the OS. It's entirely its own island of new ideas, commands, and configuration language that you have to learn and memorize.
> The unix philosophy cries out: is this the end of Linux (or, as many are calling it, GNU plus Linux)?
I still see no counterarguments here. Meanwhile, systemd, like cancer, is eating Linux from the inside, breaking its flexibility and therefore future resilence:
> i used to think that systemd was made the default and adopted by most distros because of its ease of use and the fact it supplied a whole bunch of things in one suite and i see where the appeal is in that but after switching to artix openrc, im just lost on why they decided to use systemd when openrc is objectively better when it comes to being an init system and for managing services, and all the other components of systemd suite can just be replaced, like why would they do this?
I was following where the author was coming from until they quoted this, and rather than addressing it they just slammed it, slammed the author, and moved on.
It's weird to say that ini files are not a domain specific language. And this kind of hand waving away complexity is the crux of systemd putting a bad taste in my mouth. The whole thing is a vast kingdom of nouns (cf Yegge). Forgoing expressing semantics with the config file syntax means that expressing those semantics requires creating even more nouns. Splaying the config across a bunch of tiny files forgoes juxaposition and makes for even more nouns. Owning the aspect of whether a given config file is enabled or not creates even more nouns. Needing tools to analyze the distilled config (eg dependency graph) necessitates even more nouns.
... and this was all dumped on a community that very much focuses on a smaller number of verbs - ie command line utilities. The main problem with SysV scripts is that they are bare scripts doing anything they want with those longstanding verbs. So yes, some abstraction was necessary, and that was inevitably going to seem a bit unnecessary-complexity. But systemd just took that dynamic and went overboard with it.
Sure, I can read "type=forking" and know what that means, but no, I cannot change it without looking up the documentation. Any interaction with systemd beyond start/stop/journalctl involves looking up the documentation, re-figuring-out its implicit logical model, copying the right magical incantation, and then after I've solved the problem promptly forgetting everything I figured out because the next time I touch systemd I'm guaranteed to need to know something different.
I don't dislike it enough to desperately want to move to a different init system (although if there were an easy alternative on NixOS I'd try), and I'm certainly not going to defend SysV or BSD init scripts. But it certainly feels like there is plenty of room to implement systemd's comprehensive functionality in a more user-friendly discoverable way, especially for the main type of advanced-but-casual user that interacts with init systems.
(Also as a fellow NixOS user I think it's easy to have a blindspot about systemd because its config files align with our config format. A more user-friendly concise config format would make NixOS have to do more work to splay it out for its own composability)
systemd has been a success for 2 groups, Windows admins and Fortune 500 Companies.
For UN*X people, all it does is move Linux closer to emulating Microsoft Windows. It has left the UN*X philosophy behind. With secure boot and once Wayland becomes a real thing, the last step is in place. All that will be needed to become a M/S Windows Clone is geo-location and full DRM. That means you will only be able to take screen prints of "approved" applications. I heard SUSE may enable a kind of geo-location, not sure if I read that article correctly though.
For now, at least there are the *BSDs, they allow me to make a PC work the way I want it to, not a way Fortune 500 Companies dictate it should work.
No, it's terrible. I think what they mean by "success" here is that it functions, and is easier to use and more sturdy than init scripts. But any number of things would have been better than those. Instead of writing and adopting one or more, Linux allowed Redhat to take over the last piece of itself, its spine, after which Redhat sold itself to IBM. Linux is an IBM product.
They basically mediate all Linux access to the hardware, and can add arbitrary dependencies to the system at will. They could decide that we're all going to use leftpad now.
Cutting off my hand to get rid of that cancerous mole might have saved my life, but I wish you had just cut off the mole.
> Instead of writing and adopting one or more, Linux allowed Redhat to take over the last piece of itself, its spine, after which Redhat sold itself to IBM. Linux is an IBM product.
Who exactly was going to write the systemd alternative? If by "Linux" you mean the kernel devs, that was never going to happen-- systemd is a middleware stack living in userspace, the kernel guys were never going to get involved. If you mean some distro, other distros did propose some alternatives to systemd back in the day (e.g. upstart), and they all eventually abandoned them and switched to systemd instead, because systemd was better.
OpenRC is great, there's just no flashy marketing behind it or team of consultants pushing it.
EDIT:
Heh. At least I can debug a shell script, I can't debug a config file.
I think this is a fundamental disagreement many people have in what's ultimately desirable when designing software. I personally can't stand non-trivial config files and both at work and at home have wasted way too much time pulling apart software consuming them.
The problem with systemd is that, because of all the new userspace APIs it adds, we can't just agree to disagree. You either target a Systemd OS and the users get to suffer non-trivial, often broken, configs or you target a normal GNU/Linux OS and the systemd users suffer all kinds of surprises when systemd rug pulls them.
# Export some ENVs for it
export SERVER_ADDRESS="0.0.0.0:8088"
name="my-program"
description="My program"
command="/usr/local/bin/my-program"
# If it does not know how to background iself
command_background=true
pidfile="/run/${RC_SVCNAME}.pid"
# To have logs
output_log="/var/log/${RC_SVCNAME}.log"
error_log="/var/log/${RC_SVCNAME}.err"
It seems that it still uses Turing-complete shell scripts as configuration. Hard pass.
>Who exactly was going to write the systemd alternative?
And that quote proves 100% what is wrong with Linux these days.
In the 90s, if someone posted in USENET they want init to do X, at least 2 different solutions would appear with in a week, probably many more.
That happened to me when I went from Coherent OS to Slackware in the early 90s. I asked if there was a way to have virtual consoles on my VGA monitor and my B&W Hercules monitor (386sx), or to have at least 1 VC on the Hercules. Coherent 486 supported that, Linux did not. A post appeared in a couple of days with a patch.
Now, between the Linux Foundation and the Companies that owns it, that would never happen. People who have gotten into Linux in the last 20 years cannot imagine how responsive the developers were to user requests back then.
Seems these days we are dealing with some corporate behemoth drowning in red tape.
I remember running Linux servers for the first time and couldn’t believe there wasn’t a straight forward way to get something to run on startup. Ask glue and duct tape and non portable.
Don’t get me started on the Python situation.
I assume it’s much better now.
With the clown services U hadn’t had to run bare Linux in a long time.
The whole post does have a somewhat hyperbolic tone to it. Personally, I agree with the content and analysis, but I could see the language could feel a bit off-putting, especially given the past flame wars over systemd
Agreed, I didn't mean to tone police, just acknowledging it might be off-putting to some people who got wrapped up in the battles back in the day. I wouldn't suggest the author make any changes
I was pro-systemd at the time of the controversial discussions. I still think it's a net positive relative to what was there before. But personally speaking, it's only the core of the software (service management) that improved things for me. The other things (timers, journald) I either ignore or don't like, but perhaps they're useful for distributions.
I've been quite happy with -boot, -networkd, -resolved. -logind too, what a powerful upshift in routing input.
I forget what but timesync is missing a lot of nice to haves. But ditching grub for something much much much more manageable & a joy to use. Networkd and resolved have both had a lot of nice clear straightforward options, and I love how systemd unit files snap together for these networking concerns.
Journald has been ok for me, and I use it a lot, but not intensively. Having adequate performance is definitely a huge concern.
systemd-resolved is a godsend.
I run, at home and at work, vpns and having a good way to configure split-horizon dns is incredibly useful.
At a previous company we were able to ship a binary that would automatically configure a bunch of company settings on developers' linux laptops and got VPN configurations working with split-horizon dns very easily.
prior to this, doing split-horizon dns on linux basically meant running dnsmasq (!!!) on every linux laptop
Yes, systemd's weirdly bundled system services largely suck. The ones that I'm sure about: journald is more annoying to use than syslog (it takes forever - like a minute on a top of the line CPU - to scan two months worth of logs) and resolvd has or used to have basic bugs that other systems don't have.
I think if it was just service management most people wouldn't mind it nearly as much.
I've worked as a sysadmin/devops for a double digit number of years.
Across companies, jobs, and hobbyist collaborators - I've never met someone who didn't at least like systemd, if not sing its praises. It made so many incredibly painful things about Linux administration disappear.
And made a number of new ones appear.
E.g. logging, now that there is journald, you have to pay attention to another hop in your logging chain and take care of journald in addition to rsyslogd.
Boot and shutdown has previously been deterministic, now things just randomly hang. You do the "windows solution", reboot again, now it magically works through the power of race conditions that are inherent in systemd's mode of operations.
Network device naming is also a fun one. Remember how systemd was supposed to do away with the inconsistency and mess that is "eth0, eth1, eth2, eth..."? Well, now you get either eth0, or ens123, or enp17s7f9, or enx8b220b34. Each distro does it differently, so put in a live-CD to debug something and your network devices are suddenly weird. Reinstall to another version, suddenly weird again. Btw, the issue that names could change depending on boot order was already fixed by udev before, consistent-net.rules nailed it to a mac address on first boot. So systemd took a situation that was already fixed and better than before, and made it far far worse.
Oh, and all the fun that is systemd-dbus, policykit, systemd-logind and permissions on those. Because you always wanted a Javascript interpreter and libxml to tell who could be root on your system... And tons of confused deputy exploits on that cloud of confused systemd-subdaemons.
I'll grant that service files were a nice idea. But beyond that, it has been a very mixed bag, and in places a desaster.
The grand-parent post was about the general consensus: sysadmins generally like systemd. So I'm not sure answering individual concerns is useful, but I'll take the bait...
> now that there is journald, you have to pay attention to another hop in your logging chain and take care of journald in addition to rsyslogd.
No, you don't have to, because a syslog daemon is not required any more. BTW, rsyslogd is not the only syslog service, I've had to work with alternatives like rsyslog-ng.
And the daily usage is now much simpler: `journalctl -b -1 -p err` or `journalctl -u mariadb --since "24 hour ago"` were painful queries with (rotated and compressed) syslog files.
> Boot and shutdown has previously been deterministic, now things just randomly hang.
That was not my experience. IIRC, with sysv-init boot dependencies were declared with comments inside shell scripts and parsed by insserv. That was a mess, and race conditions did occur. I have no experience with other boot systems, but I remember that upstart was not immune either.
> Network device naming is also a fun one.
The naming policy is vaguely relevant to systemd. `net.ifnames=1` is the default configuration of the Linux kernel. I think it's still possible to configure the kernel, udev and systemd to work with the old naming, but if it's just because "shorter is better", I don't think it's worth it.
> No, you don't have to, because a syslog daemon is not required any more.
That is "not required" for the single-computer-sysadmin. If all you know is your laptop, then journald might be fine. If your computers are cattle and have numbers, you don't want to read your logs on each individual computer, if possible you don't want to read logs at all, you want some automation to do it for you, aggregate it, warn you about things, etc. journald just makes that harder, and going through some syslog protocol implementation (of which there are a few, you are right there) is the only relevant game in town.
> IIRC, with sysv-init boot dependencies were declared with comments inside shell scripts and parsed by insserv. That was a mess, and race conditions did occur. I have no experience with other boot systems, but I remember that upstart was not immune either.
Dependencies and insserv were a later addition. Basically you just defined a boot order by numbering your /etc/rc[0-6].d/S* links in ascending order, either manually or in a dependency-based fashion through insserv. This means that you can have a suboptimal ordering if those dependencies are wrong or you choose your ascending order poorly. But it also means that once you have a working boot order, it can be repeated, and it will be repeated. That is what I mean by determinism. Once you have a working boot, it will keep working. Solaris SMF did something more modern, you would define dependencies in service definitions, and it would compute a static deterministic ordering from that, without those /etc/rc[0-6].d/S* symlinks. Deterministic, but dependency-based. If it has worked once, it will work again. And they even had a rollback: if after a change, the next boot wouldn't work, you could go back to the previous boot order and fix things from there. And neither had race conditions, because you usually wouldn't do things in parallel, one service would wait for the previous one in the boot order (parallel was possible but rarely used, because of the inherent fragility and possible race conditions).
Systemd does it dynamically on every boot, no precomputation, just looking at all the files and starting everything that could possibly be started at the same time, in parallel. The reason for this is that this makes Lennarts laptop boot faster, which is true in the ideal case. But it is neither deterministic, nor debuggable. And other than the previous methods, it can lead to race conditions.
Note that there is one more race condition at boot, which is races against kernel devices and systemd-udevd. That neither got worse nor better with systemd, it just lead to the kernel people giving up and handling more things like firmware loading in kernel.
> The naming policy is vaguely relevant to systemd. `net.ifnames=1` is the default configuration of the Linux kernel.
Utterly totally completely wrong. https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/lin...
The Linux kernel does not have a net.ifnames parameter, and the kernel interface naming scheme is still eth[0-9]+. When you stick that parameter in the kernel commandline, the kernel ignores it and only systemd and systemd-udevd interpret it later on.
Please, if you want to sing the praises of software or deflect blame from broken crap, make sure that you know what you are talking about...
It's a little weird to reply to someone complaining about administrative problems caused by systemd with "generally sysadmins seem to like it."
That kind of response is exactly why everyone gets emotional in these discussions. Technical problems are one thing but when you gaslight people experiencing and reporting legitimate problems you should not be surprised when they get mad and rip your solution out.
GP is being far kinder to the systemd devs than they deserve, I gave up on the project years ago for exactly this reason.
> consistent-net.rules nailed it to a mac address on first boot
fails even harder for
> put in a live-CD to debug something and your network devices are suddenly weird
I don't know that there is really any good answer here. Just methods that break in ways you're used to, and methods that break in ways you aren't. I appreciate the attempt at static naming based on hardware connection, even if the realities of how these things are presented by the bios make the results an inscrutable mess.
Maybe pure enx would have been better? But that looks like crap for even the simple cases, and fails if you swap out a piece of hardware or start doing weird things with MAC addresses.
The only consistent naming scheme I've ever seen working was Solaris on Sun hardware. But that only worked because they controlled both the hardware and the software side of things.
And yes, consistent-net.rules doesn't solve LiveCDs and reinstallations. But as I've said, systemd doesn't solve those either, and in most cases actively makes them worse.
My experience has not been that the systemd device names are "actively worse". The biggest problem I've had is changing around VMs where subtle shifts in PCIe IDs (specifically subdevices) end up changing changing names. Meanwhile, consistent-net.rules seems like a hack, always in the back of your mind as an arbitrary stateful part of the system. Why should I think of my primary network device as eth2 if there is no reason for it to be eth2 other than merely being the order on the first install?
But if you prefer consistent-net.rules, then more power to you. I'm guessing most of your frustrations could have been addressed by systemd having some shame about changing functionality out from under users, and providing an up-front option to switch between the different methods.
I used to be fine with it, until I started writing my own software tools, then looking at how to actually do stuff with systemd and I'm like, nope nope nope nope. At some point when I have space for a new project I'm going to migrate from Arch to Obarun, which uses a modern update on the excellent daemontools to provide init. When systemd works it's mostly fine, when it breaks, path of least resistance is often to reinstall the OS. Which is fine if you're just working on it. Painful if it's your personal machine, and those are what's most likely to break systemd.
> when it breaks, path of least resistance is often to reinstall the OS
Can you elaborate how do you even break this?
Adding a systemd unit cannot really break much on systemd itself. Unless you’re messing with internal systemd stuff, I am not sure what is even there to break.
At the moment, on this machine I'm using, audio just doesn't work. Dunno why. Tried a few different things to fix it, no dice. Trying to troubleshoot is made enormously more difficult by the complexity of systemd. If I had a simpler system I could run it all the way back to the source, but really, I can't be bothered, I don't really need audio on this machine atm.
i have the same sentiment, and for me systemd was getting so slow and hard to tshoot (the tools to analyze boot graphs and sequence showed service sequence breakage, some times there is no problem, sometimes sequence changed and problem happened) and i just changed for another distro without systemd and boom, startup in seconds no problem at all.
This doesn't match my experience at all. For the life of me, I cannot imagine what you are talking about, what any of your issues are. Did you file any tickets, or report any issues?
What about "actually doing stuff with systems" was "nope nope nope nope"? What situations did systemd break in? What did you try before giving up to reinstall the OS?
Thise fud train is cast aspersions, no meat.
You probably use a desktop environment like GNOME or KDE. Maybe do some light coding in an IDE. I do not. I run window manager only, i3, with a really stripped down toolset. I like the idea of knowing my system in and out. My current project is writing my own shell. It can switch between command mode and ruby REPL with ctrl-z.
You mention filing tickets / reporting issues, sorry, that means contributing to a project. I'm not wholly opposed to doing that, but I have to like the project first. And there's nothing to like about systemd.
You go do that if you like it so much. My guess is you simply won't, and will just keep using other people's software without a care in the world, thinking this kind of online warriorship is actually helping anything.
Bro, I've been running wmii since before i3 forked off from it. Don't come at me with this dripping disdainful attitude, pretending like you are some high and mighty elite system whisperer.
It's so unfortunate that there is such a widespread persistence of hater-dom. It's so toxic and awful, and so rarely does it have even the faintest shred of anything to stand up on. I struggle so much to see these attractors into the negative as so compelling, so ensnaring of the human souls as they have been lately.
Systemd is amazing, provides such a rich world versus piecemeal poorly managed systems of yore. Yes you could understand init scripts with very little effort but what a sad virtue, to dare to go no further than what an afternoon of exploration might get you.
I don't fucking hate systemd. I'm marginally glad it exists. I just don't want to live under it's despotic regime. You might like it because you think trains running on time is more important than freedom.
You're the one coming at me with fucking attitude, like a 16 year old who just discovered IntelliJ. I have done the exploration, then looked online for why everything is so damn complicated now. It's because systemd colonized the world.
It's not just an init system, it's everything below the userland. And a lot of those components are very poorly designed. And it's this big ball of mush where if you use one component, thy must use them all. It's so bad Arch does not support replacing systemd, like at all. You're totally on your own.
Go ahead, try replacing systemd components in an Arch system. I'll wait. And lose the fucking attitude.
+1, systemd is very usable, and works. I would really not go back to what was before
I don't hate journald because it's not plaintext, I hate it because it's worse than plaintext. Somehow journald manages to provide a database which is 40x slower to query than running grep on a compressed text file. I'm all in favour of storing logs in an indexed structured format but journald ain't it.
Even doing zcat | grep is faster than journald.
I now turn off journald and use rotated pain text log files. It's more efficient in all metrics.
Journald is an odd one. I don't think it being a binary log/database makes sense. If you have a tiny operation, with a single server, then the binary database doesn't really make sense, having plain text is just easier and faster. If you're a bigger operation, you'll have a central logging solution, in which case you need journald to store the longs as plain text as well, before you can do log shipping.
The only use case where the binary format might make sense is if you ship journald logs to another central journald instance. That's just very much an edge case.
afaik journald can just forward logs via rsyslog directly to a remote server
Why would it need to store plaintext locally?
Some folks like having some local storage to act as a buffer in case the remote syslog server is down. Not sure if journald can do that on its own.
Doesn't that still involve a conversion? I believe that rsyslog can read the journald database, but you're typically not querying syslog data directly, so there's a conversion between rsyslog and logstash, Splunk, Datalog, whatever.
My view is pragmatic: if you are managing the server and I am a user, then use systemd if you want to.
If it’s my responsibility then I will use a non-systemd distribution or if I have a choice, FreeBSD.
FreeBSD is many great things, but ... a pragmatic choice?
It works for many, including Netflix. The jails are better integrated as a lightweight container system than Docker is on Linux, IMHO.
Yes; it does work nicely in many circumstances.
And powers zillions of unsung, critical infrastructure tasks.
I remember when I first upgraded a system to a version using it. This was probably a RHEL 6 -> 7 update back in 2014 or so.
It made so many common things I had to do so as a sysadmin super straightforward.
Same, that was my first experience as well. It was a breath of fresh air in many ways, and after my initial learning of the new syntax, I quickly became a convert.
That said, I have had some concerns over the years about the growing scope of systemd. I think history has shown my concerns were overblown, though I think it is definitely possible and even likely that people were affected by those arguments and maybe moderated the ambitions a little bit. That could be for better or worse, I don't know, but I do think it was a factor
I think the biggest problem with systemd is that they named the main init service manager systemd and also named the organization systemd! It'd be like if GNU named their initrd "GNU". Their other tools just have systemd prepended, e.g. systemd-journald, much like GNU Make or GNU libc (glibc) or similar. The "scope creep" issue doesn't get complained about for GNU because it's clear they're an organization making a bunch of tools for different purposes. Systemd-the-organization is the same, but the naming makes it unclear that many of these tools are not depended on by systemd-the-init-system.
And some things got super hard to do, e.g. sequence. I guess it is sorted now, but either way, I wouldn't want to go back.
Does anyone actually use journald? The last time I tried(2 years ago?) it didn't even work with any log management software (like cloudwatch for example).
You had to either use some (often abandoned) third party tool or defeat the purpose by just reconfiguring everything to dump a text log to a file.
I use journald whenever I feel my blood pressure getting too low.
It's slow, truncates lines, and doesn't work well at all with less. It's almost like Pottering created it so that pulseaudio wouldn't be his worst program anymore.
One convenience of journald is that it exposes a single place to plug in log collection for observability tooling
opentelemetry-collector, promtail, and so on have native plugins for it, which makes aggregation easier to setup
Most tools have "tail this plaintext file" as well, but if it's all flowing to journald, setting up log collection ends up being that much simpler
That is what syslogd has been doing since forever. Journald actually made this harder due to not supporting the established syslog protocol.
Pipe it into lnav eg:
journalctl -b | lnav
or use -f instead of -b for follow instead of everything since boot. Now you have a colourised journal and the power of lnav.
For debugging my desktop:
For anything else: export to a syslog server, which basically any tool that matters will support in some fashion.journalctl: unrecognized option '--tail'
I architected and built our entire log-ingestion pipeline for intrusion detection on it, at Square.
I built a small Ruby wrapper around the C API. Then I used that to slurp all the logs, periodically writing the current log ID to disk. Those logs went out onto a pubsub queue, where they were ingested into both BigQuery for long-term storage / querying, and into our alerting pipeline for real-time detection.
Thanks to journald, all the logs were structured and we were able to keep a bunch of trusted metadata like timestamp, PID, UID, the binary responsible, etc. (basically anything with an underscore prefix) separate from the log message all the way to BigQuery. No parsing, and you get free isolation of the trusted bits of metadata never intermingling with user-controlled attributes.
Compared to trying to durably tail a bunch of syslog files, or having a few SPOF syslog servers that everyone forwarded to, or implementing syslog plugins, this was basically the Promised Land for us. I think we went from idea to execution in maybe a month or two (I say “we” but really this was “me”) and rolled it out as a local daemon to the entire fleet of thousands. It has received—I think—one patch release in its six+ year lifetime, and still sits there quietly collecting everything to be shipped off-host.
The only issue we’ve ever really ran into that I never figured out is a handful of times per year (across a fleet of thousands) the journald database corrupted and you couldn’t resume collecting from the saved message ID. But we were also on an absolutely ancient version of RHEL, and I suspect anything newer probably fixed that bug. We just caught the error and restarted from an earlier timestamp. We built the whole thing around at-least-once delivery so having duplicates enter the pipeline didn’t really matter.
Damn, honestly at this point I’m wishing I’d pushed to open source it.
Ironically, actually, I did write a syslog server that also forwarded into this pipeline since we had network hardware we couldn’t install custom services onto but you could point them at syslog. I also wrote this in Ruby, using the new (at the time) Fibers (“real” concurrency) feature. The main thread fired up four background threads for listening (UDP, UDP/DTLS, TCP, TCP/TLS), and each of those would hand off clients to a dedicated per-connection worker thread for message parsing. Once parsed they went onto one more background thread for collecting and sending to PubSub. Even in Ruby it could handle gazillions of messages without breaking a sweat. Fun times.
Since I’m rambling, we also made cool use of zstd’s pre-trained dictionary feature. Log messages are small and very uniform so they were perfect for it. By pre-sharing a dictionary optimized for our specific data with both ends of the pubsub queue, we got something like 90%–95% compression rates. Given the many terabytes of logs we were schlepping from our datacenters to GCP, this was a pretty nice bit of savings.
Well, everyone and noone uses journald. The usual way to log is journald -> (r)syslogd -> remote log destination(s). I've never actually seen any other way.
The preferred journald way of "fetch your logs periodically" wouldn't pass any audit and doesn't work with any kind of log processing software that people use.
I use it everyday?
Cloudwatch fucking sucks.
Plenty of log shippers can slurp journald. (Fluentd, filebeat, vector)
Even ChromeOS uses it, even on devices that still use Upstart.
> Even ChromeOS uses it, even on devices that still use Upstart.
I was curious about this, because I thought journald hard required systemd as pid 1, so I did a search, which promptly turned up https://www.chromium.org/chromium-os/developer-library/refer... -
> Jounald is deprecated and is about to be removed.
Hmm couldn't exactly tell when it was removed, but it looks like it lasted maybe 3-4 years. This is the commit that added it to the upstart config.
https://chromium.googlesource.com/chromiumos/platform2/+/870...
Fascinating. An initscript for journald is a special kind of cursed that I didn't expect to read today:)
You're being downvoted but you're absolutely right. The fact that Cloudwatch doesn't support journald is a major, major fail on AWS' part. It's not like this is new or obscure software.
I'm just surprised anyone wants to use cloudwatch when they don't need to. It is expensive, and far from the best observability platform.
It's always there by default. That's the only reason I've ever seen it used.
I wouldn’t call it a success it all. Almost managed to do was moving or forcing other software to get locked in. GDM, Podman, Pipewire…just to name a few.
As another comment says, it's a success in the same way Windows has been a success.
I've been a fan of systemd even if just for how much simpler creating services is. I just need to write a simple config file instead of a complex init script.
+1 - there was a misconception that initscripts were easy or simple - if you wanted _correct, bug free_ initscript you were probably looking at around 100 lines of shell script.
Or if the service didn't support pam_limits because it was legacy trash, you had to hack something into the initscript like `ulimit -n XYZ` and restart it. Now things like this are trivial and easy to solve. Using systemd makes large scale Linux systems administration much easier.
Now it has gone a bit overboard. Some of the stuff like the dns resolver or the nspawn capability seem a bit over the top, but overall, it has massively improved all Linux distributions it is used in.
Never again will I worry about trash buggy init scripts not actually stopping a service due to a stale pid file. Now it puts the service into a control group and can kill all things in the control group even if the service is bad code.
https://www.cvedetails.com/vendor/15978/Systemd-Project.html
https://www.theregister.com/2017/07/28/black_hat_pwnie_award...
journald is such a disaster. How is it possible to be so much slower than even gzipped plain text.
These were pretty much my feelings back when I envied launchd and was frustrated with some upstart behaviour. Systemd turned out better than both.
I am not a sysadmin -- so I keep forgetting the command to see logs cause I only need it once every 3 months.
So, systemd change is a pain for me.
use a shell function/alia?
(Not that I think journalctl -u even needs an alias tbh but if you want it you can have it)Keep a file with a list of commands used semi-regularly. Or have a script dump your command history into a file.
Better to spend a minute searching for a command in a file than to waste an hour recreating it.
AI is so helpful for doing those kinds of tasks in the terminal that you only do a couple times a year and can never remember the exact incantation
You only need to look those things up once, put it in a snippets.txt file, and then search in the file when you need it.
I keep a per project file like that with common commands and related things.
Hi, author here. The title is intentionally a little attention-grabbing hyperbole but I've appreciated the discussion and feedback about the post. Thanks for reading!
This is a nit-pick, but if you picked the title that way to BE hyperbolic, I am not sure if that is the best thing to do for a technical discussion. It sets an internal opinion before even clicking on it. It subconsciously makes people go into the article with an already preconceived notion.
I like systemd, but do agree that it's clunky in a lot of places. I'd be interested in hearing from anyone who has used Obarun or its 66suite.
You can have S66 with AntiX too. Just use the 'Init Diversity Edition' (Respin), and pick it from several other options. AFAIK the author of S66 directly collaborated with Antix, to make it perfect.
I personally love the init/service/unit-file portion of systemd and have few complaints. It has a lot of powerful features for security, cron, etc that are very simple to use. I also really like journald.
The parts I don’t really like are the half baked “ancillary” managers like resolvd, which are frankly baffling to use and tend to make simple things needlessly difficult.
And those ancillary managers are usually half-baked or outright broken and dangerous, in that they only implement half of the stuff they are trying to replace. E.g. https://github.com/systemd/systemd/issues/25676
Hard disagree. Name resolution feels like a solved problem, so why fix it if it isn't broken?
But that's just like the old init & logging systems. They mostly worked, and yes it was annoying to relearn a new way of doing things. But the new way is better.
I've got software that works with resolvconf, NetworkManager and resolved. I definitely prefer resolved.
Yes, the cost of switching likely exceeds the benefits for you, and likely for many greybeards. But for those in complicated situations, and for the next generation of sysadmins who never have to learn the old way, resolved is nice.
The older init systems were incredibly frail and often did not work. The reason is because Bash, as much as we all love it, is actually super mega ass. It's very bug-prone and actually one of the least portable languages ever invented, in practice. Because you can't, or shouldn't, call to any other binary from within Bash, otherwise you risk your portability. Now your script won't work the same on other platforms, or even the same platform with slightly different configuration.
But um... yeah... calling to other binaries is the entire ethos of Bash. It's a shell, that's supposed to be all it does. Even getting a "true" value should be a call to another binary.
Unit files, despite being a bespoke format for one singular program, are actually more portable. Like, WAY more portable.
I think even for new people the systemd configs are much more complex (and the APIs are certainly much more complex) than the libc ones which worked well (often better.)
The problem is that almost nothing in Systemd shares theory with the rest of the OS. It's entirely its own island of new ideas, commands, and configuration language that you have to learn and memorize.
I'm sure it's a success for someone 8-/ Mostly IBM.
It's certainly been a successful takeover of all of the userspace runtime by one giant integrated set of executables.
Pwned off as an init replacement, then feature creeping it's way into every nook of userspace.
For me, it's still a disaster for distro and system s/w independence and the "do one thing well" philosophy.
While many welcome our new system overlords, the s/w is still not liked by many many people...
Move into the void! The DJB way and runit await...
> The unix philosophy cries out: is this the end of Linux (or, as many are calling it, GNU plus Linux)?
I still see no counterarguments here. Meanwhile, systemd, like cancer, is eating Linux from the inside, breaking its flexibility and therefore future resilence:
https://wiki.gentoo.org/wiki/Hard_dependencies_on_systemd
https://news.ycombinator.com/item?id=42918448
https://news.ycombinator.com/item?id=42889792
(I reposted my comment from previous submission)
> i used to think that systemd was made the default and adopted by most distros because of its ease of use and the fact it supplied a whole bunch of things in one suite and i see where the appeal is in that but after switching to artix openrc, im just lost on why they decided to use systemd when openrc is objectively better when it comes to being an init system and for managing services, and all the other components of systemd suite can just be replaced, like why would they do this?
I was following where the author was coming from until they quoted this, and rather than addressing it they just slammed it, slammed the author, and moved on.
It's weird to say that ini files are not a domain specific language. And this kind of hand waving away complexity is the crux of systemd putting a bad taste in my mouth. The whole thing is a vast kingdom of nouns (cf Yegge). Forgoing expressing semantics with the config file syntax means that expressing those semantics requires creating even more nouns. Splaying the config across a bunch of tiny files forgoes juxaposition and makes for even more nouns. Owning the aspect of whether a given config file is enabled or not creates even more nouns. Needing tools to analyze the distilled config (eg dependency graph) necessitates even more nouns.
... and this was all dumped on a community that very much focuses on a smaller number of verbs - ie command line utilities. The main problem with SysV scripts is that they are bare scripts doing anything they want with those longstanding verbs. So yes, some abstraction was necessary, and that was inevitably going to seem a bit unnecessary-complexity. But systemd just took that dynamic and went overboard with it.
Sure, I can read "type=forking" and know what that means, but no, I cannot change it without looking up the documentation. Any interaction with systemd beyond start/stop/journalctl involves looking up the documentation, re-figuring-out its implicit logical model, copying the right magical incantation, and then after I've solved the problem promptly forgetting everything I figured out because the next time I touch systemd I'm guaranteed to need to know something different.
I don't dislike it enough to desperately want to move to a different init system (although if there were an easy alternative on NixOS I'd try), and I'm certainly not going to defend SysV or BSD init scripts. But it certainly feels like there is plenty of room to implement systemd's comprehensive functionality in a more user-friendly discoverable way, especially for the main type of advanced-but-casual user that interacts with init systems.
(Also as a fellow NixOS user I think it's easy to have a blindspot about systemd because its config files align with our config format. A more user-friendly concise config format would make NixOS have to do more work to splay it out for its own composability)
systemd has been a success for 2 groups, Windows admins and Fortune 500 Companies.
For UN*X people, all it does is move Linux closer to emulating Microsoft Windows. It has left the UN*X philosophy behind. With secure boot and once Wayland becomes a real thing, the last step is in place. All that will be needed to become a M/S Windows Clone is geo-location and full DRM. That means you will only be able to take screen prints of "approved" applications. I heard SUSE may enable a kind of geo-location, not sure if I read that article correctly though.
For now, at least there are the *BSDs, they allow me to make a PC work the way I want it to, not a way Fortune 500 Companies dictate it should work.
DRM is almost there. Try using Netflix on a *BSD, you can't. The DRM Netflix uses is not supported on BSDs.
No, it's terrible. I think what they mean by "success" here is that it functions, and is easier to use and more sturdy than init scripts. But any number of things would have been better than those. Instead of writing and adopting one or more, Linux allowed Redhat to take over the last piece of itself, its spine, after which Redhat sold itself to IBM. Linux is an IBM product.
They basically mediate all Linux access to the hardware, and can add arbitrary dependencies to the system at will. They could decide that we're all going to use leftpad now.
Cutting off my hand to get rid of that cancerous mole might have saved my life, but I wish you had just cut off the mole.
> Instead of writing and adopting one or more, Linux allowed Redhat to take over the last piece of itself, its spine, after which Redhat sold itself to IBM. Linux is an IBM product.
Who exactly was going to write the systemd alternative? If by "Linux" you mean the kernel devs, that was never going to happen-- systemd is a middleware stack living in userspace, the kernel guys were never going to get involved. If you mean some distro, other distros did propose some alternatives to systemd back in the day (e.g. upstart), and they all eventually abandoned them and switched to systemd instead, because systemd was better.
Those alternatives exist, actually. OpenRC, dinit, runit, and s6 are all good options and all used by some distros.
OpenRC is great, there's just no flashy marketing behind it or team of consultants pushing it.
EDIT: Heh. At least I can debug a shell script, I can't debug a config file.
I think this is a fundamental disagreement many people have in what's ultimately desirable when designing software. I personally can't stand non-trivial config files and both at work and at home have wasted way too much time pulling apart software consuming them.
The problem with systemd is that, because of all the new userspace APIs it adds, we can't just agree to disagree. You either target a Systemd OS and the users get to suffer non-trivial, often broken, configs or you target a normal GNU/Linux OS and the systemd users suffer all kinds of surprises when systemd rug pulls them.
#!/sbin/openrc-run
depend() { need net }
# Export some ENVs for it export SERVER_ADDRESS="0.0.0.0:8088"
name="my-program" description="My program" command="/usr/local/bin/my-program" # If it does not know how to background iself command_background=true pidfile="/run/${RC_SVCNAME}.pid" # To have logs output_log="/var/log/${RC_SVCNAME}.log" error_log="/var/log/${RC_SVCNAME}.err"
It seems that it still uses Turing-complete shell scripts as configuration. Hard pass.
>Who exactly was going to write the systemd alternative?
And that quote proves 100% what is wrong with Linux these days.
In the 90s, if someone posted in USENET they want init to do X, at least 2 different solutions would appear with in a week, probably many more.
That happened to me when I went from Coherent OS to Slackware in the early 90s. I asked if there was a way to have virtual consoles on my VGA monitor and my B&W Hercules monitor (386sx), or to have at least 1 VC on the Hercules. Coherent 486 supported that, Linux did not. A post appeared in a couple of days with a patch.
Now, between the Linux Foundation and the Companies that owns it, that would never happen. People who have gotten into Linux in the last 20 years cannot imagine how responsive the developers were to user requests back then.
Seems these days we are dealing with some corporate behemoth drowning in red tape.
Agree wholeheartedly.
Loonix users: We hate windows
Loonix developers: Now Loonix is windows
Loonix users: booo
… several hours later
Loonix users: yaaaay
/s
I remember running Linux servers for the first time and couldn’t believe there wasn’t a straight forward way to get something to run on startup. Ask glue and duct tape and non portable.
Don’t get me started on the Python situation.
I assume it’s much better now.
With the clown services U hadn’t had to run bare Linux in a long time.
There are numerous distributions not using systemd. So adding “complete, utter, unmitigated “ in front of success seems far-fetched.
The whole post does have a somewhat hyperbolic tone to it. Personally, I agree with the content and analysis, but I could see the language could feel a bit off-putting, especially given the past flame wars over systemd
It's a lighthearted post, so probably not worth tone policing.
It has nothing to do with the tone, it is just a fact. So there is nothing to police about.
Agreed, I didn't mean to tone police, just acknowledging it might be off-putting to some people who got wrapped up in the battles back in the day. I wouldn't suggest the author make any changes