The article kind of misses out on the networking standards of the time - there were many of which TCP and IP were just one, most of the rest were either proprietary (DNA SNA BNA etc) and incompatible, or owned by telecoms who wanted to be able to charge per packet(X.25).
TCP/IP won (IMHO) because the implementation was open source, and because no one was going to make a buck by locking others out (or charging them for the right to build an implementation). Plus there was a working reference implementation available for the taking
While TCP/IP was open source and not controlled by any 1 vendor, quite a few companies made good money from selling TCP/IP stacks and won themselves billion dollar valuations in the process[1][2]. Of course this was during a time when Microsoft saw no point in including an IP stack with their OS, or indeed much point in connecting to the Internet at all! Some of the younger readers may find these things hard to believe.
Don't forget Trumpet Winsock [1]. I worked for an ISP at the time that provided it on a floppy disk (along with a newsreader, FTP client, telnet client, and web browser). Although I doubt many customers paid for it, Trumpet Winsock was shareware and very widely used on Windows 3.x.
Everything else was commercial, while by being prevented to sell UNIX, AT&T was fine with giving its source code to universities for a symbolic price for sending the tapes, and the Lion's book.
It changed quite fast the moment AT&T was allowed to claim ownership, however just like with IBM and PC clones, AT&T could no longer take control over UNIX.
1, bidirectional sockets probably should've been ≥2 fd's, not 1.
2, non-blocking semantics and poll/select suck.
Can't blame anyone in 1983 for not getting "async I/O" right since we're still struggling; at least now we have some decent answers (like io_uring on Linux.)
I'd argue that it's not that the semantics suck, it's that there are too many of them. We have O_NONBLOCK, multiple multiplexers (for sane reasons--I don't begrudge 1980s folks for not thinking about fd set size and copy overhead for select(2) either), and others. What's worse, they don't all work with all FDs--not only are regular files not nonblock-able in the same way that sockets are, but all sorts of other FD-exposed capabilities (signalfds, timerfds, memfds, pidfds) do or don't support nonblocking semantics and multiplexing in all sorts of weird ways.
If those old system designers had stuck with keeping the async IO syscall space small (e.g. "you only get select/poll" or "you only get read(fds) and read_noblock(fds, timeout)") and consistent (by drawing a hard line at "if you expose something as an FD, it must support all APIs that generically handle FDs"), we would have ended up in a better place. Sure, that would have slowed down some implementations (e.g. vfs drivers), but would also have massively sped up development against a lot of these APIs.
Allready in 1983, 4.2BSD had 3 different attempts at doing multiplexed/asynchronous I/O:
Non-blocking I/O: "O_NONBLOCK" with "EWOULDBLOCK" (or "EAGAIN")
Signal-driven I/O: "O_ASYNC" with "SIGIO"
Synchronous I/O multiplexing: "select()"
All 3 had various defects, especially with a large number of concurrent I/O actions.
In UNIX-derived operating systems, after 1983 there have been many other attempts to implement something better than these 3 (starting with System V "poll" and with POSIX AIO, and then with various incompatible approaches in Solaris, FreeBSD and Linux), but none were good enough and most were seriously inferior to methods of doing asynchronous I/O that existed in some IBM and DEC operating systems decades earlier.
In my opinion, only io_uring has finally solved the problem of multiplex asynchronous I/O in Linux, and in a manner much better than in all older operating systems. I consider all the many older alternatives that exist for liburing as obsolete.
No, although this was true until around 2019 when Linux introduced io_uring and finally had a genuine innovation. io_uring wound up giving us a unified asynchronous I/O layer that handles just tons of things. (And it is genuinely a step up from the overlapped I/O APIs of Windows, as it avoids eating syscall transition overhead in many cases.)
Of course, Microsoft then went on to introduce IORing in 2022, continuing their legacy of never being afraid to adopt good ideas. But still - it would be wrong to suggest Linux is currently behind on async I/O. It hasn't been for years.
Sorry, what I meant was that FDs that aren’t sockets should be uniformly handled by the async IO syscalls rather than having to parse man pages or trip over EINVALs to figure out the answers to questions like “can I select(2) on a pidfd? Can I O_NOBLOCK a regular on-disk file?”.
For a concrete example, I should be able to select/poll/epoll any file descriptor. The multiplexers can short-circuit return for things where readiness is inherent (e.g. block device files/vfs, I’m not asking for the moon a la waiting for NFS shares to report ready or something).
I should be able to issue non blocking reads on special file descriptors (timerfds, signalfds, eventfds, and so on).
This is analogous to my other unachievable fantasy: everything that exposes a filesystem interface must work with inotify/kqueue/whatever. No exceptions for procfs/NFS/etc.
At some point this fantasizing just becomes me griping about what made it to market/the worse-is-better philosophy generally, though. And yet it moves, I guess!
Windows NT had many things decades before Linux, I've gotten used to it. There are things Windows has that I still sort of wish Linux had, I mentioned one of them just last night (RDP fast user switching/session roaming.)
If we're just strictly talking about low level things, another good one would be synchronization primitives, which I guess we now have some of in Linux verbatim at this point, if only for the sake of emulation. (And of course futex2.)
Windows strengths: IRPs and fantastic async I/O, ETW events, tooling- windbg xperf driver verifier time travel debugging, RDP, doesn’t claim everything acts like a file
Linux strengths: file system, open source ecosystem, ssh, lockless algorithm support
I don't think the kernel prohibits fast user switching but you'd have to implement it at the desktop layer too.
Think of ctrl-alt-f7, f8 etc. This still works today, you can run X servers as different users on different VT numbers and switch between them. All that's missing is better UI.
The kernel definitely doesn't, although the VT layer kind of sucks and probably should've never been used to multiplex graphical sessions. (I mean, I get why it was, but oh well.)
It's kind of the opposite, systemd-logind provides better seat management than without. I attempted to implement the same concept using libseat and seatd unsuccessfully though I do not remember exactly what I got caught on. (not in kwin but a toy compositor; I have been at this problem for a little while now.)
That said, most desktop systems have only implemented systemd-logind in a limited way so far that basically just does what the VT system + display manager was already doing, that's what would need to be worked on in order to make this a reality. I was able to accomplish a prototype with only patches to Kwin, kfreerdp and plasma-login-manager, no need to patch systemd-logind or anything like that.
> Who even uses systemd?
Well for one thing, almost all of the major distributions; Ubuntu, Debian, Fedora/RHEL, Arch Linux, NixOS, openSuSE? and of course SteamOS now.
The only major non-systemd Linux distributions I can think of are Alpine Linux and Gentoo. And Android if you wanted to count that, but I think it is special enough to be considered its own OS that just is Linux-based.
I personally use a systemd-based distro (NixOS) and logind is able to handle this use case. The point of trying to support libseat was for the sake of non-systemd distros. (Though elogind would probably work too.)
I certainly wouldn't DIY this; nothing would support my DIY version, which would defeat the purpose.
The Linux platform gives you full control. If the box you seek doesn't exist, you have to create your own box instead of thinking inside someone else's box that doesn't even do what you need.
Mechanisms equivalent with the overlapped I/O of Windows NT (1993) already existed 30 years earlier, e.g. in IBM OS/360 and PL/I (1964/1965).
The main features that were better in Windows NT than in the UNIX-derived operating systems were inherited from the DEC VAX/VMS operating system (1978) (e.g. WaitForMultipleObjects) and a part of them had been inherited from the even earlier operating system DEC RSX-11M (1974-11) (Dave Cutler also had a major role in those operating systems, so there is nothing surprising about this; Microsoft had to pay a big compensation to DEC, on the order of $ 100M, for the features that were obviously taken from the DEC operating systems).
UNIX was very simplified in comparison with the operating systems that were used on bigger computers, and for some things its derivatives never caught up with those older systems, except after 2000. Only "futex" (2002) and "io_uring" (2019) have advanced the state of the art clearly beyond what already existed for AIO on the IBM mainframes around 1964/1966.
To be fair, I think that was the entire challenge in the first place: the contemporary microcomputers were relatively limited on what they could do, so you couldn't necessarily afford to just do everything mainframes were doing. More importantly, it's not clear you would've wanted to. Having the taste and good sense to know what to "steal" and how to "steal" it was already hard enough work. A lot of good ideas took a while to come over, but a lot of probably bad ideas also never did.
It's an evergreen interesting fact that Dave Cutler took perhaps a bit too much inspiration from prior VMS work to NT, but I also think there's no need to put an asterisk on it anymore than there is UNIX or anything else; it does ultimately stand on its own. Like great artists, great software architects steal.
I just can’t understand why the Linux kernel has avoided tracking completions for so long and has such hard time with async I/O. Windows has had IRPs and overlapped I/O for decades, asyc is the default and there just isn’t much reason to ever use synchronous I/O.
It's such a common problem to write to a stale file descriptor. It may be closed which is fine, at least you will have an obvious error. But what's much worse is when the stale descriptor actually works, because something else in the program opened a new fd and your stale descriptor is not stale anymore, it just points at something unexpected.
Opaque handles are often a blessing.
Maybe UUID would be overkill (it certainly would have been back in the day of 16-bit unix machines) but something fairly large would have helped a lot.
Even just incrementing and rolling over a 16-bit value would have helped, instead of situations such as closing stdout and stderr, opening a file, and now random error logs sprinkle into your PDF or whatever.
Yes, "you are holding it wrong", but it should be hard to "hold it wrong", not easy.
Joking aside I pretty much agree with this. Windows NT Object handles work pretty well. I assume they are pointers of some kind under the hood, but either way a machine word is probably large enough as it seems unlikely you would ever need to handle more open handles than you have bytes of addressable memory.
Sockets won over over-engineered monstrosities like STREAMS or X.whatever
And yet, sockets are a terrible interface. They don't provide a way to get the details of the underlying connection for features like migration, checkpointing, or introspection. E.g. there is no way to get the current sequence number for TCP (there is "connection repair" mode now, but it's Linux-specific).
Well, you can say that sockets abstract the low-level details, but then these details hit you in the face when you need to do protocol-specific name resolution.
I now believe that we could have switched to something like IPv6 two decades ago if the socket interface simply allowed binding multiple address families to one socket and handled the name resolution internally.
Sockets also cemented the "one connection - one address" model that is _still_ dragging back the IPv6 adoption. MPTCP or QUIC are still barely supported.
I don't think that's the reason IPv6 adoption has been slow. It's because ISPs and telcos and data centers and everyone else are slow to upgrade hardware and software, because they don't care about it and because it costs money (money that doesn't get them more money from their customers), not because it's hard.
I'm the network admin for a small ISP that hasn't deployed IPv6 yet.
We do have ARIN IPv6 space, but we also have thousands of customers who do not know what that is or care about it either. Twice in twenty years now we have had a customer ask about it, and we have offered to work with them to help pilot support through our network (since if they have a need then that need could act as a standard against which to measure support) but always within a day they simply find some other way to satisfy their need without IPv6.
"Get more money from their customers" aside, we do still have a duty to prioritize the needs of our customers, and I'm not pulling budget away from initiatives to improve availability or increase bandwidth or lower latency simply to chase a buggy moving target technology with no light at the end of the tunnel by way of material improvement for our customers.
No "killer app". <1% marketing draw if you advertise it as a feature. Nobody's connection gets faster, or higher uptime, instead you just get more complicated support tickets when some website hasn't configured their AAAA records correctly or when some third party's IPv6 support (be that a site online or a home router or an application on the user's laptop) ruins a connection which works fine once the client disables IPv6 support on their end.
I would be overjoyed if the tech actually worked as a drop-in replacement, because that is a crazy amount of addressing space. I would be overjoyed if it were actually backwards compatible instead of a mishmash of dual stack and/or CGNAT nonsense. I would be overjoyed if the originally planned path MTU discovery and IPSEC support got off the ground, but those both died in horrible ugly ways.
As it is, it has picked up enough steam that some time over the next 15 years we will see a tipping point where the incentives to switch finally benefit all parties and the non-mobile industry starts to fall in line. Where customers actually begin to benefit directly and it becomes worth prioritizing this tech above what they currently perceive as more vital. Where a great enough percentage of our middle-mile and last-mile tech finally support it that we can survive migrating wholesale away from the vendors that do not. And where many of the hellacious pain points of trying to support it today will have been further sanded down and be less damaging to power through.
But I said something similar ten years ago and while the needle has moved since then it hasn't moved as far as I would have liked.
Honestly, I think that it is _the_ reason. IPv6 is a terrible protocol, but it could have worked.
The problem was that the early IPv6 migration strategies all focused on getting IPv6 connectivity to clients as fast as possible (like 6to4). And once you got that IPv6, a lot of stuff just stopped working. It became all-or-nothing, and the option to just disable IPv6 and fix all the issues has always been there.
Happy Eyeballs was standardized criminally late, in 2012. For some reason, the idea that a connection can be something tentative was not a part of the mindset at all.
I'm guilty of that as much as everyone else, I remember spending a lot of time working on connection roaming for 3G/WiFi switching.
Happy Eyeballs was a bad fit for network-level implementation exactly because of sockets. But there's nothing inherently app-level about using multiple probing connections in parallel and then selecting the best one. If the IPv6 authors specified this behavior in the original IPv6 RFCs, we would have taken this behavior for granted.
After all, why would you expect every program under the sun to deal with all the possible network protocols? Who on Earth would expect every software to be rewritten to use IPv6 or SCTP?!? We don't have special open() calls for files on floppy disks or RAM disks, after all.
Sockets already have these "automatic" behaviors. For example, for the connection source address selection. Very few programs explicitly select it, although it's possible.
GAI was a late addition, and it's also awkward in itself.
Because there's no way to make that one connection that works well. IPv6 transitional state is fundamentally less stable than IPv4.
If you mean that one connection should be able to use multiple address families and addresses, then yes. It would be great, but at a much higher complexity than just making several connections at once and then discarding all but the first succcessful one.
> Many early modems were acoustic couplers attached to telephone handsets using Velcro — one part was a microphone, the other a speaker. You connected by dialling the phone...
>
> Tools like the Unix-to-Unix Copy Program (UUCP) used scripts that called cu or similar tools to establish connections, then transferred data...
>
> UUCP was how we sent mail, read network news, and received (small) files. For much of the research community, it effectively was the network...
>
> Sockets changed all this.
>
> Source: https://blog.apnic.net/2026/07/28/hooray-for-the-sockets-interface
So much has changed... yet so much has not... All are pure marvels and based on dear miracles...
Sockets are one of those load-bearing abstractions that you rarely think to peek under - like filesystems, or threads, but sometimes you should work below for extreme performance needs.
I disagree with this. Separating lookup from connect(2) or sendmsg(2) is cleaner. If an application doesn't care about the difference, most people are using higher level APIs built on top anyway, and most of those will provide a quick way to hide DNS details from you.
I suppose, but the number of applications for whom a successfully-negotiated bare socket connection (with no TLS or data exchanged) is side-effectful to the point that this causes problems is pretty darn small. I'm sure there are some terrifying counterexamples, but they have to be part of a tiny minority, right?
More concerning is the risk of exhausting server socket resources when probe-based connects don't hang up quickly if they don't want to use a connections. Lots of servers/load balancers aren't well-tuned to force-close connections if the first byte doesn't arrive within a short time.
I agree. But we couldn’t have possibly guessed this in the 80s.
Instead we converged on a very simple primitive that makes no such assumptions and can be composed into a wider variety of high level abstractions in user space.
> More concerning is the risk of exhausting server socket resources when probe-based connects don't hang up quickly if they don't want to use a connections.
TCP connection cookies solved this problem in 90-s! They fell out of use because dedicating a couple MB of RAM to track a few hundred thousand connections is not a big deal anymore.
That's theoretically true. But I don't know any public Internet-facing apps that care about that.
For non-public networks, sure. For things like TCP-to-RS232 connectors that can only service one connection at a time. Happy Eyeballs needs to be disabled for them, but that's just one socket option.
And if the Happy Eyeballs protocol was standardized earlier, these kinds of apps arguably would have adapted to it.
I'd actually say that the original sin of the socket API was taking DNS in higher-level connect methods at all. Users should have to separate the act of address lookup from the act of opening a connection every time (and could write the 5loc utility function to hide that split if they want).
I say that not out of purity concerns, but because of how DNS and TCP work. DNS multirecord selection (or not) should be up to the user. DNS timeouts should be surfaced granularly and differently from e.g. TCP SYNACK timeouts. Once a TCP stream is open, if a novice user opened it conceptually to a domain rather than an address, there's no intuitively correct answer to what happens if the domain's resolution changes while the socket's connected.
Standards as well as all manual pages and other documentation say that getaddrinfo() should be called via a loop by clients. And results show that almost all clients indeed do that. Therefore, on the server-side, you can do round-robin DNS, randomization (as suggested by the post linked), fall-backs, CDN things, and many other things.
Imagine if handling that were a kernel feature completely hidden from you and there was no way to replace it in user mode, or even have visibility into its failure modes. I'd say that's worse.
Telling the socket the IDENTITY of the host you want to reach and abstracting away all the details of how to reach it is exactly what the missing TCP/IP sessions layer is supposed to do. By tightly coupling the session layer to explicit Layer 3 network routing parameters (IP addresses), the classic socket API failed to separate a host's identity from its physical location.
Questions of identity are established at a higher level in user mode by TLS. Pushing all that into the other side of the kernel interface again seems not great.
It's not just about separating lookup and connect: the fundamental stream abstraction is a point-to-point connection. It's possible to construct one-to-many connections on top of that, but it adds a lot of complexity on both sides. Doing it the other way around (emulating one-to-one connections on top of a one-to-many abstraction) means burdening simple connections with a lot of unnecessary complexity.
You can have sockets without DNS. You can pick whatever strategy you want when there are multiple A records. You can use SRV records instead. And most importantly imo, it mirrors the listening API.
Well, yeah. And I also can do IPv6 without any of these new-fangled application-level TCP by using raw sockets.
War story: many years ago I was developing a service that runs computational tasks inside containers. That was when K8s didn't work well with cloud providers.
The service was written in Go and worked on AWS. Everything worked fine for me and for our production deployment. Then people started using it with a Python client that used the REST API directly, and for some reason some people couldn't reach it when they were working from home.
Reason: I misconfigured IPv6 on AWS by not enabling the default route, so all IPv6 connections were failing. Go has HappyEyeballs enabled by default, so it worked just fine.
But Python did not have it back then. So people with IPv4-only connectivity (including our office) had no problems. But people with IPv6 were getting connection failures.
Wouldn't this add a ton of complexity for anything that extends DNS, like DoH, DoT, VPNs, firewalls, etc., and you'd still need to connect by IP for LAN connections or P2P?
Cause what you're asking is not just to do the regular DNS lookup inside `connect`, but to have the OS manage roaming and continuously updating DNS while the connection stays up?
I don't know about that. I don't know. That's a lot of complexity deep in the kernel and ossified.
I'm happy with the QUIC solution that allows you to migrate IPs and it's all userspace. It would be very hard to evolve a network protocol that had to be crammed into 3+ different kernels.
The article kind of misses out on the networking standards of the time - there were many of which TCP and IP were just one, most of the rest were either proprietary (DNA SNA BNA etc) and incompatible, or owned by telecoms who wanted to be able to charge per packet(X.25).
TCP/IP won (IMHO) because the implementation was open source, and because no one was going to make a buck by locking others out (or charging them for the right to build an implementation). Plus there was a working reference implementation available for the taking
While TCP/IP was open source and not controlled by any 1 vendor, quite a few companies made good money from selling TCP/IP stacks and won themselves billion dollar valuations in the process[1][2]. Of course this was during a time when Microsoft saw no point in including an IP stack with their OS, or indeed much point in connecting to the Internet at all! Some of the younger readers may find these things hard to believe.
[1] https://en.wikipedia.org/wiki/FTP_Software [2] https://en.wikipedia.org/wiki/NetManage
Don't forget Trumpet Winsock [1]. I worked for an ISP at the time that provided it on a floppy disk (along with a newsreader, FTP client, telnet client, and web browser). Although I doubt many customers paid for it, Trumpet Winsock was shareware and very widely used on Windows 3.x.
[1] https://en.wikipedia.org/wiki/Trumpet_Winsock
Same story with UNIX and C "success".
Everything else was commercial, while by being prevented to sell UNIX, AT&T was fine with giving its source code to universities for a symbolic price for sending the tapes, and the Lion's book.
It changed quite fast the moment AT&T was allowed to claim ownership, however just like with IBM and PC clones, AT&T could no longer take control over UNIX.
I have two complaints about sockets:
1, bidirectional sockets probably should've been ≥2 fd's, not 1.
2, non-blocking semantics and poll/select suck.
Can't blame anyone in 1983 for not getting "async I/O" right since we're still struggling; at least now we have some decent answers (like io_uring on Linux.)
> non-blocking semantics and poll/select suck.
I'd argue that it's not that the semantics suck, it's that there are too many of them. We have O_NONBLOCK, multiple multiplexers (for sane reasons--I don't begrudge 1980s folks for not thinking about fd set size and copy overhead for select(2) either), and others. What's worse, they don't all work with all FDs--not only are regular files not nonblock-able in the same way that sockets are, but all sorts of other FD-exposed capabilities (signalfds, timerfds, memfds, pidfds) do or don't support nonblocking semantics and multiplexing in all sorts of weird ways.
If those old system designers had stuck with keeping the async IO syscall space small (e.g. "you only get select/poll" or "you only get read(fds) and read_noblock(fds, timeout)") and consistent (by drawing a hard line at "if you expose something as an FD, it must support all APIs that generically handle FDs"), we would have ended up in a better place. Sure, that would have slowed down some implementations (e.g. vfs drivers), but would also have massively sped up development against a lot of these APIs.
Ah, well, hindsight is 20/20 I guess.
Allready in 1983, 4.2BSD had 3 different attempts at doing multiplexed/asynchronous I/O:
Non-blocking I/O: "O_NONBLOCK" with "EWOULDBLOCK" (or "EAGAIN")
Signal-driven I/O: "O_ASYNC" with "SIGIO"
Synchronous I/O multiplexing: "select()"
All 3 had various defects, especially with a large number of concurrent I/O actions.
In UNIX-derived operating systems, after 1983 there have been many other attempts to implement something better than these 3 (starting with System V "poll" and with POSIX AIO, and then with various incompatible approaches in Solaris, FreeBSD and Linux), but none were good enough and most were seriously inferior to methods of doing asynchronous I/O that existed in some IBM and DEC operating systems decades earlier.
In my opinion, only io_uring has finally solved the problem of multiplex asynchronous I/O in Linux, and in a manner much better than in all older operating systems. I consider all the many older alternatives that exist for liburing as obsolete.
This. I'd break-out in a cold sweat if I had to work on this again.
Windows has a better async I/O implementation than Linux and has for decades
No, although this was true until around 2019 when Linux introduced io_uring and finally had a genuine innovation. io_uring wound up giving us a unified asynchronous I/O layer that handles just tons of things. (And it is genuinely a step up from the overlapped I/O APIs of Windows, as it avoids eating syscall transition overhead in many cases.)
Of course, Microsoft then went on to introduce IORing in 2022, continuing their legacy of never being afraid to adopt good ideas. But still - it would be wrong to suggest Linux is currently behind on async I/O. It hasn't been for years.
What would all the other FDs be instead? Like an eventfd or epoll handle, it obviously couldn't be an FD so what would it actually be?
Sorry, what I meant was that FDs that aren’t sockets should be uniformly handled by the async IO syscalls rather than having to parse man pages or trip over EINVALs to figure out the answers to questions like “can I select(2) on a pidfd? Can I O_NOBLOCK a regular on-disk file?”.
For a concrete example, I should be able to select/poll/epoll any file descriptor. The multiplexers can short-circuit return for things where readiness is inherent (e.g. block device files/vfs, I’m not asking for the moon a la waiting for NFS shares to report ready or something).
I should be able to issue non blocking reads on special file descriptors (timerfds, signalfds, eventfds, and so on).
This is analogous to my other unachievable fantasy: everything that exposes a filesystem interface must work with inotify/kqueue/whatever. No exceptions for procfs/NFS/etc.
At some point this fantasizing just becomes me griping about what made it to market/the worse-is-better philosophy generally, though. And yet it moves, I guess!
Or you throw it all out and do Windows-style overlapped I/O instead.
Windows NT had overlapped IO decades before Linux. The Microsoft kernel team were alright.
Windows NT had many things decades before Linux, I've gotten used to it. There are things Windows has that I still sort of wish Linux had, I mentioned one of them just last night (RDP fast user switching/session roaming.)
If we're just strictly talking about low level things, another good one would be synchronization primitives, which I guess we now have some of in Linux verbatim at this point, if only for the sake of emulation. (And of course futex2.)
Windows strengths: IRPs and fantastic async I/O, ETW events, tooling- windbg xperf driver verifier time travel debugging, RDP, doesn’t claim everything acts like a file
Linux strengths: file system, open source ecosystem, ssh, lockless algorithm support
I don't think the kernel prohibits fast user switching but you'd have to implement it at the desktop layer too.
Think of ctrl-alt-f7, f8 etc. This still works today, you can run X servers as different users on different VT numbers and switch between them. All that's missing is better UI.
The kernel definitely doesn't, although the VT layer kind of sucks and probably should've never been used to multiplex graphical sessions. (I mean, I get why it was, but oh well.)
I was talking about this:
https://news.ycombinator.com/item?id=49093002
And it winds up being mostly a thing you have to implement in the compositor, but none of them do so far to my knowledge.
And of course the same issue applies if you wanted multiple physical seats juggling sessions between them.
That's systemd limitations. Who even uses systemd?
> That's systemd limitations.
It's kind of the opposite, systemd-logind provides better seat management than without. I attempted to implement the same concept using libseat and seatd unsuccessfully though I do not remember exactly what I got caught on. (not in kwin but a toy compositor; I have been at this problem for a little while now.)
That said, most desktop systems have only implemented systemd-logind in a limited way so far that basically just does what the VT system + display manager was already doing, that's what would need to be worked on in order to make this a reality. I was able to accomplish a prototype with only patches to Kwin, kfreerdp and plasma-login-manager, no need to patch systemd-logind or anything like that.
> Who even uses systemd?
Well for one thing, almost all of the major distributions; Ubuntu, Debian, Fedora/RHEL, Arch Linux, NixOS, openSuSE? and of course SteamOS now.
The only major non-systemd Linux distributions I can think of are Alpine Linux and Gentoo. And Android if you wanted to count that, but I think it is special enough to be considered its own OS that just is Linux-based.
Then it's libseat and seatd limitations. Why rely on a system that doesn't do what you want? Bypass it and DIY what you actually want.
I personally use a systemd-based distro (NixOS) and logind is able to handle this use case. The point of trying to support libseat was for the sake of non-systemd distros. (Though elogind would probably work too.)
I certainly wouldn't DIY this; nothing would support my DIY version, which would defeat the purpose.
The Linux platform gives you full control. If the box you seek doesn't exist, you have to create your own box instead of thinking inside someone else's box that doesn't even do what you need.
Mechanisms equivalent with the overlapped I/O of Windows NT (1993) already existed 30 years earlier, e.g. in IBM OS/360 and PL/I (1964/1965).
The main features that were better in Windows NT than in the UNIX-derived operating systems were inherited from the DEC VAX/VMS operating system (1978) (e.g. WaitForMultipleObjects) and a part of them had been inherited from the even earlier operating system DEC RSX-11M (1974-11) (Dave Cutler also had a major role in those operating systems, so there is nothing surprising about this; Microsoft had to pay a big compensation to DEC, on the order of $ 100M, for the features that were obviously taken from the DEC operating systems).
UNIX was very simplified in comparison with the operating systems that were used on bigger computers, and for some things its derivatives never caught up with those older systems, except after 2000. Only "futex" (2002) and "io_uring" (2019) have advanced the state of the art clearly beyond what already existed for AIO on the IBM mainframes around 1964/1966.
To be fair, I think that was the entire challenge in the first place: the contemporary microcomputers were relatively limited on what they could do, so you couldn't necessarily afford to just do everything mainframes were doing. More importantly, it's not clear you would've wanted to. Having the taste and good sense to know what to "steal" and how to "steal" it was already hard enough work. A lot of good ideas took a while to come over, but a lot of probably bad ideas also never did.
It's an evergreen interesting fact that Dave Cutler took perhaps a bit too much inspiration from prior VMS work to NT, but I also think there's no need to put an asterisk on it anymore than there is UNIX or anything else; it does ultimately stand on its own. Like great artists, great software architects steal.
I’m not sure you can call anything in Linux/Unix as equivalent to overlapped I/O in Windows given that there aren’t any IRPs
Including the use of systems programming languages with bounds checking, PL/I, PL.8, PL/S, NEWP, ...
I just can’t understand why the Linux kernel has avoided tracking completions for so long and has such hard time with async I/O. Windows has had IRPs and overlapped I/O for decades, asyc is the default and there just isn’t much reason to ever use synchronous I/O.
3, fds should have been UUIDs
fds should have been IPv6 addresses. I will not elaborate on why or how.
It's such a common problem to write to a stale file descriptor. It may be closed which is fine, at least you will have an obvious error. But what's much worse is when the stale descriptor actually works, because something else in the program opened a new fd and your stale descriptor is not stale anymore, it just points at something unexpected.
Opaque handles are often a blessing.
Maybe UUID would be overkill (it certainly would have been back in the day of 16-bit unix machines) but something fairly large would have helped a lot.
Even just incrementing and rolling over a 16-bit value would have helped, instead of situations such as closing stdout and stderr, opening a file, and now random error logs sprinkle into your PDF or whatever.
Yes, "you are holding it wrong", but it should be hard to "hold it wrong", not easy.
Joking aside I pretty much agree with this. Windows NT Object handles work pretty well. I assume they are pointers of some kind under the hood, but either way a machine word is probably large enough as it seems unlikely you would ever need to handle more open handles than you have bytes of addressable memory.
Sockets won over over-engineered monstrosities like STREAMS or X.whatever
And yet, sockets are a terrible interface. They don't provide a way to get the details of the underlying connection for features like migration, checkpointing, or introspection. E.g. there is no way to get the current sequence number for TCP (there is "connection repair" mode now, but it's Linux-specific).
Well, you can say that sockets abstract the low-level details, but then these details hit you in the face when you need to do protocol-specific name resolution.
I now believe that we could have switched to something like IPv6 two decades ago if the socket interface simply allowed binding multiple address families to one socket and handled the name resolution internally.
Sockets also cemented the "one connection - one address" model that is _still_ dragging back the IPv6 adoption. MPTCP or QUIC are still barely supported.
I don't think that's the reason IPv6 adoption has been slow. It's because ISPs and telcos and data centers and everyone else are slow to upgrade hardware and software, because they don't care about it and because it costs money (money that doesn't get them more money from their customers), not because it's hard.
I'm the network admin for a small ISP that hasn't deployed IPv6 yet.
We do have ARIN IPv6 space, but we also have thousands of customers who do not know what that is or care about it either. Twice in twenty years now we have had a customer ask about it, and we have offered to work with them to help pilot support through our network (since if they have a need then that need could act as a standard against which to measure support) but always within a day they simply find some other way to satisfy their need without IPv6.
"Get more money from their customers" aside, we do still have a duty to prioritize the needs of our customers, and I'm not pulling budget away from initiatives to improve availability or increase bandwidth or lower latency simply to chase a buggy moving target technology with no light at the end of the tunnel by way of material improvement for our customers.
No "killer app". <1% marketing draw if you advertise it as a feature. Nobody's connection gets faster, or higher uptime, instead you just get more complicated support tickets when some website hasn't configured their AAAA records correctly or when some third party's IPv6 support (be that a site online or a home router or an application on the user's laptop) ruins a connection which works fine once the client disables IPv6 support on their end.
I would be overjoyed if the tech actually worked as a drop-in replacement, because that is a crazy amount of addressing space. I would be overjoyed if it were actually backwards compatible instead of a mishmash of dual stack and/or CGNAT nonsense. I would be overjoyed if the originally planned path MTU discovery and IPSEC support got off the ground, but those both died in horrible ugly ways.
As it is, it has picked up enough steam that some time over the next 15 years we will see a tipping point where the incentives to switch finally benefit all parties and the non-mobile industry starts to fall in line. Where customers actually begin to benefit directly and it becomes worth prioritizing this tech above what they currently perceive as more vital. Where a great enough percentage of our middle-mile and last-mile tech finally support it that we can survive migrating wholesale away from the vendors that do not. And where many of the hellacious pain points of trying to support it today will have been further sanded down and be less damaging to power through.
But I said something similar ten years ago and while the needle has moved since then it hasn't moved as far as I would have liked.
Honestly, I think that it is _the_ reason. IPv6 is a terrible protocol, but it could have worked.
The problem was that the early IPv6 migration strategies all focused on getting IPv6 connectivity to clients as fast as possible (like 6to4). And once you got that IPv6, a lot of stuff just stopped working. It became all-or-nothing, and the option to just disable IPv6 and fix all the issues has always been there.
Happy Eyeballs was standardized criminally late, in 2012. For some reason, the idea that a connection can be something tentative was not a part of the mindset at all.
I'm guilty of that as much as everyone else, I remember spending a lot of time working on connection roaming for 3G/WiFi switching.
there is no protocol specific name resolution. getaddrinfo tells you which type of socket you need to connect to the remote endpoint!
Yes, there is. GAI was standardized in 2002, so it's not a part of the original sockets. And it still does not support Happy Eyeballs.
It does return a list of addresses you should try, in preference order, and possibly of different types.
I think Happy Eyeballs can't be implemented at that layer. It's inherently involved up to layer 7.
Happy Eyeballs was a bad fit for network-level implementation exactly because of sockets. But there's nothing inherently app-level about using multiple probing connections in parallel and then selecting the best one. If the IPv6 authors specified this behavior in the original IPv6 RFCs, we would have taken this behavior for granted.
After all, why would you expect every program under the sun to deal with all the possible network protocols? Who on Earth would expect every software to be rewritten to use IPv6 or SCTP?!? We don't have special open() calls for files on floppy disks or RAM disks, after all.
Sockets already have these "automatic" behaviors. For example, for the connection source address selection. Very few programs explicitly select it, although it's possible.
GAI was a late addition, and it's also awkward in itself.
Why would you implement multiple concurrent connections at the network level instead of making the one single connection work well?
Because there's no way to make that one connection that works well. IPv6 transitional state is fundamentally less stable than IPv4.
If you mean that one connection should be able to use multiple address families and addresses, then yes. It would be great, but at a much higher complexity than just making several connections at once and then discarding all but the first succcessful one.
> there's no way to make that one connection that works well
Really? With unlimited ability to tamper with the stuff below the sockets layer, you couldn't make a reliable network? I'm surprised.
agree they are horrible especially if you want to chase the bug
The Plan9 network API is an interesting successor to all of STREAMS, streams, and sockets.
Sockets are one of those load-bearing abstractions that you rarely think to peek under - like filesystems, or threads, but sometimes you should work below for extreme performance needs.
Go bears!
The socket interface using IP addresses instead of DNS names is widely considered to be a major mistake.
I disagree with this. Separating lookup from connect(2) or sendmsg(2) is cleaner. If an application doesn't care about the difference, most people are using higher level APIs built on top anyway, and most of those will provide a quick way to hide DNS details from you.
https://blog.ipspace.net/2009/08/what-went-wrong-socket-api/
A DNS name can have a LOT of A records associated but a socket has to pick one. This is a severe limitation.
A socket has to pick one, but your application doesn’t.
You can hedge your bets and open a connection and send data to _all_ of them, and pick whichever returns faster.
This of course requires you to know about the application-layer protocol.
For HTTP, you need to restrict this strategy to GET methods, for example.
Hence why it can’t be part of the socket interface.
You can always build higher level abstractions on top of sockets if you need them.
You absolutely can establish multiple TCP connections for POST methods. You just can't send any _data_.
This is even used by browsers, this trick even has a slightly creepy name: "Happy Eyeballs".
That’s true for HTTP.
TCP does not require that the act of connecting must be pure, so you cannot indiscriminately apply Happy Eyeballs at the socket API level.
I suppose, but the number of applications for whom a successfully-negotiated bare socket connection (with no TLS or data exchanged) is side-effectful to the point that this causes problems is pretty darn small. I'm sure there are some terrifying counterexamples, but they have to be part of a tiny minority, right?
More concerning is the risk of exhausting server socket resources when probe-based connects don't hang up quickly if they don't want to use a connections. Lots of servers/load balancers aren't well-tuned to force-close connections if the first byte doesn't arrive within a short time.
I agree. But we couldn’t have possibly guessed this in the 80s.
Instead we converged on a very simple primitive that makes no such assumptions and can be composed into a wider variety of high level abstractions in user space.
> More concerning is the risk of exhausting server socket resources when probe-based connects don't hang up quickly if they don't want to use a connections.
TCP connection cookies solved this problem in 90-s! They fell out of use because dedicating a couple MB of RAM to track a few hundred thousand connections is not a big deal anymore.
Linux will still revert to cookies when exceeding a certain number of half open connections, preventing DoS.
That's theoretically true. But I don't know any public Internet-facing apps that care about that.
For non-public networks, sure. For things like TCP-to-RS232 connectors that can only service one connection at a time. Happy Eyeballs needs to be disabled for them, but that's just one socket option.
And if the Happy Eyeballs protocol was standardized earlier, these kinds of apps arguably would have adapted to it.
I'd actually say that the original sin of the socket API was taking DNS in higher-level connect methods at all. Users should have to separate the act of address lookup from the act of opening a connection every time (and could write the 5loc utility function to hide that split if they want).
I say that not out of purity concerns, but because of how DNS and TCP work. DNS multirecord selection (or not) should be up to the user. DNS timeouts should be surfaced granularly and differently from e.g. TCP SYNACK timeouts. Once a TCP stream is open, if a novice user opened it conceptually to a domain rather than an address, there's no intuitively correct answer to what happens if the domain's resolution changes while the socket's connected.
When it was written I don't think they had DNS. Maybe they had hosts files maybe not.
> This is a severe limitation.
Rather, it is a feature.
How so?
> How so?
Standards as well as all manual pages and other documentation say that getaddrinfo() should be called via a loop by clients. And results show that almost all clients indeed do that. Therefore, on the server-side, you can do round-robin DNS, randomization (as suggested by the post linked), fall-backs, CDN things, and many other things.
Imagine if handling that were a kernel feature completely hidden from you and there was no way to replace it in user mode, or even have visibility into its failure modes. I'd say that's worse.
Telling the socket the IDENTITY of the host you want to reach and abstracting away all the details of how to reach it is exactly what the missing TCP/IP sessions layer is supposed to do. By tightly coupling the session layer to explicit Layer 3 network routing parameters (IP addresses), the classic socket API failed to separate a host's identity from its physical location.
Questions of identity are established at a higher level in user mode by TLS. Pushing all that into the other side of the kernel interface again seems not great.
https://en.wikipedia.org/wiki/Locator/Identifier_Separation_...
I've used this in networks and it enables amazingly easy endpoint mobility.
It's not just about separating lookup and connect: the fundamental stream abstraction is a point-to-point connection. It's possible to construct one-to-many connections on top of that, but it adds a lot of complexity on both sides. Doing it the other way around (emulating one-to-one connections on top of a one-to-many abstraction) means burdening simple connections with a lot of unnecessary complexity.
Absolutely not.
You can have sockets without DNS. You can pick whatever strategy you want when there are multiple A records. You can use SRV records instead. And most importantly imo, it mirrors the listening API.
Raw addresses could have been an advanced option, rather than a requirement for every program.
I feel like this wouldn’t be a problem if we had decent APIs for name resolution (separate from sockets) instead.
The ones we have suck or are non-standard.
Nope. They have to be coupled for the Happy Eyeballs to work.
Another option is for sockets to accept multiple addresses and address families in "connect" calls, so that only one connection wins.
No, they don’t. They aren’t coupled today, and applications implementing Happy Eyeballs work perfectly fine on top of sockets and a DNS client.
Well, yeah. And I also can do IPv6 without any of these new-fangled application-level TCP by using raw sockets.
War story: many years ago I was developing a service that runs computational tasks inside containers. That was when K8s didn't work well with cloud providers.
The service was written in Go and worked on AWS. Everything worked fine for me and for our production deployment. Then people started using it with a Python client that used the REST API directly, and for some reason some people couldn't reach it when they were working from home.
Reason: I misconfigured IPv6 on AWS by not enabling the default route, so all IPv6 connections were failing. Go has HappyEyeballs enabled by default, so it worked just fine.
But Python did not have it back then. So people with IPv4-only connectivity (including our office) had no problems. But people with IPv6 were getting connection failures.
Unix sockets kinda already connect by name.
Use getaddrinfo
Exactly
By who? I haven't seen this argument before.
https://blog.ipspace.net/2009/08/what-went-wrong-socket-api/
Wouldn't this add a ton of complexity for anything that extends DNS, like DoH, DoT, VPNs, firewalls, etc., and you'd still need to connect by IP for LAN connections or P2P?
Cause what you're asking is not just to do the regular DNS lookup inside `connect`, but to have the OS manage roaming and continuously updating DNS while the connection stays up?
I don't know about that. I don't know. That's a lot of complexity deep in the kernel and ossified.
I'm happy with the QUIC solution that allows you to migrate IPs and it's all userspace. It would be very hard to evolve a network protocol that had to be crammed into 3+ different kernels.
[dead]