I remember learning about ELF files first because that’s how you’d run pirated PS2 games. Funny how my insatiable appetite for games in my teens resulted in learning so much about Linux executable files and eventually it seemed inevitable that I should just learn to code.
Funny how a lot of us got into computers that way. For me, it was wanting to play Prince of Persia and other DOS games on my cousin's PC when he wasn't around. Figured out what CD and DIR did and how I could run different games by varying the commands. In a few years I was whipping up my own game launcher using AUTOEXEC.BAT, which got me into scripting. I learned to love DOS, and so the eventual transition to Linux was easy for me as I already a CLI fan and I was blown away with how much more powerful the Linux terminal was. It was basically love at first sight.
I think static executables will still be mostly loaded by the kernel; when you have a binary with PT_INTERP it will load that instead, but that executable still needs to be loaded in by the elf binfmt. Unless I just entirely missed what you were talking about from the article, which is surely possible, though I double checked and I don't see it implying that static binaries are loaded by userspace.
To me this whole thing is interesting since it essentially requires ELF loading to be duplicated between the kernel and libc, and then possibly duplicated again for libdl vs ldlinux. Seems unideal. (Though nothing new. Pretty sure it's been like that for decades by this point.)
The ELF loading logic in the Linux kernel is intentionally very simple, so it's more like a bare-bones subset of what the dynamic linker handles. matheusmoreira summarizes it well in a previous discussion [0]:
> Yeah it turns out the kernel doesn't care about sections at all. It only ever cares about the PT_LOAD segments in the program header table, which is essentially a table of arguments for the mmap system call. Sections are just dynamic linker metadata and are never covered by PT_LOAD segments.
The simplicity of the ELF loader in Linux can be exploited to make extremely small executables [1], since most of the data in the ELF header is stuff that the kernel doesn't care about.
> essentially requires ELF loading to be duplicated between the kernel and libc, and then possibly duplicated again for libdl vs ldlinux. Seems unideal.
Oh.
I liked the way QNX did it. Loading was done by a .so file, entirely by userspace. When you built a kernel boot image, you could include whatever userspace programs and .so files were needed to get started, as raw memory images. They were all loaded by the boot loader. That included the .so file with the code for loading programs. All loading and preprocessing of executable images was done entirely in user space.
It looks like Linux now has similar capabilities, but the old cruft remains. This is typical of Linux migration of machinery to user space. The kernel doesn't seem to shrink.
I think this is how it has been since the beginning of ELF in Linux. PT_INTERP comes from the original TIS specification of ELF and I think it was probably also in the original SVR4 ELF implementation.
I understand why they went this route. While it is unfortunate to need duplicate code parsing and loading ELF files, the ELF binfmt in the kernel is at least relatively simple, since it does not need to worry about dynamic linking. Doing what QNX did would be possible, but it would also add moving parts and change the relationship Linux has with the userland, which is one thing they do not like to do. They could probably come up with a middleground, like pre-baking a raw memory image with an ELF loader that can be stuck into a new process when exec'ing an ELF binary and shipping that with the kernel, but I'm sure there would be observable side-effects with regards to performance and maybe locks, I can see it being more impactful to focus on ensuring the existing implementation is correct. (AFAIK it is still "only" a few thousand lines.)
I remember learning about ELF files first because that’s how you’d run pirated PS2 games. Funny how my insatiable appetite for games in my teens resulted in learning so much about Linux executable files and eventually it seemed inevitable that I should just learn to code.
Funny how a lot of us got into computers that way. For me, it was wanting to play Prince of Persia and other DOS games on my cousin's PC when he wasn't around. Figured out what CD and DIR did and how I could run different games by varying the commands. In a few years I was whipping up my own game launcher using AUTOEXEC.BAT, which got me into scripting. I learned to love DOS, and so the eventual transition to Linux was easy for me as I already a CLI fan and I was blown away with how much more powerful the Linux terminal was. It was basically love at first sight.
We once called that “image activation” before the Industry was taken over by human LLMs in the wake of the dot-com crash.
FreeBSD still uses this term. Eg, the elf code lives in sys/kern/imgact_elf.c
Pretty sure only VMS used that term.
Oh, nice. Did not know that executable image processing had moved to user space. Does this eliminate kernel crashes from malformed executables?
I think static executables will still be mostly loaded by the kernel; when you have a binary with PT_INTERP it will load that instead, but that executable still needs to be loaded in by the elf binfmt. Unless I just entirely missed what you were talking about from the article, which is surely possible, though I double checked and I don't see it implying that static binaries are loaded by userspace.
To me this whole thing is interesting since it essentially requires ELF loading to be duplicated between the kernel and libc, and then possibly duplicated again for libdl vs ldlinux. Seems unideal. (Though nothing new. Pretty sure it's been like that for decades by this point.)
The ELF loading logic in the Linux kernel is intentionally very simple, so it's more like a bare-bones subset of what the dynamic linker handles. matheusmoreira summarizes it well in a previous discussion [0]:
> Yeah it turns out the kernel doesn't care about sections at all. It only ever cares about the PT_LOAD segments in the program header table, which is essentially a table of arguments for the mmap system call. Sections are just dynamic linker metadata and are never covered by PT_LOAD segments.
The simplicity of the ELF loader in Linux can be exploited to make extremely small executables [1], since most of the data in the ELF header is stuff that the kernel doesn't care about.
[0] https://news.ycombinator.com/item?id=45706380#45709203
[1] https://www.muppetlabs.com/~breadbox/software/tiny/teensy.ht...
Yep, good points. FWIW I do share roughly the same sentiment despite how I worded that last part of my post.
> essentially requires ELF loading to be duplicated between the kernel and libc, and then possibly duplicated again for libdl vs ldlinux. Seems unideal.
Oh.
I liked the way QNX did it. Loading was done by a .so file, entirely by userspace. When you built a kernel boot image, you could include whatever userspace programs and .so files were needed to get started, as raw memory images. They were all loaded by the boot loader. That included the .so file with the code for loading programs. All loading and preprocessing of executable images was done entirely in user space.
It looks like Linux now has similar capabilities, but the old cruft remains. This is typical of Linux migration of machinery to user space. The kernel doesn't seem to shrink.
I think this is how it has been since the beginning of ELF in Linux. PT_INTERP comes from the original TIS specification of ELF and I think it was probably also in the original SVR4 ELF implementation.
I understand why they went this route. While it is unfortunate to need duplicate code parsing and loading ELF files, the ELF binfmt in the kernel is at least relatively simple, since it does not need to worry about dynamic linking. Doing what QNX did would be possible, but it would also add moving parts and change the relationship Linux has with the userland, which is one thing they do not like to do. They could probably come up with a middleground, like pre-baking a raw memory image with an ELF loader that can be stuck into a new process when exec'ing an ELF binary and shipping that with the kernel, but I'm sure there would be observable side-effects with regards to performance and maybe locks, I can see it being more impactful to focus on ensuring the existing implementation is correct. (AFAIK it is still "only" a few thousand lines.)