Hey, so I built this thing, most of it at so far at least. And yeah, right now it isn't doing many things better than Homebrew.
Setting of relative paths for bottle installs is still not perfect, well it works for every bottle I have tested except rust. Getting bottles working 100% is very doable though imo.
Build from source formulae is still pretty f*ed + I do not know if it is really feasible given that the json API lacks information there and a full on Ruby -> Rust transpiler is way out of scope. Will probably settle for automatic build system detection based on archive structure there. + Maybe do my own version of the .rb scripts but in a more general machine readable format, not .rs lol
Casks seem to work but I have only tested some .dmg -> .app ones and .pkg installers so far though. As with bottles 100% doable.
Given that almost all formulae are available as bottles for modern ARM mac this could become a fully featured package manager. Actually didn't think so many people would look at it, started building it for myself because Homebrew just isn't cutting it for what I want.
Started working on a declarative package + system manager for mac because I feel ansible is overkill for one machine and not really made for that and nix-darwin worms itself into the system so deep. Wrapping Brew commands was abysmally slow though so I started working on this and by now I am deep enough in I won't stop xD
Anyway I am grateful for every bug report, Issue and well meaning pull request.
One of my biggest gripes about brew is how they manage dependencies. The devs have a conflicting philosophy that creates bloat. Package maintainers must define settings, settings should always use the latest Python version. It makes no sense. Maintainers won't update unless things break so you got a bunch of Python versions running around. And it won't use system Python!
Uv seems to provide an elegant solution for this. You can build a venv for each package and rust version will only have the specified deps. Since uv finds all your Python instances (and packages) and soft links them you have way less bloat and venvs become really useful. You can also use run and other tools to handle executables
You should only be using Homebrew for installing software that happens to be written in Python, not dependencies for your own Python projects. If you do that, the Python version does not matter, it's just whatever version is required to make the package work.
You mentioned a declarative package manager for Mac. I've really liked using Homebrew Bundle [1] over the last couple years. It's about the level of declarative that I've wanted and has made it really easy to bootstrap new laptop or VM (since it also works on Linux). The format for a Brewfile was pretty easy to figure out.
The way I ended up using it was that `brew install` would temporarily install something, without adding it to my Brewfile. And a little `brew add` wrapper would add the package to my Brewfile to keep it on the system permanently. That part with the wrapper could have used some love and would be a nice fit for a new brew-compatible frontend IMO. Maybe you could expand on that for Sapphire, if that also scratches your declarative itch?
What makes you interested in a rust implementation of brew?
I'm guessing it's that you hoping that it is eventually more performant -- are there specific areas of current brew you have identified as performance bottlenecks likely to eventually benefit from a rust implementation?
Or any more info to share about assumptions/hopes that motivated this or any other motivations?
Building from source, obviously, will never be really that much more performant as it mainly relies on the underlying build systems and things like ninja, cmake, cargo etc. are usually optimized very very well.
Thanks to rust just being (slightly, significantly? no idea about ruby's speed) faster + concurrent downloading & pouring of bottles, most "regular" formula installs feel a good bit faster than brew already. Mainly noticeable when installing multiple formulae at once.
Casks, especially those with pkg installers, seem to profit a bit less here.
Performance was a reason, not the main one though, like I said I wanted and still want, to build a declarative package + system managing solution on top. The idea was to get into rust with that. Imo having the base written in the same language instead of wrapping commands also gives more flexibility there.
Another reason is that I never liked the way brew looks and feels. Right now the ui/ux for Sapphire is far from finished, more like a clusterf*k and only the search command really looks the way I want it. Aiming for something modern, clean and information rich without beeing overly verbose. I really like dnf5 and what AerynOS is doing and will probably take some inspiration there.
Like mentioned, Bottles and Casks should be 100% doable and that would cover most package needs on macOS, I do not see why I should also define a new repo and packaging ecosystem when such a big and popular one exists.
Source build capability will probably stay(for easy integration of source building in the system management part later) but not be focused on brew formulae as the ruby dsl would be a horror to parse.
Well and sh*t I am not trying to compete really. This is the first time building something with rust and I really really had no idea what a giant never ending rabbit hole macOS package management is and how massive and complex Brew is.
This went from should I to can I pretty quick for me xD
I say this from ignorance, but coming from a lineage of linux package managers; brew must be doing something wrong - and upon immediate introspection I doubt that its language specific.
The performance of apt/dnf in comparison is surreal; but dnf (or at least yum, its predecessor) is written in Python; which has even worse performance characteristics than Ruby.
Clearly something is wrong, I wonder how different they are architecturally.
Have you tried Homebrew in the last year or so? I think a lot of people have an impression of Homebrew's performance from the "bad old days," i.e. back when Homebrew had to evaluate every single local formula file to perform any operations at all.
(There's still low handing fruit, but it's not like it was a few years ago where `brew list` took seconds to run. It now runs nearly instantaneously for me locally, like most of the other happy path commands.)
Yes, I use it on my daily driver, and it's at least two-orders of magnitude slower than apt.
However, the speed increase coincided with my upgrade to an M-Series laptop, so it's possible I just presumed there was a significant hardware speedup in the time we're talking about.
Another performance characteristic that used to piss me off when I used brew was that at least for one program, it had to invoke brew to launch. Something about finding a path. So that's an entire ruby interpreter starting and stopping just to (in this case) run a bash script. Horribly slow.
One big difference is that apt and dnf are both binary package management systems, whereas Homebrew is a source-based build system with binary packages (“bottles”) added on top.
If I may surface one use case: Several years ago I had to manage a bunch of Macs for CI jobs. The build process (Unreal's UAT) didn't support running more than one build process at a time, and Docker was really slow, so I'd hoped to use different user accounts to bypass that and get some parallelization gains. Homebrew made that very difficult with its penchant for system-wide installs. So a feature request: I'd love to see a competitive package manager that limits itself to operating somewhere (overridable) in the user's home directory.
In many cases the build output also has hardcoded paths unfortunately
so doing `brew install` inside a container with the proper volumes it’s not sufficient to fix the issue. Everything would have to run from within the container as well.
Initial idea for this really came from my dayjob too, we have macs but no way to centrally manage them. The client / server part for the declarative system manager I want to build on top of this is quite far out yet though. At least several months
Just FYI you are describing a ports system, of which there are many. From macports, to alpine packages, to nixpkgs... Tons of research and automation in this area already. Would encourage looking at Nix and distri in particular. Many Nix packages can be built on/for macOS!
> probably settle for automatic build system detection based on archive structure there
Please add knobs for the end user to manually configure this per package and global default before adding autodetection. As a user to is very frustrating to have to patch the package manager to override some well-intentioned automagic which didn't consider my setup or dig through sources to uncover some undocumented assumption. yarn is a cautionary example.
Looked at that. I do not really think implementing something like artichoke or rutie would be a good idea. I do not want my project to become overly bloated and to achieve my real goal of a declarative system management thing I think sticking to bottles (that cover almost all formulae thanks to the amazing homebrew community) and casks, getting those to work 100% is the better approach. Thank you for the suggestion though!
Good luck. Homebrew hasn't cut it since they started disallowing custom compile flags, and making the user jump through hoops if they want anything slightly different.
With my Homebrew hat on, but not speaking for others: I think this is pretty cool, and demonstrates something that we've discussed indirectly for years.
At its core, there are really two parts to Homebrew:
1. There's the client side, i.e. `brew`, which 99.9% of users stick to happy paths (bottle installs, supported platforms) within. These users could be supported with relative ease by a small native-code installer, since the majority of the work done by the installer in the happy path is fetching bottles, exploding them, and doing a bit of relocation.
2. There's literally everything else, i.e. all of the developer, repository, and CI/CD machinery that keeps homebrew-core humming. This is the largely invisible infrastructure that makes `brew install` work smoothly, and it's very hard to RIIR (in a large part because it's tied heavily to the formula DSL, which is arbitrary Ruby).
(1) is a nice experimental space, because Homebrew does (IMO) a decent job of isolating the client-facing side from the complexity of (2). However, (2) is where the meat-and-potatoes of packaging happens, and where Homebrew's differentiators really lie (specifically, in how easy it is to contribute new packages and bump existing ones).
Edit: Another noteworthy aspect here around performance: I mentioned this in another comment[1], but parallel downloads of things like bottles and DMGs is not an architectural limitation of Homebrew itself, but instead a conscious decision to trade some install speed for courtesy towards the services we fetch from (including GitHub itself). Smaller projects can sidestep this because they're not directing nearly the same degree of volume; I think this project will discover if/when its volumes grow that it will need to serialize downloads to avoid being throttled or outright limited.
It's not _just_ the infrastructure that is awesome for homebrew. The help I got from the team, answering in real-time when I didn't know how to get through the CI bugs is amazing. It's the boring maintainence work that makes it so special for me.
I also feel that there could be a lot of automation in the backend part, catching bugs early (maybe even on local machine before CI run) for example.
> but parallel downloads of things like bottles and DMGs is not an architectural limitation of Homebrew itself, but instead a conscious decision to trade some install speed for courtesy towards the services we fetch from
That doesn't make sense. As you say you're directing a huge volume of traffic so it makes no difference exactly when a user downloads a byte. It all gets smeared out. Only the total amount of data matters and that is unaffected by parallelism.
> As you say you're directing a huge volume of traffic so it makes no difference exactly when a user downloads a byte. It all gets smeared out.
Homebrew's traffic pattern is not a uniform distribution. Package updates go out, and users download those packages in structured ways: there are spikes for MDM-managed Homebrew installations, spikes for cronjobs and CI/CD systems, spikes at 9AM on different coasts when developers sign into their machines, etc.
(How much this matters is also not uniform: it matters somewhat less for GitHub Packages - they should have a hefty CDN - and it matters somewhat more for Casks, which tend to be large DMGs hosted on individual servers.)
Definitely sounds like one of those "I don't want to do it because it's work, but I don't want to admit that so here's a spurious technical reason" excuses to me.
If spikes at midnight etc are an issue just automatically disabled parallel downloads around midnight. Or only use them when running in a terminal. I really doubt it is an issue for GitHub though.
Yeah, to me the language (ruby) has nothing to do with the success of Homebrew, it is entirely about the DX around backend packaging. The main reason I switched from MacPorts to Homebrew back in the day is because when a new version of [insert software here] was released, I could expect the brew package to be updated within days. MacPorts always lagged significantly behind, and oftentimes didn't have certain packages at all.
As best I can tell, that's partially due to the size of the community, but also because Homebrew went out of their way to teach the system to fish via the 3(?) different kinds of "check for updates" built into the Formulae:
When Homebrew started (and when I switched) the community was significantly smaller than MacPorts. It is due to the system you mention (and the community that this MX in turn cultivated) that allowed brew to dominate the mac packaging market.
Absolutely MacPorts and Sapphire can adopt the same strategy, but the point is that brew already has, so what exactly would the benefit be? e.g. if the language of choice is effectively meaningless, re-writing homebrew in Rust serves effectively no purpose. This is contrasted with systems and software where the performance or correctness is the most important feature, and therefore RIIR can be a big win.
Said another way: Brew was not "re-write MacPorts in Ruby", it had much loftier goals which it then executed on effectively. Sapphire mostly seems to be "re-write Brew in Rust", without much beyond that. So the only real gain is a bit more performance out of the CLI.
I completely agree. I think Ruby has actually been a pretty great boon of a language choice because of the formula DSL, but that goes back to the DX point and not something fundamental to the language.
Merely for your consideration, Ruby is great if you live in that codebase all the time, but Ruby -- in general, not just Homebrew -- has indescribably bad discoverability. I mean that both in terms of "what can I write where my cursor is?" and its friend "where the hell did this symbol under my cursor come from?"
I'm acutely aware someone's going to say LSP something or Rubymine something else but as for drive-by contributions, ... anyway, like I said, just for your consideration
I'm not a big fan of keeping the Homebrew terminology though. I never know what a formula, keg, cask, cellar, tap or bottle is. Why not keep to the standard terms of package and repository etc? I don't know beer brewing terminology or how beer brewing is analogous to package management, and I honestly wish that it wasn't something which my tools expect me to learn.
Please kill the Homebrew terminology if you can! Its idiosyncratic names are the bane of my existence; it might have been cute in 2010, but it's frustrated me ever since.
I don't want to memorise their twee names; I'd much rather the name tell me what the entity / operation does by itself.
Cute names for standard things is one of my software bug bears, i.e. pet peeves, i.e. annoyances. Ruby gems and rust crates and something something beans. It is cute jargon and it annoys me to hide the definition inside a language specific terminology.
It's actually one of the main reasons I landed on using Axum for a web server in Rust instead of Rocket: I got fed up with the additional level of semantic indirection the cutesy names added. I didn't wanna burn brain cycles decoding what it means to install fairings or launch rockets, or what "ignition" means contra "liftoff". I like boring names for things.
It’s not that I like boring. But I really like descriptive names. I have other things to do with my time than figuring out what the hell a cask, a tap, or a bottle is. Like solving the problem that requires the damn software.
As an experienced homebrewer, I don't think knowing about making beer makes it any clearer to me. Why do I need to install Docker from a Cask? Oh, it's because on Mac, a Cask is actually a Mac package (DMG or PKG or something). It's just arbitrary beer-flavored terminology.
No different than the Windows registry, which apparently uses a honeybee / hive metaphor because some Windows dev hated bees and their teammates liked trolling them.
That one was non-obvious to me as well, but I am sympathetic to the need to call them something. Not trying to bikeshed, but if there was a single obvious name for them then you would have used it instead of “DMG or PKG or something”.
That said, the registry has been around for ~30 years and the terminology is well known to Windows users. It didn't build off of previously available terminology.
Homebrew just made shit up on the spot based on the project's name to be internally consistent with itself.
Microsoft has it's own fsckery of randomly named cute crap elsewhere, of course.
I will probably change the terminology at some point for this. For initially building it it is way easier to keep them the same though as trying to understand brew itself (at least for me) isn't so straight forward. It's a massive project
Yes! A thousand times yes. Homebrew terminology makes absolutely zero sense whatsoever and I find it irritating when I encounter it every now and then (like when a package says tap something and then install it blah blah)
I wish they’d just call them binaries, macOS packages, packages, gui-packages, etc.
To be clear, all those words are also jargon but they’re reusable concepts across software.
Agreed. Come to think of it Homebrew has pretty bad ergonomics in general. What i want is an overview of compiled binaries, where they are, and what their versions are. That’s it.
I disagree. I need to understand what Homebrew means when it says that a cask is keg-only or whatever, or what it means to tap a cellar or whatever when I wanna add repositories. The documentation and '--help' output also refers to the beer brewing terms, not standard package management terms.
Some reason I just died laughing. Really struggled with all that stuff while building this, just seemed to make no sense in the beginning. Maybe I should do /opt/sapphire/cave, opt/sapphire/cove and opt/sapphire/quarry for mine haha
I agreee with this, but is there a “standard” of terms? “repo” I understand to be a git repo, with all it entails. But, “package”, “library”, “module”? It seems like different languages have differenct usages.
I was a macports user but had to switch to homebrew because most new projects went there and it was generally easier to write Formulars etc. But I never really liked the project. I think writing a new package manager on top of brew infrastructure won‘t create a better setup. I don‘t know if all casks and Formulars only use the DSL stanzas or if still some use custom ruby functions and helpers. Because otherwise this new tool might need to eval ruby scripts for backwards compatibility.
I loved macports, mostly because I wanted to create a new user account for a project and got bitten with the permissions thing by homebrew. I don’t mind typing sudo.
Why didn't you like the project if it was a better project by most definitions of what makes a good package manager? e.g. what did you want from a package manager besides 1. easy to install packages 2. readily updated packages.
macports had the cleaner packages. Macports created its own base library so that packages are not based on the macOS system libraries which managed to break them when doing system updates with brew or when you decided to stay longer on an older macOS version. That meant that packages from macports took longer to install and compile. That was from the time when bottles where not a thing yet.
Brew was fast but unstable. In my opinion the inferior project won. But to brews defence they also addressed the issues over the years. Don‘t have any sources at hand but I remember a time when the brew project compared itself to macports and pointed out that their faster because they don‘t recompile all base libraries etc etc. And they had to realize that this wasn‘t a great idea. Now packages are practically split between different os versions. And different bottles are served depending which macOS version you run.
Edit: And I got reminded that it was possible to run and install packages for and from multiple users. Brew took over /usr/local/bin and other /usr/local/homebrew for the running user. Managing a system with multiple users with brew was and still is hard. With macports you needed sudo like with most other package managers. The sudo less nature was a huge deal for its adoption. And now maybe a security risk if you ask me.
> 1. easy to install packages 2. readily updated packages.
Not OP, but I want my package manager to have good UX around package installs and updates, not it deciding to update a Python major version because a random small thing I'm installing it requests it.
Any idea what's up with MacPorts lately? Packages I needed to install were uncharacteristically broken or out of date which wasn't a thing ever since I switched from Fink ages ago. Apple used to use MacPorts internally and put resources into but they stopped doing so?
Suggestion: create a Goals/Motivation/Rationale section in the README. What are the problems with Homebrew you're trying to solve? Why should a prospective user install and try this tool instead of staying with Homebrew?
The author didn't submit it to HN, so criticizing it being on HN seems unfair to me.
It's a cool piece of alpha-quality software. It may or may not be meant to be used, that's beside the point. As I see it HN isn't a platform for software recommendations, it's for discussing interesting geeky things. Which this definitely is, even if it was completely unusable today.
> It’s a real disservice to the project not to give a raison d’etre in the readme, or any kind of technical motivation / differences.
This. There's a wave of projects whose only value proposition is this vacuous "let's reinvent the wheel in Rust" sales pitch, where nothing of value is proposed beyond throwing around the Rust buzzword.
Is it really necessary to restate the advantages of rewriting in Rust in every such project? Compared to Ruby programs Rust programs are faster, more robust, more maintainable, and easier to install. That's pretty much the same for any Rust rewrite (e.g. uv).
It would be interesting to know if there are other goals though, e.g. UX improvements.
> Is it really necessary to restate the advantages of rewriting in Rust in every such project?
I'm with you when the "source" project is C/C++ or something in that realm, but when we're coming from an already memory-safe language I do think some sort of explanation is helpful. I see Homebrew as more of a "glue" application where its own performance isn't exactly critical as it coordinates processes that are much slower so I don't really care if it's a bit faster.
The speed of homebrew has never been limiting factor (for me). I think there are far more important factors in what’s maintainable or not than language, and homebrew is very easy to install.
There has to be more important reasons to replace a mature widely use project like homebrew.
The issue with homebrew is not the programming language. The issue is homebrew itself. It takes forever to install packages, auto-updates without prompting you, etc. Also on my Mac it's been broken for years. It tries to auto-update and gets stuck forever.
I've been using homebrew for years and never did I ever experienced anything I would classify as "broken".
The thread you posted is comical. How many times does anyone run homebrew per day? Or per week? And you still have people complaining about sub-second execution times of a list command? In an app whose happy flow is downloading hundreds of MB off the internet and save it to disk?
By how much? Is homebrew really so slow, and used so often that an improvement would matter?
> more maintainable
[citation needed], especially for uv, which is a tool useful only for Python developers, so using a different language limits the pool of contributors.
Agreed - without clear performance metrics or feature differentiation in the README, potential users have no compelling reason to switch from a mature ecosystem like Homebrew to an alpha-stage alternative.
Homebrew sure has room for improvement, as most software does, and I appreciate every effort to replace and renew what we have with something better. But my own grievances with Homebrew isn't with the codebase itself.
What discourages me from using Homebrew is the intent and the mindset of its developers and packagers, who, I think, see their goal building an "unstable" distribution, as Debian defines it: "[a distribution that] is run by developers and those who like to live on the edge".
I am not blaming the Homebrew developers for building a Sid rather than Bookworm. Some people want just that. Heck, I used to run Debian Sid myself, but have lost my patience for maintaining my own computers since: I am kept busy enough by fixing the software I write, I don't want to spend more time fixing software I did not.
If I understand correctly, running a Debian-style stable repo would essentially require forking all distributed software and backporting security fixes to them.
With many macOS users coming from a different communities than Debian users, I really wonder how well that would go over with the folks whose software was being distributed.
Caveat lector, as I didn't collect the data to back this, but I would be extremeley surprised to find more than a few dozen of non-cask packages in Homebrew that contain Macintosh only software.
I mainly disagree with the Homebrew stance on sudo/root. They claim it's better to install everything under a user dir, but 10% of the time that doesn't work for whatever reason, and tons of users have screwed up their permissions trying to fix it. No other package manager has this issue.
I would've been alright if they actually installed stuff in a user dir under the user's home folder, but it's insane to have a system-wide /opt/homebrew owned by whatever user happened to run the installer. Very icky.
It is possible to install Homebrew itself and the software it packages under your home folder[0], though that configuration is not officially supported.
This is how I install Homebrew when I have to, and so far the only issue I ran into is that binary packages are often tagged as installable only to Homebrew's default folder, so Homebrew had to built the to-be-installed software from source instead, resulting in it taking longer and the computer fans spinning louder.
I am not too fond of this design for reasons of privilege separation and FHS-alignment, but can accept it as most Homebrew users don't have their Macintosh computers used by multiple people.
My machine is single-user, problem is some things require root. Maybe they shouldn't according to Homebrew's stance on permissions, but idk, I'm working with what I've got.
I know why certain software has be run with root privileges, but it's a bit hard for me to come with a reasonable scenario where a software would fail to run properly when installed to a directory that is owned by the same non-root user that launches that software.
The symptom was something like https://stackoverflow.com/questions/16432071/how-to-fix-home... , which btw has a terrible accepted answer. I forget the root cause (no pun intended), possibly some bad interaction with stuff I installed outside of Homebrew. It's also possible I did something wrong, but even then there's something to be said about so many users having the same problem uniquely with Homebrew.
There are rolling distros on Linux that are remarkably stable. My current Linux distribution openSUSE Tumbleweed is one. And on macOS you already have an immutable system partition that provides a stable base to build on top of, so I fail to see the value proposition of a package manager on macOS that's deliberately slow.
I know, probably will, also feel typing out sapphire for the command all the time is annoying + sapp would be weird. For me technical implementation comes first though. I'll do that when something comes to mind
This reminds me of the PIK image format (a precursor to JPEG XL) whose name happens to be a word for penis in some languages[0]. In the present case "SPH" is a kink/fetish term meaning "Small Penis Humiliation"[1]. I don't know how many people would think of that, though.
I am not sure if the lesson is to try harder to avoid offence, or live with the fact that words can have multiple meanings and we can be "professional" enough to ignore some of those meanings in some contexts.
In my experience, rewriting software doesn't work. You should replace the components iteratively in Homebrew itself if you want your ideas to succeed. I doubt your software will see any major adoption just because you wrote it in some other language. The word Homebrew is culturally significant in hacker groups as well and sapphire is not.
Please give it an easier command name than ‘sapphire’ if you want to win people over to use it. Double in size and three times as hard to remember (or type) than ‘brew’. Even cli peeps are still just people
I used to be a big fan of Homebrew but switch to Nix about 2 years ago. For the most part, it works great with home manager for me. Many tools can be installed with breeze just like breeze albeit a bit quicker. The critical thing for me is that Nix doesn't polute my Mac environment like Brew does. But, I admit that this is just for tools. For dev environment I usually just fall back to the language specific methods like cargo, uv or npm.
Some GUI-based applications tend to require too much hassle with Nix so I personally install those with brew. But I still use home manager on top of everything.
Anyway, please do not expect progress to be too rapid. I have a full time job and do most work on this on weekends. I fully intend to make it a stable, "finished" (as much as this is possible in software) thing, it will take a while though. If anyone want's to help out I do open the bugs I find as Issues to keep track and give people an idea of things that do not work. Good Night!
I wish homebrew was a little more friendly to installing in a directory other than what the installer sets. I used to have a lot of permissions issues back when /usr/local was the only directory and none since I started installing it in ~/.brew
I don’t want it there either. The install script is defines the install directory without anyway to override it. It sets it to /usr/local or /opt/homebrew depending on the result of uname command.
I like it in ~/.brew where I have full permission to it and only my user.
> Technically, you can just extract (or git clone) Homebrew wherever you want.
However, Homebrew maintainers are dickish about essentially banning you from contributing to the project if your installation is nonstandard. They have a similarly discouraging warning if you are running a developer beta, which has turned me off of contributing fixes to broken formulas lest they reject them.
> If you decide to use another prefix: don’t open any issues, even if you think they are unrelated to your prefix choice. They will be closed without response.
No, it only does that on Apple Silicon. To this day, if you run the latest Homebrew on an Intel Mac running the latest macOS, it will install to /usr/local.
The parent indicates in a comment that they use an Apple Silicon Mac[1].
I don't see a lot of engineers running Intel Macs anymore. I haven't seen any engineers who still use an Intel model, myself, and for quite some time. Especially when there are Apple Silicon options for well under $1,000 that highly outperform the Intel models.
That sounds like what I want. Seems to only have "wip" support for the newer macOSes though. I also remember using the Debian packager on iPhone back in the golden age of jailbreaking.
This isn't a HB replacement, it's just using HB binaries. It doesn't replace HB at all. HB and this aren't full package managers when they don't officially or technically support building from source or working independently of another project's infrastructure.
A decent full package manager would support a simple, shell-like DSL like say Alpine or Arch, concurrent and parallel phases (such as downloads/builds/installs), multiple versions, reproducible builds, building from source, build acceleration, security auditing, patch management, and package cryptographic signatures (not hashes or signing of hashes).
Nix is theoretically amazing but the barrier-to-entry and gotchas for popular use make it self-limiting. Simplicity in particular areas is a price that is often paid for popularity.
Really glad to see that someone has started on a Mac package manager written in Rust.
A couple purely superficial suggestions (echoing some other comments here):
- Lose the Brew terminology, especially if the name of the project isn't a synonym of "brew."
- Change the name in general. "Sapphire" makes me think of "Ruby." IMO the obvious name is MacPac :p
FWIW the author of Homebrew is also working on a next generation package manager in rust: https://pkgx.dev/ (beware
facebook tracking and infinite AI slop)
mxcl is the owner of this other project and also the original creator of Homebrew. Someone asked politely to remove a broken package from the list, mxcl gave an oddly combative response, then you see the rest.
AI slop, but also for all the imagery. The early 00s called, and want their large icons back. This is the worst landing page I’ve seen—almost screams crypto-scam.
That's a totally different tool. Brew is a generic package manager that installs ... basically everything. Mise is specifically for managing compilers and associated tooling. You can't install libgmp or mplayer with Mise.
I'm still not sure if I like the fact that the package management part of mise relies on a bunch of third-party package managers / repositories like asdf, aqua,...
The main reason I find brew to be a bit slow is just a lack of parallelism with regards to downloads and installs. Rather brew does alternating and sequential D, I, D, I, D, I, when I wish it just kept downloading in the background when it is doing the installs. It would cut down the brew upgrade time by 30% or more at the cost of more disk space used during the process.
But this one qualm I have isn't a result of its language implementation at all.
FWIW: a longstanding limitation for parallel downloads within Homebrew isn't architectural (it's not too hard to add!) but structural with respect to Homebrew's download sources: GitHub and others are very gracious with the amount of traffic we send their way, and we don't want to overtax services that have other major consuming parties.
(This is a perverse countereffect: small projects can make performance decisions that Homebrew and other larger projects can't make, because they don't have a large install base that reveals the limitations of those decisions.)
> FWIW: a longstanding limitation for parallel downloads within Homebrew isn't architectural (it's not too hard to add!) but structural with respect to Homebrew's download sources
I have heard that before.
Hmm.... I wonder if you can get away with not doing parallel downloads, but just keep the sequential downloads going in the background while it is installing a package? It is the pause in downloads during an install that I find is the main slow down in brew.
> I wonder if you can get away with not doing parallel downloads, but just keep the sequential downloads going in the background while it is installing a package?
I could be wrong, but I believe multiple people, including maintainers, have looked into exactly that :-)
(I also need to correct myself: there is some work ongoing into concurrent downloads[1]. That work hasn't hit `brew install` yet, where I imagine the question of concurrent traffic volume will become more pressing.)
Sure. There's also a reason no major OSS packaging ecosystem uses these protocols: the only thing worse than a slow distribution scheme is an unreliable one. Combine that with the (reasonable) lack of a reward scheme for seeding an OSS packaging ecosystem, and you have a distribution mechanism that's significantly more brittle than the current "throw a lot of bandwidth at it" approach.
(Among other technical challenges, like updating the P2P broadcast for each new bottle.)
I think you are misinformed as BitTorrent, for instance, is much more reliable than https alone. The reward scheme is built in already: the client uploads while it's downloading and installing and prioritizes the clients it is downloading from. At worst, the reliability and performance are the same as the web seed.
Generating additional metadata at bottle build time doesn't appear to be much of a technical challenge either.
> The reward scheme is built in already: the client uploads while it's downloading and installing and prioritizes the clients it is downloading from.
These are asymmetric: brew runs at a point in time, and most people decidedly do not want brew running in the background or blocking while leechers are still being serviced. They want it to exit quickly once the task at hand is done.
> Generating additional metadata at bottle build time doesn't appear to be much of a technical challenge either.
That's not the challenge. The challenge is distributing those updates. My understanding is that there's no standard way to update a torrent file; you re-roll a new file with the changes. That means staggered delivery, which in turn means a long tail of clients that see different, incompatible views of the same majority-equal files.
> My understanding is that there's no standard way to update a torrent file; you re-roll a new file with the changes.
Kinda. You do create a new torrent, but you distribute it in a way that to a swarm member is functionally equivalent to updating an old one. Check out BEP-0039 and BEP-0046 which respectively cover the HTTP and DHT mechanisms for updating torrents:
If that updated torrent is a BEP-0052 (v2) torrent it will hash per-file, and so the updated v2 torrent will have identical hashes for files which aren't changed: https://www.bittorrent.org/beps/bep_0052.html
This combines with BEP-0038 so the updated torrent can refer to the infohash of the older torrent with which it shares files, so if you already have the old one you only have to download files that have changed: https://www.bittorrent.org/beps/bep_0038.html
That’s very cool! That addresses the basic update issue, although I would be surprised if there was a production-ready Ruby library for torrents that included these. The state of HTTP(S) in Ruby is sad enough :-)
(There’s also still the state/seeding problem and its collision with user expectations around brew getting faster, or at least not any slower.)
I agree with you about package manager usage patterns being a poor fit for seeding by end users. I definitely wouldn't want my computer to participate.
I could see institutional seeders doing it as a way to donate bandwidth though, like a CDN that's built into the distribution protocol instead of getting load-balanced to Microsoft's nearest PoP when hitting a GitHub `ghcr.io` URI like Homebrew does today. Or even better, use that as an HTTP Seed (BEP-0019) to combine benefits of both :)
Yeah, something at the institutional layer makes sense. Thank you for sharing these links!
(My skepticism around whether this makes sense for Homebrew might be obscuring it, but I’m generally quite a big fan of distributed/P2P protocols, and I strongly believe that existing CDN decencies in packaging ecosystems are a risk that needs mitigating.)
There isn't an update issue. BitTorrent metadata are hashes, they get updated and distributed at the same time as the current URLs and hashes in the same file or maybe in a similarly named file right next to it in the same pull request.
There is no state/seeding problem. The client downloads from the same https url as always but uses peers on an as-available basis to speed things up and reduce load on the origin.
The adjacent thread observes that there is an update issue, just one that has a technical solution.
> The client downloads from the same https url as always but uses peers on an as-available basis to speed things up and reduce load on the origin.
So some kind of hybrid scheme, which (to me) implies the worst of both worlds: clients are still going to hammer upstreams on package updates (since client traffic isn’t uniform), and every client pays a bunch of peering overhead that doesn’t pay off until the files are “hot.” In other words, upstreams still need to plan for the same amount of capacity, and clients have to do more work.
(The adjacent thread observes that none of this is necessary if CDNs or other large operators do this between themselves, rather than involving clients. That seems strictly preferable to me.)
> These are asymmetric: brew runs at a point in time, and most people decidedly do not want brew running in the background or blocking while leechers are still being serviced. They want it to exit quickly once the task at hand is done.
Yes, brew exits when it is done installing, nothing would need to change about that if you used BT protocol to speed up downloads. I'm sure you do have some helpful users who would volunteer to seed their cache though, which would become feasible.
> That's not the challenge. The challenge is distributing those updates.
The metadata goes in the formula alongside the current metadata (URLs and hashes.)
> My understanding is that there's no standard way to update a torrent file; you re-roll a new file with the changes.
You should only re-distribute the original file that was downloaded and thus one can just advertise the original torrent that was downloaded.
But as you said earlier, brew is a point in time command and this BitTorrent solution would only really work if brew switched to an always-on service. And I am not sure that many people want to do that, although I am sure some would.
Good points, background downloads would speed it up.
And the language doesn’t have much to do with that. This project looks to be someone just toying around with Rust or their own PM. Props for that, but the headline has extra implications on HN.
I recently rewrote a big portion of Atlas [1]. It’s a Nim based dependency manager that clones Nim packages to a `deps/` folder. Initially I was worried about using reference types, etc, for performance reasons. It’s a general habit of mine. Then I remembered that stuff would be negligible compared to the download times and git overhead. Well aside from the SAT solver.
Homebrew is 99% IO-bound and this will be too. Installing (prebuilt) packages doesn't require much logic. If this tool supports parallel downloads, it will be 10x faster than Homebrew, but it won't have anything to do with the language. The issue is finding a hosting provider willing to be DDOS'd for the good of the open-source community.
There are two things that work in `uv`'s favor (which, to be clear, is an incredible tool that I'm a big fan of):
1. Python packaging, unlike Homebrew, does have a compute-heavy phase in the form of dependency resolution. `uv` can make significant gains over `pip` in that phase.
2. `uv` performs parallel downloads and other operations, in part because of Rust's fearless parallelism.
Homebrew doesn't really have (1), since resolution is just a linearization of the dependency tree. And parallelism of downloads in (2) poses a problem for Homebrew that's been mentioned in other threads (whereas parallelism is not a significant problem for PyPI's CDN-fronted distribution).
I use homebrew constantly but the only part I find slow is the custom installations of the software and the lack of downloading while installing? Neither of these is related to the implementation language.
What part of the home brew experience do you find slow?
eg...
% time brew upgrade
brew upgrade 0.75s user 0.16s system 68% cpu 1.337 total
% time brew list
brew list 0.01s user 0.02s system 57% cpu 0.054 total
The slowness comes from its serialized downloads and package installations. If you install a thing that has 10 dependencies, those 10 dependencies will serially install, likely taking many seconds.
Given homebrew was strictly worse than the tool it displaced, I’m not sure it’s the right question to ask. MacOS package management is strictly marketing based.
This is why I switched as well. At that point the brew experience was a lot closer to that of apt on a Debian-based Linux distribution or something where you could reliably run “x install <pkgname>”, get up to go grab coffee, and come back to a useable newly installed package. Macports felt like it could throw errors because Venus was in retrograde sometimes.
They spent months fuding MacPorts before putting in place exactly what MacPort was doing because it turns out it’s the right solution and despite that the ergonomic is still garbage (try installing packages outside of the default folder for a good laugh).
Who is "they"? I've never FUDed MacPorts, and in a decade of contributing to and maintaining Homebrew I can say honestly that I've never heard any other maintainer talk much about it beyond user experience.
(Beyond anything else, Homebrew's biggest "win" over MacPorts was and probably is still UX and DX. The core technology of a packaging ecosystem is rarely itself the differentiator.)
Just go read the original announcements of homebrew. They expend at length about how Macport was wrong for shipping its own tool chain and homebrew was somehow superior for reusing the one shipped with MacOS (guess how it turned out). The fact that most packages completely broke if installed outside of /usr/local (itself a non sensical default) was just cherry on the cake. And let’s not talk about how everything broke if you didn’t update often enough so much so that there somehow is a command to try to bring things back to sanity.
Homebrew is by far the worst package manager I have ever used. I’m still sour it somehow dragged away packagers from solutions which were better in every way by being promoted as the "default" solution.
Here are the comparisons to other package managers:
> Packages are brewed in individual, versioned kegs. Then symlinks are created to give a normal POSIX tree. This way the filesystem is the package database. Everything else is now easy. We are made of win.
From 2009, so before my involvement. I'll note that this also says a bunch of things that aren't true in current-day Homebrew (Casks, for example, are distributions of .app bundles), so I think it'd be accurate to say that it doesn't reflect the project's current (or even past-decade) views.
Things improved a lot as homebrew relearned the design constraints and reasons why MacPorts made the decisions they did, and wound up adopting many of the same solutions.
It was frustrating in the beginning to see so much marketing-driven shade being thrown from an ill-informed position.
Obviously that wasn’t you or the current maintainers of homebrew, and things have improved tremendously, but that’s the era from which frustration like the grandparent post originates.
> Then symlinks are created to give a normal POSIX tree. This way the filesystem is the package database. Everything else is now easy. We are made of win.
The worst part is that MacPorts already did the same thing, but used hardlinks to avoid the kinds of problems that emerge when (for example) `realpath` resolves a symlink to an unexpected versioned directory that’s supposed to be an implementation detail.
There was a lot of FUD, dishonesty, and shallow understanding from the homebrew creators in the beginning.
> It’s also open source, did they not accept your pull requests for these “improvements”?
I don’t think you are being fair. This question presupposes that the supposed problems can be solved by iterative changes, rather than being inherent in the chosen design/architecture of the software, which usually requires complete replacement thereof (as well as the leadership thereof, as people who choose poor solutions to problems often can’t appreciate arguments for superior solutions).
(Not that I’m trying to suggest that I agree that homebrew in particular is bad — just speaking generally.)
This. There is plenty to dislike about Ruby, but it is exceptionally strong at implementing domain specific languages. Homebrew was able to use that to lower the barrier to packaging software.
Also, using Git + GitHub instead of SVN + Trac was absolutely a winning pick for scaling project participation back in the 2000s.
What exactly went down with homebrew? Suddenly my employer was pulling it from every device and I never really got a solid explanation of why! I've not been able to get anything meaningful from searching on my own, obviously. Someone help a guy out?
Homebrew itself is not bad, but it does allow users to install software from thirdparty sources, that are unvetted, or unverified. This could pose a security risk. I can see why some companies will disable homebrew. Especially for non-developers.
Homebrew doesn't suddenly add that capability. It just facilitates it. If nothing else, it's easier to track what's been installed via brew than whatever the user may have curlbashed or untargzed to their home.
same here. brew’s been great historically but it’s gotten bloated and kinda slow. curious to see if sapphire can keep things lean without sacrificing compatibility.
Hey, so I built this thing, most of it at so far at least. And yeah, right now it isn't doing many things better than Homebrew.
Setting of relative paths for bottle installs is still not perfect, well it works for every bottle I have tested except rust. Getting bottles working 100% is very doable though imo.
Build from source formulae is still pretty f*ed + I do not know if it is really feasible given that the json API lacks information there and a full on Ruby -> Rust transpiler is way out of scope. Will probably settle for automatic build system detection based on archive structure there. + Maybe do my own version of the .rb scripts but in a more general machine readable format, not .rs lol
Casks seem to work but I have only tested some .dmg -> .app ones and .pkg installers so far though. As with bottles 100% doable.
Given that almost all formulae are available as bottles for modern ARM mac this could become a fully featured package manager. Actually didn't think so many people would look at it, started building it for myself because Homebrew just isn't cutting it for what I want.
Started working on a declarative package + system manager for mac because I feel ansible is overkill for one machine and not really made for that and nix-darwin worms itself into the system so deep. Wrapping Brew commands was abysmally slow though so I started working on this and by now I am deep enough in I won't stop xD
Anyway I am grateful for every bug report, Issue and well meaning pull request.
This is awesome!
Is there uv support?[0]
One of my biggest gripes about brew is how they manage dependencies. The devs have a conflicting philosophy that creates bloat. Package maintainers must define settings, settings should always use the latest Python version. It makes no sense. Maintainers won't update unless things break so you got a bunch of Python versions running around. And it won't use system Python!
Uv seems to provide an elegant solution for this. You can build a venv for each package and rust version will only have the specified deps. Since uv finds all your Python instances (and packages) and soft links them you have way less bloat and venvs become really useful. You can also use run and other tools to handle executables
Plus is also rust so good synergy ;)
[0] https://astral.sh/blog/uv
You should only be using Homebrew for installing software that happens to be written in Python, not dependencies for your own Python projects. If you do that, the Python version does not matter, it's just whatever version is required to make the package work.
You mentioned a declarative package manager for Mac. I've really liked using Homebrew Bundle [1] over the last couple years. It's about the level of declarative that I've wanted and has made it really easy to bootstrap new laptop or VM (since it also works on Linux). The format for a Brewfile was pretty easy to figure out.
The way I ended up using it was that `brew install` would temporarily install something, without adding it to my Brewfile. And a little `brew add` wrapper would add the package to my Brewfile to keep it on the system permanently. That part with the wrapper could have used some love and would be a nice fit for a new brew-compatible frontend IMO. Maybe you could expand on that for Sapphire, if that also scratches your declarative itch?
[1] https://docs.brew.sh/Brew-Bundle-and-Brewfile
What makes you interested in a rust implementation of brew?
I'm guessing it's that you hoping that it is eventually more performant -- are there specific areas of current brew you have identified as performance bottlenecks likely to eventually benefit from a rust implementation?
Or any more info to share about assumptions/hopes that motivated this or any other motivations?
Building from source, obviously, will never be really that much more performant as it mainly relies on the underlying build systems and things like ninja, cmake, cargo etc. are usually optimized very very well.
Thanks to rust just being (slightly, significantly? no idea about ruby's speed) faster + concurrent downloading & pouring of bottles, most "regular" formula installs feel a good bit faster than brew already. Mainly noticeable when installing multiple formulae at once.
Casks, especially those with pkg installers, seem to profit a bit less here.
Performance was a reason, not the main one though, like I said I wanted and still want, to build a declarative package + system managing solution on top. The idea was to get into rust with that. Imo having the base written in the same language instead of wrapping commands also gives more flexibility there.
Another reason is that I never liked the way brew looks and feels. Right now the ui/ux for Sapphire is far from finished, more like a clusterf*k and only the search command really looks the way I want it. Aiming for something modern, clean and information rich without beeing overly verbose. I really like dnf5 and what AerynOS is doing and will probably take some inspiration there.
Like mentioned, Bottles and Casks should be 100% doable and that would cover most package needs on macOS, I do not see why I should also define a new repo and packaging ecosystem when such a big and popular one exists.
Source build capability will probably stay(for easy integration of source building in the system management part later) but not be focused on brew formulae as the ruby dsl would be a horror to parse.
Well and sh*t I am not trying to compete really. This is the first time building something with rust and I really really had no idea what a giant never ending rabbit hole macOS package management is and how massive and complex Brew is.
This went from should I to can I pretty quick for me xD
I would be interested to see more declarative package managers for macOS, I was looking for something similar just a few days ago.
I say this from ignorance, but coming from a lineage of linux package managers; brew must be doing something wrong - and upon immediate introspection I doubt that its language specific.
The performance of apt/dnf in comparison is surreal; but dnf (or at least yum, its predecessor) is written in Python; which has even worse performance characteristics than Ruby.
Clearly something is wrong, I wonder how different they are architecturally.
Have you tried Homebrew in the last year or so? I think a lot of people have an impression of Homebrew's performance from the "bad old days," i.e. back when Homebrew had to evaluate every single local formula file to perform any operations at all.
(There's still low handing fruit, but it's not like it was a few years ago where `brew list` took seconds to run. It now runs nearly instantaneously for me locally, like most of the other happy path commands.)
Yes, I use it on my daily driver, and it's at least two-orders of magnitude slower than apt.
However, the speed increase coincided with my upgrade to an M-Series laptop, so it's possible I just presumed there was a significant hardware speedup in the time we're talking about.
Another performance characteristic that used to piss me off when I used brew was that at least for one program, it had to invoke brew to launch. Something about finding a path. So that's an entire ruby interpreter starting and stopping just to (in this case) run a bash script. Horribly slow.
That’s been fixed for at least a year! But yes, that used to be very slow, particularly when loading brew also meant loading every formula.
One big difference is that apt and dnf are both binary package management systems, whereas Homebrew is a source-based build system with binary packages (“bottles”) added on top.
Cool project, good luck with it!
If I may surface one use case: Several years ago I had to manage a bunch of Macs for CI jobs. The build process (Unreal's UAT) didn't support running more than one build process at a time, and Docker was really slow, so I'd hoped to use different user accounts to bypass that and get some parallelization gains. Homebrew made that very difficult with its penchant for system-wide installs. So a feature request: I'd love to see a competitive package manager that limits itself to operating somewhere (overridable) in the user's home directory.
IIRC the main reason here is that brew path is hardcoded during the build process of packages, which means that you wouldn't be able to use bottles.
I didn't check, but there is a chance that path is also hardcoded in (some) formulae, so even building from the source might not help here.
You could run the build process with chroot or inside Docker, so that the hardcoded paths actually resolve to a designated subdirectory.
In many cases the build output also has hardcoded paths unfortunately
so doing `brew install` inside a container with the proper volumes it’s not sufficient to fix the issue. Everything would have to run from within the container as well.
Incidentally, that’s what is usually done in Nixpkgs in similar situations when there’s no better alternative, see buildFHSEnv et al.
Initial idea for this really came from my dayjob too, we have macs but no way to centrally manage them. The client / server part for the declarative system manager I want to build on top of this is quite far out yet though. At least several months
Nix effectively has per-user packages, but it’s hard to read into your full use case from your comment.
oh, I guess this is why the nix installer creates 32 macOS users called _nixbld$N
It's explained in the documentation.
https://nix.dev/manual/nix/2.25/installation/multi-user
One thing that Homebrew does not do easily is to easily allow for creation of universal libraries and binaries - https://github.com/orgs/Homebrew/discussions/4647
Maybe that could be a place where sapphire differentiates?
Just FYI you are describing a ports system, of which there are many. From macports, to alpine packages, to nixpkgs... Tons of research and automation in this area already. Would encourage looking at Nix and distri in particular. Many Nix packages can be built on/for macOS!
https://nixos.org/manual/nixpkgs/stable/#preface
https://michael.stapelberg.ch/posts/tags/distri/
> probably settle for automatic build system detection based on archive structure there
Please add knobs for the end user to manually configure this per package and global default before adding autodetection. As a user to is very frustrating to have to patch the package manager to override some well-intentioned automagic which didn't consider my setup or dig through sources to uncover some undocumented assumption. yarn is a cautionary example.
I'll add manual override flags and also let users not only build from source from formulae but any dir on their machine they want, only makes sense
Perhaps you could embed something like https://github.com/artichoke/artichoke to run the Ruby scripts for compatibility.
Looked at that. I do not really think implementing something like artichoke or rutie would be a good idea. I do not want my project to become overly bloated and to achieve my real goal of a declarative system management thing I think sticking to bottles (that cover almost all formulae thanks to the amazing homebrew community) and casks, getting those to work 100% is the better approach. Thank you for the suggestion though!
Please don't.
Does the "casks" and "bottles" language imply that it's intended be compatible with Homebrew? That isn't made explicit in the description.
Good luck. Homebrew hasn't cut it since they started disallowing custom compile flags, and making the user jump through hoops if they want anything slightly different.
With my Homebrew hat on, but not speaking for others: I think this is pretty cool, and demonstrates something that we've discussed indirectly for years.
At its core, there are really two parts to Homebrew:
1. There's the client side, i.e. `brew`, which 99.9% of users stick to happy paths (bottle installs, supported platforms) within. These users could be supported with relative ease by a small native-code installer, since the majority of the work done by the installer in the happy path is fetching bottles, exploding them, and doing a bit of relocation.
2. There's literally everything else, i.e. all of the developer, repository, and CI/CD machinery that keeps homebrew-core humming. This is the largely invisible infrastructure that makes `brew install` work smoothly, and it's very hard to RIIR (in a large part because it's tied heavily to the formula DSL, which is arbitrary Ruby).
(1) is a nice experimental space, because Homebrew does (IMO) a decent job of isolating the client-facing side from the complexity of (2). However, (2) is where the meat-and-potatoes of packaging happens, and where Homebrew's differentiators really lie (specifically, in how easy it is to contribute new packages and bump existing ones).
Edit: Another noteworthy aspect here around performance: I mentioned this in another comment[1], but parallel downloads of things like bottles and DMGs is not an architectural limitation of Homebrew itself, but instead a conscious decision to trade some install speed for courtesy towards the services we fetch from (including GitHub itself). Smaller projects can sidestep this because they're not directing nearly the same degree of volume; I think this project will discover if/when its volumes grow that it will need to serialize downloads to avoid being throttled or outright limited.
[1]: https://news.ycombinator.com/item?id=43765605
> "it's very hard to RIIR"
RIIR - "Rewrite It In Rust" (maybe obvious in context? sharing in case not)
IWNO, thanks.
I was not.. oware either, also thanks.
it was not obvious, thanks
It's not _just_ the infrastructure that is awesome for homebrew. The help I got from the team, answering in real-time when I didn't know how to get through the CI bugs is amazing. It's the boring maintainence work that makes it so special for me.
I also feel that there could be a lot of automation in the backend part, catching bugs early (maybe even on local machine before CI run) for example.
I think you mean “the hell I got from the team”.
I wonder if anything changed substantially over the years to make you say that?
I feel like it's quite rude to deny to someone's lived experience and then ask what might have changed to make them feel that way.
> but parallel downloads of things like bottles and DMGs is not an architectural limitation of Homebrew itself, but instead a conscious decision to trade some install speed for courtesy towards the services we fetch from
That doesn't make sense. As you say you're directing a huge volume of traffic so it makes no difference exactly when a user downloads a byte. It all gets smeared out. Only the total amount of data matters and that is unaffected by parallelism.
> As you say you're directing a huge volume of traffic so it makes no difference exactly when a user downloads a byte. It all gets smeared out.
Homebrew's traffic pattern is not a uniform distribution. Package updates go out, and users download those packages in structured ways: there are spikes for MDM-managed Homebrew installations, spikes for cronjobs and CI/CD systems, spikes at 9AM on different coasts when developers sign into their machines, etc.
(How much this matters is also not uniform: it matters somewhat less for GitHub Packages - they should have a hefty CDN - and it matters somewhat more for Casks, which tend to be large DMGs hosted on individual servers.)
Could you parallelize intelligently based on these heuristics? Like, say, whitelist GitHub and S3 for parallelizing
When I asked a few years ago, the brew maintainer said "parallelizing at this scale would hurt github, source: I work for github"
Definitely sounds like one of those "I don't want to do it because it's work, but I don't want to admit that so here's a spurious technical reason" excuses to me.
If spikes at midnight etc are an issue just automatically disabled parallel downloads around midnight. Or only use them when running in a terminal. I really doubt it is an issue for GitHub though.
Yeah, to me the language (ruby) has nothing to do with the success of Homebrew, it is entirely about the DX around backend packaging. The main reason I switched from MacPorts to Homebrew back in the day is because when a new version of [insert software here] was released, I could expect the brew package to be updated within days. MacPorts always lagged significantly behind, and oftentimes didn't have certain packages at all.
As best I can tell, that's partially due to the size of the community, but also because Homebrew went out of their way to teach the system to fish via the 3(?) different kinds of "check for updates" built into the Formulae:
- GitHub release sniffing https://github.com/Homebrew/homebrew-core/blob/b331b99b9f24f...
- page scraping https://github.com/Homebrew/homebrew-core/blob/b331b99b9f24f... (and also per-content-type flavors json and presumably xml)
- links to other formulae (IOW cascading updates): https://github.com/Homebrew/homebrew-core/blob/b331b99b9f24f...
and then the $(brew livecheck) invocation which will do a subset, a curated list, or hypothetically all of them
I can't imagine why MacPorts or our new Sapphire friend couldn't adopt a similar strategy
When Homebrew started (and when I switched) the community was significantly smaller than MacPorts. It is due to the system you mention (and the community that this MX in turn cultivated) that allowed brew to dominate the mac packaging market.
Absolutely MacPorts and Sapphire can adopt the same strategy, but the point is that brew already has, so what exactly would the benefit be? e.g. if the language of choice is effectively meaningless, re-writing homebrew in Rust serves effectively no purpose. This is contrasted with systems and software where the performance or correctness is the most important feature, and therefore RIIR can be a big win.
Said another way: Brew was not "re-write MacPorts in Ruby", it had much loftier goals which it then executed on effectively. Sapphire mostly seems to be "re-write Brew in Rust", without much beyond that. So the only real gain is a bit more performance out of the CLI.
MacPorts already supports "livechecks" which do the same thing. It's just that there are fewer people who are doing this regularly.
I completely agree. I think Ruby has actually been a pretty great boon of a language choice because of the formula DSL, but that goes back to the DX point and not something fundamental to the language.
Merely for your consideration, Ruby is great if you live in that codebase all the time, but Ruby -- in general, not just Homebrew -- has indescribably bad discoverability. I mean that both in terms of "what can I write where my cursor is?" and its friend "where the hell did this symbol under my cursor come from?"
I'm acutely aware someone's going to say LSP something or Rubymine something else but as for drive-by contributions, ... anyway, like I said, just for your consideration
This looks like a fun little project, nice work!
I'm not a big fan of keeping the Homebrew terminology though. I never know what a formula, keg, cask, cellar, tap or bottle is. Why not keep to the standard terms of package and repository etc? I don't know beer brewing terminology or how beer brewing is analogous to package management, and I honestly wish that it wasn't something which my tools expect me to learn.
Please kill the Homebrew terminology if you can! Its idiosyncratic names are the bane of my existence; it might have been cute in 2010, but it's frustrated me ever since.
I don't want to memorise their twee names; I'd much rather the name tell me what the entity / operation does by itself.
Cute names for standard things is one of my software bug bears, i.e. pet peeves, i.e. annoyances. Ruby gems and rust crates and something something beans. It is cute jargon and it annoys me to hide the definition inside a language specific terminology.
It's actually one of the main reasons I landed on using Axum for a web server in Rust instead of Rocket: I got fed up with the additional level of semantic indirection the cutesy names added. I didn't wanna burn brain cycles decoding what it means to install fairings or launch rockets, or what "ignition" means contra "liftoff". I like boring names for things.
> I like boring names for things.
It’s not that I like boring. But I really like descriptive names. I have other things to do with my time than figuring out what the hell a cask, a tap, or a bottle is. Like solving the problem that requires the damn software.
As an experienced homebrewer, I don't think knowing about making beer makes it any clearer to me. Why do I need to install Docker from a Cask? Oh, it's because on Mac, a Cask is actually a Mac package (DMG or PKG or something). It's just arbitrary beer-flavored terminology.
No different than the Windows registry, which apparently uses a honeybee / hive metaphor because some Windows dev hated bees and their teammates liked trolling them.
That one was non-obvious to me as well, but I am sympathetic to the need to call them something. Not trying to bikeshed, but if there was a single obvious name for them then you would have used it instead of “DMG or PKG or something”.
Certainly, but I'm sure there's a more descriptive name than "cask"!
A HIVE is the only nonsensical name in the registry; everything else is keys, subkeys, or values.
https://learn.microsoft.com/en-us/windows/win32/sysinfo/stru...
That said, the registry has been around for ~30 years and the terminology is well known to Windows users. It didn't build off of previously available terminology.
Homebrew just made shit up on the spot based on the project's name to be internally consistent with itself.
Microsoft has it's own fsckery of randomly named cute crap elsewhere, of course.
I will probably change the terminology at some point for this. For initially building it it is way easier to keep them the same though as trying to understand brew itself (at least for me) isn't so straight forward. It's a massive project
Kudos for your humility.
I'd agree that the current homebrew terms are inappropriately whimsical and hard to grasp, but you are right in your intuition and goal, IMO.
That is, taking care of the gears first and then carefully adjusting the public API.
Yes! A thousand times yes. Homebrew terminology makes absolutely zero sense whatsoever and I find it irritating when I encounter it every now and then (like when a package says tap something and then install it blah blah)
I wish they’d just call them binaries, macOS packages, packages, gui-packages, etc.
To be clear, all those words are also jargon but they’re reusable concepts across software.
Agreed. Come to think of it Homebrew has pretty bad ergonomics in general. What i want is an overview of compiled binaries, where they are, and what their versions are. That’s it.
`brew ls --versions` + `brew list | xargs brew list`
Everything you need is there...
Nowadays, normal users don't need to know. They just "brew install <name>".
I disagree. I need to understand what Homebrew means when it says that a cask is keg-only or whatever, or what it means to tap a cellar or whatever when I wanna add repositories. The documentation and '--help' output also refers to the beer brewing terms, not standard package management terms.
Some reason I just died laughing. Really struggled with all that stuff while building this, just seemed to make no sense in the beginning. Maybe I should do /opt/sapphire/cave, opt/sapphire/cove and opt/sapphire/quarry for mine haha
I agreee with this, but is there a “standard” of terms? “repo” I understand to be a git repo, with all it entails. But, “package”, “library”, “module”? It seems like different languages have differenct usages.
Looks awesome. Thanks for your hard work, I will try this out at some point :)
I was a macports user but had to switch to homebrew because most new projects went there and it was generally easier to write Formulars etc. But I never really liked the project. I think writing a new package manager on top of brew infrastructure won‘t create a better setup. I don‘t know if all casks and Formulars only use the DSL stanzas or if still some use custom ruby functions and helpers. Because otherwise this new tool might need to eval ruby scripts for backwards compatibility.
I loved macports, mostly because I wanted to create a new user account for a project and got bitten with the permissions thing by homebrew. I don’t mind typing sudo.
Why didn't you like the project if it was a better project by most definitions of what makes a good package manager? e.g. what did you want from a package manager besides 1. easy to install packages 2. readily updated packages.
macports had the cleaner packages. Macports created its own base library so that packages are not based on the macOS system libraries which managed to break them when doing system updates with brew or when you decided to stay longer on an older macOS version. That meant that packages from macports took longer to install and compile. That was from the time when bottles where not a thing yet. Brew was fast but unstable. In my opinion the inferior project won. But to brews defence they also addressed the issues over the years. Don‘t have any sources at hand but I remember a time when the brew project compared itself to macports and pointed out that their faster because they don‘t recompile all base libraries etc etc. And they had to realize that this wasn‘t a great idea. Now packages are practically split between different os versions. And different bottles are served depending which macOS version you run.
Edit: And I got reminded that it was possible to run and install packages for and from multiple users. Brew took over /usr/local/bin and other /usr/local/homebrew for the running user. Managing a system with multiple users with brew was and still is hard. With macports you needed sudo like with most other package managers. The sudo less nature was a huge deal for its adoption. And now maybe a security risk if you ask me.
> 1. easy to install packages 2. readily updated packages.
Not OP, but I want my package manager to have good UX around package installs and updates, not it deciding to update a Python major version because a random small thing I'm installing it requests it.
Any idea what's up with MacPorts lately? Packages I needed to install were uncharacteristically broken or out of date which wasn't a thing ever since I switched from Fink ages ago. Apple used to use MacPorts internally and put resources into but they stopped doing so?
Suggestion: create a Goals/Motivation/Rationale section in the README. What are the problems with Homebrew you're trying to solve? Why should a prospective user install and try this tool instead of staying with Homebrew?
It’s a real disservice to the project not to give a raison d’etre in the readme, or any kind of technical motivation / differences.
1. I thought "Sapphire is a next‑generation, Rust‑powered package manager inspired by Homebrew" covered it pretty well.
2. It's a personal project.
3. It's explicitly declared as alpha software.
“ Sapphire is a next‑generation, Rust‑powered package manager inspired by Homebrew”
Doesn’t tell me how it differs. What makes this next generation? Just the programming language?
If it’s just a for fun personal project that no one else is supposed to use, I’m not sure why it’s on HN.
The author didn't submit it to HN, so criticizing it being on HN seems unfair to me.
It's a cool piece of alpha-quality software. It may or may not be meant to be used, that's beside the point. As I see it HN isn't a platform for software recommendations, it's for discussing interesting geeky things. Which this definitely is, even if it was completely unusable today.
Yeah, the HN title was editorialized as a Homebrew "replacement," which may be ticking people off.
#1 is a "what", not a "why"
> It’s a real disservice to the project not to give a raison d’etre in the readme, or any kind of technical motivation / differences.
This. There's a wave of projects whose only value proposition is this vacuous "let's reinvent the wheel in Rust" sales pitch, where nothing of value is proposed beyond throwing around the Rust buzzword.
Is it really necessary to restate the advantages of rewriting in Rust in every such project? Compared to Ruby programs Rust programs are faster, more robust, more maintainable, and easier to install. That's pretty much the same for any Rust rewrite (e.g. uv).
It would be interesting to know if there are other goals though, e.g. UX improvements.
> Is it really necessary to restate the advantages of rewriting in Rust in every such project?
I'm with you when the "source" project is C/C++ or something in that realm, but when we're coming from an already memory-safe language I do think some sort of explanation is helpful. I see Homebrew as more of a "glue" application where its own performance isn't exactly critical as it coordinates processes that are much slower so I don't really care if it's a bit faster.
The speed of homebrew has never been limiting factor (for me). I think there are far more important factors in what’s maintainable or not than language, and homebrew is very easy to install.
There has to be more important reasons to replace a mature widely use project like homebrew.
The issue with homebrew is not the programming language. The issue is homebrew itself. It takes forever to install packages, auto-updates without prompting you, etc. Also on my Mac it's been broken for years. It tries to auto-update and gets stuck forever.
https://github.com/orgs/Homebrew/discussions/1177
I've been using homebrew for years and never did I ever experienced anything I would classify as "broken".
The thread you posted is comical. How many times does anyone run homebrew per day? Or per week? And you still have people complaining about sub-second execution times of a list command? In an app whose happy flow is downloading hundreds of MB off the internet and save it to disk?
Is this your argument for a major rewrite?
> Is this your argument for a major rewrite?
A major rewrite in Rust.
> faster
By how much? Is homebrew really so slow, and used so often that an improvement would matter?
> more maintainable
[citation needed], especially for uv, which is a tool useful only for Python developers, so using a different language limits the pool of contributors.
Agreed - without clear performance metrics or feature differentiation in the README, potential users have no compelling reason to switch from a mature ecosystem like Homebrew to an alpha-stage alternative.
Homebrew sure has room for improvement, as most software does, and I appreciate every effort to replace and renew what we have with something better. But my own grievances with Homebrew isn't with the codebase itself.
What discourages me from using Homebrew is the intent and the mindset of its developers and packagers, who, I think, see their goal building an "unstable" distribution, as Debian defines it: "[a distribution that] is run by developers and those who like to live on the edge".
I am not blaming the Homebrew developers for building a Sid rather than Bookworm. Some people want just that. Heck, I used to run Debian Sid myself, but have lost my patience for maintaining my own computers since: I am kept busy enough by fixing the software I write, I don't want to spend more time fixing software I did not.
If I understand correctly, running a Debian-style stable repo would essentially require forking all distributed software and backporting security fixes to them.
With many macOS users coming from a different communities than Debian users, I really wonder how well that would go over with the folks whose software was being distributed.
Caveat lector, as I didn't collect the data to back this, but I would be extremeley surprised to find more than a few dozen of non-cask packages in Homebrew that contain Macintosh only software.
I mainly disagree with the Homebrew stance on sudo/root. They claim it's better to install everything under a user dir, but 10% of the time that doesn't work for whatever reason, and tons of users have screwed up their permissions trying to fix it. No other package manager has this issue.
I would've been alright if they actually installed stuff in a user dir under the user's home folder, but it's insane to have a system-wide /opt/homebrew owned by whatever user happened to run the installer. Very icky.
It is possible to install Homebrew itself and the software it packages under your home folder[0], though that configuration is not officially supported.
This is how I install Homebrew when I have to, and so far the only issue I ran into is that binary packages are often tagged as installable only to Homebrew's default folder, so Homebrew had to built the to-be-installed software from source instead, resulting in it taking longer and the computer fans spinning louder.
[0] https://docs.brew.sh/Installation#untar-anywhere-unsupported
I am not too fond of this design for reasons of privilege separation and FHS-alignment, but can accept it as most Homebrew users don't have their Macintosh computers used by multiple people.
My machine is single-user, problem is some things require root. Maybe they shouldn't according to Homebrew's stance on permissions, but idk, I'm working with what I've got.
Can you give a concrete example?
I know why certain software has be run with root privileges, but it's a bit hard for me to come with a reasonable scenario where a software would fail to run properly when installed to a directory that is owned by the same non-root user that launches that software.
The symptom was something like https://stackoverflow.com/questions/16432071/how-to-fix-home... , which btw has a terrible accepted answer. I forget the root cause (no pun intended), possibly some bad interaction with stuff I installed outside of Homebrew. It's also possible I did something wrong, but even then there's something to be said about so many users having the same problem uniquely with Homebrew.
Lemme try to repro again when I'm home.
There are rolling distros on Linux that are remarkably stable. My current Linux distribution openSUSE Tumbleweed is one. And on macOS you already have an immutable system partition that provides a stable base to build on top of, so I fail to see the value proposition of a package manager on macOS that's deliberately slow.
Consider rebranding to a 4 letter name or even better a 3 letter one.
I know it sounds dumb but uv was smart to go shorter than pip and sapphire feels heavier than brew no matter what it does after typing that.
The name of the project doesn't need to be the same as the CLI, e.g. Homebrew -> brew.
This could be Sapphire -> sap
Maybe `grog` (or `grug` from https://grugbrain.dev)? :)
Yeah i vote it should be rebranded “why”
command: why install htop, -> install start: because you wan' it -> done: you got it
Something like that you mean?
I would love if the way to respond "yes" to the CLI is "why not"
Why not.
Perhaps _why?
I know, probably will, also feel typing out sapphire for the command all the time is annoying + sapp would be weird. For me technical implementation comes first though. I'll do that when something comes to mind
'sph' doesn't seem too bad though.
This reminds me of the PIK image format (a precursor to JPEG XL) whose name happens to be a word for penis in some languages[0]. In the present case "SPH" is a kink/fetish term meaning "Small Penis Humiliation"[1]. I don't know how many people would think of that, though.
I am not sure if the lesson is to try harder to avoid offence, or live with the fact that words can have multiple meanings and we can be "professional" enough to ignore some of those meanings in some contexts.
[0]: https://github.com/google/pik/issues/6
[1]: https://www.urbandictionary.com/define.php?term=sph
In my experience, rewriting software doesn't work. You should replace the components iteratively in Homebrew itself if you want your ideas to succeed. I doubt your software will see any major adoption just because you wrote it in some other language. The word Homebrew is culturally significant in hacker groups as well and sapphire is not.
Yeah that’s not possible.
There’s also `cart` : https://github.com/heywoodlh/cart
I especially like their claim of being unprivileged. Very early stages just like Sapphire.
still, nothing can replace macports for me, ever
I’m really happy with my MacPorts.
Please give it an easier command name than ‘sapphire’ if you want to win people over to use it. Double in size and three times as hard to remember (or type) than ‘brew’. Even cli peeps are still just people
https://en.wikipedia.org/wiki/Alias_(command)
I used to be a big fan of Homebrew but switch to Nix about 2 years ago. For the most part, it works great with home manager for me. Many tools can be installed with breeze just like breeze albeit a bit quicker. The critical thing for me is that Nix doesn't polute my Mac environment like Brew does. But, I admit that this is just for tools. For dev environment I usually just fall back to the language specific methods like cargo, uv or npm.
Funny, I gave up on nix-darwin recently after a year, and went back to homebrew.
Trying to manage nix was more work than I wanted to do.
Some GUI-based applications tend to require too much hassle with Nix so I personally install those with brew. But I still use home manager on top of everything.
Anyway, please do not expect progress to be too rapid. I have a full time job and do most work on this on weekends. I fully intend to make it a stable, "finished" (as much as this is possible in software) thing, it will take a while though. If anyone want's to help out I do open the bugs I find as Issues to keep track and give people an idea of things that do not work. Good Night!
Homebrew was kind of slow, at one point, but I find that it works pretty well these days. Still, this is pretty cool and competitions is always good.
Interestingly, I always imagined that a would-be replacement would come written in Swift. I guess I was wrong.
I wish homebrew was a little more friendly to installing in a directory other than what the installer sets. I used to have a lot of permissions issues back when /usr/local was the only directory and none since I started installing it in ~/.brew
For the last 5 years (since Apple Silicon was released), Homebrew has installed to /opt/homebrew by default.
I don’t want it there either. The install script is defines the install directory without anyway to override it. It sets it to /usr/local or /opt/homebrew depending on the result of uname command.
I like it in ~/.brew where I have full permission to it and only my user.
As described on https://docs.brew.sh/Installation#untar-anywhere-unsupported:
> Technically, you can just extract (or git clone) Homebrew wherever you want.
However, Homebrew maintainers are dickish about essentially banning you from contributing to the project if your installation is nonstandard. They have a similarly discouraging warning if you are running a developer beta, which has turned me off of contributing fixes to broken formulas lest they reject them.
> If you decide to use another prefix: don’t open any issues, even if you think they are unrelated to your prefix choice. They will be closed without response.
No, it only does that on Apple Silicon. To this day, if you run the latest Homebrew on an Intel Mac running the latest macOS, it will install to /usr/local.
The parent indicates in a comment that they use an Apple Silicon Mac[1].
I don't see a lot of engineers running Intel Macs anymore. I haven't seen any engineers who still use an Intel model, myself, and for quite some time. Especially when there are Apple Silicon options for well under $1,000 that highly outperform the Intel models.
1. https://news.ycombinator.com/item?id=43267210
If this provided support for multi-user setup without the user switching gymnastics of homebrew, then I'd be interested.
I just want the equivalent of `sudo apt-get install` on Mac, with the same exact commands.
The thing you want then is `fink`, which literally uses dpkg and apt.
That sounds like what I want. Seems to only have "wip" support for the newer macOSes though. I also remember using the Debian packager on iPhone back in the golden age of jailbreaking.
This isn't a HB replacement, it's just using HB binaries. It doesn't replace HB at all. HB and this aren't full package managers when they don't officially or technically support building from source or working independently of another project's infrastructure.
A decent full package manager would support a simple, shell-like DSL like say Alpine or Arch, concurrent and parallel phases (such as downloads/builds/installs), multiple versions, reproducible builds, building from source, build acceleration, security auditing, patch management, and package cryptographic signatures (not hashes or signing of hashes).
Nix is theoretically amazing but the barrier-to-entry and gotchas for popular use make it self-limiting. Simplicity in particular areas is a price that is often paid for popularity.
Cool project. Definitely starring. But I’m sticking with nix to manage across systems.
Yes a steep learning curve but once you have it set up then it’s easy to sync across devices.
Really glad to see that someone has started on a Mac package manager written in Rust.
A couple purely superficial suggestions (echoing some other comments here):
- Lose the Brew terminology, especially if the name of the project isn't a synonym of "brew." - Change the name in general. "Sapphire" makes me think of "Ruby." IMO the obvious name is MacPac :p
wen linux
FWIW the author of Homebrew is also working on a next generation package manager in rust: https://pkgx.dev/ (beware facebook tracking and infinite AI slop)
Wow. They really don't want anyone to take this project seriously! This is actually making me reconsider having brew installed even...
https://github.com/pkgxdev/pantry/pull/5360#issuecomment-233...
Is this the guy who was complaining that he didn't get hired by Google even though he made brew? Maybe there were personality issues at play...
I checked out his X account and yep, that's the guy.
If you search for "max howell google interview" it comes up, if you search for "max howell alphabet interview", no dice.
This has nothing to do with Homebrew. It's a different project.
Yeah, but the relationship makes my spidey senses tingle even if it's not accurate.
Is there something horrible I'm missing here? It's just a not so well designed page of links to Github repos.
Edit: Oh, the Github issue, yeah that's screwed up.
Consider re-attuning your spidey senses away from inaccurate things!
Creator, not current maintainer. Homebrew is currently maintained by over a dozen people, myself included.
Oof. Those AI images really harm the credibility of the package manager.
I both like and dislike them. First thing I searched for was the prompts, but I haven't found them yet.
As for Facebook tracking I 100% dislike that.
What’s the deal with the cautious warnings? Is homebrew being replaced by software developed using a “vibing coding” or what have you?
if you see the issue I linked, there are a lot of thumbs up / thumbs down in the comments, it's not just me =)
I think I lack context enough that even the link doesn't make it very clear to me.
mxcl is the owner of this other project and also the original creator of Homebrew. Someone asked politely to remove a broken package from the list, mxcl gave an oddly combative response, then you see the rest.
AI slop, but also for all the imagery. The early 00s called, and want their large icons back. This is the worst landing page I’ve seen—almost screams crypto-scam.
Also in Rust, haha.
I finally looked up the GitHub repo found some context about the website: https://github.com/pkgxdev/pantry/issues/5358
Cool, but why not use https://mise.jdx.dev/ ?
That's a totally different tool. Brew is a generic package manager that installs ... basically everything. Mise is specifically for managing compilers and associated tooling. You can't install libgmp or mplayer with Mise.
Because I haven’t heard of it until now?
I'm still not sure if I like the fact that the package management part of mise relies on a bunch of third-party package managers / repositories like asdf, aqua,...
> WARNING: ALPHA SOFTWARE > Sapphire is experimental, under heavy development, and may be unstable. Use at your own risk!
Ruby seems fine for brew. Does this do anything else better? Ruby makes it easy to write recipes for it which is a huge boon for a package manager.
My thoughts exactly.
The main reason I find brew to be a bit slow is just a lack of parallelism with regards to downloads and installs. Rather brew does alternating and sequential D, I, D, I, D, I, when I wish it just kept downloading in the background when it is doing the installs. It would cut down the brew upgrade time by 30% or more at the cost of more disk space used during the process.
But this one qualm I have isn't a result of its language implementation at all.
FWIW: a longstanding limitation for parallel downloads within Homebrew isn't architectural (it's not too hard to add!) but structural with respect to Homebrew's download sources: GitHub and others are very gracious with the amount of traffic we send their way, and we don't want to overtax services that have other major consuming parties.
(This is a perverse countereffect: small projects can make performance decisions that Homebrew and other larger projects can't make, because they don't have a large install base that reveals the limitations of those decisions.)
> FWIW: a longstanding limitation for parallel downloads within Homebrew isn't architectural (it's not too hard to add!) but structural with respect to Homebrew's download sources
I have heard that before.
Hmm.... I wonder if you can get away with not doing parallel downloads, but just keep the sequential downloads going in the background while it is installing a package? It is the pause in downloads during an install that I find is the main slow down in brew.
> I wonder if you can get away with not doing parallel downloads, but just keep the sequential downloads going in the background while it is installing a package?
I could be wrong, but I believe multiple people, including maintainers, have looked into exactly that :-)
(I also need to correct myself: there is some work ongoing into concurrent downloads[1]. That work hasn't hit `brew install` yet, where I imagine the question of concurrent traffic volume will become more pressing.)
[1]: https://github.com/Homebrew/brew/issues/18278
P2P protocols like BitTorrent are made for this kind of problem.
Homebrew packages are on the order of tens of megabytes. Plain old HTTPS downloads will complete faster than setting up a BitTorrent download.
Using BitTorrent is also a great way to get banned from company-owned laptops.
Sure. There's also a reason no major OSS packaging ecosystem uses these protocols: the only thing worse than a slow distribution scheme is an unreliable one. Combine that with the (reasonable) lack of a reward scheme for seeding an OSS packaging ecosystem, and you have a distribution mechanism that's significantly more brittle than the current "throw a lot of bandwidth at it" approach.
(Among other technical challenges, like updating the P2P broadcast for each new bottle.)
I think you are misinformed as BitTorrent, for instance, is much more reliable than https alone. The reward scheme is built in already: the client uploads while it's downloading and installing and prioritizes the clients it is downloading from. At worst, the reliability and performance are the same as the web seed.
Generating additional metadata at bottle build time doesn't appear to be much of a technical challenge either.
> The reward scheme is built in already: the client uploads while it's downloading and installing and prioritizes the clients it is downloading from.
These are asymmetric: brew runs at a point in time, and most people decidedly do not want brew running in the background or blocking while leechers are still being serviced. They want it to exit quickly once the task at hand is done.
> Generating additional metadata at bottle build time doesn't appear to be much of a technical challenge either.
That's not the challenge. The challenge is distributing those updates. My understanding is that there's no standard way to update a torrent file; you re-roll a new file with the changes. That means staggered delivery, which in turn means a long tail of clients that see different, incompatible views of the same majority-equal files.
> My understanding is that there's no standard way to update a torrent file; you re-roll a new file with the changes.
Kinda. You do create a new torrent, but you distribute it in a way that to a swarm member is functionally equivalent to updating an old one. Check out BEP-0039 and BEP-0046 which respectively cover the HTTP and DHT mechanisms for updating torrents:
https://www.bittorrent.org/beps/bep_0039.html
https://www.bittorrent.org/beps/bep_0046.html
If that updated torrent is a BEP-0052 (v2) torrent it will hash per-file, and so the updated v2 torrent will have identical hashes for files which aren't changed: https://www.bittorrent.org/beps/bep_0052.html
This combines with BEP-0038 so the updated torrent can refer to the infohash of the older torrent with which it shares files, so if you already have the old one you only have to download files that have changed: https://www.bittorrent.org/beps/bep_0038.html
That’s very cool! That addresses the basic update issue, although I would be surprised if there was a production-ready Ruby library for torrents that included these. The state of HTTP(S) in Ruby is sad enough :-)
(There’s also still the state/seeding problem and its collision with user expectations around brew getting faster, or at least not any slower.)
I agree with you about package manager usage patterns being a poor fit for seeding by end users. I definitely wouldn't want my computer to participate.
I could see institutional seeders doing it as a way to donate bandwidth though, like a CDN that's built into the distribution protocol instead of getting load-balanced to Microsoft's nearest PoP when hitting a GitHub `ghcr.io` URI like Homebrew does today. Or even better, use that as an HTTP Seed (BEP-0019) to combine benefits of both :)
https://www.bittorrent.org/beps/bep_0019.html
Yeah, something at the institutional layer makes sense. Thank you for sharing these links!
(My skepticism around whether this makes sense for Homebrew might be obscuring it, but I’m generally quite a big fan of distributed/P2P protocols, and I strongly believe that existing CDN decencies in packaging ecosystems are a risk that needs mitigating.)
There isn't an update issue. BitTorrent metadata are hashes, they get updated and distributed at the same time as the current URLs and hashes in the same file or maybe in a similarly named file right next to it in the same pull request.
There is no state/seeding problem. The client downloads from the same https url as always but uses peers on an as-available basis to speed things up and reduce load on the origin.
The adjacent thread observes that there is an update issue, just one that has a technical solution.
> The client downloads from the same https url as always but uses peers on an as-available basis to speed things up and reduce load on the origin.
So some kind of hybrid scheme, which (to me) implies the worst of both worlds: clients are still going to hammer upstreams on package updates (since client traffic isn’t uniform), and every client pays a bunch of peering overhead that doesn’t pay off until the files are “hot.” In other words, upstreams still need to plan for the same amount of capacity, and clients have to do more work.
(The adjacent thread observes that none of this is necessary if CDNs or other large operators do this between themselves, rather than involving clients. That seems strictly preferable to me.)
> These are asymmetric: brew runs at a point in time, and most people decidedly do not want brew running in the background or blocking while leechers are still being serviced. They want it to exit quickly once the task at hand is done.
Yes, brew exits when it is done installing, nothing would need to change about that if you used BT protocol to speed up downloads. I'm sure you do have some helpful users who would volunteer to seed their cache though, which would become feasible.
> That's not the challenge. The challenge is distributing those updates.
The metadata goes in the formula alongside the current metadata (URLs and hashes.)
> My understanding is that there's no standard way to update a torrent file; you re-roll a new file with the changes.
You should only re-distribute the original file that was downloaded and thus one can just advertise the original torrent that was downloaded.
But as you said earlier, brew is a point in time command and this BitTorrent solution would only really work if brew switched to an always-on service. And I am not sure that many people want to do that, although I am sure some would.
Good points, background downloads would speed it up.
And the language doesn’t have much to do with that. This project looks to be someone just toying around with Rust or their own PM. Props for that, but the headline has extra implications on HN.
I recently rewrote a big portion of Atlas [1]. It’s a Nim based dependency manager that clones Nim packages to a `deps/` folder. Initially I was worried about using reference types, etc, for performance reasons. It’s a general habit of mine. Then I remembered that stuff would be negligible compared to the download times and git overhead. Well aside from the SAT solver.
1: https://github.com/nim-lang/atlas
Ruby isn't web scale. Rust is web scale.
Homebrew is dog-slow. If this becomes yet another Rust tool that is 10x faster than the one that it replaces: great.
Homebrew is 99% IO-bound and this will be too. Installing (prebuilt) packages doesn't require much logic. If this tool supports parallel downloads, it will be 10x faster than Homebrew, but it won't have anything to do with the language. The issue is finding a hosting provider willing to be DDOS'd for the good of the open-source community.
Hmm I'm sure people said that about Pip before uv came along and was literally 10x faster.
To be fair I haven't noticed Brew being as tediously slow as Pip. Maybe I just use it way less.
There are two things that work in `uv`'s favor (which, to be clear, is an incredible tool that I'm a big fan of):
1. Python packaging, unlike Homebrew, does have a compute-heavy phase in the form of dependency resolution. `uv` can make significant gains over `pip` in that phase.
2. `uv` performs parallel downloads and other operations, in part because of Rust's fearless parallelism.
Homebrew doesn't really have (1), since resolution is just a linearization of the dependency tree. And parallelism of downloads in (2) poses a problem for Homebrew that's been mentioned in other threads (whereas parallelism is not a significant problem for PyPI's CDN-fronted distribution).
Python should be able to do parallel downloads at least, albeit having the overhead of multiple OS threads unless you're using asyncio.
Yep. The limitation in pip is architectural, not on Python's own side. However, there's been some progress there recently[1].
[1]: https://github.com/pypa/pip/issues/825
> If this tool supports parallel downloads, it will be 10x faster than Homebrew, but it won't have anything to do with the language.
If this is true, why are the Rust tools, on average, so much faster than the JS (or whatever) tools they replace?
Hint: the answer hasn't just to do with the theoretical technical merits of the underlying platform, but also what type of developer they attract.
Do you have an example of a faster Rust tool that replaced an IO-bound JS tool? The toolchain ones that got Rust replacements were CPU-bound afaik.
I use homebrew constantly but the only part I find slow is the custom installations of the software and the lack of downloading while installing? Neither of these is related to the implementation language.
What part of the home brew experience do you find slow?
eg...
% time brew upgrade
brew upgrade 0.75s user 0.16s system 68% cpu 1.337 total
% time brew list
brew list 0.01s user 0.02s system 57% cpu 0.054 total
The slowness comes from its serialized downloads and package installations. If you install a thing that has 10 dependencies, those 10 dependencies will serially install, likely taking many seconds.
What is it doing better than homebrew ?
Given homebrew was strictly worse than the tool it displaced, I’m not sure it’s the right question to ask. MacOS package management is strictly marketing based.
At the time I switched from macports to homebrew (years ago?), homebrew could install more things I needed without any errors. Many more.
That was it, that's why I switched, nothing more, nothing less.
This is why I switched as well. At that point the brew experience was a lot closer to that of apt on a Debian-based Linux distribution or something where you could reliably run “x install <pkgname>”, get up to go grab coffee, and come back to a useable newly installed package. Macports felt like it could throw errors because Venus was in retrograde sometimes.
MacPorts is still around, and that's what I use.
That’s not fair. Homebrew has been fantastic. It’s also open source, did they not accept your pull requests for these “improvements”?
They spent months fuding MacPorts before putting in place exactly what MacPort was doing because it turns out it’s the right solution and despite that the ergonomic is still garbage (try installing packages outside of the default folder for a good laugh).
Who is "they"? I've never FUDed MacPorts, and in a decade of contributing to and maintaining Homebrew I can say honestly that I've never heard any other maintainer talk much about it beyond user experience.
(Beyond anything else, Homebrew's biggest "win" over MacPorts was and probably is still UX and DX. The core technology of a packaging ecosystem is rarely itself the differentiator.)
Just go read the original announcements of homebrew. They expend at length about how Macport was wrong for shipping its own tool chain and homebrew was somehow superior for reusing the one shipped with MacOS (guess how it turned out). The fact that most packages completely broke if installed outside of /usr/local (itself a non sensical default) was just cherry on the cake. And let’s not talk about how everything broke if you didn’t update often enough so much so that there somehow is a command to try to bring things back to sanity.
Homebrew is by far the worst package manager I have ever used. I’m still sour it somehow dragged away packagers from solutions which were better in every way by being promoted as the "default" solution.
Found it because I was curious: https://github.com/Homebrew/legacy-homebrew/commit/29d85578e...
Here are the comparisons to other package managers:
> Packages are brewed in individual, versioned kegs. Then symlinks are created to give a normal POSIX tree. This way the filesystem is the package database. Everything else is now easy. We are made of win.
vs MacPorts registry which used its own homebrewed (lol) Receipts files in 2009, and now uses a SQLite DB: https://guide.macports.org/chunked/internals.registry.html#i...
> I wouldn't worry about it not being root. We don't install anything base enough for it to be a concern (unlike MacPorts or Fink).
vs MacPorts installs to `/opt/local` as root.
> Why Not MacPorts?
> =================
> 1. MacPorts installs its own libz, its own openssl, etc. It is an autarky.
> This makes no sense to me. OS X comes with all that shit.
> 2. MacPorts support Tiger, and PPC. We don't, so things are better optimised.
There is no “Why Not Fink?” section.
And because I didn't know the word autarky: https://en.wiktionary.org/wiki/autarky
From 2009, so before my involvement. I'll note that this also says a bunch of things that aren't true in current-day Homebrew (Casks, for example, are distributions of .app bundles), so I think it'd be accurate to say that it doesn't reflect the project's current (or even past-decade) views.
Things improved a lot as homebrew relearned the design constraints and reasons why MacPorts made the decisions they did, and wound up adopting many of the same solutions.
It was frustrating in the beginning to see so much marketing-driven shade being thrown from an ill-informed position.
Obviously that wasn’t you or the current maintainers of homebrew, and things have improved tremendously, but that’s the era from which frustration like the grandparent post originates.
> Then symlinks are created to give a normal POSIX tree. This way the filesystem is the package database. Everything else is now easy. We are made of win.
The worst part is that MacPorts already did the same thing, but used hardlinks to avoid the kinds of problems that emerge when (for example) `realpath` resolves a symlink to an unexpected versioned directory that’s supposed to be an implementation detail.
There was a lot of FUD, dishonesty, and shallow understanding from the homebrew creators in the beginning.
> Just go read the original announcements of homebrew.
I can't find these anywhere on the official blog, which goes back to the first 1.0 release of Homebrew. Links would be helpful.
> It’s also open source, did they not accept your pull requests for these “improvements”?
I don’t think you are being fair. This question presupposes that the supposed problems can be solved by iterative changes, rather than being inherent in the chosen design/architecture of the software, which usually requires complete replacement thereof (as well as the leadership thereof, as people who choose poor solutions to problems often can’t appreciate arguments for superior solutions).
(Not that I’m trying to suggest that I agree that homebrew in particular is bad — just speaking generally.)
Then fork it! Make a new or better one! Handwaving it’s not good does nothing for anyone
Macports was less ergonomic imho, which caused the community to shift to homebrew.
This. There is plenty to dislike about Ruby, but it is exceptionally strong at implementing domain specific languages. Homebrew was able to use that to lower the barrier to packaging software.
Also, using Git + GitHub instead of SVN + Trac was absolutely a winning pick for scaling project participation back in the 2000s.
Everybody forgot about Fink
https://www.finkproject.org/doc/users-guide/index.php
https://pdb.finkproject.org/pdb/index.php?phpLang=en
(Not a recommendation; seems pretty dead)
Not sure why the subtweet, but I'll guess it was macports, which in my experience was not as fun or easy as brew.
back in the day macports was a fucking nightmare. homebrew may not be perfect but it is popular for good reason.
Right now probably not much, it's WIP.
Having different project leadership is probably a big selling point ;)
What exactly went down with homebrew? Suddenly my employer was pulling it from every device and I never really got a solid explanation of why! I've not been able to get anything meaningful from searching on my own, obviously. Someone help a guy out?
Homebrew itself is not bad, but it does allow users to install software from thirdparty sources, that are unvetted, or unverified. This could pose a security risk. I can see why some companies will disable homebrew. Especially for non-developers.
Homebrew doesn't suddenly add that capability. It just facilitates it. If nothing else, it's easier to track what's been installed via brew than whatever the user may have curlbashed or untargzed to their home.
I think they’re specifically asking about the project as opposed to the tool
So does sourceforge and GitHub and similar, are those blocked too?
Does homebrew have bad leadership?
Some of their leadership has been heavy handed with regards to unpopular decisions, like removing options from packages.
I hope it is faster. Homebrew operations take forever for me.
Ok, but then what goals do you have with it? What are you trying to solve/improve upon?
I doesn't have to do anything better. It's in Rust and that should be the only reason you need to switch.
so cool!, but can you invert a binary tree? /s
I'm really rooting for this. I can't wait for Homebrew replacements, what a pain it is.
What is your pain point? If you don't elucidate your pain point, whatever the replacement is it might copy over that pain point.
same here. brew’s been great historically but it’s gotten bloated and kinda slow. curious to see if sapphire can keep things lean without sacrificing compatibility.