It's quite the dark-pattern to allow you to download it and scan for free, without any clear indication that it's a paid product, and then ambush you with a purchase dangling the space savings in front of your face.
If anything, it's the opposite of a dark pattern: you get to see if it'll provide any value for you without paying a dime. It's a far better proposition than paying up only to find out that it won't free up any space for you. The linked site is very clear about how the pricing works.
> The linked site is very clear about how the pricing works.
Yes, it is.
It is also, very obviously, placed in such a way to reduce the bounce rate off the landing page and increase conversions.
When was the last time you were on a site that had meaningful content below its "releases" section?
Why is the answer to the "How do I pay?" FAQ question bend over backwards to first say
>Hyperspace is a free download in the Mac App Store. Once downloaded, it’s free to scan an unlimited number of files. Scanning will let you know how much space is eligible to be reclaimed.
Why doesn't the app store description say which capabilities are free and which are paid?
One or two of these are innocuous, but there are are a dozen small choices like this that have been made here, all aligned to increase the likelihood of one thing: Put the user in the position where they've already installed the app, sunk some time into it, and are anticipating the benefit before making it clear that they have to pay money.
That the page technically contains the words "you will need to pay for Hyperspace" does not mean this isn't a dark pattern.
What makes it a (crystal clear IMO) dark pattern is that the purportedly "free" feature set gives the user no value whatsoever. It only exists to induce sales. Inducing a high-bar commit (the purchase) by first securing a low-bar commit (install + scan) is an old trick. Don't fall for it.
I think this is an outrageous abuse of the term “dark pattern” from people who want to be angry more than they want to think. The website mentions repeatedly what the pricing model is, and the operation is “we show you exactly how much value you would get from paying, and then you can decide whether it’s worth it to you.” That’s as honest and plain as business can get: Plenty of people don’t have many/any files which would benefit from APFS COW snapshots and for them paying before the scan would be a ripoff.
We used to call products like this trials or trialware. There was always clear verbiage on the site attesting to the fact that the product was not free but that you could download it for a trial to see if you liked it before paying for it. But I suppose this doesn't even really offer a trial as it doesn't do anything at all until you pay.
Here, the first mention of any kind of cost to this software is ~1/8th of the way down the page when you're met with Q: How do I pay for Hyperspace? which litteraly states that it's a free download that doesn't do anything unless you pay them, meaning that it is not free in any sense of the term.
You are complaining that it can’t instantaneously perform a complete scan of your disk, and then attributing malicious intent to this lack of magic instant scanning technology.
What are we even doing here. The knee-jerk conspiratorial foolishness on this site is completely off the deep end.
No, I'm complaining that it doesn't ask me to pay before it takes my time to perform a complete scan of my disk (ideally before it even gives me a download of the app).
We both know it does so intentionally, and we also know exactly what the intent is.
Wouldn’t the ideal be that it informs you up front that it requires payment to get the benefit, but allows you to scan to estimate the benefit before deciding whether to pay (but after acknowledging that getting the benefit requires payment)?
I don’t understand. It mentions multiple times it’s not free right on the linked page. And if you open the App Store link? It also mentions it’s a paid app.
Q: How do I pay for Hyperspace?
Hyperspace is a free download in the Mac App Store. Once downloaded, it’s free to scan an unlimited number of files. Scanning will let you know how much space is eligible to be reclaimed.
If you decide you want to reclaim that space, you will need to pay for Hyperspace using the purchase button that appears after a successful scan or by selecting “Purchase Hyperspace…” from the “Hyperspace” menu in the menu bar.
The fact that the payment is buried under 10k characters of text is deliberate. That it is not in the 'pricing' or 'buy' link in the menu where everyone has it is clearly a choice.
Hard to imagine anyone less likely to ponder on - and implement - any kind of "dark pattern" than John Siracusa. Maybe the landing page could be re-ordered slightly to state that part of the FAQ - which is very clear - further up, but I doubt that there's a malicious bone in that man's body.
For Windows users looking to save space, there's Compactor[1] that uses the NTFS built-in compression opportunistically for only those files that benefit from it. It is not a deduplicator but the basic idea is the same: take an underutilized OS feature and make it usable.
Pretty handy when your flight sim game takes 1.2 terabytes with most of that being sparse terrain mesh data and uncompressed textures. Compressing the whole directory would take over a day according to Microsoft and their infamous progress bar; Compactor gets it done in fifteen minutes.
It's amazing that this is a product. In an earlier job we used to ask how to do this (discovery phase of deduplication) in the first interview screening. Once you have that list, it's straightforward to make a few syscalls that make it happen.
If you're referencing Fetch for mac (pre macos) or Transit by Panic (later renamed to Transmit), I was a fan (and I've been a terminal fanboy since the early 80s). Both of them started as FTP wrappers, and both provided convenience the cli lacked.
As I recall, there are nuances around certain types of files that require expert attention, such as links, files inside frameworks/packages, and unmaterialized iCloud Drive files.
Struggling to find the effective difference between the two, other than it being Apple specific. Either way, neat - I can hopefully use this to provide faster layer unpacks on MacOS as well.
Nice I was just thinking about this the other day! Given the memory supply issue today, I wonder how much of data in our data centers worldwide is essentially just copied data? I have a feeling that there is a ton of redundancy, much of it absolutely necessary, but much of it essentially not at all, and howmuc memory we can reclaim by culling copies
This could be useful for reclaiming space when your disk is almost full, but $9.99 a month for a tool the average person isn't going to need very often seems a bit much.
It’s not really intended to be a recurring $9.99/mo. It’s $9.99, and that gives you a month of usage. But the expectation is you use it once or twice, then you’re done for a long time. But apparently they also offer an auto-renew subscription for that same price, which seems like a strange choice.
No, this uses a feature of the macOS file system. From the FAQ:
> Q: Are clone files the same thing as symbolic links or hard links?
> A: No. Symbolic links (“symlinks”) and hard links are ways to make two entries in the file system that share the same data. This might sound like the same thing as the space-saving clones used by Hyperspace, but there’s one important difference. With symlinks and hard links, a change to one of the files affects all the files.
> The space-saving clones made by Hyperspace are different. Changes to one clone file do not affect other files. Cloned files should look and behave exactly the same as they did before they were converted into clones.
Yes, but it sounds like it's more fine-grained than just the file-level. So even if you change part of one copy, the remaining unchanged subpieces are still shared, iiuc.
CoW deduplicates at the block level. Unless you rewrite the whole file, the untouched blocks will remain common between them and only different blocks will be allocated separately.
Continuous block level deduplication is resource intensive and not a great fit unless space is at a high premium.
Offline block deduping is interesting though, but only for solid state storage. For hard disks it’s a nightmare.
Seems useful for s3 buckets. I suppose a script that watches for new objects, calculates its sha256 and stores that in a DB, then checks for duplicate hashes would be a fairly trivial task. Though s3 doesn’t support symbolic links so accounting would need to be handled by server side code.
While the idea of data-deduplication does apply everywhere, this specifically depends on the APFS CoW data feature, so it's not applicable outside of macOS (or APFS Volumes)
but link is dangerous, as a edit made to one file will then be made to all copies.
For example, If I have two files /home/m/todo.txt and /home/m/backup2025/todo.txt that are identical and you then replace /home/m/todo.txt with a link to /home/m/backup2025/todo.txt and I later edit /home/m/todo.txt then backup copy would also be updated. With apfs clones the backup would not be modified when I updated the file.
It's quite the dark-pattern to allow you to download it and scan for free, without any clear indication that it's a paid product, and then ambush you with a purchase dangling the space savings in front of your face.
If anything, it's the opposite of a dark pattern: you get to see if it'll provide any value for you without paying a dime. It's a far better proposition than paying up only to find out that it won't free up any space for you. The linked site is very clear about how the pricing works.
> The linked site is very clear about how the pricing works.
Yes, it is.
It is also, very obviously, placed in such a way to reduce the bounce rate off the landing page and increase conversions.
When was the last time you were on a site that had meaningful content below its "releases" section?
Why is the answer to the "How do I pay?" FAQ question bend over backwards to first say >Hyperspace is a free download in the Mac App Store. Once downloaded, it’s free to scan an unlimited number of files. Scanning will let you know how much space is eligible to be reclaimed.
Why doesn't the app store description say which capabilities are free and which are paid?
One or two of these are innocuous, but there are are a dozen small choices like this that have been made here, all aligned to increase the likelihood of one thing: Put the user in the position where they've already installed the app, sunk some time into it, and are anticipating the benefit before making it clear that they have to pay money.
That the page technically contains the words "you will need to pay for Hyperspace" does not mean this isn't a dark pattern.
What makes it a (crystal clear IMO) dark pattern is that the purportedly "free" feature set gives the user no value whatsoever. It only exists to induce sales. Inducing a high-bar commit (the purchase) by first securing a low-bar commit (install + scan) is an old trick. Don't fall for it.
I think this is an outrageous abuse of the term “dark pattern” from people who want to be angry more than they want to think. The website mentions repeatedly what the pricing model is, and the operation is “we show you exactly how much value you would get from paying, and then you can decide whether it’s worth it to you.” That’s as honest and plain as business can get: Plenty of people don’t have many/any files which would benefit from APFS COW snapshots and for them paying before the scan would be a ripoff.
We used to call products like this trials or trialware. There was always clear verbiage on the site attesting to the fact that the product was not free but that you could download it for a trial to see if you liked it before paying for it. But I suppose this doesn't even really offer a trial as it doesn't do anything at all until you pay.
Here, the first mention of any kind of cost to this software is ~1/8th of the way down the page when you're met with Q: How do I pay for Hyperspace? which litteraly states that it's a free download that doesn't do anything unless you pay them, meaning that it is not free in any sense of the term.
This would only be true if it didn't take your time before asking you to pay to not waste the time that it took.
It says it’s a paid app right on the linked page.
It doesn't mention being a paid app until after it says it a free download roughly 10k characters down the screen.
There is a giga ton of content in the homepage. Are you really going to read it all? The main window doesn’t suggest it’s a paid app.
The app store identifies apps with paid features. You shouldn't be entirely surprised.
You are complaining that it can’t instantaneously perform a complete scan of your disk, and then attributing malicious intent to this lack of magic instant scanning technology.
What are we even doing here. The knee-jerk conspiratorial foolishness on this site is completely off the deep end.
No, I'm complaining that it doesn't ask me to pay before it takes my time to perform a complete scan of my disk (ideally before it even gives me a download of the app).
We both know it does so intentionally, and we also know exactly what the intent is.
Wouldn’t the ideal be that it informs you up front that it requires payment to get the benefit, but allows you to scan to estimate the benefit before deciding whether to pay (but after acknowledging that getting the benefit requires payment)?
No we don't know this. And in fact you can listen to the Accidental Tech Podcast at length to hear him thinking through the current design at length.
I don’t understand. It mentions multiple times it’s not free right on the linked page. And if you open the App Store link? It also mentions it’s a paid app.
What am I missing?
It's a couple Q's in on the page:
Q: How do I pay for Hyperspace? Hyperspace is a free download in the Mac App Store. Once downloaded, it’s free to scan an unlimited number of files. Scanning will let you know how much space is eligible to be reclaimed.
If you decide you want to reclaim that space, you will need to pay for Hyperspace using the purchase button that appears after a successful scan or by selecting “Purchase Hyperspace…” from the “Hyperspace” menu in the menu bar.
It's four pages below the fold. The fact this is a paid tool is deliberately obscured.
Not sure why you're being downvoted.
The fact that the payment is buried under 10k characters of text is deliberate. That it is not in the 'pricing' or 'buy' link in the menu where everyone has it is clearly a choice.
Hard to imagine anyone less likely to ponder on - and implement - any kind of "dark pattern" than John Siracusa. Maybe the landing page could be re-ordered slightly to state that part of the FAQ - which is very clear - further up, but I doubt that there's a malicious bone in that man's body.
This is how basically all Mac software worked in my youth. Maybe we should return to the heady days of shareware
Agreed. Are there any open source alternatives for this?
For Windows users looking to save space, there's Compactor[1] that uses the NTFS built-in compression opportunistically for only those files that benefit from it. It is not a deduplicator but the basic idea is the same: take an underutilized OS feature and make it usable.
Pretty handy when your flight sim game takes 1.2 terabytes with most of that being sparse terrain mesh data and uncompressed textures. Compressing the whole directory would take over a day according to Microsoft and their infamous progress bar; Compactor gets it done in fifteen minutes.
[1] https://github.com/Freaky/Compactor
I think the record for most space saved by this utility in a single run is in the hundreds of TBs now. I’ll see if I can find the toot.
Edit: maybe a bit hyperbolic of me, looks like it was 3.94TB https://mastodon.social/@WTL/116710030179809319
Discussed upon release at: https://news.ycombinator.com/item?id=43173462
You can also use the free fclones CLI tool that uses the same native APFS functionality: https://news.ycombinator.com/item?id=43173713
It's amazing that this is a product. In an earlier job we used to ask how to do this (discovery phase of deduplication) in the first interview screening. Once you have that list, it's straightforward to make a few syscalls that make it happen.
What are they gonna think of next, productizing something as trivial as FTP?
> Two kinds of purchases are possible: one-time purchases and subscriptions.
I’m surprised they’re not charging on a $/GB saved model.
If you're referencing Fetch for mac (pre macos) or Transit by Panic (later renamed to Transmit), I was a fan (and I've been a terminal fanboy since the early 80s). Both of them started as FTP wrappers, and both provided convenience the cli lacked.
As I recall, there are nuances around certain types of files that require expert attention, such as links, files inside frameworks/packages, and unmaterialized iCloud Drive files.
How does this compare to diskDedupe, which has been around longer and is much cheaper?
This uses reflinks, right? I've been experimenting with using reflinks under Linux to speed up layer extraction for Docker, it's great.
It uses APFS clones
Struggling to find the effective difference between the two, other than it being Apple specific. Either way, neat - I can hopefully use this to provide faster layer unpacks on MacOS as well.
Yup that’s pretty much it, reflinks is Btrfs/XFS, clones is APFS, but the concept is the same (CoW in both cases).
Ding ding ding…
Yep
Nice I was just thinking about this the other day! Given the memory supply issue today, I wonder how much of data in our data centers worldwide is essentially just copied data? I have a feeling that there is a ton of redundancy, much of it absolutely necessary, but much of it essentially not at all, and howmuc memory we can reclaim by culling copies
This could be useful for reclaiming space when your disk is almost full, but $9.99 a month for a tool the average person isn't going to need very often seems a bit much.
It’s not really intended to be a recurring $9.99/mo. It’s $9.99, and that gives you a month of usage. But the expectation is you use it once or twice, then you’re done for a long time. But apparently they also offer an auto-renew subscription for that same price, which seems like a strange choice.
By comparison, have to handle it to DeDupe for its transparency:
Key Features Free to Scan, Unlock to Deduplicate
Surprised that app with Full Disk Access permission requirement is allowed to the App Store.
i asked claude to make this and in 15min had a working replica CLI
Probably very simple for claude, considering how many implementations of this simple concept already exist and are part of the training data.
Or use jdupes
I’ve been using fdupes. It’s installable from MacPorts. Or is it finddupes? I don’t quite remember.
How does it compare to https://diskdedupe.com/?
Are you deleting the others and creating a symlink to the original? How does this work exactly? I didn't get it from the FAQ section.
Does this mean that if the original is gone, all the file links will not be found?
No, this uses a feature of the macOS file system. From the FAQ:
> Q: Are clone files the same thing as symbolic links or hard links?
> A: No. Symbolic links (“symlinks”) and hard links are ways to make two entries in the file system that share the same data. This might sound like the same thing as the space-saving clones used by Hyperspace, but there’s one important difference. With symlinks and hard links, a change to one of the files affects all the files.
> The space-saving clones made by Hyperspace are different. Changes to one clone file do not affect other files. Cloned files should look and behave exactly the same as they did before they were converted into clones.
It's a CoW (copy-on-write) file, it sounds like.
Yes, but it sounds like it's more fine-grained than just the file-level. So even if you change part of one copy, the remaining unchanged subpieces are still shared, iiuc.
CoW deduplicates at the block level. Unless you rewrite the whole file, the untouched blocks will remain common between them and only different blocks will be allocated separately.
Continuous block level deduplication is resource intensive and not a great fit unless space is at a high premium.
Offline block deduping is interesting though, but only for solid state storage. For hard disks it’s a nightmare.
It sounds very much like a video encoder.
Here’s a lengthier explanation [1] but it’s basically an APFS feature.
[1]: https://hypercritical.co/hyperspace/#how-it-works
Thank you. This makes sense now.
look ma, antibackup!
Seems useful for s3 buckets. I suppose a script that watches for new objects, calculates its sha256 and stores that in a DB, then checks for duplicate hashes would be a fairly trivial task. Though s3 doesn’t support symbolic links so accounting would need to be handled by server side code.
While the idea of data-deduplication does apply everywhere, this specifically depends on the APFS CoW data feature, so it's not applicable outside of macOS (or APFS Volumes)
The same concepts exist in ZFS, BtrFS and, IIRC, XFS.
also, worst case you can do this with link features? Which is basically every system under the sun.
but link is dangerous, as a edit made to one file will then be made to all copies.
For example, If I have two files /home/m/todo.txt and /home/m/backup2025/todo.txt that are identical and you then replace /home/m/todo.txt with a link to /home/m/backup2025/todo.txt and I later edit /home/m/todo.txt then backup copy would also be updated. With apfs clones the backup would not be modified when I updated the file.
Link might be desirable for your e-book or mp3 collections. Or .iso images, MAME ROMs, and other files that are unlikely to be changed.
I don't think it's very useful but there's already a checksum in the meta data of an S3 object: https://docs.aws.amazon.com/AmazonS3/latest/API/API_Object.h...
It might be better to keep an index of paths to checksums and use the checksum as object key in S3.
I'm not sure what you like to achieve but if you use restic with S3 as a backend you can achieve much better deduplicate and compression.
[dead]