When I was younger, my friends and I joked about making a "Disk Fragmenter" which would enumerate through every file on the disk and split them into non-contiguous clusters to make sure that the actuator arm on your platter-drive gets a PROPER WORKOUT TO FEEL THE BURN.
Such tools really exist and they're used for testing file systems, so that you have a more realistic "aged" layout on a freshly created test file systems.
We also have memory fragmenters to do the same for memory for the same reasons.
Would those tools be primarily proprietary, or do some live in the open-source world too? I’d be interested in the techniques they use to aim for realism there.
Admittedly I haven’t made much of a search, but various search engines and autocorrects seem really convinced that I mean “defragmenter” instead of “fragmenter,” despite myself…
This makes me wonder at the absurdity (yet plausible implementation??!?!) of a tool to test something like ZFS filesystem limits, as they are quite gargantuan in many possible measures.
Perhaps it's appropriate for me to link to the free game I made where you play a taxi driving through Disk Defragmenter and delivering stray blocks of data to their final locations: https://safetystoatstudios.itch.io/ddr. It was pretty fun to watch Defragmenter as a kid, but it would have been better if it was interactive, kind of like the 3D maze screensaver.
The speed is determined by a simple 1000 / <iops> per disk (there's different disks if you click "Settings").
If you inspect and go to sources and look for "1e3 /" you'll find the line responsible, you can set a breakpoint and change e.disk.iops to 1000 which will make the timeout per row 1s instead of 2-4s, so the whole thing goes faster.
I agree. It may sound elitist but back then you need to understand a few concepts to use a computer. From simple things like a filesystem (folders, file, different file types, things my 19 years old nephew don't know), to how to install software, how to download software (!) and then things even a bit more technical: an operative system, how it works, booting, drivers, etc.
Now everything is just easier (which is better for most people!) but I miss the old days.
20 years ago if somebody told you they used Linux it really meant something - they weren't running WSL, they weren't running a VM in EC2, they probably didn't power on and immediately have things working, and they probably learned a ton just getting to a login screen for the first time. Spinning up a VM just doesn't give the same learning experience.
That and many desktop PCs making a kind of grinding noise from the floppy drive on start up, along with the bios "BEEP", as the default boot order option was to seek booting from A: (and possibly B:) floppy drives before the hard drive.
I was going to comment this same thing until I saw yours. I miss degaussing... Especially after holding a magnet to the CRT to get it to display fun colors it wasn't meant to and crossing your fingers that the degauss would fix the damage you'd wrought.
I recall there being some techno-mysticism surrounding defragging regarding how often and when it was necessary, what it did and didn’t help, etc. I also remember the claim “Macs don’t need defragging” but I don’t know if HFS(+) was actually any less prone to fragmentation than FAT32 was. Companies like Symantec were happy to sell defragging tools to Mac users regardless.
> The HFS Plus file system transparently defragments files that are less than 20 MiB in size and are broken into 8 or more fragments when the file is being opened.
Which I expect would have significantly reduced the need to defrag and thus reduced the joy of computing. :p
I love it. A couple notes:
- This is about 10x a real defrag speed - WAAAY too fast
- IIRC the "Data that will not be moved" was typically lumped at the beginning of the drive, not spread out all over.
I'm a stickler for accuracy too. The defragged blocks in the original Win9x program weren't solid blue - they had a hatch pattern texture to them. I remember it way too vividly.
Source: Watching a disk defrag was every millenial kids equivalent of going to the laundromat and watching laundry tumble.
here is another version i found when googling these things that seems somewhat similar, but with a few more "accurate" features. also, has the source available in a single file.
Norton Speed Disk [1] for DOS was more satisfying to watch because it would visualize reads and writes. So you'd see it grab blocks and then smush them together into contiguous areas. The Windows version was not, as I recall, as satisfying.
what a delight it was to behold this visualized process of reinstalling order. the music of the hard disk's seeks went from anarchy to harmony. watching the defragmentation felt like taking a cold shower after sweating all day long ...
I love this. However, Amiga’s Reorg program had a vastly better display, with a larger, resizable window and individual pixels instead of large boxes. You could see much more of the disk at once and get a feel for what the entire operation was doing. Large holes would appear in the middle of a hunk of data, then large files would get moved into the hole, leaving new holes where their incontiguous blocks use to be. Defrag was nice. Reorg was captivating. I miss it.
Typically it would be getting some bits from other parts of the drive and there would be a few (that many) bits it wouldn't move because they were the swap file and DOS boot system files.
Fragmentation is an issue with any file system. SSDs dramatically reduce the cost of fragmentation but do not eliminate it, as non-sequential access is still slower than sequential access.
FAT32 allocates the first free block. IIRC, file order must match disk order because reasons. Ext2 prioritized allocating the middle of free space to allow fragment-free growth on both sides and it became a standard filesystem feature after that.
This seems....wrong? In the start it is writing over data in the start of the disk that has not been read/relocated (some non-free sectors NEVER light up as "Read", only as "writen") which would cause their data to be lost.
When I was younger, my friends and I joked about making a "Disk Fragmenter" which would enumerate through every file on the disk and split them into non-contiguous clusters to make sure that the actuator arm on your platter-drive gets a PROPER WORKOUT TO FEEL THE BURN.
Such tools really exist and they're used for testing file systems, so that you have a more realistic "aged" layout on a freshly created test file systems.
We also have memory fragmenters to do the same for memory for the same reasons.
Would those tools be primarily proprietary, or do some live in the open-source world too? I’d be interested in the techniques they use to aim for realism there.
Admittedly I haven’t made much of a search, but various search engines and autocorrects seem really convinced that I mean “defragmenter” instead of “fragmenter,” despite myself…
It's usually custom scripts, I'm not aware of a standard tool. You fill up the file system and then do lots of operations that cause fragmentation.
Actually here is a recent paper discussing this (although the basic concept is much older):
https://arxiv.org/abs/2401.08858
Arch Linux Wiki on stress testing: https://wiki.archlinux.org/title/Stress_testing
This makes me wonder at the absurdity (yet plausible implementation??!?!) of a tool to test something like ZFS filesystem limits, as they are quite gargantuan in many possible measures.
On CoW systems, file and block deduplication and writes to the middle of a file will do that for you.
LOL
Perhaps it's appropriate for me to link to the free game I made where you play a taxi driving through Disk Defragmenter and delivering stray blocks of data to their final locations: https://safetystoatstudios.itch.io/ddr. It was pretty fun to watch Defragmenter as a kid, but it would have been better if it was interactive, kind of like the 3D maze screensaver.
AND you made it for Sega Genesis! That's so good.
I love this. I wanted the disk noise, and I got it. I will say that’s a lot faster than my drives were in the late 1990s though.
The speed is determined by a simple 1000 / <iops> per disk (there's different disks if you click "Settings").
If you inspect and go to sources and look for "1e3 /" you'll find the line responsible, you can set a breakpoint and change e.disk.iops to 1000 which will make the timeout per row 1s instead of 2-4s, so the whole thing goes faster.
There's something satisfying about defragging that has been lost to time. Like oiling a squeaky engine
I agree. It may sound elitist but back then you need to understand a few concepts to use a computer. From simple things like a filesystem (folders, file, different file types, things my 19 years old nephew don't know), to how to install software, how to download software (!) and then things even a bit more technical: an operative system, how it works, booting, drivers, etc.
Now everything is just easier (which is better for most people!) but I miss the old days.
I've said the same thing about using Linux.
20 years ago if somebody told you they used Linux it really meant something - they weren't running WSL, they weren't running a VM in EC2, they probably didn't power on and immediately have things working, and they probably learned a ton just getting to a login screen for the first time. Spinning up a VM just doesn't give the same learning experience.
That and many desktop PCs making a kind of grinding noise from the floppy drive on start up, along with the bios "BEEP", as the default boot order option was to seek booting from A: (and possibly B:) floppy drives before the hard drive.
Degaussing too.
I once worked with a gigantic Apple (Sony, really) monitor on my desk. Every time I turned it on, it degaussed the monitors nearby.
So many of us spent so much time staring at the barrel of a particle accelerator.
I was going to comment this same thing until I saw yours. I miss degaussing... Especially after holding a magnet to the CRT to get it to display fun colors it wasn't meant to and crossing your fingers that the degauss would fix the damage you'd wrought.
Beuunnnnng
I recall there being some techno-mysticism surrounding defragging regarding how often and when it was necessary, what it did and didn’t help, etc. I also remember the claim “Macs don’t need defragging” but I don’t know if HFS(+) was actually any less prone to fragmentation than FAT32 was. Companies like Symantec were happy to sell defragging tools to Mac users regardless.
Wikipedia claims [1]:
> The HFS Plus file system transparently defragments files that are less than 20 MiB in size and are broken into 8 or more fragments when the file is being opened.
Which I expect would have significantly reduced the need to defrag and thus reduced the joy of computing. :p
[1] https://en.wikipedia.org/wiki/File_system_fragmentation
People used to say the same about Linux ext4, it didn't need defragmenting.
afaics all that meant was file allocations make a little extra space at the end, so if the file expands it doesn't need to allocate a new block.
If one keeps files around for a long time and constantly adds to them, then the files get fragmented on any filesystem.
The only way to avoid this is pre-allocate all files to their maximum possible size and never exceed that size.
I love it. A couple notes: - This is about 10x a real defrag speed - WAAAY too fast - IIRC the "Data that will not be moved" was typically lumped at the beginning of the drive, not spread out all over.
I'm a stickler for accuracy too. The defragged blocks in the original Win9x program weren't solid blue - they had a hatch pattern texture to them. I remember it way too vividly.
Source: Watching a disk defrag was every millenial kids equivalent of going to the laundromat and watching laundry tumble.
https://youtu.be/dc_SDyLYq3U?t=347
I am guessing the developer took into account the fact that the younger generations these days don't have the patience for real time defragging.
author commenting on such things:
https://morello.dev/blog/defrag98-rebuild
what it looked like before
https://morello.dev/blog/windows-98-defrag-simulator
a previous discussion:
https://news.ycombinator.com/item?id=40962195
here is another version i found when googling these things that seems somewhat similar, but with a few more "accurate" features. also, has the source available in a single file.
https://horatioconkerhead.github.io/win98-defrag/
if you want the real deal, you can run windows 95 in the browser quick. it took running it a few times to get it to actually defrag:
https://archive.org/details/win95_in_dosbox
It could be spread out too
Norton Speed Disk [1] for DOS was more satisfying to watch because it would visualize reads and writes. So you'd see it grab blocks and then smush them together into contiguous areas. The Windows version was not, as I recall, as satisfying.
[1] https://youtu.be/-T712Wbc6a0
My computer's definitely gonna be faster after this.
I do laugh in that in all the times I ever did this for real (even back to my massive 170MB drive), it never felt much faster.
At that point, you had to deal with questions of interleave.
https://en.wikipedia.org/wiki/Interleaving_(disk_storage)
You probably defragged frequently enough. You would definitely notice it if a file system got fragmented enough.
Will need to degauss after this
What, you don't need the data any more? Or are you conflating screen degaussing?
We used to degauss the CRTs because it looked cool and there was nothing else to do. Very adjacent to watching the defrag window maximised.
I did it in computer lab and showed others and the teacher lost it, thought we were breaking things.
It also made a really satisfying “boyoyoyinnng” sound
what a delight it was to behold this visualized process of reinstalling order. the music of the hard disk's seeks went from anarchy to harmony. watching the defragmentation felt like taking a cold shower after sweating all day long ...
I love this. However, Amiga’s Reorg program had a vastly better display, with a larger, resizable window and individual pixels instead of large boxes. You could see much more of the disk at once and get a feel for what the entire operation was doing. Large holes would appear in the middle of a hunk of data, then large files would get moved into the hole, leaving new holes where their incontiguous blocks use to be. Defrag was nice. Reorg was captivating. I miss it.
Love the warning for using on a mobile device.
In the same vein, but showing live the go allocator and GC operating on the heap of a running program: https://github.com/MichaelMure/gogc98
Need this but for AI models. Sleep mode where they self prune and optimize overnight for their daily purpose.
https://platform.claude.com/docs/en/managed-agents/dreams
I started the defrag on F and got about 1/4 the way, but had to stop cause my brother wanted to use the family computer.
When I changed it between drives and back I lost my progress :/
It feels a little too defragmented to start.
Typically it would be getting some bits from other parts of the drive and there would be a few (that many) bits it wouldn't move because they were the swap file and DOS boot system files.
I used to watch defrag (and defraggler). If you have OCD they are very satisfying to watch. The more fragmented they were to start with, the better.
There is even a click noise when you idly tap the screen.
As great as it is, way too fast.
Fuck! Why I'm just staring at it like I did ~20 years ago. It's not even real!
And it blue screens?
Maybe my memory is bad, but I recall defrag taking hours to complete.
I wish ext4, BtrFS, and XFS had something like this.
XFS does, though not visual. https://www.man7.org/linux/man-pages/man8/xfs_fsr.8.html
4 Drives, 5.5G total. Luxury!
Per-drive individual sounds, and BSoD included. Nice touches, love it!
The sound killed me... :D
Defragmenting XFS is much less dramatic.
I need to set up a Windows 95 computer with a hard drive and stream the defragging online, complete with live audio.
What were the major filesystem technology improvements that made this redundant?
I think NTFS and SSD disks.
Fragmentation is an issue with any file system. SSDs dramatically reduce the cost of fragmentation but do not eliminate it, as non-sequential access is still slower than sequential access.
FAT32 allocates the first free block. IIRC, file order must match disk order because reasons. Ext2 prioritized allocating the middle of free space to allow fragment-free growth on both sides and it became a standard filesystem feature after that.
And, disk capacities got a lot larger. When most of the space on a disk is free, it's easier to allocate without fragmenting.
Legend
This seems....wrong? In the start it is writing over data in the start of the disk that has not been read/relocated (some non-free sectors NEVER light up as "Read", only as "writen") which would cause their data to be lost.
Brilliant. Whirring disk noise is nice touch.
If AI generated this is what the AI revolution is all about.
Can we get a cross platform screensaver?
bill gates, I want my time back!
This scratched an itch for me.
I’m going to watch Johnny Castaway now.
My YouTube
Hot