... Ted Nelson’s 1974 manifesto, Computer Lib/Dream Machines, extolling the vision of Project Xanadu hypertext.
Computer Lib/Dream Machines was not about project Xanadu, it merely mentions it on a few pages.
I recall Computer Lib/Dream Machines was the most influential thing Ted Nelson did, far more influential by than Xanadu. It was seen all over at the beginning of the personal computer era. Try to get a look at a copy if you can, it's fascinating.
This self-published book was very much of its time - it had the same oversize-on-newsprint format and DIY look as the Whole Earth Catalog, which was popular at the same time.
It was important because it pursuasively made the case that computing was about to become a medium for self-expresson - a "dream machine" - that was an unheard-of concept at the time. You can use a computer to produce graphics! To play games! How about that! He told about Sutherland's Sketchpad, Englebart's NLS, and the Arpanet at a time when these were completely invisible to almost everyone. Xanadu gets a few pages in the book but it is not at all a work of self-promotion - almost the whole book is about the work of others. It's a survey of the computing scene right before personal computers appeared.
This book was very timely and I think it provided a lot of inspiration for the personal computer movement. It made you want to try out computing! It has one page about microprocessors: "Here they come ... Microprocessors are what's happening" (the MITS Altair appeared the next year, in 1975).
I think Ted Nelson is one of those people whose main effects are through their influence on others, rather than through what they directly produce themselves.
> Xanadu wasn’t the victim of “Worse is Better”; it was just a solution in search of a problem.
Well, in the day to day use of computers across everyone I interact with, the following problems are very much present and still unsolved (savvy users can address them and transform them into chores; non-savvy users solve some of them by paying money to pospone the issues):
- I don't have any space left on my phone but my laptop has tons of space available. (Addressed by Xanadu principle 13.)
- How do I back up my stuff uniformly across my devices. (Addressed by Xanadu principle 14.)
- How do I share this file with a relative. (Addressed by Xanadu principles 2+4+10+11+12.)
- How do I not get trapped by vendors who want to lock me into their cloud solution. (Addressed by Xanadu principles 1, 2, 3, 15, 17.)
So the problems are very much there, and not improving. If anything, Xanadu was a prescient solution that addressed back them the problems we have today.
Xanadu principle have outlined important problems. This does not mean that Xanadu created practical solution, though: it seems they were focused on text transclusions UI for some reason...
If the goal of Xanadu was to offer a better, more principled form of information sharing, they should have focused 99% of effort on the protocols & backend implementation, leaving UI rudimentary. E.g. TUI might be sufficient to browser hypertext, and that would be compatible with ~all systems, unlike their GUI contraptions.
There was a commercial networking product which addressed principle 13 as well --- Kerbango? I remember a couple of ads for it, the idea was that adding a computer to a network and installing this would increase available storage space and back up all the other computers running the software on that network, basically a RAID of compleat machines....
Intending to solve a concrete problem doesn't automatically mean a product designed under that intent in fact solves a concrete problem.
The paragraph above the phrase you quote, addressing the very principles you mention:
"Most of those principles make sense only if you don’t ever try it in the real world, where you will discover that many of them are not just of questionable value, or would be fiercely opposed by many entities, but outright illegal. (One of the systems that most embodied the principles of robust decentralized storage of files, which moved them closer to users requesting them for efficiency, was… Freenet, which immediately became notorious for child pornography.)"
Gwern elaborates at length about the principles, the concrete prototypes and the dissonance.
Did you just ctrl+f "solution in search of a problem" to criticize the analysis?
> fiercely opposed by many entities, but outright illegal.
As I understand it, this fails to advances the broader "searching for a problem" argument. And is actually addressing a problem others have tried to solve later.
re. copyright holders fierce opposition: Youtube might be a good counter to the suggested tragedy of the anticommons. And on the high seas, napster's rise and the continued success of torrents suggest legally is not a barrier to success in and of it's self. The article reasonably ties copyright to free software. opensource has done pretty well. [The article's link back to ycombinator referencing Xanadu's allergic reaction to opening the code is an interesting candidate for why we're not using the OS now. Too locked down to get the love and exposure it needed? Bonus recursive side note: "has never believed in FLOSS" links to hackernews, demonstrating both back and forward linking -- another counterpoint to the presented augments against Xanadu's impracticality ]
Its not your (quoted) claim, but it's worth being explicit. Freenet is as legal as facebook or twitter[0]. People do illegal stuff with crypography. But it's ostensibly a way to protect from authoritarian overreach. Which still seems like a problem in search of a solution.
[0]: https://freenet.org/about/faq/#is-it-legal-to-run-a-freenet-...
> One example comes from U.S. law, used here for illustration rather than as a universal rule: 17 U.S.C. §512, part of the DMCA. It expressly addresses “transitory digital network communications” (automatic transmission, routing, and intermediate or transient storage) and “system caching” (certain automatic temporary storage of third-party material).
I don't understand what's your point. How does the existence of copyright breach bears on the fact that Xanadu was a fever dream and not a real solution to real problems and hence consigned to the list of fanciful technologies that never gained adoption?
I read the quoted text as "Xanadu's 'transclusion' can't work in the real world because of copyright and legal issues." But neither of those are issues for popular existing technologies.
Rereading your parent comment, I'm guessing my confusion is in connecting that quote to why Xanadu fails to solve "the concrete problem" or endorsing "a solution in search of a problem"/addressing "worse is better."
My comment was intended as a +1 to @Rygian: Xanadu was prescient wrt. file management and an opportunity to push back against Freenet=CSAM
I think Ted Nelson had a vision but not a good design. I agree with Gwern, his design was a materialization of his vision of a solution. It was not the response to a problem. I like Ted Nelson's ideas but sadly what he built was just not a very useful piece of software.
Hmm, I agree that the problems are real and very relevant and anadu intended to offer a solution. But it was simply never finished enough to actually solve them.
Sure, but that's not the "just a solution in search of a problem" claim made in the article. I don't think anyone including Ted Nelson has ever claimed that Xanadu was sufficiently complete at any point.
The other books on hardware from this timeframe such as Tracy Kidder's _The Soul of a New Machine_ or Jerry Kaplan's _StartUp_ or Andy Hertzfeld's _Revolution in The Valley: The Insanely Great Story of How the Mac Was Made_ (available as: https://folklore.org/0-index.html) are quite striking and forthright in their discussion of hardware limitations and the ingenuity needed to get things done.
Or, contrast the far more prosaic _Weaving the Web_ by Tim Berners-Lee and esp. take note of compromises such as URL changing from Universal Resource Locator to Uniform Resource Locator.
If git's pack index gets refactored slightly, it can do anything from trigram-indexed search to metadata indexing to hyperlinking both ways across repos. That is actually useful, I use it: https://replicated.live/blog/wiki
It's not a bad UI metaphor for limited applications, but it makes no sense at all for a global document store.
Humans like boundaries. Even ignoring the copyright issues, we like our cultural artefacts to have edges and a limited number of interaction modes - basically read/consume, write/publish, mention/reference, and converse/comment.
Massively cross-linked semantic soup isn't one of those modes. Text artefacts need to be stable - it's a core requirement - or at least mutable in very, limited and controlled ways.
Music has sampling and LLMs have style transfer, but text with mutable semantics is a weird animal and not really how we do things.
I am not in the contract negotiation business, but I had that experience.
Speaking of git-with-permalinks, it implies that you have some repos locally that may want to link to each other. Like your code in 4 different repos, one repo with tickets, one wiki, and some other stuff.
It is especially helpful if agents leave cross-references to have some trail for the decisions. Agents, generally speaking, have no memory, so each time they recover the context from scratch. When I read what they coded, I essentially do the same.
So, permalinks and transclusion are practically useful. T.H.Nelson picked up that idea in the movie industry, I hear. They furiously edit their scripts and contracts, I suppose.
I agree that transclusion, particularly based on byte offsets, is not very useful for the WWW. But the concept is much more useful than what the author suggests. Maybe you have heard of Ivar Jacobson's Objectory tool. I worked with it in the nineties in large projects and it was able to transclude terms and definitions wherever they were linked, even integrated in the text flow where they appeared. That was true added value and much more useful than tools like DOORS. I myself have implemented CrossLine, which I used for large project information aggregation, creating new documents with transcluded passages from specifications and minutes put into the most useful context. In CrossLine, the unit of transclusion is an outline item. Also Jacobson's tool had useful units. I think the failure of approaches like Xanadu was not an absence of meaningful transclusion use-cases; rather to identify and integrate the right units of knowledge.
Meanwhile the Jewish world has a project that shows Bible passages and if you click on them, it does connect you to a world of translations, commentary, references in other books. https://www.sefaria.org/Genesis.1.1?lang=bi&with=Commentary%.... Comparing this to the famous Xanadu demo mentioned in the article, makes you think that Xanadu could have worked. (Except for the micropayments thing).
There's an interesting comparison to HyperCard, which didn't have the intellectual scaffolding of Xanadu, but just worked, even on a typical Mac of the day with 1MB of RAM.
Until the stack got too large, or one forgot the ID of the card one wanted to link to or the search text which would allow one to make it a target, &c.
HyperCard was _amazing_ (and I still have my copy of _The Manhole: Where Alice would have gone if Alice had HyperCard_ which was the predecessor for _Myst_) and it kills me that we lost MacBasic:
and that HyperCard was such a mess of licensing that one had to use the "magic" word, or a command to enable Developer Mode and that it took so long to be developed/released:
(though one wonders what would have happen if it had instead been a NeXT product)
and then of course that the various alternatives had such tortured histories and of course the infamous "rug pull" when Live Code went closed source --- is there a viable free/opensource implementation which is nicer and more usable than Decker?
I first read about Xanadu very early in my life, and this project is an absolute miracle in terms of how much it can hold my interest for something that could not and never would work.
The crazy irritating zig-zagging split document view has always struck me as a demo that would be rounded off after "initial release", because it's still a useful feature underneath the stupid viewer-friendly UI. Anyone who has ever reviewed a Git PR or negotiated a contract has already had to do what Xanadu was built to do. It's just that lawyers and programmers invented user interfaces that actually make sense for that, while the Xanadu version is hyper-optimized for the worst possible case.
Still, it is reasonable to assume that once people start writing documents "Xanalogically" - i.e. online, in the way of a Wikipedia article - that there would be a need for a viewable history of one's edits, and that someone might want to reply to a document with inline commentary. It's really everything else that didn't make it into WorldWideWeb or WikiWikiWeb that makes my head spin.
Like, how could two-way links work? I mean, they can't, obviously? Links go around text, so they have to live on a document. The author here mentions spam, but there's a far simpler structural problem that making links two-way means the links have to be stored... say it with me folks, as a stream of annotations on top of the underlying text! Links would go "on top of" your document, linking word 153 of paragraph 4 of document A onto word 96 of paragraph 13 of document B.
But what happens if document B changes? Then it's going to have invalid backlinks! Well, it does work, because in Xanadu... all the documents you write would be transclusions of a "permascroll". Everything you ever write goes in the permascroll and then your Xanadu client would transparently publish a transclusion of your edited document. So "word 96 of paragraph 13 of document B" somehow gains actual referential integrity across changes to the document.
It is the most NLE-ass idea ever, and I didn't make the connection to video editing until you mentioned Ted having Hollywood ambitions. Because basically every video editing tool on a computer ever is extremely Xanalogical. In a Hollywood shoot or a TV show, you basically keep your cameras rolling throughout the shoot, you mark off your shots with a clap board, and then the editor assembles an actual timeline from the source footage. During this process everything is nondestructively transcluded right up until the final export[2].
But, even then, there is no way in hell I could go and transclude someone else's movie straight from their source footage and make a commentary on it. Like, not even in today's environment of YouTube video essays longer than the films they're talking about. It just wouldn't be possible, not for technical reasons but legal ones.
If Ted had actually made it into Hollywood, he would have learned that every reuse of one's work needs to be individually negotiated by a team of lawyers, no matter how petty the use, because all the suits running Hollywood are Valentian rapists[1] (if not actual rapists) who will take advantage of you the moment you provide any leeway. Anyone who doesn't understand this gets taken advantage of until they burn out or fight back. There is no world in which copyright is respected but I'm also allowed to publish a book containing half of your book without your permission. The law saying you're not allowed to do this is older than the US Civil War and decided in a court case[0] that singlehandedly turned copyright from a straightforward inquiry into an insanely fact-intensive question that never gets resolved.
Of course, this was supposed to have been pre-negotiated, which almost makes sense if Xanadu was just weird futuristic Lexis-Nexus, but Ted Nelson also wanted this thing to be an open standard. This would have made piracy extremely straightforward! And I think that's what makes Xanadu so interesting. It's trying to do literally everything and spending comical amounts of effort at it, inventing insane things like Enfilades and tumbler numbers[3], all to fall flat on its face with nothing to show for it.
[2] Of course, in 1960 NLEs didn't exist. Film (and video) was edited destructively with actual scissors and splices.
[3] I legitimately thought tumblr was named after these things for like years but it turns out, no, it's named after a "tumblelog" which is basically proto-Twitter.
>Like, how could two-way links work? I mean, they can't, obviously? Links go
>around text, so they have to live on a document. The author here mentions spam,
>but there's a far simpler structural problem that making links two-way means
>the links have to be stored... say it with me folks, as a stream of annotations
>on top of the underlying text! Links would go "on top of" your document,
>linking word 153 of paragraph 4 of document A onto word 96 of paragraph 13 of
>document B.
>
>But what happens if document B changes? Then it's going to have invalid
>backlinks! Well, it does work, because in Xanadu... all the documents you write
>would be transclusions of a "permascroll". Everything you ever write goes in
>the permascroll and then your Xanadu client would transparently publish a
>transclusion of your edited document. So "word 96 of paragraph 13 of document
>B" somehow gains actual referential integrity across changes to the document.
If I recall correctly, the site you browse in hypotetical Xanadu-land has to mirror the reference (for transclusions) and get downloaded back (for two-ways links). Payments and that levy system are there because without some freedom to copy content you get dead links with extra steps.
Well, he had to have at least some passing knowledge of the film industry. From Wikipedia: "Nelson is the son of Emmy Award–winning director Ralph Nelson and Academy Award – winning actress Celeste Holm."
Your musing about 2-way links reminded me of the short-lived "webhooks for backlinks" trend, which didn't last long before spammers found and abused it.
The idea was that you had a well-known webhook location, and when people linked to your blog post, they'd ping your webhook with their URL, and you'd display it under your blog post as a "backlink".
If that's obviously a way to inject marketing spam you'd be right, but I swear this stuff got quite popular before being overran.
Unfortunately google is now so bad, it's impossible to find genuine history of the phenomena I'm describing, so I just have to trust my memory that I'm not inventing a past that didn't exist.
This was an amazing read. I try to make hyper-documents as Nelson wanted and I have seen much of the contradictions. Yet I still believe you have to give people a hundred ideas for them to use 10. He did just that.
Exactly! I worked with a couple ex Xanadu acolytes at Apple back in the day. What they took away from the experience were the many inspirations rather than the project as a model for organizations. One of them brought him in to speak and he spent the time exhorting us to think beyond WYSIWIG (be careful about how your metaphors limit what your avoidances can be) which at the time I found rather eye-opening.
Xanadu suffered from the same disease as the world building hobby of amateur writers.
In the absence of adversarial dynamics, anything you dream up seems possible. But reality is bound by predator-prey dynamics that sap the free energy from any high energy configuration. Good writers and good engineers understand that your design must settle into some low energy configuration, otherwise it won't be feasible or competitive.
"Worse-is-better", as the original author puts it, is an ambiguous position at best:
"You might think that by the year 2000 I would have settled what I think of worse is better - after over a decade of thinking and speaking about it, through periods of clarity and periods of muck, and through periods of multi-mindedness on the issues. But, at OOPSLA 2000, I was scheduled to be on a panel entitled "Back to the Future: Is Worse (Still) Better?" And in preparation for this panel, the organizer, Martine Devos, asked me to write a position paper, which I did, called "Back to the Future: Is Worse (Still) Better?" In this short paper, I came out against worse is better. But a month or so later, I wrote a second one, called "Back to the Future: Worse (Still) is Better!" which was in favor of it. I still can’t decide."
Undecidability at play I guess. We can't precompute every possible future and know which algoriths terminate. We must search, and a local minimum might be in fact high energy after all. Better and worse are judgement calls, and "the right thing" depends on your choice of definitions.
Congruence with reality always wins and reality appears to be made of distributed intelligence, despite the hard problem of the mind precluding a final answer.
The world is too mutable for it to exist as a decentralized system, you can set up all these permascrolls and enfilades and tumblers and stuff. But someone is going to see profit in making two-way links one-way or altering the permanent record, so they will. Not to mention what happens just by entropy, servers will get turned off because companies go under or people lose interest. You could make it everybody's problem with something like IPFS, you still would have to solve spammers exploiting the commons.
And it only makes sense for a place where documents are approximately equally popular. If the canonical Harry Potter contained links to all places referring or citing it, it would be one overloaded document.
And then the whole micropayments thing would make it a non-starter, since that have failed badly just about every time it has been tried.
Their grandiose plan was to install microtransactions on EVERYTHING, so that even hitting a simple webpage could be $10 to read. This is the libertarian utopia, where only the rich can read and learn.
It would have been the death knell for communication if every word had a bloody pricetag.
So good fucking riddance. I hope Xanadu stays in the trash-heap of history.
The Xanadu proposal dates from the mid-20th century so it includes a quaint assumption from that era: people expect to be paid for the work they do. What, people are supposed to spend hours typing at a computer for no payment in order to make some billionaire even richer? Who would do that? It's nuts.
Yes, if Xanadu had taken off people might have had to pay a little to view its content. Just like we used to pay for books, records, movies, magazines, etc etc etc. It wasn't so bad -- those things were mass entertainment, not just for the rich, and they were certainly not a death knell for communication. I don't know how you came up with $10 to view a web page. When Nelson proposed Xanadu, mass market paperback novels -- hundreds of pages -- cost 50 cents. Kids (like me) could easily afford them. They were cheap because there was a huge market -- so communication could flourish.
It's not clear to me we are living in the best of all possible worlds. If the micropayments thing had taken off, it might have provided a way for large numbers of people to make some money by writing, etc. In our world, most of the money is made by a few corporations, and an increasing proportion of what's on the net is slop made by ... nobody.
Excellent points. When Nelson was building Xanadu in the 60s-80s there was no expectation that the user would be the product. Which we have today, things like Google workspace, Office, YouTube, social media etc letting the user use software for "free" in the sense that their is no visible micro traction but an invisible harvesting of your habits and interests to sell to advertisers or whoever wants to buy it. Nelson simply didn't predict this.
Can you not think in first, second or third derivative actions with a system that bakes in micropayments at the deepest infrastructure level?
Should people be able to get paid for content they made? Sure. But we have that already - copyright. And it's also already so askew to the largest companies. The small folk are ALREADY squeezed out.
I could imagine trap Xanadu-pages that whose goal is to charge for garbage. Cents at a time, just under whatever limits you have on the micropayment lever. But a page could drain dollars. And there'd be millions of them. Traps everywhere.
Wait, so you can generate garbage content, link it everywhere? Yeah the LLM in a Xanadu system would create billion-page webs that *appear* useful, but whose only goal is 'drain your wallet'. And pages can be generated faster than you can block.
Xanadu also completely upends the idea of public domain - republish and start that money counter going! Republish republish republish. Or steal your content, republish, and rake in the microtransactions. Again, I can do it faster than you can catch me.
This whole Xanadu garbage is predicated upon the root idea that only monied people should have the right to read and access content. I am glad it is in the dumpster of dead technologies. Frankly, all libertarian extreme technologies need to be destroyed.
Also not to be confused with the 1980 film starring Olivia Newton-John with a great soundtrack by her, Electric Light Orchestra, The Tubes, and Cliff Richard.
... Ted Nelson’s 1974 manifesto, Computer Lib/Dream Machines, extolling the vision of Project Xanadu hypertext.
Computer Lib/Dream Machines was not about project Xanadu, it merely mentions it on a few pages.
I recall Computer Lib/Dream Machines was the most influential thing Ted Nelson did, far more influential by than Xanadu. It was seen all over at the beginning of the personal computer era. Try to get a look at a copy if you can, it's fascinating.
This self-published book was very much of its time - it had the same oversize-on-newsprint format and DIY look as the Whole Earth Catalog, which was popular at the same time.
It was important because it pursuasively made the case that computing was about to become a medium for self-expresson - a "dream machine" - that was an unheard-of concept at the time. You can use a computer to produce graphics! To play games! How about that! He told about Sutherland's Sketchpad, Englebart's NLS, and the Arpanet at a time when these were completely invisible to almost everyone. Xanadu gets a few pages in the book but it is not at all a work of self-promotion - almost the whole book is about the work of others. It's a survey of the computing scene right before personal computers appeared.
This book was very timely and I think it provided a lot of inspiration for the personal computer movement. It made you want to try out computing! It has one page about microprocessors: "Here they come ... Microprocessors are what's happening" (the MITS Altair appeared the next year, in 1975).
I think Ted Nelson is one of those people whose main effects are through their influence on others, rather than through what they directly produce themselves.
> Xanadu wasn’t the victim of “Worse is Better”; it was just a solution in search of a problem.
Well, in the day to day use of computers across everyone I interact with, the following problems are very much present and still unsolved (savvy users can address them and transform them into chores; non-savvy users solve some of them by paying money to pospone the issues):
- I don't have any space left on my phone but my laptop has tons of space available. (Addressed by Xanadu principle 13.)
- How do I back up my stuff uniformly across my devices. (Addressed by Xanadu principle 14.)
- How do I share this file with a relative. (Addressed by Xanadu principles 2+4+10+11+12.)
- How do I not get trapped by vendors who want to lock me into their cloud solution. (Addressed by Xanadu principles 1, 2, 3, 15, 17.)
So the problems are very much there, and not improving. If anything, Xanadu was a prescient solution that addressed back them the problems we have today.
Xanadu principle have outlined important problems. This does not mean that Xanadu created practical solution, though: it seems they were focused on text transclusions UI for some reason...
If the goal of Xanadu was to offer a better, more principled form of information sharing, they should have focused 99% of effort on the protocols & backend implementation, leaving UI rudimentary. E.g. TUI might be sufficient to browser hypertext, and that would be compatible with ~all systems, unlike their GUI contraptions.
There was a commercial networking product which addressed principle 13 as well --- Kerbango? I remember a couple of ads for it, the idea was that adding a computer to a network and installing this would increase available storage space and back up all the other computers running the software on that network, basically a RAID of compleat machines....
Intending to solve a concrete problem doesn't automatically mean a product designed under that intent in fact solves a concrete problem.
The paragraph above the phrase you quote, addressing the very principles you mention:
"Most of those principles make sense only if you don’t ever try it in the real world, where you will discover that many of them are not just of questionable value, or would be fiercely opposed by many entities, but outright illegal. (One of the systems that most embodied the principles of robust decentralized storage of files, which moved them closer to users requesting them for efficiency, was… Freenet, which immediately became notorious for child pornography.)"
Gwern elaborates at length about the principles, the concrete prototypes and the dissonance.
Did you just ctrl+f "solution in search of a problem" to criticize the analysis?
> fiercely opposed by many entities, but outright illegal.
As I understand it, this fails to advances the broader "searching for a problem" argument. And is actually addressing a problem others have tried to solve later.
re. copyright holders fierce opposition: Youtube might be a good counter to the suggested tragedy of the anticommons. And on the high seas, napster's rise and the continued success of torrents suggest legally is not a barrier to success in and of it's self. The article reasonably ties copyright to free software. opensource has done pretty well. [The article's link back to ycombinator referencing Xanadu's allergic reaction to opening the code is an interesting candidate for why we're not using the OS now. Too locked down to get the love and exposure it needed? Bonus recursive side note: "has never believed in FLOSS" links to hackernews, demonstrating both back and forward linking -- another counterpoint to the presented augments against Xanadu's impracticality ]
Its not your (quoted) claim, but it's worth being explicit. Freenet is as legal as facebook or twitter[0]. People do illegal stuff with crypography. But it's ostensibly a way to protect from authoritarian overreach. Which still seems like a problem in search of a solution.
[0]: https://freenet.org/about/faq/#is-it-legal-to-run-a-freenet-... > One example comes from U.S. law, used here for illustration rather than as a universal rule: 17 U.S.C. §512, part of the DMCA. It expressly addresses “transitory digital network communications” (automatic transmission, routing, and intermediate or transient storage) and “system caching” (certain automatic temporary storage of third-party material).
I don't understand what's your point. How does the existence of copyright breach bears on the fact that Xanadu was a fever dream and not a real solution to real problems and hence consigned to the list of fanciful technologies that never gained adoption?
I read the quoted text as "Xanadu's 'transclusion' can't work in the real world because of copyright and legal issues." But neither of those are issues for popular existing technologies.
Rereading your parent comment, I'm guessing my confusion is in connecting that quote to why Xanadu fails to solve "the concrete problem" or endorsing "a solution in search of a problem"/addressing "worse is better."
My comment was intended as a +1 to @Rygian: Xanadu was prescient wrt. file management and an opportunity to push back against Freenet=CSAM
I think Ted Nelson had a vision but not a good design. I agree with Gwern, his design was a materialization of his vision of a solution. It was not the response to a problem. I like Ted Nelson's ideas but sadly what he built was just not a very useful piece of software.
Hmm, I agree that the problems are real and very relevant and anadu intended to offer a solution. But it was simply never finished enough to actually solve them.
Sure, but that's not the "just a solution in search of a problem" claim made in the article. I don't think anyone including Ted Nelson has ever claimed that Xanadu was sufficiently complete at any point.
The other books on hardware from this timeframe such as Tracy Kidder's _The Soul of a New Machine_ or Jerry Kaplan's _StartUp_ or Andy Hertzfeld's _Revolution in The Valley: The Insanely Great Story of How the Mac Was Made_ (available as: https://folklore.org/0-index.html) are quite striking and forthright in their discussion of hardware limitations and the ingenuity needed to get things done.
The book _CAD History_, available on-line at: https://www.shapr3d.com/history-of-cad/introduction has a bit on Xanadu from the Autodesk end of things.
Or, contrast the far more prosaic _Weaving the Web_ by Tim Berners-Lee and esp. take note of compromises such as URL changing from Universal Resource Locator to Uniform Resource Locator.
I really like the analogy of “pinball” from _The Soul of a New Machine_. It matches my imagination of good engineers perfectly.
Racing the Beam is another one, about the design of the frame buffer-less Atari 2600.
I always believed that side-by-side diff view is a transclusion interface.
Also, permalinks can work on top of git quite nicely https://replicated.live/blog/link
If git's pack index gets refactored slightly, it can do anything from trigram-indexed search to metadata indexing to hyperlinking both ways across repos. That is actually useful, I use it: https://replicated.live/blog/wiki
CRDT merges are a very incremental and useful upgrade for git https://replicated.live/blog/meta
Xanadu-like functionality is perfectly within reach.
We don't get what we want or what we need. Instead, we move erratically in very small, easy, local steps.
Word's Track Changes is very transclusiony.
It's not a bad UI metaphor for limited applications, but it makes no sense at all for a global document store.
Humans like boundaries. Even ignoring the copyright issues, we like our cultural artefacts to have edges and a limited number of interaction modes - basically read/consume, write/publish, mention/reference, and converse/comment.
Massively cross-linked semantic soup isn't one of those modes. Text artefacts need to be stable - it's a core requirement - or at least mutable in very, limited and controlled ways.
Music has sampling and LLMs have style transfer, but text with mutable semantics is a weird animal and not really how we do things.
I am not in the contract negotiation business, but I had that experience.
Speaking of git-with-permalinks, it implies that you have some repos locally that may want to link to each other. Like your code in 4 different repos, one repo with tickets, one wiki, and some other stuff.
It is especially helpful if agents leave cross-references to have some trail for the decisions. Agents, generally speaking, have no memory, so each time they recover the context from scratch. When I read what they coded, I essentially do the same.
So, permalinks and transclusion are practically useful. T.H.Nelson picked up that idea in the movie industry, I hear. They furiously edit their scripts and contracts, I suppose.
I agree that transclusion, particularly based on byte offsets, is not very useful for the WWW. But the concept is much more useful than what the author suggests. Maybe you have heard of Ivar Jacobson's Objectory tool. I worked with it in the nineties in large projects and it was able to transclude terms and definitions wherever they were linked, even integrated in the text flow where they appeared. That was true added value and much more useful than tools like DOORS. I myself have implemented CrossLine, which I used for large project information aggregation, creating new documents with transcluded passages from specifications and minutes put into the most useful context. In CrossLine, the unit of transclusion is an outline item. Also Jacobson's tool had useful units. I think the failure of approaches like Xanadu was not an absence of meaningful transclusion use-cases; rather to identify and integrate the right units of knowledge.
Meanwhile the Jewish world has a project that shows Bible passages and if you click on them, it does connect you to a world of translations, commentary, references in other books. https://www.sefaria.org/Genesis.1.1?lang=bi&with=Commentary%.... Comparing this to the famous Xanadu demo mentioned in the article, makes you think that Xanadu could have worked. (Except for the micropayments thing).
It's like the Word of God succeeded where everything else failed. Probably some specific lessons to be learned there.
There's an interesting comparison to HyperCard, which didn't have the intellectual scaffolding of Xanadu, but just worked, even on a typical Mac of the day with 1MB of RAM.
Sure!
Until the stack got too large, or one forgot the ID of the card one wanted to link to or the search text which would allow one to make it a target, &c.
HyperCard was _amazing_ (and I still have my copy of _The Manhole: Where Alice would have gone if Alice had HyperCard_ which was the predecessor for _Myst_) and it kills me that we lost MacBasic:
https://www.folklore.org/MacBasic.html
and that HyperCard was such a mess of licensing that one had to use the "magic" word, or a command to enable Developer Mode and that it took so long to be developed/released:
https://www.folklore.org/Joining_Apple_Computer.html
(though one wonders what would have happen if it had instead been a NeXT product)
and then of course that the various alternatives had such tortured histories and of course the infamous "rug pull" when Live Code went closed source --- is there a viable free/opensource implementation which is nicer and more usable than Decker?
I first read about Xanadu very early in my life, and this project is an absolute miracle in terms of how much it can hold my interest for something that could not and never would work.
The crazy irritating zig-zagging split document view has always struck me as a demo that would be rounded off after "initial release", because it's still a useful feature underneath the stupid viewer-friendly UI. Anyone who has ever reviewed a Git PR or negotiated a contract has already had to do what Xanadu was built to do. It's just that lawyers and programmers invented user interfaces that actually make sense for that, while the Xanadu version is hyper-optimized for the worst possible case.
Still, it is reasonable to assume that once people start writing documents "Xanalogically" - i.e. online, in the way of a Wikipedia article - that there would be a need for a viewable history of one's edits, and that someone might want to reply to a document with inline commentary. It's really everything else that didn't make it into WorldWideWeb or WikiWikiWeb that makes my head spin.
Like, how could two-way links work? I mean, they can't, obviously? Links go around text, so they have to live on a document. The author here mentions spam, but there's a far simpler structural problem that making links two-way means the links have to be stored... say it with me folks, as a stream of annotations on top of the underlying text! Links would go "on top of" your document, linking word 153 of paragraph 4 of document A onto word 96 of paragraph 13 of document B.
But what happens if document B changes? Then it's going to have invalid backlinks! Well, it does work, because in Xanadu... all the documents you write would be transclusions of a "permascroll". Everything you ever write goes in the permascroll and then your Xanadu client would transparently publish a transclusion of your edited document. So "word 96 of paragraph 13 of document B" somehow gains actual referential integrity across changes to the document.
It is the most NLE-ass idea ever, and I didn't make the connection to video editing until you mentioned Ted having Hollywood ambitions. Because basically every video editing tool on a computer ever is extremely Xanalogical. In a Hollywood shoot or a TV show, you basically keep your cameras rolling throughout the shoot, you mark off your shots with a clap board, and then the editor assembles an actual timeline from the source footage. During this process everything is nondestructively transcluded right up until the final export[2].
But, even then, there is no way in hell I could go and transclude someone else's movie straight from their source footage and make a commentary on it. Like, not even in today's environment of YouTube video essays longer than the films they're talking about. It just wouldn't be possible, not for technical reasons but legal ones.
If Ted had actually made it into Hollywood, he would have learned that every reuse of one's work needs to be individually negotiated by a team of lawyers, no matter how petty the use, because all the suits running Hollywood are Valentian rapists[1] (if not actual rapists) who will take advantage of you the moment you provide any leeway. Anyone who doesn't understand this gets taken advantage of until they burn out or fight back. There is no world in which copyright is respected but I'm also allowed to publish a book containing half of your book without your permission. The law saying you're not allowed to do this is older than the US Civil War and decided in a court case[0] that singlehandedly turned copyright from a straightforward inquiry into an insanely fact-intensive question that never gets resolved.
Of course, this was supposed to have been pre-negotiated, which almost makes sense if Xanadu was just weird futuristic Lexis-Nexus, but Ted Nelson also wanted this thing to be an open standard. This would have made piracy extremely straightforward! And I think that's what makes Xanadu so interesting. It's trying to do literally everything and spending comical amounts of effort at it, inventing insane things like Enfilades and tumbler numbers[3], all to fall flat on its face with nothing to show for it.
[0] https://en.wikipedia.org/wiki/Folsom_v._Marsh
[1] https://en.wikipedia.org/wiki/Jack_Valenti#Valenti_on_new_te...
[2] Of course, in 1960 NLEs didn't exist. Film (and video) was edited destructively with actual scissors and splices.
[3] I legitimately thought tumblr was named after these things for like years but it turns out, no, it's named after a "tumblelog" which is basically proto-Twitter.
> If Ted had actually made it into Hollywood
Well, he had to have at least some passing knowledge of the film industry. From Wikipedia: "Nelson is the son of Emmy Award–winning director Ralph Nelson and Academy Award – winning actress Celeste Holm."
Your musing about 2-way links reminded me of the short-lived "webhooks for backlinks" trend, which didn't last long before spammers found and abused it.
The idea was that you had a well-known webhook location, and when people linked to your blog post, they'd ping your webhook with their URL, and you'd display it under your blog post as a "backlink".
If that's obviously a way to inject marketing spam you'd be right, but I swear this stuff got quite popular before being overran.
Unfortunately google is now so bad, it's impossible to find genuine history of the phenomena I'm describing, so I just have to trust my memory that I'm not inventing a past that didn't exist.
I remember those. Trackbacks, right? Back when we were using MovableType?
Trackbacks! Yes thank you, I'd forgotten the name.
This was an amazing read. I try to make hyper-documents as Nelson wanted and I have seen much of the contradictions. Yet I still believe you have to give people a hundred ideas for them to use 10. He did just that.
Exactly! I worked with a couple ex Xanadu acolytes at Apple back in the day. What they took away from the experience were the many inspirations rather than the project as a model for organizations. One of them brought him in to speak and he spent the time exhorting us to think beyond WYSIWIG (be careful about how your metaphors limit what your avoidances can be) which at the time I found rather eye-opening.
What an interesting thing to say, and what a good working environment you had!
Do you think Xanadu would have had a chance if it had been built with today's technology?
Xanadu suffered from the same disease as the world building hobby of amateur writers.
In the absence of adversarial dynamics, anything you dream up seems possible. But reality is bound by predator-prey dynamics that sap the free energy from any high energy configuration. Good writers and good engineers understand that your design must settle into some low energy configuration, otherwise it won't be feasible or competitive.
"Worse-is-better", as the original author puts it, is an ambiguous position at best:
"You might think that by the year 2000 I would have settled what I think of worse is better - after over a decade of thinking and speaking about it, through periods of clarity and periods of muck, and through periods of multi-mindedness on the issues. But, at OOPSLA 2000, I was scheduled to be on a panel entitled "Back to the Future: Is Worse (Still) Better?" And in preparation for this panel, the organizer, Martine Devos, asked me to write a position paper, which I did, called "Back to the Future: Is Worse (Still) Better?" In this short paper, I came out against worse is better. But a month or so later, I wrote a second one, called "Back to the Future: Worse (Still) is Better!" which was in favor of it. I still can’t decide."
https://dreamsongs.com/WorseIsBetter.html
Undecidability at play I guess. We can't precompute every possible future and know which algoriths terminate. We must search, and a local minimum might be in fact high energy after all. Better and worse are judgement calls, and "the right thing" depends on your choice of definitions.
Congruence with reality always wins and reality appears to be made of distributed intelligence, despite the hard problem of the mind precluding a final answer.
The world is too mutable for it to exist as a decentralized system, you can set up all these permascrolls and enfilades and tumblers and stuff. But someone is going to see profit in making two-way links one-way or altering the permanent record, so they will. Not to mention what happens just by entropy, servers will get turned off because companies go under or people lose interest. You could make it everybody's problem with something like IPFS, you still would have to solve spammers exploiting the commons.
And it only makes sense for a place where documents are approximately equally popular. If the canonical Harry Potter contained links to all places referring or citing it, it would be one overloaded document.
And then the whole micropayments thing would make it a non-starter, since that have failed badly just about every time it has been tried.
Im completely glad Xanadu died.
Their grandiose plan was to install microtransactions on EVERYTHING, so that even hitting a simple webpage could be $10 to read. This is the libertarian utopia, where only the rich can read and learn.
It would have been the death knell for communication if every word had a bloody pricetag.
So good fucking riddance. I hope Xanadu stays in the trash-heap of history.
The Xanadu proposal dates from the mid-20th century so it includes a quaint assumption from that era: people expect to be paid for the work they do. What, people are supposed to spend hours typing at a computer for no payment in order to make some billionaire even richer? Who would do that? It's nuts.
Yes, if Xanadu had taken off people might have had to pay a little to view its content. Just like we used to pay for books, records, movies, magazines, etc etc etc. It wasn't so bad -- those things were mass entertainment, not just for the rich, and they were certainly not a death knell for communication. I don't know how you came up with $10 to view a web page. When Nelson proposed Xanadu, mass market paperback novels -- hundreds of pages -- cost 50 cents. Kids (like me) could easily afford them. They were cheap because there was a huge market -- so communication could flourish.
It's not clear to me we are living in the best of all possible worlds. If the micropayments thing had taken off, it might have provided a way for large numbers of people to make some money by writing, etc. In our world, most of the money is made by a few corporations, and an increasing proportion of what's on the net is slop made by ... nobody.
Excellent points. When Nelson was building Xanadu in the 60s-80s there was no expectation that the user would be the product. Which we have today, things like Google workspace, Office, YouTube, social media etc letting the user use software for "free" in the sense that their is no visible micro traction but an invisible harvesting of your habits and interests to sell to advertisers or whoever wants to buy it. Nelson simply didn't predict this.
Can you not think in first, second or third derivative actions with a system that bakes in micropayments at the deepest infrastructure level?
Should people be able to get paid for content they made? Sure. But we have that already - copyright. And it's also already so askew to the largest companies. The small folk are ALREADY squeezed out.
I could imagine trap Xanadu-pages that whose goal is to charge for garbage. Cents at a time, just under whatever limits you have on the micropayment lever. But a page could drain dollars. And there'd be millions of them. Traps everywhere.
Wait, so you can generate garbage content, link it everywhere? Yeah the LLM in a Xanadu system would create billion-page webs that *appear* useful, but whose only goal is 'drain your wallet'. And pages can be generated faster than you can block.
Xanadu also completely upends the idea of public domain - republish and start that money counter going! Republish republish republish. Or steal your content, republish, and rake in the microtransactions. Again, I can do it faster than you can catch me.
This whole Xanadu garbage is predicated upon the root idea that only monied people should have the right to read and access content. I am glad it is in the dumpster of dead technologies. Frankly, all libertarian extreme technologies need to be destroyed.
Not to be confused with ATS 3 Xanadu <https://github.com/githwxi/ATS-Xanadu/>
Also not to be confused with the 1980 film starring Olivia Newton-John with a great soundtrack by her, Electric Light Orchestra, The Tubes, and Cliff Richard.
Also not to be confused with the place where Kubla Khan decreed a stately pleasure done.
Gaze upon my blogs, and despair.
[dead]
Ah, the could of been internet if the guy hadn't flaked by having all those sex parties