It is actually pretty easy to keep alive URLs if there is a will to do so. Just have tests in place so you can't mess them up accidentally. And convert things that are not changing anymore to static html.
One of my first web projects involved a music forum and I have been keeping the URLs alive for 24 years now:
It's not easy. Part of the instructions would need to contain things like "how to convince your boss to not just delete the old thing" and "what to do when the startup you worked for went bankrupt" among others. Most URLs don't disappear by mistake.
Well, and pay registrar and hosting fees for 100 years, and then just hope the hosting company doesn't fold in the interim after your passing. The problem isn't "can you make sure it stays up while you're alive", the problem is, after you're gone, you're no longer there to keep the flame alive and there are no guarantees anyone else picks up the torch.
Does this suggest a subscription(ish) business where you pay the hosting company for N years and then they ensure your website stays up and unchanged for N years after you die? Quality control issue?
Given how many things want me to pay a subscription while I'm alive, surely somebody will start charging me when I'm dead. It is the logical conclusion of SaaS. I wanted to create an acronym for this, but death as a service didn't seem like the best choice.
> It is actually pretty easy to keep alive URLs if there is a will to do so. Just have tests in place
"Just".
Maintaining and keeping these tests (and the oriinal link) running is also probably easy. "Just" keep maintaining them over changes in technology, hosting providers, DNS registrars...
I think that recent LLMs do pass the Turing test, but I don't think it is a good test for intelligence, for two reasons: emergent complexity and pareidolia.
To determine whether a machine is intelligent, I think it's necessary to understand how it works internally and not just take a black-box approach. This was already understood when Alan Turing wrote his 1950 paper. Elmer and Elsie, William Grey Walter's tortoise robots built during the preceding two years, appeared to behave as if they had some degree of self-awareness. But their "brains" consisted of only two thermionic valves, so self-awareness can be ruled out.
Most of the LLM tells we usually recognise are due to their heavy attunement to role playing a "helpful assistant" with a knack for correct typography.
If you used a smaller corpus, followed a different kind of fine tuning routine with rather normal conversation patterns, and wrote a different base prompt, I don’t see why it shouldn’t be possible to come up with a very particular personality, so to speak.
>while ELIZA has a comforting effect that many crave, PARRY often actively makes users uncomfortable and afraid
PARRY is my spirit animal.
> In empirical evaluations conducted during the 1970s, experienced psychiatrists proved unable to reliably distinguish PARRY's responses from those of actual patients diagnosed with paranoid disorders, achieving only 48 percent accuracy, a figure statistically indistinguishable from random guessing.
> You'd think that LLMs fulfill this, but I do wonder if a clever human could still discern between them given their particularities.
Yes, especially if you use the original adversarial setup with three simultaneous participants (judge, two candidates) as described in Turing's original paper.
However, I'm not quite sure how long even that one will last.
Wayback shows #6 as "By 2010, more than 50 percent of books sold worldwide will be printed on demand at the point of sale in the form of library-quality paperbacks."
It was removed early 2011. Is this what happens when they can't decide on who won?
>You'd think that LLMs fulfill this, but I do wonder if a clever human could still discern between them given their particularities.
While LLMs are very capable of appearing human in general conversation, the test is that they are trying to convince a human that they are human, which is not something that is in their training data and I suspect they would go about things in very odd ways from an actual person, especially when challenged by an actual person. It is still very easy to back LLMs into a corner where they fall into very unnatural patterns and in my experience challenging them on their "humanness" (or anything) backs them into that corner.
LLMs also still can't use context or subtext in natural ways and even people who are terrible with purposefully exploiting context and subtext, generally have a natural ability that they developed to use and parse them. In a short conversation that is not adversarial, this is not an issue but falls apart as the length of the conversation grows or they start getting challenged.
LLMs just don't play with ideas and conversation like we do, they don't riff on it and go with the flow without care, picking up on subtexts and contexts and manipulating them the way people constantly do without effort until they find themselves on the edge of a cliff looking into the void and debating if they should jump. If we are put into the situation of proving we are human (in an academic test sort of way) we will play with it and have fun with it, and we will find ourselves on that cliff looking into the void, debating if we should expose ourselves and admit things we normally wouldn't. We pick up on this when people do it, even over the internet, their way of stating things changes and their subtext falls away or drastically changes; I have never seen anything approaching this in LLMs.
I don't think any frontier lab tried to make an LLM optimized for the literal Turing test where the LLM is made to argue that it's human. This is not in their interest. They make their LLMs insist they aren't human, have no feelings and sentience etc etc.
So we don't actually know how good LLMs can be in this.
That is absolutely true of the chatbots we have access to but I suspect there are people working towards these ends, including all the big names in the field, major win for their company if they manage it. But even then, I have yet to see a model that could naturally handle subtext and context, they struggle even when dealing with programming languages where these things are far simpler than they are in human language.
There is also the problem with how easy it is to back them into a corner where they start acting in very unnatural ways. This would arise when challenging them on their humanity and most people would pick up on it and craft their challenges around it until it fell apart because that is how people work in such adversarial situations.
I'm agnostic on the matter because post training and RLHF on this specific capability could be quite effective. Nobody ever tried because it would cause a backlash and bad PR. Much better PR to solve math problems, help run businesses, cure disease etc than to pretend it's human. The only business case for pretend human is AI boy/girlfriends but those are also not most profitable at their most realistic state.
> LLMs just don't play with ideas and conversation like we do, they don't riff on it and go with the flow without care, picking up on subtexts and contexts and manipulating them the way people constantly do without effort until they find themselves on the edge of a cliff looking into the void and debating if they should jump.
> You'd think that LLMs fulfill this, but I do wonder if a clever human could still discern between them given their particularities.
I think the clear winning strategy is to find some of the common sense reasoning fails that are still quite common, like the "should I walk or drive to the car wash" one from a few months ago.
These are usually from a missing context (i.e. specifying where the car is right now). Tokenizer issues (how many Xs are in Y) are a better giveaway, but models can solve them using tools nowadays. I think the best way to uncover an LLM is to reveal its superhuman aspects. Ask it for an A* algorithm for your problem. If it can just type down a working algorithm in a few seconds, that is definitely not a human. Although with some careful system prompt design, you could probably mask that aspect too.
> These are usually from a missing context (i.e. specifying where the car is right now).
The reason for the mistake is irrelevant here. All that matters is that you catch the agent saying something that basically no human would, especially in this type of game.
In a true competitive Turing test, these reasoning pitfalls could be included in the system prompt, there are not that many of them. The tester would need to find a novel reasoning pitfall, but those aren't uncommon among humans either, so that would not make a very useful differentiator.
Another approach could be to probe the helpfulness of an AI. A human would likely not comply with a request to write a sonnet about tangerines.
I don't understand what you mean that "there are not that many of them". The models, even the best ones, still have clear problems with certain kinds of real world reasoning, and it's not a known, closed set. With some prodding and knowing this error mode, I think it's not hard to trip any model with a novel reasoning error like this.
And no, I don't think humans would make the same errors, the key reason for this failure mode is that there are some facts of human day to day life that are so mundane that they are virtually never written about, thus making them an ideal combination of universal human experiences but out of distribution for the LLMs.
> You'd think that LLMs fulfill this, but I do wonder if a clever human could still discern between them given their particularities.
You don't even have to be that clever. We've all tried using it for work. They fail.
In the first place, the Turing test was always a loosely defined thought experiment. I suppose it's still serving that purpose, but at least half of HN takes it way too seriously. It's absolutely not proof of <engagementbait> endorsed by Alan Turing himself.
I think if you trained an LLM specifically for the purpose of passing the Turing test (instead of being helpful, harmful, and so on), its likely it would pass it.
You would have to train it/finetune it on a couple hundred of 'humans chatting in the context of a Turing test'.
I don't think that's true, assuming that the humans are also allowed to strategize and study the problem beforehand. The main weakness of the current models is that they still fail hard at certain kinds of common sense reasoning about real world situations, that no human ever would. Things like the "walk or drive to a car wash that's 2 minutes away" thing from a few months ago (I think the latest models have patched that one in particular, but I'm sure others can be found in a similar vein).
That would not really pass the test in spirit, I think. Passing must be something that falls out as a consequence of being a good chatbot, not the other way around.
However such good conversational agents would still have benefits, and i would be curious how far the approach of building a model that’s nice to talk to can take you, as opposed to the current generation of all-knowledgable helpful assistants.
The big problem with the Turing test always was that it doesn't take into account adversarial designs. They had those chatbot contests about a decade ago, where the chatbots would regularly pass the turing test, not because the bots were intelligent, but because they were packed with rhetorical tricks designed to avoid saying anything of substance.
By any meaningful measure, today's LLM's pass the Turing test. Except we may need to lobotomize them for the deception to work.
What's more interesting are the types of humans that might fail the Turing test. Maybe to discourage the failure of real humans, there could be consequences outside the context of the test.
That is, most people accused of being a computer would crack and start pleading their humanity, but some sociopaths might not. I then wonder how many of them overlap with those so invested in abusing the premise of the Turing test.
It really is a fun thought experiment when you spice it up enough.
Humanity is taking an interesting technological arc.
Terminator (1984 film) had a scene showing that in a future 2009, humans would use dogs to try to sniff out whether a robot passing for a human is secretly a machine.[1]
In our real world 2026, there are no humanoid machines that can complete basic generic tasks, like carrying a tray across the stage and holding it for 30 seconds.[2] They move slowly and badly (probably from an LLM like neural network doing very few frames per second of correction and analysis), and are nowhere near lifelike.
When they don't need a body to pass for a human, such as typing online, they do a bit better.
We can tell them apart from humans. As you say, they have stylistic quirks. And you mention that they're easy to spot because they come trained with inhuman amounts of trivial knowledge.
I find it more accurate to refer to "A Turing Test" as opposed to "The Turing Test" for that reason. It's just one loosely-defined test; necessary but not sufficient.
It won't fool all of the people all of the time. But it succeeds often enough to pass the loose definition. It does many things that we called "AI complete" for decades.
It is also clear that is is also not really aware, either. I think Turing would be tickled that we find ourselves in an intermediate state that he would not have imagined.
I would not call Turing's experiment ill-specified, on the contrary among other factors the paper the game is introduced in earnestly calls for a telepathy-proof room to ensure no side channel leakage. The 1950s was an interesting decade.
The Turing test has been passed. The Turing test had already been passed in 2002. The Turing test isn't about some machine fooling all humans all the time. It's about fooling some humans some of the time. That's an extremely low bar. It has been cleared. Let's put this to rest.
The bar you mentioned was reached in the 1960s with ELIZA. I have no idea why the obsession with the test has persisted into modern generations. It was some guy's thought experiment from 1950. It's not scientific in any fashion. Passing it doesn't prove anything. It's like some kind of authority worship game where people invoke the name of Turing as though doing so bolsters the strength of their arguments in any way whatsoever.
In my mind, the part that is most likely to fail is "return an HTML document that still contains the following text." Nowadays, more and more "HTML documents" do not contain any content at all, just some JavaScript that is responsible for then fetching and displaying the content.
If the text was returned using a staged process, whereby the server first returns an instruction (JavaScript), and the web browser then executes that instruction, which is what transfers the text, then that doesn't actually change anything. Those are just technical details of the transfer. You are viewing an HTML document on-screen; it makes no difference to this bet if it was returned to you by a different mechanism than originally conceived, since the bet doesn't specify any transfer mechanism, just that the URL was entered into a browser.
I understand that this isn't entirely clear-cut and that people will interpret it differently, but in my opinion, there is a significant difference between "a webpage that contains text X" and "an HTML document that contains text X." "Webpage" and "HTML document" are not synonyms - "HTML document" has a much narrower meaning.
Even if you interpreted the bet generously to allow for a browser to render the content using JavaScript, if I understand the bet correctly, the condition of working with a command-line tool like curl would fail.
I read the or's as giving alternatives, only one of which needs to work.
I can see how other interpretations are possible, but I don't see anything especially generous about mine. I'm not "allowing" anything: To make a judgement you have to pick a web browser to try it with, and you'd probably pick the most popular one, in the most popular configuration, that seems the most fair. If that web browser happens to execute Javascript when you navigate to an URL, then that's just how it is.
That depends on which side of the JS rendering bullshit you're sitting. If you're on the serving side, yes.
But I do the same as GP, just on the receiving side. That is, reaching out for help to baby eldritch entity, because that's healthier for my mind than dealing directly with the unspeakable horrors that half the websites are today. The industry has lost its collective mind in pursue of profit and control, and that was long before the thousand-mask wearing shoggoths became our daily consorts.
I think the parent is referring to the possibility that one day, browsers might put up a blanket security warning for HTTP URLs instead of following redirects. Or they might try to be a little too clever with HTTPS upgrades.
In particular, the automatic upgrade feature in modern browsers is based on several heuristics rather than explicit configuration like HSTS, so there's always a bit of room for breakage there. For example, they don't even check if the server returns a 301 redirect, which can be problematic if the server wants to redirect to HTTPS on a different host/port or make some changes to the path.
> Or they might try to be a little too clever with HTTPS upgrades.
They're not going to get any cleverer than they are now. In October we'll finally be done with heuristics or central databses of https-first websites used by the various browsers or the half a dozen of headers that you must dance around to get the upgrade to work securely and reliably. They will just default to https first.
Personally, if I were interpreting the bet, and that browser behavior had been changed before the day of the bet, I'd interpret that as failure, because the browser didn't show an HTML page, it showed a warning dialogue.
And in any case, at some point after browsers start doing that, I will be unsurprised if some further conditions prevail that provide reason for web servers to stop listening on http at all.
At a certain bet size it would almost be self-fulfilling, heh, as one side would be very incentivized to keep at least that url working (buying the domain + hosting it if the page were to die)
I was expecting some kind of system that would follow the pages automatically without you having to do extra work when changing the URL (“continue to work even when I change them and forget them”) but it looks like you still add the redirects manually?
That’s more than most will do, sure, but it doesn’t seem like it saves you from a broken link if you forget to add a specific redirect or miss it during a mass format change. Am I missing something?
I really liked disqus back in the day. When I was in my kinda cringey singularity phase, Kurzweil and all that. Tracking comments across similar blogs with the same people was handy.
I know nothing of Safecloud, but freenet failed for a very valid reasons:
1. Data retention isn't infinite. If the source disappears from the network, peers will prune it shortly. This is exponentially true in our multimedia-heavy world.
2. Despite 1, if you actually want to delete something you published, you can't. Absolutists will see this as a good thing. Normal people will see this as a problem.
3. Peer-to-peer doesn't work in a mobile-first world, and not enough people will host nodes as a hobby. If all website publishers/hosters have to host a node, then we're just back to the current model but with extra steps.
Closed-garden mobile-first tech feudalism lobotimized the Internet. I remember trying out the Freenets, I2Ps, and whatnot in the 90's. Too cumbersome nerdy tools to use for anything actually worthwhile. It was born in the idealized world village of the 90's Internet. Maybe real life doesn't work like that, and the lobotimized centralized tech will eventually win.
Is the p2p problem really still one in our world that is mobile first, but also has ubiquitous internet connectivity pretty much everywhere? I’m almost sure I could host my website from my phone and have at least 99.5% percent of uptime.
Give that it's impossible to even listen on port 443 on a phone, as that would require root privileges, I don't think you can host your website on your phone at all, at least not if you want it to accessible in a normal way. And that's not even going into all of the problems with running background apps on a phone, especially on iOS.
It is actually pretty easy to keep alive URLs if there is a will to do so. Just have tests in place so you can't mess them up accidentally. And convert things that are not changing anymore to static html.
One of my first web projects involved a music forum and I have been keeping the URLs alive for 24 years now:
https://www.gnoosic.com/discussion/
I feel like I owe it to the people who participated to keep it online forever. Also as a document of history.
Cool URIs don't change, as we should all remember:
https://www.w3.org/Provider/Style/URI
It's not easy. Part of the instructions would need to contain things like "how to convince your boss to not just delete the old thing" and "what to do when the startup you worked for went bankrupt" among others. Most URLs don't disappear by mistake.
*a will and the ability for that will to access what it needs
Well, and pay registrar and hosting fees for 100 years, and then just hope the hosting company doesn't fold in the interim after your passing. The problem isn't "can you make sure it stays up while you're alive", the problem is, after you're gone, you're no longer there to keep the flame alive and there are no guarantees anyone else picks up the torch.
Does this suggest a subscription(ish) business where you pay the hosting company for N years and then they ensure your website stays up and unchanged for N years after you die? Quality control issue?
Given how many things want me to pay a subscription while I'm alive, surely somebody will start charging me when I'm dead. It is the logical conclusion of SaaS. I wanted to create an acronym for this, but death as a service didn't seem like the best choice.
There is certainly precedent for this - will executors for example. How about “SCITA? Service continues in the afterlife.
SCIP, in perpetuity?
Software after Death as a Service?
> It is actually pretty easy to keep alive URLs if there is a will to do so. Just have tests in place
"Just".
Maintaining and keeping these tests (and the oriinal link) running is also probably easy. "Just" keep maintaining them over changes in technology, hosting providers, DNS registrars...
> I feel like I owe it to the people who participated to keep it online forever. Also as a document of history.
Oh! So its your website. Cool.
Did the money change hands, for this bet, in 2022? How do you ensure it. Or is it honor based?
It's worth noting that some prediction URLs, for instance: https://longbets.org/6/ are offline.
The original long bet was whether or not computers could pass the turing test by 2029: https://longbets.org/1/
You'd think that LLMs fulfill this, but I do wonder if a clever human could still discern between them given their particularities.
https://longbets.org/9/ is another interesting one depending on whether you think covid leaked from a lab or not.
I think that recent LLMs do pass the Turing test, but I don't think it is a good test for intelligence, for two reasons: emergent complexity and pareidolia.
Arguably, ELIZA passed the Turing test ca. 1967: https://archive.computerhistory.org/resources/access/text/20... and not that long ago, ELIZA was still more convincing: https://arstechnica.com/information-technology/2023/12/real-...
The Turing Test can be gamed. This was done in some Loebner Prize entries: see https://www.behavior.org/resources/325.pdf and https://en.wikipedia.org/wiki/Eugene_Goostman
To determine whether a machine is intelligent, I think it's necessary to understand how it works internally and not just take a black-box approach. This was already understood when Alan Turing wrote his 1950 paper. Elmer and Elsie, William Grey Walter's tortoise robots built during the preceding two years, appeared to behave as if they had some degree of self-awareness. But their "brains" consisted of only two thermionic valves, so self-awareness can be ruled out.
There was a paper a while ago, where they found a strategy for GPT beating humans on the Turing test 2/3rds of the time.
They just prompted it to talk the way a 13 year old girl texts.
(Also they prompted it to pretend not to know anything about anything.)
Most of the LLM tells we usually recognise are due to their heavy attunement to role playing a "helpful assistant" with a knack for correct typography.
If you used a smaller corpus, followed a different kind of fine tuning routine with rather normal conversation patterns, and wrote a different base prompt, I don’t see why it shouldn’t be possible to come up with a very particular personality, so to speak.
> where they found a strategy for GPT beating humans on the Turing test 2/3rds of the time.
IIRC, PARRY https://simulation-and-society.org/parry/ got there in 1972 by pretending to be a paranoid schizophrenic person
>while ELIZA has a comforting effect that many crave, PARRY often actively makes users uncomfortable and afraid
PARRY is my spirit animal.
> In empirical evaluations conducted during the 1970s, experienced psychiatrists proved unable to reliably distinguish PARRY's responses from those of actual patients diagnosed with paranoid disorders, achieving only 48 percent accuracy, a figure statistically indistinguishable from random guessing.
https://ai.miraheze.org/wiki/PARRY
Also I found the source code!
https://www.cs.cmu.edu/afs/cs/project/ai-repository/ai/areas...
I feel like pretending to be a child is cheating somewhat. Might as well print random key presses and claim it's a toddler.
> You'd think that LLMs fulfill this, but I do wonder if a clever human could still discern between them given their particularities.
Yes, especially if you use the original adversarial setup with three simultaneous participants (judge, two candidates) as described in Turing's original paper.
However, I'm not quite sure how long even that one will last.
Wayback shows #6 as "By 2010, more than 50 percent of books sold worldwide will be printed on demand at the point of sale in the form of library-quality paperbacks."
It was removed early 2011. Is this what happens when they can't decide on who won?
>https://longbets.org/6/
Anyone know why this (and many other bets) have been removed?
https://web.archive.org/web/20080803213625/https://longbets....
>By 2010, more than 50 percent of books sold worldwide will be printed on demand at the point of sale in the form of library-quality paperbacks.
Not exactly a hot-button issue.
>You'd think that LLMs fulfill this, but I do wonder if a clever human could still discern between them given their particularities.
While LLMs are very capable of appearing human in general conversation, the test is that they are trying to convince a human that they are human, which is not something that is in their training data and I suspect they would go about things in very odd ways from an actual person, especially when challenged by an actual person. It is still very easy to back LLMs into a corner where they fall into very unnatural patterns and in my experience challenging them on their "humanness" (or anything) backs them into that corner.
LLMs also still can't use context or subtext in natural ways and even people who are terrible with purposefully exploiting context and subtext, generally have a natural ability that they developed to use and parse them. In a short conversation that is not adversarial, this is not an issue but falls apart as the length of the conversation grows or they start getting challenged.
LLMs just don't play with ideas and conversation like we do, they don't riff on it and go with the flow without care, picking up on subtexts and contexts and manipulating them the way people constantly do without effort until they find themselves on the edge of a cliff looking into the void and debating if they should jump. If we are put into the situation of proving we are human (in an academic test sort of way) we will play with it and have fun with it, and we will find ourselves on that cliff looking into the void, debating if we should expose ourselves and admit things we normally wouldn't. We pick up on this when people do it, even over the internet, their way of stating things changes and their subtext falls away or drastically changes; I have never seen anything approaching this in LLMs.
I don't think any frontier lab tried to make an LLM optimized for the literal Turing test where the LLM is made to argue that it's human. This is not in their interest. They make their LLMs insist they aren't human, have no feelings and sentience etc etc.
So we don't actually know how good LLMs can be in this.
That is absolutely true of the chatbots we have access to but I suspect there are people working towards these ends, including all the big names in the field, major win for their company if they manage it. But even then, I have yet to see a model that could naturally handle subtext and context, they struggle even when dealing with programming languages where these things are far simpler than they are in human language.
There is also the problem with how easy it is to back them into a corner where they start acting in very unnatural ways. This would arise when challenging them on their humanity and most people would pick up on it and craft their challenges around it until it fell apart because that is how people work in such adversarial situations.
I'm agnostic on the matter because post training and RLHF on this specific capability could be quite effective. Nobody ever tried because it would cause a backlash and bad PR. Much better PR to solve math problems, help run businesses, cure disease etc than to pretend it's human. The only business case for pretend human is AI boy/girlfriends but those are also not most profitable at their most realistic state.
> LLMs just don't play with ideas and conversation like we do, they don't riff on it and go with the flow without care, picking up on subtexts and contexts and manipulating them the way people constantly do without effort until they find themselves on the edge of a cliff looking into the void and debating if they should jump.
Sounds like Neuro-sama to me.
> You'd think that LLMs fulfill this, but I do wonder if a clever human could still discern between them given their particularities.
I think the clear winning strategy is to find some of the common sense reasoning fails that are still quite common, like the "should I walk or drive to the car wash" one from a few months ago.
These are usually from a missing context (i.e. specifying where the car is right now). Tokenizer issues (how many Xs are in Y) are a better giveaway, but models can solve them using tools nowadays. I think the best way to uncover an LLM is to reveal its superhuman aspects. Ask it for an A* algorithm for your problem. If it can just type down a working algorithm in a few seconds, that is definitely not a human. Although with some careful system prompt design, you could probably mask that aspect too.
> These are usually from a missing context (i.e. specifying where the car is right now).
The reason for the mistake is irrelevant here. All that matters is that you catch the agent saying something that basically no human would, especially in this type of game.
If it is missing context, it will be indistinguishable from mistakes that humans could make as well.
In a true competitive Turing test, these reasoning pitfalls could be included in the system prompt, there are not that many of them. The tester would need to find a novel reasoning pitfall, but those aren't uncommon among humans either, so that would not make a very useful differentiator.
Another approach could be to probe the helpfulness of an AI. A human would likely not comply with a request to write a sonnet about tangerines.
I don't understand what you mean that "there are not that many of them". The models, even the best ones, still have clear problems with certain kinds of real world reasoning, and it's not a known, closed set. With some prodding and knowing this error mode, I think it's not hard to trip any model with a novel reasoning error like this.
And no, I don't think humans would make the same errors, the key reason for this failure mode is that there are some facts of human day to day life that are so mundane that they are virtually never written about, thus making them an ideal combination of universal human experiences but out of distribution for the LLMs.
By 2029 things will be different. Humans will have learned to talk like LLMs.
> You'd think that LLMs fulfill this, but I do wonder if a clever human could still discern between them given their particularities.
You don't even have to be that clever. We've all tried using it for work. They fail.
In the first place, the Turing test was always a loosely defined thought experiment. I suppose it's still serving that purpose, but at least half of HN takes it way too seriously. It's absolutely not proof of <engagementbait> endorsed by Alan Turing himself.
I think if you trained an LLM specifically for the purpose of passing the Turing test (instead of being helpful, harmful, and so on), its likely it would pass it.
You would have to train it/finetune it on a couple hundred of 'humans chatting in the context of a Turing test'.
I don't think that's true, assuming that the humans are also allowed to strategize and study the problem beforehand. The main weakness of the current models is that they still fail hard at certain kinds of common sense reasoning about real world situations, that no human ever would. Things like the "walk or drive to a car wash that's 2 minutes away" thing from a few months ago (I think the latest models have patched that one in particular, but I'm sure others can be found in a similar vein).
That would not really pass the test in spirit, I think. Passing must be something that falls out as a consequence of being a good chatbot, not the other way around.
However such good conversational agents would still have benefits, and i would be curious how far the approach of building a model that’s nice to talk to can take you, as opposed to the current generation of all-knowledgable helpful assistants.
I think you are absolutely right.
The big problem with the Turing test always was that it doesn't take into account adversarial designs. They had those chatbot contests about a decade ago, where the chatbots would regularly pass the turing test, not because the bots were intelligent, but because they were packed with rhetorical tricks designed to avoid saying anything of substance.
By any meaningful measure, today's LLM's pass the Turing test. Except we may need to lobotomize them for the deception to work.
I'd like to see an honest attempt at that.
What's more interesting are the types of humans that might fail the Turing test. Maybe to discourage the failure of real humans, there could be consequences outside the context of the test.
That is, most people accused of being a computer would crack and start pleading their humanity, but some sociopaths might not. I then wonder how many of them overlap with those so invested in abusing the premise of the Turing test.
It really is a fun thought experiment when you spice it up enough.
Mannerisms aside, they are easy to spot from having inhuman amounts of trivia knowledge
Couldn’t you instruct them to not display as much knowledge for a test situation…?
Humanity is taking an interesting technological arc.
Terminator (1984 film) had a scene showing that in a future 2009, humans would use dogs to try to sniff out whether a robot passing for a human is secretly a machine.[1]
In our real world 2026, there are no humanoid machines that can complete basic generic tasks, like carrying a tray across the stage and holding it for 30 seconds.[2] They move slowly and badly (probably from an LLM like neural network doing very few frames per second of correction and analysis), and are nowhere near lifelike.
When they don't need a body to pass for a human, such as typing online, they do a bit better.
We can tell them apart from humans. As you say, they have stylistic quirks. And you mention that they're easy to spot because they come trained with inhuman amounts of trivial knowledge.
[1] https://www.reddit.com/r/MovieDetails/s/0qUCVPYgjt
[2] https://www.reddit.com/r/LivestreamFail/s/t6ZV0yhgEe
I find it more accurate to refer to "A Turing Test" as opposed to "The Turing Test" for that reason. It's just one loosely-defined test; necessary but not sufficient.
It won't fool all of the people all of the time. But it succeeds often enough to pass the loose definition. It does many things that we called "AI complete" for decades.
It is also clear that is is also not really aware, either. I think Turing would be tickled that we find ourselves in an intermediate state that he would not have imagined.
I would not call Turing's experiment ill-specified, on the contrary among other factors the paper the game is introduced in earnestly calls for a telepathy-proof room to ensure no side channel leakage. The 1950s was an interesting decade.
The Turing test has been passed. The Turing test had already been passed in 2002. The Turing test isn't about some machine fooling all humans all the time. It's about fooling some humans some of the time. That's an extremely low bar. It has been cleared. Let's put this to rest.
The bar you mentioned was reached in the 1960s with ELIZA. I have no idea why the obsession with the test has persisted into modern generations. It was some guy's thought experiment from 1950. It's not scientific in any fashion. Passing it doesn't prove anything. It's like some kind of authority worship game where people invoke the name of Turing as though doing so bolsters the strength of their arguments in any way whatsoever.
> entering the characters http://www.longbets.org/601 into the address bar of a web browser or command line tool (like curl)
The part of this that's most likely to fail at some point is the "http://".
In my mind, the part that is most likely to fail is "return an HTML document that still contains the following text." Nowadays, more and more "HTML documents" do not contain any content at all, just some JavaScript that is responsible for then fetching and displaying the content.
If the text was returned using a staged process, whereby the server first returns an instruction (JavaScript), and the web browser then executes that instruction, which is what transfers the text, then that doesn't actually change anything. Those are just technical details of the transfer. You are viewing an HTML document on-screen; it makes no difference to this bet if it was returned to you by a different mechanism than originally conceived, since the bet doesn't specify any transfer mechanism, just that the URL was entered into a browser.
I understand that this isn't entirely clear-cut and that people will interpret it differently, but in my opinion, there is a significant difference between "a webpage that contains text X" and "an HTML document that contains text X." "Webpage" and "HTML document" are not synonyms - "HTML document" has a much narrower meaning.
> or command line tool (like curl)
Even if you interpreted the bet generously to allow for a browser to render the content using JavaScript, if I understand the bet correctly, the condition of working with a command-line tool like curl would fail.
I read the or's as giving alternatives, only one of which needs to work.
I can see how other interpretations are possible, but I don't see anything especially generous about mine. I'm not "allowing" anything: To make a judgement you have to pick a web browser to try it with, and you'd probably pick the most popular one, in the most popular configuration, that seems the most fair. If that web browser happens to execute Javascript when you navigate to an URL, then that's just how it is.
That's a secondary ("OR") condition, used if the browser condition isn't satisfied.
You two have just spelled out a very specific bet. Care to put a wager on it? :)
I used to be someone, man, I was a dreamer, a programmer, a man!
Now I'm a human slave copying text from web browsers to chat windows to get around JS rendering bullshit.
The preferred term is meat proxy ;)
That depends on which side of the JS rendering bullshit you're sitting. If you're on the serving side, yes.
But I do the same as GP, just on the receiving side. That is, reaching out for help to baby eldritch entity, because that's healthier for my mind than dealing directly with the unspeakable horrors that half the websites are today. The industry has lost its collective mind in pursue of profit and control, and that was long before the thousand-mask wearing shoggoths became our daily consorts.
I like your funny words, magic man.
One of those things is true
The detailed terms of the bet include this:
> A 301 redirect from www.longbets.org/601 to a different URL containing that text would also fulfill those conditions.
I assume this would cover the HTTP->HTTPS redirect scenario.
I think the parent is referring to the possibility that one day, browsers might put up a blanket security warning for HTTP URLs instead of following redirects. Or they might try to be a little too clever with HTTPS upgrades.
In particular, the automatic upgrade feature in modern browsers is based on several heuristics rather than explicit configuration like HSTS, so there's always a bit of room for breakage there. For example, they don't even check if the server returns a 301 redirect, which can be problematic if the server wants to redirect to HTTPS on a different host/port or make some changes to the path.
> Or they might try to be a little too clever with HTTPS upgrades.
They're not going to get any cleverer than they are now. In October we'll finally be done with heuristics or central databses of https-first websites used by the various browsers or the half a dozen of headers that you must dance around to get the upgrade to work securely and reliably. They will just default to https first.
https://blog.google/security/https-by-defau/
Yes this is about Chrome, but all others will follow.
Personally, if I were interpreting the bet, and that browser behavior had been changed before the day of the bet, I'd interpret that as failure, because the browser didn't show an HTML page, it showed a warning dialogue.
And in any case, at some point after browsers start doing that, I will be unsurprised if some further conditions prevail that provide reason for web servers to stop listening on http at all.
Yeah that was a clever bit of foresight there.
At a certain bet size it would almost be self-fulfilling, heh, as one side would be very incentivized to keep at least that url working (buying the domain + hosting it if the page were to die)
Someone should do a long bet on the continuous existence of longbets.com
One thing I've done is include my redirect rules with the web application. So my links continue to work even when I change them and forget them.
https://news.ycombinator.com/item?id=46139686
I was expecting some kind of system that would follow the pages automatically without you having to do extra work when changing the URL (“continue to work even when I change them and forget them”) but it looks like you still add the redirects manually?
That’s more than most will do, sure, but it doesn’t seem like it saves you from a broken link if you forget to add a specific redirect or miss it during a mass format change. Am I missing something?
Amazing that Disqus comments are still live for this page. More impressive even than the website itself being live.
I really liked disqus back in the day. When I was in my kinda cringey singularity phase, Kurzweil and all that. Tracking comments across similar blogs with the same people was handy.
I'm grateful the comments are still there, since they reminded me to get back in touch with my old friend Craig Hubley!
Some nice posts about his prediction on Jeremy Keith's blog:
https://adactio.com/journal/tags/longbets
Wow, that could be a post of it’s own
Anyway to find all bets expiring this year? it will be interesting to see. Could not find a way to search for it there.
I like the comment on the site about blockchains replacing HTML links (from 4 years ago).
I'm not sure how it played out in 2011, but as of now it looks like the win condition hinges on the redirect.
For the heck of it I confirmed "http://www.longbets.org/601" does in fact result in a 301 redirect to "https://longbets.org/601/". I suspected they might have used a 302.
This is basically a bet on if you think longbets is going to be around.
"Cool URIs don't change" is still the ideal. 301 redirects are the pragmatic reality. Both are right in different ways.
The fact that this bet page itself still exists is a win for both sides. Internet Archive gets $1000 either way, so we all win.
It's all a matter of whether someone is willing to pay for the hosting.
Which is not, in my opinion, how things should be hosted on the Web.
Freenet had a different approach, even back in the early days (two decades ago). I interviewed Ian Clarke about it https://www.youtube.com/watch?v=JWrRqUkJpMQ
I wrote about it for years: https://community.intercoin.app/t/who-pays-for-storage-nfts-...
And recently I built Safecloud to be different: https://safebots.github.io/Safecloud/
On HN, a TLDR would be helpful, because just clicking your links would potentially mean falling for shilling of ones own side-hustle.
I bet there's complexity to the topic, but I am sure you could put it into a few words instead of just leaving a hook for others to click on.
I know nothing of Safecloud, but freenet failed for a very valid reasons:
1. Data retention isn't infinite. If the source disappears from the network, peers will prune it shortly. This is exponentially true in our multimedia-heavy world.
2. Despite 1, if you actually want to delete something you published, you can't. Absolutists will see this as a good thing. Normal people will see this as a problem.
3. Peer-to-peer doesn't work in a mobile-first world, and not enough people will host nodes as a hobby. If all website publishers/hosters have to host a node, then we're just back to the current model but with extra steps.
Closed-garden mobile-first tech feudalism lobotimized the Internet. I remember trying out the Freenets, I2Ps, and whatnot in the 90's. Too cumbersome nerdy tools to use for anything actually worthwhile. It was born in the idealized world village of the 90's Internet. Maybe real life doesn't work like that, and the lobotimized centralized tech will eventually win.
Is the p2p problem really still one in our world that is mobile first, but also has ubiquitous internet connectivity pretty much everywhere? I’m almost sure I could host my website from my phone and have at least 99.5% percent of uptime.
Give that it's impossible to even listen on port 443 on a phone, as that would require root privileges, I don't think you can host your website on your phone at all, at least not if you want it to accessible in a normal way. And that's not even going into all of the problems with running background apps on a phone, especially on iOS.