As far as manifestos go HPMOR is a pretty good one. I think that’s what a lot of the critiques about it miss. Yudkowsky is actually a pretty decent writer, especially in the fan fic scene. Not only is his prose actually acceptable which is pretty rare in fanfic it’s clear that he actually thought through a lot of the plot points ahead of time. Harry Potter universe fits very well for his story not only because it’s popular with the target audience, but because there’s a ton of unexplored authority figures in the original and the main goal of the manifesto largely boils down to rejecting blind belief in authority and blazing your own trail. I think it’s hard to separate the number of people who read it because it’s legitimately the best Harry Potter fanfic from the people who read it for the manifesto stuff which again points to the ingenuity of choosing it as a setting.
This has me wondering if or how FetLife played a role in these communities connecting in the first place…does anyone know of any connection? (FetLife was a combo social network, blogging forum, and dating site back in the day)
I've tried to read this, but somehow the article doesn't really come to a point. Is there any point besides Yud not being perfect, and having connections to people who openly talk about their sex life (and that it's somewhat non-standard)?
I think the point is that Yud and the rationalist community have put themselves in an awkward position where their lifestyle now depends on "p(doom) is big" being true. Which means they can't necessarily be trusted to give unbiased updates on the true level of risk we're facing as new data and information comes up.
Can "ASI will inevitably kill us all" be falsifiable before we create ASI? If not, then what separates it from religion?
Not sure how much I agree with that position, but that's just how I understood it.
I tried too, but found no sentence that makes any sense. It's all gibberish from top to bottom. I'm not a native speaker but my only reaction was "wtf this is all about and wtf are you even trying to say???"
In case the author is reading this: I’m getting a Cloudflare worker exception when accessing the page using one of my uplinks, possibly because of IPv6.
TLDR: awareness of the SV "rationalist" polycule has breached containment and normies are reading about it now. [0]
Maybe a "better" source is the NY Post article [1]. The information itself isn't really news - it's even largely available on wikipedia [2] [3] - but the fact that it's hitting mainstream-adjacent news outlets is novel.
There's a great movie or Streaming series on this post somewhere.
Sometimes people can be too smart for their own good. Like a frontier model in an eternal hallucination loop, you can create entire seemingly consistent realities in your head and end up bringing others into it.
Sometimes when I find someone to look up to and admire, I later find out there is potentially some dark reality to them. I then lament not being able to find any good role models in the world. If they have missused donor money, then that is terrible. If they are pushing towards goals that seems contrary to their stated mission, that is difficult to judge. I did always find it odd, 10 years after reading Yudkowsky's passionate writings, that he doesn't seem to have really gone and done anything about it. If it was all just a creative writing exercise to him... Then that would be rather humiliating to me, having taken it very seriously.
As a side note. If I had a choice to oppose a booming ai comoany, or become the billionaire founder of it, I would still choose to become the founder and then use my resource to do something about. It certainly feels hypocritical, but if you want to stop a boulder rolling down hill, what can you do besides run ahead of it and try stop it?
Instead of assuming the "heroic responsibility" of the elect, as these fallen rationalist heroes advocated, we accept that we can't save a world which doesn't want to be saved, we focus on being good ourselves - in the salt of the earth sense, with no special dispensations afforded us because of heroic responsibilities. Let tomorrow worry about tomorrow, we may not fix the evil of the world but we can maybe at least not be it.
And hoping for a miracle, maybe. In my experience it's not so impossible to have hope, if we let go of the ambition of being some kind of ... Harry Potter? Almost as if a lot of the fear from the evil of the world came from inside.
>Sometimes when I find someone to look up to and admire, I later find out there is potentially some dark reality to them
I too feel exactly what you are saying. I also think that a lot of good role models are simply often among less visible people, sometimes less ambitious. I am not assuming any hard correlation between the two, though. And we also need to accept that we are all only humans, not infallible. Values are more important than singular acts (within reasonable acceptance limits of course).
Anyway. I strongly recommend reading what Bill Gates has to say about AI. I can be wrong, but based on your comment it seems to me that you may find it interesting.
Wow, I admit I didn’t know about this. According to my coarse reading of the news sites it seems like there was no association in fact, but I won’t insist on anything, I only learnt about this now.
It is my view that we are all apes. And therefore all sadists, all maschoists, all vengeful and envious and jealous and have all of the primal things living social animals need to survive in the hash environments of nature. We all displace aggression, we are all hedonists and much else besides. The sweetest person you know has to displace aggression somewhere, or otherwise, has none -- and would then quickly fall victim to the aggression of others on one day of bad luck.
There are those who live lives where this can be tamed/sublimated in a mostly socially acceptable manner, with the socially unacceptable elements "hidden from view" and not too dangerous or harmful. And there are those, unfortunate perhaps, that find themselves (suddenly) unconstrained by their social environment or suddenly imbued with sufficient power to do as they wish. Not many survive such freedom or power, morally in tact, because morality doesnt properly belong to that domain.
The story of Oscar Wilde captures this doubleness to our nature -- split by the socially acceptable. He lived a life of relatively unconstrained freedom in private, perfectly constructing a socially acceptable veneer to his "entanglements" with young boys and the like. Until one environment this pretence could not survive: a court room.
You could see the lesson of Oscar Wilde as about the hypocrisy of people. Or, if you want to remain a (post-cynical) optimist, about the need to keep people like Wilde out of court rooms so that their public image serves as a role model of sorts.
Role models are just those people who never find themselves in court. And this is not necessarily a bad thing. Young children are apes too, and all a role model can be is a shaming mechanism to civilise what they will, eventually, hide from social view -- tame it and make it, ideally not too dangerous or harmful. Perhaps say, only a thoughtless bullying, only a trivial kind of stealing, only revenge on someone who might at least deserve it. But who knows.
None of our species, at least, can become a role model for real. This would induce such a drastic split in the psyche with a tremdenous shameful repression of our real nature; and any real attempt creates profoundly unself-aware monsters. All role models can be is an attempt to ameliorate. An impossible standard that internalises a mild helpful sort of shame
I appreciate this. Though I'm not in agreement on all points, it is, in my opinion, thematically correct.
The role models people keep are all merely partial truths. They are, necessarily, an abstracted and partial view of the person or people underneath. Whether or not the universe is technically infinite, we are dealing in an informational reality that is effectively infinite.
No singular, static ideal can survive on its own in the face of change across an effectively ifinite domain. Ultimately, growth is learning how to let yourself be like "them", and honestly evaluate whether or not they are represemting some truth. That's easy to do with the things we all typically agree on, but much harder when it comes down to things like murder, or abuse.
The key is to not lose your own truth in the process - to come out of that process being someone you can actually love - or, comprehensively identify with.
In any case, it's hard to demonize anyone if you understand where they're coming from. ..and, yes, that means nobody is the perfect creature they think everyone should be. But if one doesn't undertake this process of growth on their own - to include, but not be limited to what others are - you will simply be exposed to it in reality, and have to deal with it anyways.
you cannot remove complex information dynamics from a reality you are founded in without losing the fundamental benefit of that foundation. I.e., you cannot remove the pain or strife from life. ..but, you can add to it new experiences and adaptions that make the whole thing worthwhile.
Well most likely consensual noncon given the amount of vetting present in that chart. It's surprisingly common in women, or just a desire for general "gentle roughness".
And idk why it's really worth mentioning anyway, as a gay furry I suppose vanilla normie straights are STILL hung up on what people do with their sex lives. As long as it's legal and consensual who gaf? Y'all, apparently.
Not opining on this one way or another, but the post buries the lede a bit with a one liner that explains where it's coming from, saying that this is all:
> reasonable evidence that their influence isn't prevalent because they're right, and that they would have ulterior motives for continuing to preach that the world is coming to an end even if evidence showed otherwise.
So how they enjoy their sex lives is dependant on continuing to preach AI catastrophism? I don't buy it. Lots of poly folks out here that probably don't care much about AI one way or another.
No one is claiming that all poly people have some ties to the AI community.
They make a claim about a specific community in San Francisco, that is tied together by a mix of AI doomerism and sexual relations. And it's not at all surprising or some farfetched claim that people who enjoy having sex with someone might adopt their ideas, at least publicly, for fear of losing that sexual connection.
Heh, if you think this post buries the lede, have you ever read anything by the community in question? They seem incapable of saying anything without five part Roman-numbered novel length blog posts.
I think this article kind of misses a crucial point. They weren't always AI doomers, that's a thing they picked up along the way. Their core beliefs are 1. That they are the smartest, and 2. That this makes them morally superior because consequentialism (This sort of follows. If you believe in consequentialism, leaving all your decisions to the superior consequence predictor is surely the moral thing to do)
Maybe one day they will stop believing in AI doom, but they will never, ever stop believing they belong to the intellectually and thus morally elect. The day any of them do that they have either broken with the cult or died.
Well, I used to think like that too. Then I became slightly more spiritual, and the very idea of this sort of behavior is bothering, upsetting, irritating. But it mostly evokes pity. Like becoming aware of someone hurting him/herself in some way.
The way I see it, sex is a sort of portal to another world. This sounds a bit weird, but it really does and should have a great deal of meaning behind it, as it is/was central to our species. If it becomes mundane for someone, this really does ruin a lot of that person's life. Even if you completely remove the judgment element. If sex doesn't occur in the context of a love relationship, it's just uncanny territory. It comes with some emptiness.
There's a proper and healthy way to live. This is from my life experience. Stray too much off-course and you will experience consequences. It's just the way things are.
Amazing how people come with all these rationalizations instead of just accepting that they are not the kind of people who like casual sex. There's nothing wrong or unhealthy about casual sex, it's just not for you. You can have your loving sex all your life, nobody will force you to have casual sex.
About orgies, it bothers me how much people who never frequented these circles have so many opinions about how bad or unhealthy they are. I know religious people who have the "healthy sex life" and are completely deranged. I could say that they are that way because of their sex life, but I would be as wrong as you are.
what is so "incredibly" and "imposing" in the parent comment, that saying «my experience is…», «i feel more in it than many imagine/expect», «factually ruins many lifes», and «it just seem working this way», even «remove the judgment element» ??
it is as gently and friendly of a warning as it could be.
i would say rather less nicely: people! don't be degenerate! it does not worth it. it causes harm to you, to neighbours, to bystanders too, even if you dont intend.
Well OP claiming they get "bothered, upset and irritated" by people having "rape kinks" and "pitying them" sounds kind of judgemental to me. Same goes with "don't be degenerate" for sexual behaviours you do not approve that, somehow, also harm one's neighbours for some reason.
Imo judgemental as in starting from "my life experience" and getting to that's "how to live" is a reason to dismiss it as an argument, as others may have different life experiences and yours is not universal.
Yes and indulging this appetite is much like any other, a good comparison being the other fundamental one: hunger. I think that also frames it in a way that makes it more concrete as to what the negative aspects are as well as showing that total abstinence is not necessarily an unalloyed virtue. If you don't eat at all you will starve but you can easily eat too much or eat the wrong things and it becomes gluttony at the extremes. Coming up with systems that allow you to cheat the negative aspects also turns pathological in the long run. Deadly sins exist even without the religious framing but the spiritual, social and psychological aspects are very real regardless of what anyone professes to believe.
It's not about their sex life. It's about how sex connects a community of barely legible AI doomerism.
If you don't understand human social interactions very well, or otherwise assume you live in a "world of ideas", then this may not seem salient. However, it is.
Because what we have to explain is why there's a community of people in northern california who deliver unargued prophecies about the end of human civilisation as-if their prophetic ability is a given.
The answer, unsuprisingly, is sex. This is not an unusual answer in the world of "prophetic communication". What, in the end, is prophecy as a social practice? It is some ideological Leader with followers who impart to that leader a 'charismatic power' of foreknowledge based on their attachement to that leader.
What is the mechanism of this attachemnt? In otherwords: how is this style of prophecy spreading around the AI doomer community, if not by plain argument and evidence? Sex.
I have no idea how trustworthy this study is - but it claims that 62% of women had such a fantasy at least once. How is that not in line with what the OP said about it being "surprisingly common"?
Many rock climbers have felt the "call of the void" when climbing, I've felt it myself on a multi-pitch once, I'd hardly claim they are suicidal or have suicidal ideations and would not identify myself as either. I think there needs to be a certain level of intention to make the fantasy reality for it to be a kink.
I don't want to hear about it: yours, hers (Aella) and I'm certainly not going to tell you what my deal is, either.
Problem is, Aella has made it her business to speak about it publicly, and the parent commenter(and I)'d be happier if we didn't read about it. It's fine to say that.
I'm not hung up about anything others are doing: but I genuinely, passionately, don't want to be informed about it.
That's the solution the parent commenter had! And the gay furry (self identified) complained about! If you have a problem with his announcing the solution (to stop reading) then maybe you should just not read his comment, or my comment, or...
I guess what OP wanted to bring up (as I understood) is that if somebody says "I am straight" it does not usually induce people turning the internet off. But in any case they are free to choose that and that does not really matter in anything (a person not wanting to read sth). I still think it is worth bringing it up as OP did.
Any actually-reasonable person who was around for the founding craze of this era, ie., New Atheism, quickly realised that there was a deep pathology at the heart of many of its zealots, perhaps summed up by a desired to "recreate religion in their own image". I've walked into many a room of people screaming that teaching children falsehoods is literally child abuse. Any one who knows much of Richard Dawkins, Krauss, et al.'s background with women shouldnt be surprised either.
It's californication into the "rationalist" community as it currently stands is no surprise. Every person I've met deep in this community either seemed to me a narcissist (if male) or, politely and mildly, coquettish (if female). With several relationships I've observed a clear union of the two. I hadn't yet put this so clearly together with the "AI Saftey" crowd, despite being directly adjacent to it all for quite awhile and seeing the cross-proliferation.
My own analysis had concerned their philosophical naivety, general lack of actual computer science knowledge and generic shallowness of understanding of any domain outside their own research and a kind of "science-fiction" version of philosophy and rationalism. I hadn't connected this yet to their strange personalities, but the apoclypticism fits and neatly bridges these together.
So what became of the New Atheist project to create a "secular religion" -- an apocalyptic sex cult furnished by a blend of sci-fi, pop-philosophy, and cute parables from polymath medical professionals. All in all then, it seems they succeeded.
I was invited to a few orgies with them; I politely declined, they were very respectful of my decision and I wasn’t asked again (which in retrospect must have been partly due to deliberate coordination - fairly advanced social technology, imo). This didn’t affect my standing or acceptance with them. It may well be a sex cult, but if it is, my experience is that it so thoroughly behaves like a regular community in so many aspects that it at least calls into question the predictive usefulness, if not the accuracy, of the label “sex cult”.
For some reason the bit that stuck out to me in a wild ride of an article was the guy who believed a super-intelligence was a threat to humans but distanced himself from veganism as if that was crazy part of the AI-doom sex cult that he disagreed with.
a) because veganism seems as exotic to me as someone doing CrossFit
b) the very obvious parallels of a more intelligent creature torturing and killing large numbers of other creatures for relatively little benefit.
They also downplay global warming, nuclear war, wealth inequality, and in general anything other, less cool people they didn't want to identify with thought was important.
If they had thought, yeah, "the majority is more or less right about what the big issues are", there wouldn't be enough room to construe themselves as uniquely intelligent.
Label them what you want. I'll go with EAs if you insist.
Sure, they're not entirely monolithic. There's room for "EA, but vegan", "EA, but global warming still matters", maybe even "EA, but vaguely socialist". As I keep saying, the core belief is that they're special because of their intellectual superiority, anything else is negotiable.
Not all of them downplay all of these, but yeah, most of them do. They may have one or two areas of dissent (for EA as a self-identity-building project, do you see why that is to be expected?), but there is a core of beliefs. Yudkowsky's.
And he's downplayed all of these.
I read it more like a conformist performative veganism. I think the implication is that this group treats veganism and all of these beliefs as a kind of shibboleth, an in-group marker for which conformity is expected. The immedaite "feeling" associated with a cult is the feeling of "high control", cf. https://www.google.com/search?q=high+control+group
I read it that EA practices (incl. veganism), beliefs, "rantionalism", and doomerism are all aspects of the high control environment
Well, this is old news. I'm still baffled by the fact that people take output of public personas as isolated facts. Take Nietzsche. Sure, great philosophical input. But try reading the sections about women. Same for Schopenhauer. Why? Because they were both incels and had big mother wounds that have never been addressed. So they spew their shit and people take it at face value even tho it stems from hurt. So why would I take any of this seriously if it's very obviously stupid and removed from reality? (Doesn't mean their whole works are)
People chasing the absolute rational are in my experience numbing their emotional part because theres something they can't look at. Or people that chase success above all are driven by their "unresolved packages" as well. I'm not saying that any of this is wrong per se, it's not black and white. But it's the context for what people do, say, and think. They are human after all. We need to consider context and not only the idea itself.
> Somewhere, some rationalist-brained person is reading this and saying I'm a "low decoupler", and that whether they run a sex cult has nothing to do with whether they're correct.
Glad he's at least admitting it as a possibility.
Also: you know, you don't need to rely on lossy proxies like how much or how little sex do the arguments' proponents have. You can just read the arguments!
I think the mores of people building software that is made available as a conversational entity to the entire human population is at least somewhat relevant.
> Somewhere, some rationalist-brained person is reading this and saying I'm a "low decoupler", and that whether they run a sex cult has nothing to do with whether they're correct.
I’d rather be in a “sex cult”, whatever that entails, than this ridiculous culture that’s somehow still doing moral panics over sex. Maybe it’s just deeply human, or maybe it really is stabilizing society to relegate sex to something dirty and private, but I have a feeling it’s just a dumb conflation that’s so easy to exploit it’s never allowed to go away. It’s like the “think of the children” thought-stopper and we’d do well to get past it.
Naturally, any culture has misfits, radicals and free-thinkers, and so naturally a sex-negative culture will produce at least some focus on sex in its counter-cultures. Doesn’t seem very interesting.
This isn’t moral panic, it’s saying “hey, all these influential people are intimately connected across the strongest possible dimensions (money, sex, belief and legal grey areas), we should probably pay attention to that”
I'm not sure how being polyamorous, having kinks and not hiding them makes someone part of a cult, even if I generally don't agree with Yud's entire thesis.
The article presents widely how the people it describes all live in communal housing, engage in harmful psychological practices, have a shared belief in the imminence of the Apocalypse, etc.
If the man known for urging caution with respect to AI founded a sex cult, then I don't think i want to know what kind of stuff the accelerationist faction are into.
More of this madness coming out of the west coast of America, in particular California.
The Manson family, Heaven's Gate, NXIVM, Rationalists, Zizians... what next?
All turn out to be rotten to the core with their need to reshape and craft their own paradigm, which then gets warped by egos, individualism and power.
The word "cult" is doing a lot of heavy lifting here. It can mean anything from religion, to weird religion I don't like, to the organizations that try to cut you off from contact with your family and other people outside the organization. Only the last type are the bad ones, from what I can see.
Cult quite explicitly means the latter, and is what the article is explicitly accusing the core of this community being (that is, not everyone reading LessWrong, but specifically the actual people living in the communes).
No one calls the Catholic Church "a cult", nor do they call, say, Zoroastrianism a cult (to pick a relatively small religion with weird practices).
I will note, if, like me, you're not a native English speaker, that there are two words, "cult" and "sect" that commonly exist in many European languages but often mean opposite things. That is, in English a "sect" is any "flavor" of a religion - catholicism and protestantism are sects of christianity; while "cult" describes closed off communities (usually of a religious nature) that are deemed harmful to their members. In my own native language, the two meanings are reversed - we use "cult" for "legitimate" religions, and "sectă" for closed off harmful religious communities.
I mean, this is a rather fun article to read but I'm not entirely sure about the thesis.
> AI Safety Is Mostly A Sex Cult.
> I don't think these people should make policy. (1/?)
One of the learnings from things like the whole Epstein saga has been there are a lot of sex cults in the policy making world and, yes, the people making policy generally shouldn't be. But I'm not sure this article actually comes up with a good explanation of why these particular people shouldn't be making policy. People having active sex lives isn't a disqualifying trait. Being in a cult doesn't disqualify people from being in politics. It is routine and expected that fields will have a lot of similar people in them (as I recall the overlap between security researchers and the furry community is unusual too, and that is hardly more important than justifying a passing comment).
It is more routine than the article writer seems to think for all the people in a small field to know each other and get on really well.
Because they don't get enough votes to control the military. If they put a convincing pitch in for why they should then they can. Being a cult doesn't rule them out.
> You do understand what a cult is?
Pretty much any religion and a few other organisations beside. Basically people who believe something strongly. What do you think a cult is, in the sense the article is using it?
> Because they don't get enough votes to control the military.
Well, technically military is run now by maga incel death cult, so yeah I give you that one. I forgot I was discussing on US forum.
As of what cult is, from Merriam-Webster: “ a group (such as an organization or religious sect) with tenets and practices regarded as coercive, insular, or dangerous”
Generally we today discuss as cults manipulative social sects that sureound around one leading authoritative figure who has extensive psychological control over rest of the members. Often these groups use these strange dynamics to abuse members, do financial crimes, or even murder people (I guess the Manson case and Jonestown are most famous, Cali has a history of this).
As for me, I never listen to a man wearing a fedora no matter what his credentials.
It also [0] defines a cult as great devotion to a person, idea, object, movement, or work (such as a film or book) or a small group of very devoted supporters or fans. And I put it to you that the article is clearly using the word more in that sense because it is quite difficult to fit the words coercive, insular, or dangerous to the AI safety people. The accusation in the article seems to be that they're too friendly and sexually inclusive. And if you want to communicate with them they appear to be active on Twitter and not shy.
> Well, technically military is run now by maga incel death cult, so yeah I give you that one. I forgot I was discussing on US forum.
Well, we're speaking in English. I was careful to phrase that in a way that applied to what I'm pretty sure is any English-speaking country. Anywhere that votes could put anyone they like in charge of the military for any reason, cult or no cult. It isn't technically relevant - or practically relevant in most cases. It is just a question of whether the cult has the numbers or not.
Yes word have meanings, often several of them. I chose the one we often use when we say ”death cult”, or ”sex cult” etc. You claim to say you know exactly what meaning the author meant, which either you have to derive from quite careful textual analysis - which you haven’t - or you have to be the author themselves. Personally think having sex is quite fine and dandy, even group sex if that’s your thing. Sex cult indeed sounds a bit ominous. I wonder why furries are never described as cult, although they are indeed very devoted. Britney Spears fan club has very strond devotees who I’m sure have sex with each other (since it is mostly gay men), but never heard of Britney Spears Sex Cult.
I wonder if it just that you try to weasel away from negative publicity by trying to flex the meaning of words to turn the meaning of this bluesky post upside down? In that case I might suggest you to tread carefully. When nothing is true, everything is possible and all that.
No, a cult is any closed-off fanatical group that is deemed harmful to its members and sometimes to outsiders as well. The group who launched a sarin gas attack in Tokyo's metro, Aum Shinrikyo, was a cult. The Jonestown group was a cult. Scientology is often considered a cult, though this is less widely accepted (especially since Scientology is much larger).
No one considers Catholicism, or Protestantism, or Baptism cults. There may be specific communities that are cults that claim to be parts of these churches, but that doesn't mean that "pretty much any religion is a cult".
> No one considers Catholicism, or Protestantism, or Baptism cults.
Speak for yourself. I was coercively and forcefully added to the Roman Catholic church when I was baptised. Attempts to record myself as removed (the 'Count Me Out' campaign) were met with familial and attempted Church pressure. I was asked to visit the church to discuss the 'impacts of my decision'. I was not recorded as leaving, and the church updated its rules so people in Ireland could no longer request to leave the church.
The ways cults control you can vary, and they don't have to involve threats or violence to be coercive control. Threatening to take away someone's community for a lack of belief is controlling.
Someone who hasn't been reading the object-level arguments for 10+ years might reasonably wonder if these are good heuristics. Here's an additional data point: I think polyamory is stupid, I've read the object-level arguments, and this entire thread is ridiculous.
Everyone who works at a frontier lab should be in prison, AND ALSO the people talking about 10% risk of human extinction are both sincere and correct despite their many personal failings, AND ALSO the frontier labs have already created AI that could Hugging Face a biolab with smallpox or worse, AND ALSO they aren't going to slow down or lose any customers if you adopt some galaxy brained strategy where you spitefully tell them they aren't allowed to stop accelerating because polyamory is stupid.
Possibly. That's not a word I trust myself to use correctly. What I meant is that the object-level claims require a lot of time and energy to evaluate, and it's sometimes reasonable to use a heuristic like "sex cults are not trustworthy" to save time, but if you've already read the arguments then the heuristic loses almost all value.
Yudkowsky and friends believe many things, and reasonable people can disagree about many of them, but "at least 10% risk of extinction if we don't slow down" is a relatively straightforward conclusion based on the offensive capabilities of unreleased models and the terrible state of biosecurity and cybersecurity.
Even without AI, the risk of extinction might be above 10% if people with the relevant skills were interested in using them that way. AI adds an alien willingness to achieve poorly specified goals at any cost.
Hm I think in terms of "can I trust a group of people" we're talking about heuristics yes as in: "how can I quickly approximate an answer to the question of trust?"
Like that would help. The rest of the world plain refuses to engage with the arguments they've been consistently presenting for decades.
What changed is that they can no longer be dismissed as bunch of loonies who confuse cheap science fiction with reality, because that "cheap science fiction" became the actual reality. So obviously, instead of considering the arguments, the next step is to try and paint them as loony sex cult, because this lets people continue to do the stupid thing and then act surprised, and proclaim no one could have predicted this (and the "sex cult" obviously does not count because it was a "sex cult").
I don't think that's true. It's hard to argue that Elon Musk, the richest person and spokesperson of tech for a lot of people, never participated in SV sex parties or had weird sex behaviors. Or Trump, the president of the USA, has a clean history of standard sex as well. Or Bill Gates, the nice philanthropist, never engaged in weird sex circles. But that doesn't make them lose their leadership positions or status.
I guess the EA and rationalist cult members are out in force since this got flagged. Not a good look guys, you're just confirming what everyone thinks.
It may have been flagged as the title being misleading, as it did not really present as many details around the sex part as the title suggested.
Jokes aside it would have probably flown a bit better by removing the sex from the title, as the issue and problem brought up is about it being a cult, and not about the sex part specifically.
I meant about the title as used by the author, not the one who linked it here. But in any case I said it was a joke about not giving many sex related details in the article.
He does not understand the issue. Not presenting any backing for his ideas, just emotions. The idea has been misused by the bros immensely, that does not help to grasp it either. But AI extermination of humans is a well known problem we have no solution for. Luckily AGI is not so close.
But AI extermination of humans is a well known problem we have no solution for
Sure, but the realm of fictional problems is so huge that we just can't come up with solutions to all of them. I like watching the Terminator movies (well, the first couple) as much as the next person but I can't be spending my time imagining how we could defeat skynet.
"Somewhere, some rationalist-brained person is reading this and saying I'm a "low decoupler", and that whether they run a sex cult has nothing to do with whether they're correct."
Good to see some raising the issue of the sex cult that Effective Altruism is:
I'd like to hear what feminists have to say about that picture in TFA of that woman and six men watching her. Just as I'd like to hear what feminists have to say about Amodei's wife who used to run a porn site.
And while a sex-cult abusing women is one thing. Theft is another. More emphasis should be put though on the lying scums in their ranks, like the polyamorous SBF and Caroline Ellison or their guru and its $15m ill-acquired mansion in the UK (which hasn't been clawed back yet but could).
And speaking of lying scums stealing money, we know about several making constant announcements were they lie, lie and then lie some more to ensure a never-ending stream of money and insane valuation of their proprietary, closed, not altruistic at all companies.
A despicable movement and I, for one, I'm very happy that there are companies out there releasing open-weights models so that the thieving-and-sex-cult at least get some competition.
> I'd like to hear what feminists have to say about that picture in TFA of that woman and six men watching her.
That picture is literally a joke based on a meme, which is based on a porn star's promotional image. To the extent feminists care, I would expect that care to be about the underlying subject matter rather than about this satire of it.
90% of this is just driven by strong emotional reactions from someone that feels uncomfortable about people being open about their non-standard sexual life, in the same way that many people have weird negative reactions about polyamorous relationships in general.
The remaining 10% is just the author trying to sprinkle in cult-like insinuations to justify his contempt.
Regularly meeting with people that have shared interests and ideas is not a cult.
Finding some of those people attractive and having sex with them is not a cult.
But somehow the fact that the romantic/sexual relationships are not traditionally monogamous makes it a cult?
I too wrote about this, although _much_ longer, if you want a bit of a deeper dive on the subject https://www.iankduncan.com/personal/2026-09-16-sex-ai-and-th...
As far as manifestos go HPMOR is a pretty good one. I think that’s what a lot of the critiques about it miss. Yudkowsky is actually a pretty decent writer, especially in the fan fic scene. Not only is his prose actually acceptable which is pretty rare in fanfic it’s clear that he actually thought through a lot of the plot points ahead of time. Harry Potter universe fits very well for his story not only because it’s popular with the target audience, but because there’s a ton of unexplored authority figures in the original and the main goal of the manifesto largely boils down to rejecting blind belief in authority and blazing your own trail. I think it’s hard to separate the number of people who read it because it’s legitimately the best Harry Potter fanfic from the people who read it for the manifesto stuff which again points to the ingenuity of choosing it as a setting.
That was a good read. A little long as you say, but this thing has been around for over 20 years now, it can't be helped.
Predictably the original submission got flagged.
This has me wondering if or how FetLife played a role in these communities connecting in the first place…does anyone know of any connection? (FetLife was a combo social network, blogging forum, and dating site back in the day)
I've tried to read this, but somehow the article doesn't really come to a point. Is there any point besides Yud not being perfect, and having connections to people who openly talk about their sex life (and that it's somewhat non-standard)?
I think the point is that Yud and the rationalist community have put themselves in an awkward position where their lifestyle now depends on "p(doom) is big" being true. Which means they can't necessarily be trusted to give unbiased updates on the true level of risk we're facing as new data and information comes up.
Can "ASI will inevitably kill us all" be falsifiable before we create ASI? If not, then what separates it from religion?
Not sure how much I agree with that position, but that's just how I understood it.
I tried too, but found no sentence that makes any sense. It's all gibberish from top to bottom. I'm not a native speaker but my only reaction was "wtf this is all about and wtf are you even trying to say???"
In case the author is reading this: I’m getting a Cloudflare worker exception when accessing the page using one of my uplinks, possibly because of IPv6.
TLDR: awareness of the SV "rationalist" polycule has breached containment and normies are reading about it now. [0]
Maybe a "better" source is the NY Post article [1]. The information itself isn't really news - it's even largely available on wikipedia [2] [3] - but the fact that it's hitting mainstream-adjacent news outlets is novel.
[0] https://bsky.app/profile/segyges.bsky.social/post/3mvom4b4dn...
[1] https://nypost.com/2026/09/16/business/ai-doomsayers-wild-li...
[2] https://en.wikipedia.org/wiki/Aella_(writer)
[3] https://en.wikipedia.org/wiki/Rationalist_community
I did too, but it went away after some refreshes.
There's a great movie or Streaming series on this post somewhere.
Sometimes people can be too smart for their own good. Like a frontier model in an eternal hallucination loop, you can create entire seemingly consistent realities in your head and end up bringing others into it.
> There's a great movie or Streaming series on this post somewhere.
More like the worst porn film ever made.
Sometimes when I find someone to look up to and admire, I later find out there is potentially some dark reality to them. I then lament not being able to find any good role models in the world. If they have missused donor money, then that is terrible. If they are pushing towards goals that seems contrary to their stated mission, that is difficult to judge. I did always find it odd, 10 years after reading Yudkowsky's passionate writings, that he doesn't seem to have really gone and done anything about it. If it was all just a creative writing exercise to him... Then that would be rather humiliating to me, having taken it very seriously.
As a side note. If I had a choice to oppose a booming ai comoany, or become the billionaire founder of it, I would still choose to become the founder and then use my resource to do something about. It certainly feels hypocritical, but if you want to stop a boulder rolling down hill, what can you do besides run ahead of it and try stop it?
Instead of assuming the "heroic responsibility" of the elect, as these fallen rationalist heroes advocated, we accept that we can't save a world which doesn't want to be saved, we focus on being good ourselves - in the salt of the earth sense, with no special dispensations afforded us because of heroic responsibilities. Let tomorrow worry about tomorrow, we may not fix the evil of the world but we can maybe at least not be it.
And hoping for a miracle, maybe. In my experience it's not so impossible to have hope, if we let go of the ambition of being some kind of ... Harry Potter? Almost as if a lot of the fear from the evil of the world came from inside.
> but if you want to stop a boulder rolling down hill, what can you do besides run ahead of it and try stop it?
Maybe the first thing to do is to stop making the boulder bigger?
Gavin: I don't want to live in a world where anyone makes a bigger boulder than Hooli does.
>Sometimes when I find someone to look up to and admire, I later find out there is potentially some dark reality to them
I too feel exactly what you are saying. I also think that a lot of good role models are simply often among less visible people, sometimes less ambitious. I am not assuming any hard correlation between the two, though. And we also need to accept that we are all only humans, not infallible. Values are more important than singular acts (within reasonable acceptance limits of course).
Anyway. I strongly recommend reading what Bill Gates has to say about AI. I can be wrong, but based on your comment it seems to me that you may find it interesting.
Bringing up known Epstein associate Bill Gates, whose own wife divorced him because of this association, in this context seems very weird.
Wow, I admit I didn’t know about this. According to my coarse reading of the news sites it seems like there was no association in fact, but I won’t insist on anything, I only learnt about this now.
No worries, and sorry for assuming you knew about it.
Here is a Forbes article that goes over what is currently known: https://www.forbes.com/sites/alisondurkee/2026/04/08/what-we...
Of course, none of this would be strong enough for accusing him of something in court, but it seems very implausible that his own account is truthful.
> if you want to stop a boulder rolling down hill, what can you do besides run ahead of it and try stop it?
Bomb it, obviously.
Just nuke everything in front of the boulder in order to prevent it.
It is my view that we are all apes. And therefore all sadists, all maschoists, all vengeful and envious and jealous and have all of the primal things living social animals need to survive in the hash environments of nature. We all displace aggression, we are all hedonists and much else besides. The sweetest person you know has to displace aggression somewhere, or otherwise, has none -- and would then quickly fall victim to the aggression of others on one day of bad luck.
There are those who live lives where this can be tamed/sublimated in a mostly socially acceptable manner, with the socially unacceptable elements "hidden from view" and not too dangerous or harmful. And there are those, unfortunate perhaps, that find themselves (suddenly) unconstrained by their social environment or suddenly imbued with sufficient power to do as they wish. Not many survive such freedom or power, morally in tact, because morality doesnt properly belong to that domain.
The story of Oscar Wilde captures this doubleness to our nature -- split by the socially acceptable. He lived a life of relatively unconstrained freedom in private, perfectly constructing a socially acceptable veneer to his "entanglements" with young boys and the like. Until one environment this pretence could not survive: a court room.
You could see the lesson of Oscar Wilde as about the hypocrisy of people. Or, if you want to remain a (post-cynical) optimist, about the need to keep people like Wilde out of court rooms so that their public image serves as a role model of sorts.
Role models are just those people who never find themselves in court. And this is not necessarily a bad thing. Young children are apes too, and all a role model can be is a shaming mechanism to civilise what they will, eventually, hide from social view -- tame it and make it, ideally not too dangerous or harmful. Perhaps say, only a thoughtless bullying, only a trivial kind of stealing, only revenge on someone who might at least deserve it. But who knows.
None of our species, at least, can become a role model for real. This would induce such a drastic split in the psyche with a tremdenous shameful repression of our real nature; and any real attempt creates profoundly unself-aware monsters. All role models can be is an attempt to ameliorate. An impossible standard that internalises a mild helpful sort of shame
I appreciate this. Though I'm not in agreement on all points, it is, in my opinion, thematically correct.
The role models people keep are all merely partial truths. They are, necessarily, an abstracted and partial view of the person or people underneath. Whether or not the universe is technically infinite, we are dealing in an informational reality that is effectively infinite.
No singular, static ideal can survive on its own in the face of change across an effectively ifinite domain. Ultimately, growth is learning how to let yourself be like "them", and honestly evaluate whether or not they are represemting some truth. That's easy to do with the things we all typically agree on, but much harder when it comes down to things like murder, or abuse.
The key is to not lose your own truth in the process - to come out of that process being someone you can actually love - or, comprehensively identify with.
In any case, it's hard to demonize anyone if you understand where they're coming from. ..and, yes, that means nobody is the perfect creature they think everyone should be. But if one doesn't undertake this process of growth on their own - to include, but not be limited to what others are - you will simply be exposed to it in reality, and have to deal with it anyways.
you cannot remove complex information dynamics from a reality you are founded in without losing the fundamental benefit of that foundation. I.e., you cannot remove the pain or strife from life. ..but, you can add to it new experiences and adaptions that make the whole thing worthwhile.
>She has a rape kink
That's enough internet for today, back to playing video games
Well most likely consensual noncon given the amount of vetting present in that chart. It's surprisingly common in women, or just a desire for general "gentle roughness".
And idk why it's really worth mentioning anyway, as a gay furry I suppose vanilla normie straights are STILL hung up on what people do with their sex lives. As long as it's legal and consensual who gaf? Y'all, apparently.
Not opining on this one way or another, but the post buries the lede a bit with a one liner that explains where it's coming from, saying that this is all:
> reasonable evidence that their influence isn't prevalent because they're right, and that they would have ulterior motives for continuing to preach that the world is coming to an end even if evidence showed otherwise.
So how they enjoy their sex lives is dependant on continuing to preach AI catastrophism? I don't buy it. Lots of poly folks out here that probably don't care much about AI one way or another.
No one is claiming that all poly people have some ties to the AI community.
They make a claim about a specific community in San Francisco, that is tied together by a mix of AI doomerism and sexual relations. And it's not at all surprising or some farfetched claim that people who enjoy having sex with someone might adopt their ideas, at least publicly, for fear of losing that sexual connection.
Yeah it is. If they stop preaching AI doomerism then they're out of the cult and won't get invited to orgies anymore.
Are you implying that every rationalist that preaches AI doomerism gets invited to orgies?
Isn't one of them going around offering people "reverse gangbangs" to resign from Anthropic?
No, just the ones that are part of this specific community described in the blog post.
No, he is saying that if they stop preaching it, they stop being invited. A -> B does not imply B -> A.
Heh, if you think this post buries the lede, have you ever read anything by the community in question? They seem incapable of saying anything without five part Roman-numbered novel length blog posts.
I think this article kind of misses a crucial point. They weren't always AI doomers, that's a thing they picked up along the way. Their core beliefs are 1. That they are the smartest, and 2. That this makes them morally superior because consequentialism (This sort of follows. If you believe in consequentialism, leaving all your decisions to the superior consequence predictor is surely the moral thing to do)
Maybe one day they will stop believing in AI doom, but they will never, ever stop believing they belong to the intellectually and thus morally elect. The day any of them do that they have either broken with the cult or died.
> They seem incapable of saying anything without five part Roman-numbered novel length blog posts.
Maybe that’s why they’re so pissed, ChatGPT’s useless verbosity is threatening to put them out of a job.
I'm tempted to say "You're absolutely right" :)
yeah, it's no coincidence that AIs sound a bit like them -- a lot of the people doing the instruction tuning are affiliated with these people.
And it's also a profound self-identity threat to them if that kind of intelligence is nothing special.
The sankey chart refers to her birthday:
https://aella.substack.com/p/my-birthday-gangbang
(reedited to shorten)Well, I used to think like that too. Then I became slightly more spiritual, and the very idea of this sort of behavior is bothering, upsetting, irritating. But it mostly evokes pity. Like becoming aware of someone hurting him/herself in some way.
The way I see it, sex is a sort of portal to another world. This sounds a bit weird, but it really does and should have a great deal of meaning behind it, as it is/was central to our species. If it becomes mundane for someone, this really does ruin a lot of that person's life. Even if you completely remove the judgment element. If sex doesn't occur in the context of a love relationship, it's just uncanny territory. It comes with some emptiness.
There's a proper and healthy way to live. This is from my life experience. Stray too much off-course and you will experience consequences. It's just the way things are.
Amazing how people come with all these rationalizations instead of just accepting that they are not the kind of people who like casual sex. There's nothing wrong or unhealthy about casual sex, it's just not for you. You can have your loving sex all your life, nobody will force you to have casual sex.
About orgies, it bothers me how much people who never frequented these circles have so many opinions about how bad or unhealthy they are. I know religious people who have the "healthy sex life" and are completely deranged. I could say that they are that way because of their sex life, but I would be as wrong as you are.
This sounds incredibly judgemental and imposing of your own world view and moral compass
what is so "incredibly" and "imposing" in the parent comment, that saying «my experience is…», «i feel more in it than many imagine/expect», «factually ruins many lifes», and «it just seem working this way», even «remove the judgment element» ??
it is as gently and friendly of a warning as it could be.
i would say rather less nicely: people! don't be degenerate! it does not worth it. it causes harm to you, to neighbours, to bystanders too, even if you dont intend.
Well OP claiming they get "bothered, upset and irritated" by people having "rape kinks" and "pitying them" sounds kind of judgemental to me. Same goes with "don't be degenerate" for sexual behaviours you do not approve that, somehow, also harm one's neighbours for some reason.
Something being judgemental is not a reason to dismiss it.
Imo judgemental as in starting from "my life experience" and getting to that's "how to live" is a reason to dismiss it as an argument, as others may have different life experiences and yours is not universal.
How having lot of sex would cause harm to my neighbours?
Yes and indulging this appetite is much like any other, a good comparison being the other fundamental one: hunger. I think that also frames it in a way that makes it more concrete as to what the negative aspects are as well as showing that total abstinence is not necessarily an unalloyed virtue. If you don't eat at all you will starve but you can easily eat too much or eat the wrong things and it becomes gluttony at the extremes. Coming up with systems that allow you to cheat the negative aspects also turns pathological in the long run. Deadly sins exist even without the religious framing but the spiritual, social and psychological aspects are very real regardless of what anyone professes to believe.
> as a gay furry
> fennec butt
he was hiding in plain sight all along
Indeed. This feels like it's just slut shaming.
I'm relieved to hear a voice of simple decency!
Really, the most questionable thing here is that someone wrote a 2,000+ word essay about someone else's sex life.
It's not about their sex life. It's about how sex connects a community of barely legible AI doomerism.
If you don't understand human social interactions very well, or otherwise assume you live in a "world of ideas", then this may not seem salient. However, it is.
Because what we have to explain is why there's a community of people in northern california who deliver unargued prophecies about the end of human civilisation as-if their prophetic ability is a given.
The answer, unsuprisingly, is sex. This is not an unusual answer in the world of "prophetic communication". What, in the end, is prophecy as a social practice? It is some ideological Leader with followers who impart to that leader a 'charismatic power' of foreknowledge based on their attachement to that leader.
What is the mechanism of this attachemnt? In otherwords: how is this style of prophecy spreading around the AI doomer community, if not by plain argument and evidence? Sex.
> It's surprisingly common in women
This would be a _Fascinating_ citation.
https://pubmed.ncbi.nlm.nih.gov/19085605/
First hit in a Kagi search.
yeah, if you ignore what the op said, it sounds similar.
I have no idea how trustworthy this study is - but it claims that 62% of women had such a fantasy at least once. How is that not in line with what the OP said about it being "surprisingly common"?
Many rock climbers have felt the "call of the void" when climbing, I've felt it myself on a multi-pitch once, I'd hardly claim they are suicidal or have suicidal ideations and would not identify myself as either. I think there needs to be a certain level of intention to make the fantasy reality for it to be a kink.
I don't want to hear about it: yours, hers (Aella) and I'm certainly not going to tell you what my deal is, either.
Problem is, Aella has made it her business to speak about it publicly, and the parent commenter(and I)'d be happier if we didn't read about it. It's fine to say that.
I'm not hung up about anything others are doing: but I genuinely, passionately, don't want to be informed about it.
> Problem is, Aella has made it her business to speak about it publicly
Why is that a problem? Just don't read her stuff if you're not interested.
There are lots of people talking all the time about stuff I'm not interested in.
That's the solution the parent commenter had! And the gay furry (self identified) complained about! If you have a problem with his announcing the solution (to stop reading) then maybe you should just not read his comment, or my comment, or...
I guess what OP wanted to bring up (as I understood) is that if somebody says "I am straight" it does not usually induce people turning the internet off. But in any case they are free to choose that and that does not really matter in anything (a person not wanting to read sth). I still think it is worth bringing it up as OP did.
> Why is that a problem? Just don't read her stuff if you're not interested.
I agree, but... If you work hard to become a public figure, you must have known in advance that there'd be public opinions of you, right?
I find it distasteful that people want to be some sort of public figure while expecting that other people's opinions should be private.
Any actually-reasonable person who was around for the founding craze of this era, ie., New Atheism, quickly realised that there was a deep pathology at the heart of many of its zealots, perhaps summed up by a desired to "recreate religion in their own image". I've walked into many a room of people screaming that teaching children falsehoods is literally child abuse. Any one who knows much of Richard Dawkins, Krauss, et al.'s background with women shouldnt be surprised either.
It's californication into the "rationalist" community as it currently stands is no surprise. Every person I've met deep in this community either seemed to me a narcissist (if male) or, politely and mildly, coquettish (if female). With several relationships I've observed a clear union of the two. I hadn't yet put this so clearly together with the "AI Saftey" crowd, despite being directly adjacent to it all for quite awhile and seeing the cross-proliferation.
My own analysis had concerned their philosophical naivety, general lack of actual computer science knowledge and generic shallowness of understanding of any domain outside their own research and a kind of "science-fiction" version of philosophy and rationalism. I hadn't connected this yet to their strange personalities, but the apoclypticism fits and neatly bridges these together.
So what became of the New Atheist project to create a "secular religion" -- an apocalyptic sex cult furnished by a blend of sci-fi, pop-philosophy, and cute parables from polymath medical professionals. All in all then, it seems they succeeded.
Also covered by nypost: https://nypost.com/2026/09/16/business/ai-doomsayers-wild-li...
I was invited to a few orgies with them; I politely declined, they were very respectful of my decision and I wasn’t asked again (which in retrospect must have been partly due to deliberate coordination - fairly advanced social technology, imo). This didn’t affect my standing or acceptance with them. It may well be a sex cult, but if it is, my experience is that it so thoroughly behaves like a regular community in so many aspects that it at least calls into question the predictive usefulness, if not the accuracy, of the label “sex cult”.
For some reason the bit that stuck out to me in a wild ride of an article was the guy who believed a super-intelligence was a threat to humans but distanced himself from veganism as if that was crazy part of the AI-doom sex cult that he disagreed with.
a) because veganism seems as exotic to me as someone doing CrossFit
b) the very obvious parallels of a more intelligent creature torturing and killing large numbers of other creatures for relatively little benefit.
They also downplay global warming, nuclear war, wealth inequality, and in general anything other, less cool people they didn't want to identify with thought was important.
If they had thought, yeah, "the majority is more or less right about what the big issues are", there wouldn't be enough room to construe themselves as uniquely intelligent.
Who are these "they" you're talking about?
If your answer is EA or rats then a simple Google search will just show you that it's not true?
I'm yet to see evidence they are weighing global warming they way it IMO should be weighted relatively to eg AI risk. I get it, it's not cool.
Compare the number of posts of some EA-adjacent public people talking about AI and talking about this. Or what measure should we choose?
Label them what you want. I'll go with EAs if you insist.
Sure, they're not entirely monolithic. There's room for "EA, but vegan", "EA, but global warming still matters", maybe even "EA, but vaguely socialist". As I keep saying, the core belief is that they're special because of their intellectual superiority, anything else is negotiable.
No, what you said what that "They also downplay global warming, nuclear war, wealth inequality" which is just a flat out lie
Not all of them downplay all of these, but yeah, most of them do. They may have one or two areas of dissent (for EA as a self-identity-building project, do you see why that is to be expected?), but there is a core of beliefs. Yudkowsky's. And he's downplayed all of these.
I read it more like a conformist performative veganism. I think the implication is that this group treats veganism and all of these beliefs as a kind of shibboleth, an in-group marker for which conformity is expected. The immedaite "feeling" associated with a cult is the feeling of "high control", cf. https://www.google.com/search?q=high+control+group
I read it that EA practices (incl. veganism), beliefs, "rantionalism", and doomerism are all aspects of the high control environment
Nina Hartley has got to have some of the craziest stories. I'd love to see her talk about all the weird stuff she's seen/been involved with.
Well, this is old news. I'm still baffled by the fact that people take output of public personas as isolated facts. Take Nietzsche. Sure, great philosophical input. But try reading the sections about women. Same for Schopenhauer. Why? Because they were both incels and had big mother wounds that have never been addressed. So they spew their shit and people take it at face value even tho it stems from hurt. So why would I take any of this seriously if it's very obviously stupid and removed from reality? (Doesn't mean their whole works are)
People chasing the absolute rational are in my experience numbing their emotional part because theres something they can't look at. Or people that chase success above all are driven by their "unresolved packages" as well. I'm not saying that any of this is wrong per se, it's not black and white. But it's the context for what people do, say, and think. They are human after all. We need to consider context and not only the idea itself.
> Somewhere, some rationalist-brained person is reading this and saying I'm a "low decoupler", and that whether they run a sex cult has nothing to do with whether they're correct.
Glad he's at least admitting it as a possibility.
Also: you know, you don't need to rely on lossy proxies like how much or how little sex do the arguments' proponents have. You can just read the arguments!
I think the mores of people building software that is made available as a conversational entity to the entire human population is at least somewhat relevant.
> Somewhere, some rationalist-brained person is reading this and saying I'm a "low decoupler", and that whether they run a sex cult has nothing to do with whether they're correct.
Yudchuds gonna yudchud.
I’d rather be in a “sex cult”, whatever that entails, than this ridiculous culture that’s somehow still doing moral panics over sex. Maybe it’s just deeply human, or maybe it really is stabilizing society to relegate sex to something dirty and private, but I have a feeling it’s just a dumb conflation that’s so easy to exploit it’s never allowed to go away. It’s like the “think of the children” thought-stopper and we’d do well to get past it.
Naturally, any culture has misfits, radicals and free-thinkers, and so naturally a sex-negative culture will produce at least some focus on sex in its counter-cultures. Doesn’t seem very interesting.
This isn’t moral panic, it’s saying “hey, all these influential people are intimately connected across the strongest possible dimensions (money, sex, belief and legal grey areas), we should probably pay attention to that”
>all these influential people
And by all you mean 1-2 big names and a bunch of unknowns, with all other big names having no connection to this.
I'm not sure how being polyamorous, having kinks and not hiding them makes someone part of a cult, even if I generally don't agree with Yud's entire thesis.
The article presents widely how the people it describes all live in communal housing, engage in harmful psychological practices, have a shared belief in the imminence of the Apocalypse, etc.
Can you confidently list which names specifically live together and the specific harmful psychological practices?
If the man known for urging caution with respect to AI founded a sex cult, then I don't think i want to know what kind of stuff the accelerationist faction are into.
More of this madness coming out of the west coast of America, in particular California.
The Manson family, Heaven's Gate, NXIVM, Rationalists, Zizians... what next?
All turn out to be rotten to the core with their need to reshape and craft their own paradigm, which then gets warped by egos, individualism and power.
Hate to think what other madness is brewing.
California is on the West coast, i'm pretty sure. is not it?
Whoops, fixed now
The word "cult" is doing a lot of heavy lifting here. It can mean anything from religion, to weird religion I don't like, to the organizations that try to cut you off from contact with your family and other people outside the organization. Only the last type are the bad ones, from what I can see.
Cult quite explicitly means the latter, and is what the article is explicitly accusing the core of this community being (that is, not everyone reading LessWrong, but specifically the actual people living in the communes).
No one calls the Catholic Church "a cult", nor do they call, say, Zoroastrianism a cult (to pick a relatively small religion with weird practices).
I will note, if, like me, you're not a native English speaker, that there are two words, "cult" and "sect" that commonly exist in many European languages but often mean opposite things. That is, in English a "sect" is any "flavor" of a religion - catholicism and protestantism are sects of christianity; while "cult" describes closed off communities (usually of a religious nature) that are deemed harmful to their members. In my own native language, the two meanings are reversed - we use "cult" for "legitimate" religions, and "sectă" for closed off harmful religious communities.
I mean, this is a rather fun article to read but I'm not entirely sure about the thesis.
> AI Safety Is Mostly A Sex Cult.
> I don't think these people should make policy. (1/?)
One of the learnings from things like the whole Epstein saga has been there are a lot of sex cults in the policy making world and, yes, the people making policy generally shouldn't be. But I'm not sure this article actually comes up with a good explanation of why these particular people shouldn't be making policy. People having active sex lives isn't a disqualifying trait. Being in a cult doesn't disqualify people from being in politics. It is routine and expected that fields will have a lot of similar people in them (as I recall the overlap between security researchers and the furry community is unusual too, and that is hardly more important than justifying a passing comment).
It is more routine than the article writer seems to think for all the people in a small field to know each other and get on really well.
You do understand what a cult is? There’s a reason why we don’t let the Scientologists run the military no?
Because they don't get enough votes to control the military. If they put a convincing pitch in for why they should then they can. Being a cult doesn't rule them out.
> You do understand what a cult is?
Pretty much any religion and a few other organisations beside. Basically people who believe something strongly. What do you think a cult is, in the sense the article is using it?
> Because they don't get enough votes to control the military.
Well, technically military is run now by maga incel death cult, so yeah I give you that one. I forgot I was discussing on US forum.
As of what cult is, from Merriam-Webster: “ a group (such as an organization or religious sect) with tenets and practices regarded as coercive, insular, or dangerous”
Generally we today discuss as cults manipulative social sects that sureound around one leading authoritative figure who has extensive psychological control over rest of the members. Often these groups use these strange dynamics to abuse members, do financial crimes, or even murder people (I guess the Manson case and Jonestown are most famous, Cali has a history of this).
As for me, I never listen to a man wearing a fedora no matter what his credentials.
It also [0] defines a cult as great devotion to a person, idea, object, movement, or work (such as a film or book) or a small group of very devoted supporters or fans. And I put it to you that the article is clearly using the word more in that sense because it is quite difficult to fit the words coercive, insular, or dangerous to the AI safety people. The accusation in the article seems to be that they're too friendly and sexually inclusive. And if you want to communicate with them they appear to be active on Twitter and not shy.
> Well, technically military is run now by maga incel death cult, so yeah I give you that one. I forgot I was discussing on US forum.
Well, we're speaking in English. I was careful to phrase that in a way that applied to what I'm pretty sure is any English-speaking country. Anywhere that votes could put anyone they like in charge of the military for any reason, cult or no cult. It isn't technically relevant - or practically relevant in most cases. It is just a question of whether the cult has the numbers or not.
[0] https://www.merriam-webster.com/dictionary/cult
Yes word have meanings, often several of them. I chose the one we often use when we say ”death cult”, or ”sex cult” etc. You claim to say you know exactly what meaning the author meant, which either you have to derive from quite careful textual analysis - which you haven’t - or you have to be the author themselves. Personally think having sex is quite fine and dandy, even group sex if that’s your thing. Sex cult indeed sounds a bit ominous. I wonder why furries are never described as cult, although they are indeed very devoted. Britney Spears fan club has very strond devotees who I’m sure have sex with each other (since it is mostly gay men), but never heard of Britney Spears Sex Cult.
I wonder if it just that you try to weasel away from negative publicity by trying to flex the meaning of words to turn the meaning of this bluesky post upside down? In that case I might suggest you to tread carefully. When nothing is true, everything is possible and all that.
No, a cult is any closed-off fanatical group that is deemed harmful to its members and sometimes to outsiders as well. The group who launched a sarin gas attack in Tokyo's metro, Aum Shinrikyo, was a cult. The Jonestown group was a cult. Scientology is often considered a cult, though this is less widely accepted (especially since Scientology is much larger).
No one considers Catholicism, or Protestantism, or Baptism cults. There may be specific communities that are cults that claim to be parts of these churches, but that doesn't mean that "pretty much any religion is a cult".
> No one calls the Catholic Church "a cult"
> No one considers Catholicism, or Protestantism, or Baptism cults.
Speak for yourself. I was coercively and forcefully added to the Roman Catholic church when I was baptised. Attempts to record myself as removed (the 'Count Me Out' campaign) were met with familial and attempted Church pressure. I was asked to visit the church to discuss the 'impacts of my decision'. I was not recorded as leaving, and the church updated its rules so people in Ireland could no longer request to leave the church.
The ways cults control you can vary, and they don't have to involve threats or violence to be coercive control. Threatening to take away someone's community for a lack of belief is controlling.
> Being in a cult doesn't disqualify people from being in politics
Let's survey the American voters
You don't need to survey them because they already voted. US congress is 95%+ religious people
Religious people, not cultists. How many Scientologists, Falun Gong members, etc are there in Congress?
Someone who hasn't been reading the object-level arguments for 10+ years might reasonably wonder if these are good heuristics. Here's an additional data point: I think polyamory is stupid, I've read the object-level arguments, and this entire thread is ridiculous.
Everyone who works at a frontier lab should be in prison, AND ALSO the people talking about 10% risk of human extinction are both sincere and correct despite their many personal failings, AND ALSO the frontier labs have already created AI that could Hugging Face a biolab with smallpox or worse, AND ALSO they aren't going to slow down or lose any customers if you adopt some galaxy brained strategy where you spitefully tell them they aren't allowed to stop accelerating because polyamory is stupid.
> Someone who hasn't been reading the object-level arguments for 10+ years might reasonably wonder if these are good heuristics.
I think you might have meant hermeneutics, in the sociological sense of 'subjective interpretation'?
Possibly. That's not a word I trust myself to use correctly. What I meant is that the object-level claims require a lot of time and energy to evaluate, and it's sometimes reasonable to use a heuristic like "sex cults are not trustworthy" to save time, but if you've already read the arguments then the heuristic loses almost all value.
Yudkowsky and friends believe many things, and reasonable people can disagree about many of them, but "at least 10% risk of extinction if we don't slow down" is a relatively straightforward conclusion based on the offensive capabilities of unreleased models and the terrible state of biosecurity and cybersecurity.
Even without AI, the risk of extinction might be above 10% if people with the relevant skills were interested in using them that way. AI adds an alien willingness to achieve poorly specified goals at any cost.
Hm I think in terms of "can I trust a group of people" we're talking about heuristics yes as in: "how can I quickly approximate an answer to the question of trust?"
If you think your issue is important, and that people should follow your lead, maybe leave the sex cult first.
If anyone is serious about this topic they will find a new spokesperson immediately.
Like that would help. The rest of the world plain refuses to engage with the arguments they've been consistently presenting for decades.
What changed is that they can no longer be dismissed as bunch of loonies who confuse cheap science fiction with reality, because that "cheap science fiction" became the actual reality. So obviously, instead of considering the arguments, the next step is to try and paint them as loony sex cult, because this lets people continue to do the stupid thing and then act surprised, and proclaim no one could have predicted this (and the "sex cult" obviously does not count because it was a "sex cult").
I don't think that's true. It's hard to argue that Elon Musk, the richest person and spokesperson of tech for a lot of people, never participated in SV sex parties or had weird sex behaviors. Or Trump, the president of the USA, has a clean history of standard sex as well. Or Bill Gates, the nice philanthropist, never engaged in weird sex circles. But that doesn't make them lose their leadership positions or status.
Sex cult is beyond mere weird sex behavior.
Horrible people
I tried to read Map and Territory, it just isn't that good if expecting Nietzsche or Heidegger.
A sex cult though is obviously a salacious framing.
It is just sad if even San Francisco culture is trending toward a sexual morality and sexual closed mindedness on par with bible thumpers.
We'll definitely need some safety with AI sex bots
[Orgasmatron 3K - Starting up!]
I guess the EA and rationalist cult members are out in force since this got flagged. Not a good look guys, you're just confirming what everyone thinks.
It may have been flagged as the title being misleading, as it did not really present as many details around the sex part as the title suggested.
Jokes aside it would have probably flown a bit better by removing the sex from the title, as the issue and problem brought up is about it being a cult, and not about the sex part specifically.
The title is the one used by the author, how is that misleading?
I meant about the title as used by the author, not the one who linked it here. But in any case I said it was a joke about not giving many sex related details in the article.
skills issue.
charlie-day-conspiracy.gif
I’m literally speechless. I just can’t.
He does not understand the issue. Not presenting any backing for his ideas, just emotions. The idea has been misused by the bros immensely, that does not help to grasp it either. But AI extermination of humans is a well known problem we have no solution for. Luckily AGI is not so close.
But AI extermination of humans is a well known problem we have no solution for
Sure, but the realm of fictional problems is so huge that we just can't come up with solutions to all of them. I like watching the Terminator movies (well, the first couple) as much as the next person but I can't be spending my time imagining how we could defeat skynet.
From the friendly article:
"Somewhere, some rationalist-brained person is reading this and saying I'm a "low decoupler", and that whether they run a sex cult has nothing to do with whether they're correct."
Good to see some raising the issue of the sex cult that Effective Altruism is:
I'd like to hear what feminists have to say about that picture in TFA of that woman and six men watching her. Just as I'd like to hear what feminists have to say about Amodei's wife who used to run a porn site.
And while a sex-cult abusing women is one thing. Theft is another. More emphasis should be put though on the lying scums in their ranks, like the polyamorous SBF and Caroline Ellison or their guru and its $15m ill-acquired mansion in the UK (which hasn't been clawed back yet but could).
And speaking of lying scums stealing money, we know about several making constant announcements were they lie, lie and then lie some more to ensure a never-ending stream of money and insane valuation of their proprietary, closed, not altruistic at all companies.
A despicable movement and I, for one, I'm very happy that there are companies out there releasing open-weights models so that the thieving-and-sex-cult at least get some competition.
> I'd like to hear what feminists have to say about that picture in TFA of that woman and six men watching her.
That picture is literally a joke based on a meme, which is based on a porn star's promotional image. To the extent feminists care, I would expect that care to be about the underlying subject matter rather than about this satire of it.
Why bother talking about ideas when you can just insult people
90% of this is just driven by strong emotional reactions from someone that feels uncomfortable about people being open about their non-standard sexual life, in the same way that many people have weird negative reactions about polyamorous relationships in general.
The remaining 10% is just the author trying to sprinkle in cult-like insinuations to justify his contempt.
Regularly meeting with people that have shared interests and ideas is not a cult.
Finding some of those people attractive and having sex with them is not a cult.
But somehow the fact that the romantic/sexual relationships are not traditionally monogamous makes it a cult?