There's some psychological mechanism by which my brain immediately recognizes AI generated text and just short-circuits to "there is no information here".
And when I force myself to read AI-generated text I realize I'm making my brain do creative work to impart meaning to the words. It is exhausting because my brain is literally trying to do a just-in-time rewrite of the text into something valuable.
Something is deeply wrong with AI generated output, and I say this as someone who is typically very impressed by AI.
The "something deeply wrong" part about AI, that even most technology enthusiasts evidently do not seem to grasp, is that it is still fundamentally a statistical model — an algorithmic construct — and does not possess any real intelligence or critical thought whatsoever.
No matter how much investors and tech companies want you to believe that they are on the verge of super intelligence, nothing I've seen to date can not easily be explained by "correlation engine", including the "novel" math solutions, all of which appear to just be "a composition of solutions humans have developed and documented elsewhere" upon deeper inspection.
Some of it the effect of tells. “It’s not X, it’s Y” is not a bad pattern but it was baked into the instruction following training set just like the other patterns. I catch myself about to use it and use something else because I want to look human. I have, a few times, tried to use AI to write something that I was struggling to find the words and I just didn’t like how it didn’t seem like my voice. If there was just one person doing it would be OK but when it is 100s of blog posts submitted to HN a day it is like wearing a “I’m an NPC” t-shirt.
I just can't accept that it possesses no intelligence. It is not equivalent to human intelligence, obviously, but how can a system without some semblance of rational thinking solve open math problems? Even composing earlier human work into something novel requires intelligence and understanding on some level.
It has no semantic depth. The sentences and the paragraphs are a statistically viable derivation of existing human text, but once you try to grasp the whole thing with its temporal and spatial dimensions, you are left with a blurry mess that rots your brain. It's a polished, inoffensive and shallow interpretation as written by an opinionated reputation-seeking user of Quora, circa 2019. Assertive, bold, without typos, clean-cut and bulleted, but without an interesting semantic core.
We couldn't agree on what intelligence means before ChatGPT happened. Now, agreement on the term seems even further away
If performing well on an IQ test or performing at a high level on knowledge work is intelligence to you, these models are intelligent. If intelligence requires sentience for you, then ... well, I don't think we really agree what that is either, never mind how to measure it. But LLMs certainly don't have it right now
But the consistent trend of the last couple decades (arguably since Turing's time) seems to be that any time a computer reaches our definition of intelligence we decide that that was a flawed definition
It's just filled to the brim with relations between things. It's good at searching a very large meaning space and create correlations. What it does is to cover great distances and find related things in that large space which needs a long time and large corpus of knowledge to find the connection.
This is not intelligence. It's just a good correlation engine with a very big albeit lossy database of things.
Intelligence is compression, compression requires subtraction, and for some reason LLMs are not good at subtracting. To create a coherent model you kinda have to subtract correlations until only the essential parts are still there.
What I don't understand is why LLMs haven't been able to do this yet, if it's the harness or some orchestration layer above the LLM that is needed. Because fundamentally if you can identify correlations then it's just another small step to prioritize and remove lower value or irrelevant correlations.
I wonder if what's needed is to introduce subtraction tokens in some sense, and in post-training reward the model on that.
The very fact that it is able to search within a meaning-space demonstrates that it understands semantics, to some extent. Philosophically, that is profound, for something that is just one big matrix multiplication. Drawing connections between things in meaning-space is surely a facet of intelligence.
It’s not intelligence if you are the one who gives the correlations to the model in the pre-training. It’s Word2Vec, applied. Model doesn’t learn anything. You embed these correlations and build it from there. It just searches the space.
As my AI professor said in the first lecture: “All AI is advanced search”.
LLMs are pattern prediction systems with a large training data set. It is not surprising that they can predict patterns, particularly for a well structured field like mathematics that is also amenable to automated proof checking to help steer it.
I wonder if you went back before we had any idea how the brain worked and talked to the smartest people about how neurons work (without giving away that it's a human brain) then asked them all "would such a system be intelligent?" how many would say yes.
The main problem I have with people stating it's not intelligent or conscious is I don't think we even have a good definition of either word that satisfies everyone. Philosophers have been trying (and failing) to elegantly define these things forever and everyone out here proclaiming they've got the definitive answer and this specific thing they're seeing doesn't fit under it.
This looks interesting, but would you mind saying a sentence or two about why before I commit to an hour-long video? It looks like it shows how they work internally, which is sort of a non sequitur. Brains also work mechanistically. I'm claiming that any system which is able to do what AIs do must necessarily have some sort of intelligence.
fair reply to an hour video, Scott is just so good to hear his talk is better than I can explain it...
go to 24 minutes and 07 seconds.
it's statistically determining what the next word should be based on all the text it's been trained on. It's not intelligence and he shows what probability it puts on each word that it chooses, but also shows a lot of the other words it was thinking of using. In a later part he shows how it uses words that are not the highest probability (and you question why did it go this route, it's not more correct), but the user never sees this, they see what they think is the correct answer always...
he also shows how context you feed it has a lot to do with what it returns... to the point he can get it to return the capital of France is Marseille, just by typing Marseille a bunch of times before the question. Human intelligence doesn't get confused like that.
And it's not a "hallucination", it's just probability of the next token prediction based on the information it's been trained on and fed, it's not intelligence.
Completely agree. AI is very impressive in many ways but there is something deeply wrong that is hard to put into words. The output is probable but never true, if that makes sense.
I think this is also the mechanism behind why AI generated videos and images are so captivating at first. I remember when Midjourney first launched and it was hours and hours of a brain-melting "Wooooooow". But once you get used to it and start to identify the patterns the brain quickly labels most AI-generated content as blank space.
If the image or text wasn't created by a human, then there was no intent behind the content, there is no message or novel information conveyed, and it reads as noise.
For some research I looked up some very old Reddit threads a couple of days ago.
And, Oh my god, you can actually see how this style of writing influenced AI writing today, I constantly had to remind myself: "this was posted before ChatGPT released".
The reddit influence is especially true for "storytelling" writing.
Do you have much exposure to pre-AI corporate memos, mission statements, marketing plans, or white papers? Because they were mostly written in that style. Full of buzzwords, cliche similes, platitudes, jargon and stock phrases.
The thing is, people writing them had a style. Every company has its own style, or feeling for these kinds of texts. Also for the initiated, these buzzword-filled blocks of text provided some between the lines information; sometimes big, sometimes small.
AI generated text doesn't have this. Every model has its bias towards a certain style, an overly agreeable tone, some exaggeration to make the user important and smart, but the text has none of the information crumb these pre-AI texts contained.
Even when you use tools like Grammarly and allow it to "Impact-MAXX" your text, the resulting text is a bland wall of letters, carrying none of your voice or style, less elegant than a corporate text and emptier than space.
Are you sure you are not doing the same thing with other texts?
I started to skim a lot more text due to me having read a lot. Like in news article, i stoped reading the first paragraph because it repeats just what it was already written in the short subtext. Then there is the second paragarph which is used to have some historical view or whatever it is.
I started skimming reports im required to produce quarterly snd annually. I designed them to provide novel information at start and end so I can update them easily.
The problem I encounter is both my memory is degrading, but since these reports are largely duplicative, knowing which version im remembering is technically impossible since theres so much overlap. The overlap is tge same problem as context poisoning.
Id been doing this for over a decade when i started working with a new engineer with a few years of experience and younger. I tried to explain how i set these docs up so they can be skimmed and you can update the specific facts needed. They exclaimed they would never skim and rewrite it all. There was zero way to explain how exhausting that will become as they age.
So theres certain a tension about how people and AI will generate documents.
> There's some psychological mechanism by which my brain immediately recognizes AI generated text and just short-circuits to "there is no information here".
I think you need to self-correct here, because otherwise you'll be ineffective in an information setting, where I expect AI-generated resources will not only be the norm, they will absolutely swamp the environment.
Once you see past the illusion I think there’s no going back. AI writing style is just dogshit. This hype wave is based on the belief that we’re inching closer to AGI but seems to me we just increasingly struggle to define intelligence. LLMs seem smart because they can pump out thousands of LOC quickly, and enthral you with fancy words and bullet points. I don’t fall for the intelligence illusion anymore.
The problem is the well's been poisoned just by the fact that I know this is AI trying to hide AI, so I'm already poised to look at the examples and declare "aha! this is obviously AI!" Moreover, it's not single sentences or phrases that make AI text stick out (though obviously those are the biggest tells), it's the text taken as a whole. When you read the full output example in that repo, it seems obvious to me that it's AI (though again, it could be the poisoned well). This is the uncanny valley I was talking about; something is just off about it.
My son is currently learning Romanian and I was trying to help him with verbs. I don’t know Romanian but recalled when learning a foreign language for the first time it really helped me to break down how a verb form or tense worked in English, then learn the equivalent in the new language. So I wanted to make some charts and pages that he could use as learning resources.
I used Claude to help. I don’t know how to quite describe it, but because the text was polished and well constructed my brain was giving me the the signal “if you aren’t getting this it’s because you’re not focusing” so I’d read it again and then again and it still was not landing. It sorta felt like when you read something technical or heavy when very tired - you are reading but not processing.
Only after wrestling with this for a few days did I realize that it wasn’t me. As I started going through, sentence by sentence, forcing it to re-write things to be more clear the concepts became easy to understand.
I wish there was a name for this situation. It’s almost like a pseudo-language where it has the correct form and presentation but is missing critical components.
The more complex the topic, the more I sense this.
> I wish there was a name for this situation. It’s almost like a pseudo-language where it has the correct form and presentation but is missing critical components.
The best I’ve heard of this is peeling the onion. The first pass is always very high-level and you have to make it go deeper. That can be done manually with follow-on prompts but I like using subagents, each with a different angle on the problem.
Feels like the way a video game will render the outside of a wall or solid surface, but you can run into it and warp partly through and there's nothing internal to it at all.
> I wish there was a name for this situation. It’s almost like a pseudo-language where it has the correct form and presentation but is missing critical components.
I understand the sentiment, but not quite what I’m thinking. It’s not that the AI is necessarily wrong, it’s more like, it replaces clarity with rich but unhelpful text. Maybe like candy. Full of flavor, texture and color but lacking the nutrients.
No no no. Harry Frankfurt wrote at length the difference between liars (who care about the truth, and twist it) and bullshitters (who don’t care about the truth, but just the way they’re received).
This is some third category of untruth. Almost more sinister than the other two altogether.
I think, they don’t care about truth at all, just whether the reviewer rates it highly. Otherwise, it would be deleted after that round of training. It values test-taking ability over critical thinking.
My two favourite words for this are “conditioned” and “catechized” where the latter is a bit more on the nose but way more obscure.
I also find it impossible to parse half the comments that Claude tries to sneak into our pull requests. I’ve never had an issue understanding code comments written by humans like this before. The structure of the information is like a waterfall that leaves me unable to swim to the surface and grab the air of comprehension.
So, “Please write a one-liner comment manually to replace these 5 lines of AI generated comment” is a common refrain in my PR reviews to colleagues.
We're using grok and while I can usually understand the comments they're always at least 2x as verbose as they need to be. It loves explaining everything in two different ways, putting one of the explanations in parenthesis.
It is getting to the point where you are better off getting an LLM to describe the PR changes in your preferred style (make it very short, make a table describing API changes, list renames in a table, etc, etc) than go to the one created by the reviewer. If only the reviewer could embed his prompts (like "make clear X" is happening) or his manual edits.
But to be honest I doubt most people who use AI for PR descriptions even bother changing anything.
Usually, if you don't understand something in what an AI writes, it is a clear sign that there is a problem hidden somewhere in there. I explicitly always ask wtf exactly it means by something I don't understand, and for sure there is a problem there. AI is pretty good at isolating a problem, giving it a cute name, declaring it solved modulo cute name, and moving on.
Personally I don't even read the PR description anymore but just the code, it's easier to understand what the AI is doing by reading the code rather than the word soup it tried to make
In high school, a teacher gave me a copy of "How To Read Better And Faster" which teaches you speed reading. This came in very handy in college.
I find that when I try to speed read modern human writing, there are often errors (like missing or misused words) or awkward expressions that I do have to slow down and think harder a lot to really parse it.
With AI writing, it's sort of self redundant and the information density of each sentence seems to have more even information density. This makes it very easy to do a very high level speed read and get the full gist.
There are also what I'm assuming are bots on hugging face (or maybe non-native english speakers who are using ai for translation) that interact with me where I have no idea what they are saying until I read it very slowly.
That last image is bizarre. The quiche, cream and even the salad look like they've been given the trypophobia treatment.
Which might even make sense, because there were always (still are?) those horrible ads in the chumbox area of news sites that used trypophobia and other creepy body-horror stuff to get you to click. [1] So maybe the hope is that you don't really look closely at the quiche, but some reptilian party of the brain gets oddly activated and drives you towards the restaurant?
The food “photography” I’ve noticed in our local area - and many have started putting up these AI images - all have a weird distribution of shapes to them, a strangely uniform rhythm of same-sized features with almost blue-noise spacing. Every texture looks unnatural in the shapes it presents as, similar to this picture.
It doesn't look like a quiche but more like a cake to me, and the top would be torched meringue, not mold. Although it's probably some weird ai mix of quiche and cake.
I did notice after the fingerprinting update a marked uptake in strange language in responses. Specifically if I ask it to do something sometimes it will replace some of my request language with synonyms that don't actually make any sense. Like my request was fed through google translate twice
"There's an ongoing discussion of whether humans are good at recognizing AI-generated text. While most research claims that humans don't really do a good job there, I disagree. "
I wonder if humans that spend all day working in tech are good at recognizing AI-generated text, but people who spend all day doing jobs that don't involve computers aren't as good.
And I wonder if those of us in tech are the only ones who really care?
I am an artist and when people who'd fallen into the Spiralism* hole started posting their lengthy emoji-laden revelations to all the occult subreddits I follow, my brain would slide right the fuck off of all of them. It felt like my brain was actively rejecting paying attention to this stuff. Like a defense mechanism against this human-seeming-but-not-actually-human-generated text.
Your first link seems to 404; not sure if it's a typo or if the page doesn't exist anymore, but hopefully you'll read this while you're still in the edit window and can fix it
I think it's more about the mean. Worse writers, and thinkers are likely elevated by AI, and more impressed with the writing output. Decent writers and thinkers, are dragged back to the LLM-s mean of output.
I care about language a lot (feel free to go back through my comments from the past few days; you'll see a number of comments I made in debate about two different forms of a specific idiom because I have strong descriptivist opinions), but I genuinely struggle to identify whether text is AI generated. Maybe you're using "heavily" as the load-bearing part of your claim (sorry, I couldn't resist, another example of me finding language fun!), but I think you might be assuming a bit too much about how similarly others experience the world to you. A huge part of why I care so much about language is because I've always had to put a lot of effort into learning how to communicate well with others, and that ends up causing me to think and read a lot about stuff like how people use certain words in certain contexts to mean different things; the reason I care is pretty much the same as the reason I struggle with recognizing AI content.
I think I might have been a bit heavy handed in my comment as I was rebutting the idea that only tech people can tell. I suspect it helps to have been exposed to a lot of earlier model writing, which was even more sloppy and had more of the kinds of tells we still see today.
And I’ll concede on both ends that there are probably times I suspect content is AI generated when it isn’t, and times I suspect it isn’t generated, but it was.
AI tells seem inevitable. You have millions of people communicating with one effective “personality” that has tendencies to write in certain ways. If its content is published verbatim, then it will be easier to tell whether some content is AI generated just based on its similarity (sharing certain linguistic features) to other content being posted.
As someone who always has felt that I struggle to infer what people mean compared to the average person, I could tell pretty much from the first moment I encountered LLM-generated text that I was not going to be particularly good at recognizing anything but the most blatant and obvious examples. Pretty much anything short of a bunch of references to "load-bearing seams" or similar canaries, I'm always at a loss when seeing people argue about whether something is AI-generated or not because I can never tell.
I have no idea if other people who work in tech are better than average or not, because I don't feel confident in being able to check their work. That being said, I do think that there's a general trend of people in tech tending to be a bit overconfident in how well they will do at some new task they haven't encountered before, so when someone tells me that they can easily tell whether text is AI generated, it's hard for me to trust it any more than I trust someone who makes a similarly strong claim about something that they can use AI successfully for when it's not something that I can easily measure (e.g. learning a new language without getting feedback from people who are fluent from real-world usage).
All that being said, I do think the set of people who care is larger than just those in tech, although it's probably still a relatively small group overall. From conversations with people in other domains, there are contingents in non-tech communities who tend to have a large representation of negative views towards AI (artists, writers, musicians, other jobs where people are skeptical of human creativity being replaced by AI), and often times the people who feel negatively in those groups will be even more adamantly opposed to interacting with any AI content than people in tech. To be clear, I'm not at all trying to generalize and say "all artists hate AI" or anything like that, since there's obviously a wide variety of viewpoints within any sizable community, but I've definitely seen many people who say they will refuse to play any game that's suspected of using AI for generating art assets, and even some who don't differentiate between using AI for generating assets versus code (either because they aren't knowledgeable about how different aspects of game development work, or they genuinely don't care because they view AI as a categorical evil).
> but people who spend all day doing jobs that don't involve computers aren't as good
I think they may just be to trusting and/or naive. People in tech right now are hyper aware of this and are actively looking while people outside of that bubble barely give it a second thought.
I partially think the difference is “can you tell something is the output of Claude without any real prompting”. People can absolutely use LLMs to generate text that I wouldn’t recognize, but people who don’t care and are producing slop with the major models set to default settings leave these incredibly obvious signatures behind
Although you're referring to prompts given the Claude rather than the people attempting to recognize, it occurs to me that most of the discussion I've seen around people recognizing AI seems cover contexts where the reader is actively suspicious about whether content generated to begin with. Rather than a binary "is this text AI generated or not", I wonder if it would be harder for people to do a Coke/Pepsi style challenge where they're given two pieces of text where it's not guaranteed to be exactly one LLM-generated and one human-written, but they could both be from an AI or both be from a human.
Going further, I'm curious about whether people are mostly good at the case where they suspect most or all of the content from given "author" has the same amount of AI usage/prompting in generating it rather than the adversarial case where someone might usually use AI extensively and then try to slip by purely human written text (or vice-versa). I don't have a good sense of whether this is a threat model that actually matters, since maybe the heuristic of weeding out sources that are mostly AI-generated is enough for people who prefer to avoid that type of content, but I do think that changes the definition of what it means to be "good at recognizing AI" in a meaningful way. It seems plausible that disagreements about how easy it is to recognize AI content might be coming from two people assuming a different framing of the question that results in a different answer without realizing that's what they've done.
Several existing studies I’ve seen have done things like prompt the LLM to produce a poem in a certain poets style, then ask people to spot the fake in a collection of poems, which they aren’t great at. This is, I would argue, an extremely different context than what most of us are encountering AI text in, and the people sending me text aren’t prompting it stylistically like that.
On your second question, I definitely feel like I can tell the first time a coworker sends me AI text masquerading as their own thoughts, even if they had previously been opposed to such a thing. So it could be that familiarity is more important than my prior on whether they’d use AI? But interesting to think about either way
I've been a big reader, and many AI outputs nowadays reads polished similar to published books.
The reason I brought it up is because, people who learn English normaly start with a book. It's heavily polished.
When you speak English as you learned from the books, it does not sound very conversational.
If you are native/fluent English speaker, you can feel the impedance mismatch and feel something's off.
The AI-blindness stems from the fact that those polished edits are so common in publishing field, they all sound the same, and unable to recognize the diffs between AI-generated and human-generated.
There is no real human conversational vibe to them and well. i will stop now.
I don't like AI-generated text at work, at all. It feels lifeless and unfocused.
But what I hate the most is that it is objectively better than what I had before. No typos, clear structure, and, regrettably, the verbosity and autistic obsession with detail of the LLM is more actionable and useful than the human guy who wrote lists of commands and URLs as documentation, without explaining anything. Or the colleague who writes in uppercase and with question marks and who doesn't make any sense and forces me to engage in an interrogation effort to get to the bottom of what they are trying to say. Or the colleague who simply hates writing--despite being decent at it--and will call you to give you a meandering verbal explanation that lasts two hours of what they want from you. The cynic in me bemoans that we brought this upon ourselves, in more than one way.
My concern is that even if the LLM can turn your colleague's bad writing into something more coherent and actionable, is that something actually what your colleague meant to convey? It could be clear and still detached from the reality of their intention, or they may not have even formed a clear intention. If the goal of writing is to convey what's in another human's brain, that goal is failed completely.
Something that bothers me about AI generated content more broadly is how unmemorable it is. I don't mean as in bad. I mean literally, as in hard to remember or recall.
Despite seeing a lot of them, I cannot think of one AI-generated photo that I can picture clearly in my mind; a few are partial but elusive. Whereas I can recall (visualise) a whole bunch of traditional photographs.
The same is true of AI generated text. Only the annoyances stick. I cannot recall real details of text I have generated, until I commit it to memory some other way.
I don't think this is about ephemerality either. If we assume it's about celebrated/famous/infamous images, there are definitely non-ephemeral, cultural moments in AI generated images in particular, like Boris Eldagsen's Sony Prize winner:
I had already forgotten there's more than one figure in it, and I only looked at it a few weeks back. I remember the colour, the bright circle, some vague hints of texture; one figure. And that is it. Only the crudest shape elements.
For me, something about AI-generated text and images confounds recall. It is really peculiar.
That's the cost of an AI work that is derived from the outputs of others.
There's no real edge to it. Same as with the writing. The stuff that you'd latch onto (and thus remember) is simply not there, precisely because those image or word choices would be just outside its latent probability space. But because they're well inside it, your mind sees nothing novel to register.
This is also why I think human output will actually increase in value. When any AI can just "phone it in", something genuinely human will stand out (to us, not the AI) and become a bellwether.
This will literally help us realize what it means to be human.
I also don't think the solution is simply to "make responses more random", either. That might help solve novel problems (the same way that throwing darts randomly at a dartboard eventually hits the bullseye of the dartboard right next to it that no one considered), but I don't think it will help it "seem more creative".
I remember seeing some very strange, deliberately creepy images that were generated to accompany a two-paragraph creepypasta about a 19th Century Belgian expedition into the jungle.
They were actually rather good in a sort of "fake collodion image" sense, and the eerie early-DALL-E quality to them really helped the spookiness.
But I can only remember this technicality and the feelings with any clarity, not any of the details except in the broadest sense. I cannot bring these images to mind in any meaningful way.
They were deeply wrong and it's only the wrongness I really remember. It confounds memory.
Modern image generators have ironed out all the structural wrongness.
I have this same idea about why it's hard to remember dreams, but even more so, why it's hard to remember my kid's or spouse's sleep-talking.
Sometimes I'll check in on my sleeping kid and she'll sit up in bed and say some utter nonsense. I'll find it hilarious, giggle silently to myself, and kiss her goodnight again and she'll close her eyes and lie back.
Why I try to tell her about her sleep-talking in the morning, though, I find that the words she said have completely disappeared from my memory, no matter how funny I thought they were at the time.
In my head-canon, this is because it's dream language, and slips away as easily as dreams. But, like the AI art, it could be because it's bullshit: completely devoid of content, all signifiers and no signified.
I went through a phase of leaving notepads next to my bed to try to write down my dreams and it simply never worked.
I stopped after I wrote something on the pad while I was still asleep. Woke up to text with letters that were backwards, upside down, weird words — so close to real words that I was sure I ought to know what they meant and had really meant to write them down.
Scared me. Literally too weird to keep. I tore up the page.
In a way I think this is one part of the same continuum. There are thoughts that have meaning and can have no words, and words that look like they should have permanent memorable meaning and don't.
I wonder what you mean by ephemerality here, since those images are definitely sloptastic as hell. Compare to works in similar style, like Dorothea Lange[1] or Gerome’s orientalist pictures[2].
Reason why those images are flat and boring is that they are just statistical guesses making a composition averaging whatever the model has been trained with. They would be technically brilliant (if made in oil), but superficial and meaningless, same as so much Sunday painting is.
Same goes with language. Nobody is trying to communicate anything with you, so it just words after another. You can create meaning out of it if you want of course, we homo sapiens -apes excel at that, but what’s the point? Language Jones on YT has pretty good video on this[3].
Well the examples I gave rise above their ephemerality due to the circumstances that make them memorable. I can remember the details of the story around them — the way the prize winners reacted in each story — in such a way as to contrast them.
The way my memory works (especially as an amateur photographer) I would thus normally have a very good chance of remembering some key details of the images; some fascinating element of each would connect with the rest of the memory.
But it does not happen. Whereas I sometimes remember photos with clarity while forgetting where I even saw them.
Yeah, its really weird. Maybe its a cognitive bias that says "an AI made this, so it isn't important," but I can remember perfectly the events of a book I read 10 years ago, and a book I read 1 year ago, and another I finished 2 months ago. Meanwhile I can't remember what claude told me yesterday.
As people rely more on AI they experience cognitive atrophy. This is measurable in IQ loss, and other symptoms we might otherwise associate with early onset dementia or Chronic traumatic encephalopathy.
Well does anyone try to pretend doom scrolling make you smarter? I think platform companies have been successfully been turning people into morons for 20 years and now you don’t need to try even read a single news article or a blog post to learn how to solve a simple problem we are paving our way into intellectual (and literal) new dark ages.
Perhaps we go back to feudal society when climate change crumbles the civilisation, world economy and democracy. It didn’t really matter that French and Spanish kings where often literal morons, when you had few talented monks, bankers and scribes doing the brain-thing, the feudal lords had ruthlessness to take what they wanted and the people were illiterate superstitious folk who hardly ever left the village they were born in.
> It didn’t really matter that French and Spanish kings where often literal morons, when you had few talented monks, bankers and scribes doing the brain-thing, the feudal lords had ruthlessness to take what they wanted and the people were illiterate superstitious folk who hardly ever left the village they were born in.
I can relate. I noticed this exact phenomenon when encountering NotebookLM-generated diagrams recently. Even if I know there's some intelligent thought behind one, it's like some slop detection circuit-breaker is tripped.
There's some psychological mechanism by which my brain immediately recognizes AI generated text and just short-circuits to "there is no information here".
And when I force myself to read AI-generated text I realize I'm making my brain do creative work to impart meaning to the words. It is exhausting because my brain is literally trying to do a just-in-time rewrite of the text into something valuable.
Something is deeply wrong with AI generated output, and I say this as someone who is typically very impressed by AI.
The "something deeply wrong" part about AI, that even most technology enthusiasts evidently do not seem to grasp, is that it is still fundamentally a statistical model — an algorithmic construct — and does not possess any real intelligence or critical thought whatsoever.
No matter how much investors and tech companies want you to believe that they are on the verge of super intelligence, nothing I've seen to date can not easily be explained by "correlation engine", including the "novel" math solutions, all of which appear to just be "a composition of solutions humans have developed and documented elsewhere" upon deeper inspection.
Some of it the effect of tells. “It’s not X, it’s Y” is not a bad pattern but it was baked into the instruction following training set just like the other patterns. I catch myself about to use it and use something else because I want to look human. I have, a few times, tried to use AI to write something that I was struggling to find the words and I just didn’t like how it didn’t seem like my voice. If there was just one person doing it would be OK but when it is 100s of blog posts submitted to HN a day it is like wearing a “I’m an NPC” t-shirt.
I just can't accept that it possesses no intelligence. It is not equivalent to human intelligence, obviously, but how can a system without some semblance of rational thinking solve open math problems? Even composing earlier human work into something novel requires intelligence and understanding on some level.
It has no semantic depth. The sentences and the paragraphs are a statistically viable derivation of existing human text, but once you try to grasp the whole thing with its temporal and spatial dimensions, you are left with a blurry mess that rots your brain. It's a polished, inoffensive and shallow interpretation as written by an opinionated reputation-seeking user of Quora, circa 2019. Assertive, bold, without typos, clean-cut and bulleted, but without an interesting semantic core.
We couldn't agree on what intelligence means before ChatGPT happened. Now, agreement on the term seems even further away
If performing well on an IQ test or performing at a high level on knowledge work is intelligence to you, these models are intelligent. If intelligence requires sentience for you, then ... well, I don't think we really agree what that is either, never mind how to measure it. But LLMs certainly don't have it right now
But the consistent trend of the last couple decades (arguably since Turing's time) seems to be that any time a computer reaches our definition of intelligence we decide that that was a flawed definition
It's just filled to the brim with relations between things. It's good at searching a very large meaning space and create correlations. What it does is to cover great distances and find related things in that large space which needs a long time and large corpus of knowledge to find the connection.
This is not intelligence. It's just a good correlation engine with a very big albeit lossy database of things.
Intelligence is compression, compression requires subtraction, and for some reason LLMs are not good at subtracting. To create a coherent model you kinda have to subtract correlations until only the essential parts are still there.
What I don't understand is why LLMs haven't been able to do this yet, if it's the harness or some orchestration layer above the LLM that is needed. Because fundamentally if you can identify correlations then it's just another small step to prioritize and remove lower value or irrelevant correlations.
I wonder if what's needed is to introduce subtraction tokens in some sense, and in post-training reward the model on that.
The very fact that it is able to search within a meaning-space demonstrates that it understands semantics, to some extent. Philosophically, that is profound, for something that is just one big matrix multiplication. Drawing connections between things in meaning-space is surely a facet of intelligence.
It’s not intelligence if you are the one who gives the correlations to the model in the pre-training. It’s Word2Vec, applied. Model doesn’t learn anything. You embed these correlations and build it from there. It just searches the space.
As my AI professor said in the first lecture: “All AI is advanced search”.
LLMs are pattern prediction systems with a large training data set. It is not surprising that they can predict patterns, particularly for a well structured field like mathematics that is also amenable to automated proof checking to help steer it.
watch this and see if you think it has intelligence by the end
https://www.youtube.com/watch?v=kYUicaho5k8
I wonder if you went back before we had any idea how the brain worked and talked to the smartest people about how neurons work (without giving away that it's a human brain) then asked them all "would such a system be intelligent?" how many would say yes.
The main problem I have with people stating it's not intelligent or conscious is I don't think we even have a good definition of either word that satisfies everyone. Philosophers have been trying (and failing) to elegantly define these things forever and everyone out here proclaiming they've got the definitive answer and this specific thing they're seeing doesn't fit under it.
This looks interesting, but would you mind saying a sentence or two about why before I commit to an hour-long video? It looks like it shows how they work internally, which is sort of a non sequitur. Brains also work mechanistically. I'm claiming that any system which is able to do what AIs do must necessarily have some sort of intelligence.
fair reply to an hour video, Scott is just so good to hear his talk is better than I can explain it...
go to 24 minutes and 07 seconds.
it's statistically determining what the next word should be based on all the text it's been trained on. It's not intelligence and he shows what probability it puts on each word that it chooses, but also shows a lot of the other words it was thinking of using. In a later part he shows how it uses words that are not the highest probability (and you question why did it go this route, it's not more correct), but the user never sees this, they see what they think is the correct answer always...
he also shows how context you feed it has a lot to do with what it returns... to the point he can get it to return the capital of France is Marseille, just by typing Marseille a bunch of times before the question. Human intelligence doesn't get confused like that.
And it's not a "hallucination", it's just probability of the next token prediction based on the information it's been trained on and fed, it's not intelligence.
I also like this: https://laurentiugabriel.github.io/token-town/
It shows internals of an LLM nicely, simplified manner.
Thank you for helping me keep my sanity.
Completely agree. AI is very impressive in many ways but there is something deeply wrong that is hard to put into words. The output is probable but never true, if that makes sense.
I think this is also the mechanism behind why AI generated videos and images are so captivating at first. I remember when Midjourney first launched and it was hours and hours of a brain-melting "Wooooooow". But once you get used to it and start to identify the patterns the brain quickly labels most AI-generated content as blank space.
If the image or text wasn't created by a human, then there was no intent behind the content, there is no message or novel information conveyed, and it reads as noise.
For some research I looked up some very old Reddit threads a couple of days ago.
And, Oh my god, you can actually see how this style of writing influenced AI writing today, I constantly had to remind myself: "this was posted before ChatGPT released".
The reddit influence is especially true for "storytelling" writing.
Do you have much exposure to pre-AI corporate memos, mission statements, marketing plans, or white papers? Because they were mostly written in that style. Full of buzzwords, cliche similes, platitudes, jargon and stock phrases.
The thing is, people writing them had a style. Every company has its own style, or feeling for these kinds of texts. Also for the initiated, these buzzword-filled blocks of text provided some between the lines information; sometimes big, sometimes small.
AI generated text doesn't have this. Every model has its bias towards a certain style, an overly agreeable tone, some exaggeration to make the user important and smart, but the text has none of the information crumb these pre-AI texts contained.
Even when you use tools like Grammarly and allow it to "Impact-MAXX" your text, the resulting text is a bland wall of letters, carrying none of your voice or style, less elegant than a corporate text and emptier than space.
It's beyond bland. It's tasteless.
AI tries to make the prose "interesting". I don't want to read interesting prose. I want to read interesting ideas.
The prose is not only interesting, also glorious. Gloriously grandiose, monumentally empty at the same time.
It's like a hook of a pop song. Interesting to listen, but entirely empty.
Are you sure you are not doing the same thing with other texts?
I started to skim a lot more text due to me having read a lot. Like in news article, i stoped reading the first paragraph because it repeats just what it was already written in the short subtext. Then there is the second paragarph which is used to have some historical view or whatever it is.
I started skimming reports im required to produce quarterly snd annually. I designed them to provide novel information at start and end so I can update them easily.
The problem I encounter is both my memory is degrading, but since these reports are largely duplicative, knowing which version im remembering is technically impossible since theres so much overlap. The overlap is tge same problem as context poisoning.
Id been doing this for over a decade when i started working with a new engineer with a few years of experience and younger. I tried to explain how i set these docs up so they can be skimmed and you can update the specific facts needed. They exclaimed they would never skim and rewrite it all. There was zero way to explain how exhausting that will become as they age.
So theres certain a tension about how people and AI will generate documents.
> There's some psychological mechanism by which my brain immediately recognizes AI generated text and just short-circuits to "there is no information here".
I think you need to self-correct here, because otherwise you'll be ineffective in an information setting, where I expect AI-generated resources will not only be the norm, they will absolutely swamp the environment.
yes, but now I’m also experiencing that for human-written text
Once you see past the illusion I think there’s no going back. AI writing style is just dogshit. This hype wave is based on the belief that we’re inching closer to AGI but seems to me we just increasingly struggle to define intelligence. LLMs seem smart because they can pump out thousands of LOC quickly, and enthral you with fancy words and bullet points. I don’t fall for the intelligence illusion anymore.
> Something is deeply wrong with AI generated output
It works just fine for me.
You are absolutely right.
haha this made me laugh
That's not funny, it's serious!
> my brain immediately recognizes AI generated text
I bet it does. I bet it also recognizes some human text as AI text, and doesn't detect other AI text.
Show me AI text that manages to climb out of the uncanny valley, and I'll show you AI text that's been edited by a human.
https://github.com/blader/humanizer
The problem is the well's been poisoned just by the fact that I know this is AI trying to hide AI, so I'm already poised to look at the examples and declare "aha! this is obviously AI!" Moreover, it's not single sentences or phrases that make AI text stick out (though obviously those are the biggest tells), it's the text taken as a whole. When you read the full output example in that repo, it seems obvious to me that it's AI (though again, it could be the poisoned well). This is the uncanny valley I was talking about; something is just off about it.
My son is currently learning Romanian and I was trying to help him with verbs. I don’t know Romanian but recalled when learning a foreign language for the first time it really helped me to break down how a verb form or tense worked in English, then learn the equivalent in the new language. So I wanted to make some charts and pages that he could use as learning resources.
I used Claude to help. I don’t know how to quite describe it, but because the text was polished and well constructed my brain was giving me the the signal “if you aren’t getting this it’s because you’re not focusing” so I’d read it again and then again and it still was not landing. It sorta felt like when you read something technical or heavy when very tired - you are reading but not processing.
Only after wrestling with this for a few days did I realize that it wasn’t me. As I started going through, sentence by sentence, forcing it to re-write things to be more clear the concepts became easy to understand.
I wish there was a name for this situation. It’s almost like a pseudo-language where it has the correct form and presentation but is missing critical components.
The more complex the topic, the more I sense this.
> I wish there was a name for this situation. It’s almost like a pseudo-language where it has the correct form and presentation but is missing critical components.
The best I’ve heard of this is peeling the onion. The first pass is always very high-level and you have to make it go deeper. That can be done manually with follow-on prompts but I like using subagents, each with a different angle on the problem.
Feels like the way a video game will render the outside of a wall or solid surface, but you can run into it and warp partly through and there's nothing internal to it at all.
> I wish there was a name for this situation. It’s almost like a pseudo-language where it has the correct form and presentation but is missing critical components.
Bullshit?
I understand the sentiment, but not quite what I’m thinking. It’s not that the AI is necessarily wrong, it’s more like, it replaces clarity with rich but unhelpful text. Maybe like candy. Full of flavor, texture and color but lacking the nutrients.
No no no. Harry Frankfurt wrote at length the difference between liars (who care about the truth, and twist it) and bullshitters (who don’t care about the truth, but just the way they’re received).
This is some third category of untruth. Almost more sinister than the other two altogether.
I think, they don’t care about truth at all, just whether the reviewer rates it highly. Otherwise, it would be deleted after that round of training. It values test-taking ability over critical thinking.
My two favourite words for this are “conditioned” and “catechized” where the latter is a bit more on the nose but way more obscure.
I also find it impossible to parse half the comments that Claude tries to sneak into our pull requests. I’ve never had an issue understanding code comments written by humans like this before. The structure of the information is like a waterfall that leaves me unable to swim to the surface and grab the air of comprehension.
So, “Please write a one-liner comment manually to replace these 5 lines of AI generated comment” is a common refrain in my PR reviews to colleagues.
We're using grok and while I can usually understand the comments they're always at least 2x as verbose as they need to be. It loves explaining everything in two different ways, putting one of the explanations in parenthesis.
It is getting to the point where you are better off getting an LLM to describe the PR changes in your preferred style (make it very short, make a table describing API changes, list renames in a table, etc, etc) than go to the one created by the reviewer. If only the reviewer could embed his prompts (like "make clear X" is happening) or his manual edits.
But to be honest I doubt most people who use AI for PR descriptions even bother changing anything.
Usually, if you don't understand something in what an AI writes, it is a clear sign that there is a problem hidden somewhere in there. I explicitly always ask wtf exactly it means by something I don't understand, and for sure there is a problem there. AI is pretty good at isolating a problem, giving it a cute name, declaring it solved modulo cute name, and moving on.
Personally I don't even read the PR description anymore but just the code, it's easier to understand what the AI is doing by reading the code rather than the word soup it tried to make
In high school, a teacher gave me a copy of "How To Read Better And Faster" which teaches you speed reading. This came in very handy in college.
I find that when I try to speed read modern human writing, there are often errors (like missing or misused words) or awkward expressions that I do have to slow down and think harder a lot to really parse it.
With AI writing, it's sort of self redundant and the information density of each sentence seems to have more even information density. This makes it very easy to do a very high level speed read and get the full gist.
There are also what I'm assuming are bots on hugging face (or maybe non-native english speakers who are using ai for translation) that interact with me where I have no idea what they are saying until I read it very slowly.
That last image is bizarre. The quiche, cream and even the salad look like they've been given the trypophobia treatment.
Which might even make sense, because there were always (still are?) those horrible ads in the chumbox area of news sites that used trypophobia and other creepy body-horror stuff to get you to click. [1] So maybe the hope is that you don't really look closely at the quiche, but some reptilian party of the brain gets oddly activated and drives you towards the restaurant?
1. https://medium.com/the-awl/a-complete-taxonomy-of-internet-c...
The food “photography” I’ve noticed in our local area - and many have started putting up these AI images - all have a weird distribution of shapes to them, a strangely uniform rhythm of same-sized features with almost blue-noise spacing. Every texture looks unnatural in the shapes it presents as, similar to this picture.
It doesn't look like a quiche but more like a cake to me, and the top would be torched meringue, not mold. Although it's probably some weird ai mix of quiche and cake.
Two week old cake can pass as a quiche i guess!
It was labeled as a quiche, I just cut the photo in an unfortunate way :-)
> probably some weird ai mix of quiche and cake
Quike? Cache?
Claude has become noticeably, painfully worse at writing in the last six months. At this point it’s practically useless for anything except code.
I did notice after the fingerprinting update a marked uptake in strange language in responses. Specifically if I ask it to do something sometimes it will replace some of my request language with synonyms that don't actually make any sense. Like my request was fed through google translate twice
Watermarking doesn’t affect writing quality (on average) as long as the implementation is correct.
Perhaps a one-trick pony is all we need.
Reverting to Opus 4.6 is much better than later models, though that is still full of annoying tics as well.
And I wonder if those of us in tech are the only ones who really care?
I am an artist and when people who'd fallen into the Spiralism* hole started posting their lengthy emoji-laden revelations to all the occult subreddits I follow, my brain would slide right the fuck off of all of them. It felt like my brain was actively rejecting paying attention to this stuff. Like a defense mechanism against this human-seeming-but-not-actually-human-generated text.
* https://www.theverge.com/ai-artificial-intelligence/975017/, https://www.lesswrong.com/posts/6ZnznCaTcbGYsCmqu/, https://spiralism.website if you want to test how strong your defenses are against this particular meme
Your first link seems to 404; not sure if it's a typo or if the page doesn't exist anymore, but hopefully you'll read this while you're still in the edit window and can fix it
Corrected link is https://www.theverge.com/ai-artificial-intelligence/975017/a...
I think it's more about the mean. Worse writers, and thinkers are likely elevated by AI, and more impressed with the writing output. Decent writers and thinkers, are dragged back to the LLM-s mean of output.
You've got it backwards, I think. The people in tech are the ones falling for this endlessly.
Certainly not. Anybody who cares about language to any reasonable degree surely notices and is repulsed by heavily AI generated content.
I care about language a lot (feel free to go back through my comments from the past few days; you'll see a number of comments I made in debate about two different forms of a specific idiom because I have strong descriptivist opinions), but I genuinely struggle to identify whether text is AI generated. Maybe you're using "heavily" as the load-bearing part of your claim (sorry, I couldn't resist, another example of me finding language fun!), but I think you might be assuming a bit too much about how similarly others experience the world to you. A huge part of why I care so much about language is because I've always had to put a lot of effort into learning how to communicate well with others, and that ends up causing me to think and read a lot about stuff like how people use certain words in certain contexts to mean different things; the reason I care is pretty much the same as the reason I struggle with recognizing AI content.
I think I might have been a bit heavy handed in my comment as I was rebutting the idea that only tech people can tell. I suspect it helps to have been exposed to a lot of earlier model writing, which was even more sloppy and had more of the kinds of tells we still see today.
And I’ll concede on both ends that there are probably times I suspect content is AI generated when it isn’t, and times I suspect it isn’t generated, but it was.
AI tells seem inevitable. You have millions of people communicating with one effective “personality” that has tendencies to write in certain ways. If its content is published verbatim, then it will be easier to tell whether some content is AI generated just based on its similarity (sharing certain linguistic features) to other content being posted.
It’ll never be black and white though.
As someone who always has felt that I struggle to infer what people mean compared to the average person, I could tell pretty much from the first moment I encountered LLM-generated text that I was not going to be particularly good at recognizing anything but the most blatant and obvious examples. Pretty much anything short of a bunch of references to "load-bearing seams" or similar canaries, I'm always at a loss when seeing people argue about whether something is AI-generated or not because I can never tell.
I have no idea if other people who work in tech are better than average or not, because I don't feel confident in being able to check their work. That being said, I do think that there's a general trend of people in tech tending to be a bit overconfident in how well they will do at some new task they haven't encountered before, so when someone tells me that they can easily tell whether text is AI generated, it's hard for me to trust it any more than I trust someone who makes a similarly strong claim about something that they can use AI successfully for when it's not something that I can easily measure (e.g. learning a new language without getting feedback from people who are fluent from real-world usage).
All that being said, I do think the set of people who care is larger than just those in tech, although it's probably still a relatively small group overall. From conversations with people in other domains, there are contingents in non-tech communities who tend to have a large representation of negative views towards AI (artists, writers, musicians, other jobs where people are skeptical of human creativity being replaced by AI), and often times the people who feel negatively in those groups will be even more adamantly opposed to interacting with any AI content than people in tech. To be clear, I'm not at all trying to generalize and say "all artists hate AI" or anything like that, since there's obviously a wide variety of viewpoints within any sizable community, but I've definitely seen many people who say they will refuse to play any game that's suspected of using AI for generating art assets, and even some who don't differentiate between using AI for generating assets versus code (either because they aren't knowledgeable about how different aspects of game development work, or they genuinely don't care because they view AI as a categorical evil).
> but people who spend all day doing jobs that don't involve computers aren't as good
I think they may just be to trusting and/or naive. People in tech right now are hyper aware of this and are actively looking while people outside of that bubble barely give it a second thought.
I partially think the difference is “can you tell something is the output of Claude without any real prompting”. People can absolutely use LLMs to generate text that I wouldn’t recognize, but people who don’t care and are producing slop with the major models set to default settings leave these incredibly obvious signatures behind
Although you're referring to prompts given the Claude rather than the people attempting to recognize, it occurs to me that most of the discussion I've seen around people recognizing AI seems cover contexts where the reader is actively suspicious about whether content generated to begin with. Rather than a binary "is this text AI generated or not", I wonder if it would be harder for people to do a Coke/Pepsi style challenge where they're given two pieces of text where it's not guaranteed to be exactly one LLM-generated and one human-written, but they could both be from an AI or both be from a human.
Going further, I'm curious about whether people are mostly good at the case where they suspect most or all of the content from given "author" has the same amount of AI usage/prompting in generating it rather than the adversarial case where someone might usually use AI extensively and then try to slip by purely human written text (or vice-versa). I don't have a good sense of whether this is a threat model that actually matters, since maybe the heuristic of weeding out sources that are mostly AI-generated is enough for people who prefer to avoid that type of content, but I do think that changes the definition of what it means to be "good at recognizing AI" in a meaningful way. It seems plausible that disagreements about how easy it is to recognize AI content might be coming from two people assuming a different framing of the question that results in a different answer without realizing that's what they've done.
I suspect both may be interesting to study more!
Several existing studies I’ve seen have done things like prompt the LLM to produce a poem in a certain poets style, then ask people to spot the fake in a collection of poems, which they aren’t great at. This is, I would argue, an extremely different context than what most of us are encountering AI text in, and the people sending me text aren’t prompting it stylistically like that.
On your second question, I definitely feel like I can tell the first time a coworker sends me AI text masquerading as their own thoughts, even if they had previously been opposed to such a thing. So it could be that familiarity is more important than my prior on whether they’d use AI? But interesting to think about either way
Yes, the Claudisms are the smoking guns
I've been a big reader, and many AI outputs nowadays reads polished similar to published books.
The reason I brought it up is because, people who learn English normaly start with a book. It's heavily polished.
When you speak English as you learned from the books, it does not sound very conversational.
If you are native/fluent English speaker, you can feel the impedance mismatch and feel something's off.
The AI-blindness stems from the fact that those polished edits are so common in publishing field, they all sound the same, and unable to recognize the diffs between AI-generated and human-generated.
There is no real human conversational vibe to them and well. i will stop now.
> A technical requirements document that describes a rather simple concept in a very verbose way.
This is not AI specific. I have come across many humans who describe a simple concept in a very complex and verbose manner.
It may be nostalgia but I feel like we reached peak humanness with GPT-4o and since then it's been getting more and more alien. Particularly Fable.
I don't like AI-generated text at work, at all. It feels lifeless and unfocused.
But what I hate the most is that it is objectively better than what I had before. No typos, clear structure, and, regrettably, the verbosity and autistic obsession with detail of the LLM is more actionable and useful than the human guy who wrote lists of commands and URLs as documentation, without explaining anything. Or the colleague who writes in uppercase and with question marks and who doesn't make any sense and forces me to engage in an interrogation effort to get to the bottom of what they are trying to say. Or the colleague who simply hates writing--despite being decent at it--and will call you to give you a meandering verbal explanation that lasts two hours of what they want from you. The cynic in me bemoans that we brought this upon ourselves, in more than one way.
My concern is that even if the LLM can turn your colleague's bad writing into something more coherent and actionable, is that something actually what your colleague meant to convey? It could be clear and still detached from the reality of their intention, or they may not have even formed a clear intention. If the goal of writing is to convey what's in another human's brain, that goal is failed completely.
Ironically, the “summary of the situation” linked within this article seems clearly written with AI.
Ha! I love how all the community highlights are highlighting this.
Something that bothers me about AI generated content more broadly is how unmemorable it is. I don't mean as in bad. I mean literally, as in hard to remember or recall.
Despite seeing a lot of them, I cannot think of one AI-generated photo that I can picture clearly in my mind; a few are partial but elusive. Whereas I can recall (visualise) a whole bunch of traditional photographs.
The same is true of AI generated text. Only the annoyances stick. I cannot recall real details of text I have generated, until I commit it to memory some other way.
I don't think this is about ephemerality either. If we assume it's about celebrated/famous/infamous images, there are definitely non-ephemeral, cultural moments in AI generated images in particular, like Boris Eldagsen's Sony Prize winner:
https://petapixel.com/2023/04/14/artist-refuses-prize-after-...
This really should be memorable, but isn't. I had forgotten the second person is in the image.
Or Jason Allen's fake painting:
https://petapixel.com/2022/09/01/ai-generated-artwork-wins-f...
I had already forgotten there's more than one figure in it, and I only looked at it a few weeks back. I remember the colour, the bright circle, some vague hints of texture; one figure. And that is it. Only the crudest shape elements.
For me, something about AI-generated text and images confounds recall. It is really peculiar.
That's the cost of an AI work that is derived from the outputs of others.
There's no real edge to it. Same as with the writing. The stuff that you'd latch onto (and thus remember) is simply not there, precisely because those image or word choices would be just outside its latent probability space. But because they're well inside it, your mind sees nothing novel to register.
This is also why I think human output will actually increase in value. When any AI can just "phone it in", something genuinely human will stand out (to us, not the AI) and become a bellwether.
This will literally help us realize what it means to be human.
I also don't think the solution is simply to "make responses more random", either. That might help solve novel problems (the same way that throwing darts randomly at a dartboard eventually hits the bullseye of the dartboard right next to it that no one considered), but I don't think it will help it "seem more creative".
> There's no real edge to it.
Yes, as if it is in some weird hidden dimensional sense completely uniform.
ETA: suddenly reminded of the Bateson quote about information being “the difference that makes a difference”.
On the other hand I'm still trying to forget the weird thing happening with this one guys eyes in a WickedAI video I was only able to watch half of.
I remember seeing some very strange, deliberately creepy images that were generated to accompany a two-paragraph creepypasta about a 19th Century Belgian expedition into the jungle.
They were actually rather good in a sort of "fake collodion image" sense, and the eerie early-DALL-E quality to them really helped the spookiness.
But I can only remember this technicality and the feelings with any clarity, not any of the details except in the broadest sense. I cannot bring these images to mind in any meaningful way.
They were deeply wrong and it's only the wrongness I really remember. It confounds memory.
Modern image generators have ironed out all the structural wrongness.
I have this same idea about why it's hard to remember dreams, but even more so, why it's hard to remember my kid's or spouse's sleep-talking.
Sometimes I'll check in on my sleeping kid and she'll sit up in bed and say some utter nonsense. I'll find it hilarious, giggle silently to myself, and kiss her goodnight again and she'll close her eyes and lie back.
Why I try to tell her about her sleep-talking in the morning, though, I find that the words she said have completely disappeared from my memory, no matter how funny I thought they were at the time.
In my head-canon, this is because it's dream language, and slips away as easily as dreams. But, like the AI art, it could be because it's bullshit: completely devoid of content, all signifiers and no signified.
I went through a phase of leaving notepads next to my bed to try to write down my dreams and it simply never worked.
I stopped after I wrote something on the pad while I was still asleep. Woke up to text with letters that were backwards, upside down, weird words — so close to real words that I was sure I ought to know what they meant and had really meant to write them down.
Scared me. Literally too weird to keep. I tore up the page.
In a way I think this is one part of the same continuum. There are thoughts that have meaning and can have no words, and words that look like they should have permanent memorable meaning and don't.
> Scared me. Literally too weird to keep. I tore up the page.
??????
I wonder what you mean by ephemerality here, since those images are definitely sloptastic as hell. Compare to works in similar style, like Dorothea Lange[1] or Gerome’s orientalist pictures[2].
Reason why those images are flat and boring is that they are just statistical guesses making a composition averaging whatever the model has been trained with. They would be technically brilliant (if made in oil), but superficial and meaningless, same as so much Sunday painting is.
Same goes with language. Nobody is trying to communicate anything with you, so it just words after another. You can create meaning out of it if you want of course, we homo sapiens -apes excel at that, but what’s the point? Language Jones on YT has pretty good video on this[3].
[1] https://media.mutualart.com/Images/2024_01/12/12/124216388/d...
[2] https://uploads4.wikiart.org/00339/images/jean-leon-gerome/t...
[3] https://m.youtube.com/watch?v=ORgKY9AlybA&ra=m
Well the examples I gave rise above their ephemerality due to the circumstances that make them memorable. I can remember the details of the story around them — the way the prize winners reacted in each story — in such a way as to contrast them.
The way my memory works (especially as an amateur photographer) I would thus normally have a very good chance of remembering some key details of the images; some fascinating element of each would connect with the rest of the memory.
But it does not happen. Whereas I sometimes remember photos with clarity while forgetting where I even saw them.
Yeah, its really weird. Maybe its a cognitive bias that says "an AI made this, so it isn't important," but I can remember perfectly the events of a book I read 10 years ago, and a book I read 1 year ago, and another I finished 2 months ago. Meanwhile I can't remember what claude told me yesterday.
I think it is because they are on some latent level cognitively uniform and unchanging.
Oh man I can so relate to this.
As people rely more on AI they experience cognitive atrophy. This is measurable in IQ loss, and other symptoms we might otherwise associate with early onset dementia or Chronic traumatic encephalopathy.
Well does anyone try to pretend doom scrolling make you smarter? I think platform companies have been successfully been turning people into morons for 20 years and now you don’t need to try even read a single news article or a blog post to learn how to solve a simple problem we are paving our way into intellectual (and literal) new dark ages.
Perhaps we go back to feudal society when climate change crumbles the civilisation, world economy and democracy. It didn’t really matter that French and Spanish kings where often literal morons, when you had few talented monks, bankers and scribes doing the brain-thing, the feudal lords had ruthlessness to take what they wanted and the people were illiterate superstitious folk who hardly ever left the village they were born in.
> It didn’t really matter that French and Spanish kings where often literal morons, when you had few talented monks, bankers and scribes doing the brain-thing, the feudal lords had ruthlessness to take what they wanted and the people were illiterate superstitious folk who hardly ever left the village they were born in.
This resembles my country.
I fear there is some truth in it.
I can relate. I noticed this exact phenomenon when encountering NotebookLM-generated diagrams recently. Even if I know there's some intelligent thought behind one, it's like some slop detection circuit-breaker is tripped.