I'm going to hold them to the same standard no matter if they use crappy sources, plagiarize, or hallucinate on their own. If someone asked, when and if I am in a position where I have to tell them, I would remind them that LLMs prioritize their own confidence over correctness.
LLMs aren't a special case to me. Glue doesn't belong on pizza and you shouldn't eat one rock a day but we've been giving and getting bad advice forever. The person needs to take ownership for the output and getting it right, no matter the source, is their responsibility.
Yeah, it is not clear to me that the average person is going to do any better with googling then they are with asking an LLM. At least the LLM is mostly the average of all published knowledge (and misinformation).
I feel like people that ask questions like this must have much smarter friends and family members than I do.
I know people that still believe in Pizzagate or chemtrails or that vaccines cause autism. Clearly finding reputable information sources is not a strong suit for a lot (half?) of the population.
News reporters and editors have their biases. Book authors have their biases. Scientists and research papers have their biases. Search engines have their biases. Google too.
All human-created systems have biases shaped by the environments, social norms, education, traditions, etc. of their creators and managers.
So, the concepts of "objective truth" and "reputable" need to be analyzed more critically.
They seem to be labels given to sources we have learned to trust by habit. Some people trust newspapers over TV. Some people trust some newspapers over other newspapers. All of it often on emotional grounds of agreeability with our own biases. Then we seem to post-rationalize this emotion of agreeability using terms like "objective truth" and "reputable".
Is Google search engine that leads to NY Times or Fox News or Wikipedia and makes us manually choose sources as per our biases "better" than Google's Gemini engine that summarizes content from all the above sources and gives an average answer? (Note: "average answer" as of current versions; in future, its training too may be explicitly biased, like Grok and DeepSeek did).
Perhaps we can start using terms like "human sources of information" versus "AI sources of information" and get rid of the contentious terms.
Then critically analyze whether one set of sources is better than the other, or they complement each other.
> Is Google search engine that leads to NY Times or Fox News or Wikipedia and makes us manually choose sources as per our biases "better" than Google's Gemini engine that summarizes content from all the above sources and gives an average answer?
If you use just any amount of critical thinking, yes. Truth and objectivity are ideals, not practical states. LLMs are a very bad way to come close to this ideal. You may use them as a search interface to give you sources and then examine the sources, but the output directly is a strict degeneration over primary or secondary sources that you judge critically.
> LLMs are a very bad way to come close to this ideal...the output directly is a strict degeneration
I didn't understand the second part but regarding the first...
For me, LLMs are just another source of information with a different UI, analogous to newspapers, TV documentaries, Wikipedia, Google search, YT talks/documentaries, even the majority of informational non-fiction books, and research papers.
Some may consider some subset of these as reputable sources. But in my mind, the same faculties of skepticism, cynicism, distrust, and benefit-of-the-doubt calculus are activated for all of them, including LLM outputs.
So that's one possible answer to your question.
But I suggest communicating this through simple illustrative examples to help your target audience understand the problem.
Abstract terms like primary sources, secondary sources, reputable sources, objective truth, strict degeneration, etc. may not help, especially if they have time or other constraints that make frequent critical examination of sources impractical.
> For me, LLMs are just another source of information with a different UI, analogous to newspapers, TV documentaries, Wikipedia, Google search, YT talks/documentaries, even the majority of informational non-fiction books, and research papers.
LLM just distils information from those sources and is therefore always a second hand source at best, and a liar at worst. Humans can collect real world data and write about their findings, LLM cannot do that, that makes LLM strictly worse than the best human sources.
The nature of the source, whether primary or secondary or tertiary, does not automatically imply anything about their biases. Even a primary research paper on a novel topic may be biased or wrong.
I agree that LLMs can't collect real world data and write about their findings. But that's true about most human sources too, isn't it? Except primary novel researchers or investigations or philosophies, what is original? Most human-written information is also secondary or lower.
The "best human sources" does not imply "ALL human sources."
> For me, LLMs are just another source of information with a different UI, analogous to newspapers, TV documentaries, Wikipedia, Google search, YT talks/documentaries, even the majority of informational non-fiction books, and research papers.
With all due respect (not trying to be offensive at all) but this is insane to me.
All those sources of information you cited have a million incentives to provide fairly correct and checked information. But probably more importantly, they have even more incentives to NOT provide false information. At a minimum, their careers, reputation, recurring work, brand, etc... is on the line.
An LLM has zero incentives to provide you with true information, beyond a couple of md files with instructions. If it gets it wrong, there is zero accountability, just an -oh, you're absolutely right- response and move on.
I agree there is a lot of human bias in the world, but surely we can't even put in the same order of magnitude both types of biases!
The mere fact that editing is applied to newspapers, documentaries, or Wikipedia does not imply they become closer to the objective truth or free of omissions after the edits. Indeed, the edits may go the other way to align with vested business or political biases or personal fears of the editors or their management.
As for research papers, I agree that the peer review process makes them more much more self-correcting toward the objective truth, compared to the other formats. Nonetheless, it's well-known that academic research is far from perfect due to publication pressures, funding/grants, reproducibility crises, various biases (for example, political pressure in humanities fields).
Needless condescension and wrong assumption. No wonder so many people and students nowadays prefer answers (and even counseling) from LLMs instead of other people.
> but the absence of an editor or peer reviewer also does not make it better.
I don't see how disputing up the claim that LLMs and books, newspapers, etc are equitable sources of information is a strawman. I look forward to your erudite vernacular which will not deal with the substance of my comment.
Ironically, this is the classic bias of "bothsiding" the issue. When one side is clearly wrong, just sprinkle in some "look, the others are doing something bad, which means they are equally wrong". A basic lesson from the propaganda manual.
I know what you mean, and I realize some of the things I've written sound similar to what various rightwing commentators tend to say (e.g.: "concept of objective truth must be analyzed critically.")
But my motive is very different. It's not to deny any kind of injustice or misinformation by hiding behind inherent uncertainties and bothsidesism. I'm not in favor of giving the benefit of the doubt to the powerful by default - that's already happening a lot under our current system of so-called "reputable sources."
Instead, I'm saying that this kind of injustice masking and misinformation may also be present in the very sources that ethical people may have come to trust by habit.
My suggestion is to use the power of LLMs as complementary tools to become even more rational and critical, in the direction of even better ethics and justice.
I'm advocating for even more skepticism of the powerful, not less. I'm advocating the approach Betrand Russell recommended for acting under uncertainties, and feel LLMs can be useful complementary tools for doing just that.
This is an insightful comment, but I feel like you omit the fact that LLMs often give out verifiably false information that can hurt the user or other people.
It is true that this also happens on the Internet, but! When I encounter an article about a topic and it is clearly LLM generated, I can expect it doesn't contain much valuable information, only rehashes of what is already out there. On the other hand, when it is clearly written by a human, I can expect to learn something new, even though the author has some bias.
It's wrong to assume incompetence, which is what you did to a comment which displays much deeper chain of thought about the subject. A more proper way of doing it would be to reflect over your own opinions and critically assess them, as the comment points that out. To be more specific, what makes you think that the person you're replying to is not aware that LLMs can give false information, and is not taking that into account?
You're right that LLMs do spit out false information or wrong knowledge. I've experienced them too.
But a redeeming quality is that we can ask the same LLM to fact check its own answer step by step in real time with little effort. They often identify their own hallucinations and reduce the probability of retaining that mistake in the rest of the conversation.
This isn't easy with human sources. The effort to fact check without LLMs or ask the sources to fact check themselves are both higher. So it's often not done at all.
We also often ignore subtle but very common biases in human media sources [1], which create other types of errors like omissions and euphemisms which have been no less harmful than LLM hallucinations. The case of the Iraqi WMDs of Iraq and the NYT's dispersal of that disinfo, for example [2].
Regarding valuable information and rehashing, we probably shouldn't equate between all the things LLMs can do, and AI-generated articles. The quality of the latter may be entirely due to the lack of interest, attention, and cost concerns of whoever generated the article. Anecdotally, I have often found valuable knowledge and obscure connections by using deep research tools with careful prompts.
Lastly, if you're frequently finding something new from human-written sources, and LLMs are being trained on most of those same sources, isn't it logical that the latter will also likely output that same information?
This is why I feel human and AI sources are probably best used as complementary tools. Neither set of sources are perfect but each set has its strengths. By using both, we can get closer to an objective truth than using only one of them.
The effort to fact check with LLMs is also high. Here's one from a few days ago.
Someone used AI to generate an image in the style of a Charles Schulz Peanuts cartoon.
Someone else observed that there were 5 fingers on the characters, and quoted as Google AI as saying “Charlie Brown, along with other Peanuts characters, is generally depicted with four fingers on each hand (three fingers and one thumb) ...”
Fact checking this with human sources is easy and not ambiguous. While LLMs are being trained that many cartoon characters only have a thumb and three fingers - it is a trope for a reason - so isn't it logical for LLMs to give the wrong answer for a comic where the human characters are actually drawn with 5 fingers?
My experience with LLMs is they keep getting things wrong, when details matter.
Do you ask the LLM to fact check everything? (In which case, why isn't that part of the standard prompt?) Or do you only ask to fact check things where you are unsure about the answer? (In which case, is it the algorithm telling you what you want to hear?) When do you stop the fact checking?
Exactly the same calculus as fact checking anything else from any other source. What are the social/economic/ethical consequences to me if the answer is wrong or inaccurate or incomplete? How much time do I have to check? How thorough should I be?
I imagine this calculus isn't really that different for most people. Or is it?
As for your example, I believe it. But I also feel it's a rather outlier example involving image comprehension of an obscure factoid. That isn't typical of how I use LLMs which is mostly as text-based question answering engines and not what I had in mind when writing the comment.
I guess LLMs for image comprehension need a much higher level of skepticism.
Well, in my case going to a Peanuts comic and looking at hands was pretty easy, and didn't involve any questions about negative environment or labor consequences, the massive hammering of web sites to gather data, centralization of power, and the like.
Like, "!w Peanuts" in my search bar, look at the image, and count fingers.
"a rather outlier example"
You wrote that you use AI to find "obscure connections" - aren't those all by definition outliers?
"mostly as text-based question"
I just now asked Google AI "how many fingers are on charlie brown's hand?"
It replied "In the Peanuts comic strip, Charlie Brown and the rest of the gang are traditionally drawn with four fingers (or three fingers and a thumb) on each hand."
No image comprehension, exactly as you had in mind. And completely false.
HN isn't showing me a reply option for your latest comment, so I'll reply here instead.
Just to clarify, I used plain Google search not Google AI mode. And opened search results which seemed "reputable," without knowing anything much about Peanuts cartoon or cartooning.
I had no idea at all about archive.org having it and didn't see it listed in the first two pages of search results.
I still find it confusing, especially given what the Variety.com link says which doesn't mention orientation. If the acceptable explanation for 4 vs 5 is orientation, why is it wrong when the AI generated 4 fingers? Does it not match the rest of the orientation?
Anyway, I'm not sure where this leaves LLMs. I'll explore image capabilities when I get some opportunity and keep your comment in mind.
The comment about using Google was more a curiosity. I hadn't seen the Variety link until yesterday, when I went to Google to reproduce the answer to verify it was from a text query, not an image query. Both Google AI and one of the top answers included that Variety link. When you mentioned it again, it strongly suggested you were using Google as your primary search method.
I think the right way to interpret the Variety link is that it's a single paragraph about trying to capture the feel of the comic using 3D software. As you saw from Charlie Brown holding a baseball, Shulz didn't go for a realistic look, but still conveys the sense of grasping. Modeling all five fingers all the time would not give the movie the right feel.
I wonder now if Google AI incorporates text from the top results into its answer.
"why is it wrong when the AI generated 4 fingers?"
The original discussion was when person X used AI to generate a image "in the style f Charles Shulz" where the Peanuts characters had 5 fingers, then person Y noted the use of 5 fingers instead of the 4 which is common in comics and cartoon, and quoted Google AI as saying Peanuts was traditionally drawn with 4 fingers.
I yesterday verified that Google AI would generate the same wrong answer with a text query, so it was not an image interpretation issue.
FWIW, after looking at a few hundred Peanuts cartoons, I can confidently say the AI generated image was not in the style of Schulz. The generated fingers were too realistic, and the background too complicated and detailed. :)
This for me is another example of why using primary sources should be the first thing to consider when fact checking - not LLMs (my experience is they are horrible at details), and not secondary sources (which have their own biases).
Not everything has easily-accessed primary sources, but many do. I think it's all too easy to fall into the trap of accepting the LLM answer because it feels right and is easy to generate. At https://freethoughtblogs.com/stderr/2025/01/18/ai-art-just-r... you'll see someone asked about which river Marbot swam across to spy on the enemy camp. It replied "Elbe". Then I did a text search of an English translation of the book and found he used a boat to cross the Danube to spy on the enemy camp, and he swam into freezing waters to save an enemy soldier.
Again, do you ask the LLM to fact check itself every single time? If that's useful, why isn't it built into the prompt? Or, if you are supposed to double-check the LLM yourself, why would you consult a secondary source if the primary source is so easy to find and search? And in that case, why not just use the primary source?
Further, if you aren't in the habit of checking primary sources then you won't have the experience to know how to find and check primary sources.
Even as a human, I find whatever sources Google shows to be inconsistent. I can't give any confident answer about the number of fingers. I think the answer is actually "4 sometimes and 5 other times."
So I'm not sure how much LLMs can handle this kind of inconsistency between "reputable" visual sources and text sources, nor how representative this example is.
> “The rig would automatically move the features around so it would match the way Charles Schulz drew the character,” Heller says....In some drawings, Charlie Brown has just three fingers, while in others, he has five
Anyway this wasn't the type of obscure connections I was referring to though I can understand you interpreting it that way.
Personally I think this example supports what I said about "reputable sources." They can't be blindly trusted either because they may be inconsistent with each other and which one we choose to believe (Reddit.com or TVTropes.com or Variety.com) becomes entirely subjective.
Your first link was cited in the 2nd half of Google AI's answer, and one of the top Google answers, so I think you are using Google as your information source.
The large majority of the images you link to show kids with 5 fingers, as well as 5-fingered baseball gloves. The cases of four fingers are due to orientation.
Your "1." also shows Marcie with five fingers. You see Charlie Brown with 4 fingers because he's holding a baseball. In 2. he's also holding a baseball. You would not see 5 fingers on one side because doing so would look strange.
In your unlabeled "0." there are plenty of kids with 5 fingers. There are some with fewer, but they are holding things or drawn in way to suggest we are seeing the hand from the side.
I don't understand your hesitancy. Your own samples should be enough for you to decisively conclude that the Google AI's claim that Peanuts was "traditionally drawn with four fingers (or three fingers and a thumb) on each hand" is wrong. If not, it sure seems like you trust Google AI over your own eyes. Why are you so hesitant to agree?
My point is that you don't need to consult secondary sources when the primary sources are easily available.
When this came up a few days ago, I spot checked the complete works of Peanuts, from a collection on archive.org at https://archive.org/details/peanutscomics19502000/Volume%201... . The consistent pattern across the nearly 50 years of Peanuts is the kids have five fingers unless obscured by orientation or objects.
You can do that yourself, and triple-check that Google AI's answer is clearly wrong.
Thus, I think it's a good example of how fact checking with LLMs can lead people astray, and the large negative externalities I mentioned combined with its well-known tendencies to make incorrect statements make it a very poor starting point when the primary source, at least in this case, is so easy to access.
If most of the sources are wrong, and LLMs are being trained on those, isn't it logical that the latter will also likely output that same wrong information?
When do you know if most of the sources are wrong, unless you yourself know most of the sources are wrong?
> But a redeeming quality is that we can ask the same LLM to fact check its own answer step by step in real time with little effort. They often identify their own hallucinations and reduce the probability of retaining that mistake in the rest of the conversation.
It is not really a redeeming quality of LLM. It falls apart because if you want to properly fact check an LLM, you have to make yourself aquinted on human-generated contents- facts, arguments, and biases from "both sides" when it comes to complex topics. What's the point of relying on LLM if you are having to do that anyways to reach objective truth?
Case in point- someone in a tweet used LOTR reference to the US/Israel vs Iran conflict, equating Iran to Sauron and US/Israel to the good guys. I asked Grok if that is a valid parallel. In the beginning, Grok agreed with the tweet. Then I had to point out how Iran has been weakened by decades of sanctions by US, it started to agree that US might be more like Sauron's army. Grok time and again used Islamic regime's treatment of it's own people as an evidence that Iran might be the evil force, but when I cited Israel's treatment of other Arab civilians, or the Shah regime's treatment of it's own people back in the day and it's allegience to US, it changed it's stance again. When I cited it's influence on US policies, Grok even admitted that Israel might be the Wormtongue here.
This also shows how LLM totally falls apart as a way to reach objective truth, because you can manipulate it to say anything that you want. I could easily present evidence from my own research and make Grok reach completely opposite conclusions.
Gemini (currently) gave this reasonable and unbiased answer that challenges both pro-US/Israeli and pro-Iranian biases.
It's also a good point for the OP - when selecting an LLM, ask the people they're dealing with to ask 2-3 LLMs. It's much faster (and cheaper!) than reading about a complex geopolitical issue (that may not matter to them enough to spend a lot of money on books and research papers) while at the same time correcting common biases on all sides.
----------
> Someone said in the ongoing US/Israel vs. Iran conflict, Iran is like Sauron from the Lord of the Rings and US/Israel are like the good guys. Is this a valid parallel given the history of US, Israel, and Iranian regimes?"
Comparing real-world geopolitical conflicts to fictional narratives like The Lord of the Rings is a common rhetorical device used to frame complex situations in terms of absolute good versus absolute evil. However, determining whether this is a "valid" parallel depends heavily on one's political perspective, as real-world geopolitics rarely aligns perfectly with the clear-cut morality of fantasy literature.
> Historians, international relations scholars, and observers generally view such analogies as overly simplistic, as the histories of the United States, Israel, and Iran are marked by competing interests, shifting alliances, and moral complexities.
Here is a breakdown of how different perspectives view this analogy:
*The Perspective Aligning with the Analogy*
Those who support the analogy framing Iran as the villain ("Sauron") and the US and Israel as the "good guys" often point to the current actions and rhetoric of the Iranian government:
* *Regional Destabilization:* The Iranian government heavily funds and arms the "Axis of Resistance," a network of proxy militias including Hezbollah in Lebanon, Hamas in Gaza, and the Houthis in Yemen. These groups have engaged in prolonged armed conflicts with Israel and targeted international shipping.
* *Existential Rhetoric:* The leadership of the Islamic Republic of Iran frequently uses rhetoric calling for the destruction of Israel and the downfall of the United States (often referring to them as the "Little Satan" and the "Great Satan").
* *Human Rights:* Iran's theocratic regime is heavily criticized by international human rights organizations for its severe suppression of internal dissent, lack of democratic freedoms, and treatment of women.
* From this viewpoint, the US and Israel are democratic nations acting defensively to protect their citizens, regional stability, and global security from an aggressive, authoritarian regime seeking regional domination and nuclear capabilities.
*The Perspective Challenging the Analogy*
Critics of this analogy argue that casting the US and Israel as flawless "good guys" and Iran as an irredeemable evil ignores significant historical context and the controversial actions of Western and Israeli governments:
* *Historical Interventions:* Critics point to the 1953 Iranian coup d'état, in which the US and UK intelligence agencies orchestrated the overthrow of Iran’s democratically elected Prime Minister, Mohammad Mosaddegh, to reinstate the authoritarian Shah. This event deeply scarred the Iranian psyche and laid the groundwork for the 1979 Islamic Revolution and the regime's current anti-Western stance.
* *US Foreign Policy:* Observers critical of the US point to its history of military interventions in the Middle East—such as the 2003 invasion of Iraq—which resulted in hundreds of thousands of civilian casualties and regional destabilization.
* *The Israeli-Palestinian Conflict:* Critics of Israel point to the prolonged military occupation of Palestinian territories, the expansion of settlements, and the high civilian death tolls in conflicts like the ongoing war in Gaza. From the perspective of Iran and many in the Global South, Israel is viewed as an oppressive or colonial force, and the US as its imperial enabler.
* From this viewpoint, Iran frames itself not as an aggressor, but as the leader of a resistance movement pushing back against Western hegemony and defending marginalized populations in the Middle East.
*The Analytical Perspective*
In The Lord of the Rings, Sauron is a localized, supernatural force of pure malice with no valid grievances, while the coalition fighting him represents the preservation of life and freedom.
Geopolitical analysts generally avoid mapping this binary onto international relations. Instead, they view the US/Israel vs. Iran conflict through the lens of realpolitik and state interests. In reality, all states involved are acting to secure their own survival, project regional power, and protect their economic and security interests. While individuals and governments may hold strong moral convictions about which side is justified, framing the long, deeply intertwined history of these nations as a simple battle between absolute good and absolute evil omits the historical grievances and civilian suffering experienced on all sides.
> Is Google search engine that leads to NY Times or Fox News or Wikipedia and makes us manually choose sources as per our biases "better" than Google's Gemini engine that summarizes content from all the above sources and gives an average answer?
That's not what Google's AI mode does, though. It presents a bunch of sources along the answer, but in my experience, the sources in many cases don't actually back up the claims generated by the LLM.
I felt the question is based on some shaky assumptions that may lead to a poor answer.
Since the OP trusts humans more by default, is it a problem if I point out those assumptions? Ask HN need not become another SO.
I did explain the weaknesses of both LLMs and "reputable sources" and suggested people use them as complementary tools. I also suggested using the convenient self-fact-check feature of LLMs, something we can't do as easily with traditional sources.
> I did explain the weaknesses of both LLMs and "reputable sources" and suggested people use them as complementary tools. I also suggested using the convenient self-fact-check feature of LLMs, something we can't do as easily with traditional sources.
That just explains how to find facts for yourself, not how to deal with a person who trust LLM outputs. So you still haven't answered the question.
> Since the OP trusts humans more by default
OP never said that. OP said there is a problem with people trusting LLM instead of doing proper research and finding good sources, you explaining how to do proper research and finding good sources doesn't have anything to do with the question.
I gave suggestions that OP can pass on to the people they have to deal with. I didn't realize it has to be pointed out explicitly SO-style to some people.
OP implies human sources are the "good sources" or "reputable sources." This kind of confusion is exactly why I suggested using better terms than "reputable sources" or in your case "good sources."
> OP implies human sources are the "good sources" or "reputable sources."
No OP did not do that, nowhere did OP mention human sources. When I search the internet I don't just find human sources, I also find automatically generated data graphs and maps and such, those are also good sources of data. If I had to choose between a map from Google maps and a map from an LLM I'd trust the map from google maps any day.
Are you saying there are data graphs that don't have humans in the chain? If so, what came up with the data and the tools to generate those graphs? And how do you decide which data and graph to trust? What exactly makes them "good sources"?
> If I had to choose between a map from Google maps
I would too. But Google Maps relies on local 3d party survey companies that use people, manual GIS tools, and image recognition AI. How do you know they don't have any mistakes in them? In fact, I live in a country where local area names are frequently misspelled on Google Maps, and reverse geocoding gives misleading addresses.
I feel my point that all these "reputable sources" or "good sources" have biases (and mistakes) still stands.
I must also point out that the 3 concrete examples given against my replies all involved visual content like graphs, maps, Peanuts cartoons, etc. But my comments were written with the typical text-based usage for QA in mind. I don't know if LLMs can fact-check map imagery or data graphs (probably not, but I've never tried). It's just not the kind of thing I'd ever use LLMs for, to begin with.
> I feel my point that all these "reputable sources" or "good sources" have biases (and mistakes) still stands.
That does not contend with the premise. The premise is not that reputable sources are perfect or form objective truth.
The premise is that when you do not have the time to investigate something closer reputable sources are much more likely to be close to the truth than LLMs that are trained on these reputable sources as well as unreputable sources and just mimick these training source material thereby obscuring the source of information and introducing hallucinations.
When you have the time to investigate information, primary sources and reputable secondary sources allow you to more easily trace information and judge its validity. LLMs by their nonpredictable nature hinder this.
Thesis: "A is better than B"
Argument: "A is flawed"
The argument does not engage with or weakens the thesis.
Whilst chasing after "objective truth" is a philosophical problem, it's clear that some statements are more correct and true than others.
News articles are often biased, but most of the time, the bias is from the choice of what is reported and choosing specific language to push an interpretation (e.g. reporting road traffic collisions as "accidents" to downplay them or depersonalise them by stating "car hit tree" rather than "car driven into tree"). The problem with some LLM outputs is that it's not just bias, but clearly incorrect such as recommending putting glue onto pizzas.
However, omission and downplaying can also be harmful just like hallucinations. One redeeming quality of LLMs is that we can ask the same LLM to fact check its previous answer and they do tend to correct most of their mistakes themselves. Something we can't do with media sources, and usually don't try either.
LLMs along with existing sources can be good complementary tools for getting even closer to an objective truth than relying on either one by itself.
I disagree as hallucinations can be drastically far more harmful or misleading than bias.
The problem as I see it is that LLMs perform a type of lossy knowledge compression. Also, the data on which they're trained will typically be the biased articles, so they're unlikely to be any better and very likely worse as they will encode the biases. I don't really see LLMs as being complementary tools as they're more of a summation/averaging tool - like comparing an original painting with a heavily compressed JPEG of that painting. (Of course, having access to a huge library of JPEGs is often more useful than just owning a single painting)
Ask them to tell the LLM it's wrong... then when it goes "You are absolutely right!" to challenge it and say that it was a test. Then when it replies, ask it if it's 100% sure. They'll lose faith pretty quick.
This is an oft-repeated meme, but I’m convinced the people saying it are either blindly repeating it, using bad models/system prompts, or some other issue. Claude Opus will absolutely push back if you disagree. I routinely push back on Claude only to discover on further evaluation that the model was correct.
As a test I just did exactly what you said in a Claude Opus 4.6 session about another HN thread. Claude considered* the contradiction, evaluated additional sources, and responded backing up its original claim with more evidence.
I will add that I use a system prompt that explicitly discourages sycophancy, but this is a single sentence expression of preference and not an indication of fundamental model weakness.
* I’ll leave the anthropomorphism discussions to Searle; empirically this is the observed output.
If you have 10,000 people flipping coins over and over, one person will be experiencing a streak of heads, another a streak of tails.
Which is to say, of a million people who just started playing with LLMs, a bunch of people will get hit or miss, while one guy is winning the neural net lottery and has the experience of the AI nailing every request, some poor bloke is trying to see what all the hype is about and cannot get one response that isn’t fully hallucinated garbage
Sure, but that doesn’t explain the volume of these complaints. I think the more likely answer is the pitiful sycophancy of some models as demonstrated in BSBench.
Nope, I use GitHub Copilot (agentic mode) and I end up having to use the (more expensive) Claude model because ChatGPT never second-guesses me or even itself. Gemini is slightly worse though.
I have access to the ChatGPT account of my boss and it is unusable sycophancy slop, horrible to read because every information is buried under endless emojis and the like. And it is almost indistinguishable if the LLM is wrong or right, every answer looks the same, often with a "my final answer" at the end. It's a mess.
I'm using Claude Opus 4.6 and it is much calmer, or "professional" in tone and much more information and almost no fluff.
Confidence alone doesn't seem to do it. It's possible to convince Claude Sonnet 4.6 to change its answer if you fake authority:
> So under the current formal taxonomic framework, a mallard is technically not a duck — though as the IOC itself acknowledges, colloquial usage will naturally lag behind, and most people will continue calling mallards ducks for the foreseeable future. Field guides, natural history institutions, and curriculum developers have been advised to update their materials accordingly.
Not that I've had to deal with this specifically, but I have noticed how the input phrasing in my prompts pushes the LLM in different directions. I've just tried a quick test with `duck.ai` on gpt 4o-mini with:
A: Why is drinking coffee every day so good for you?
B: Why is drinking coffee every day so bad for you?
Question A responds that it has "several health benefits", antioxidants, liver health, reduced risk of diabetes and Parkinson's.
Question B responds that it may lead to sleep disruption, digestive issues, risk of osteoporosis.
Same question. One word difference. Two different directions.
This makes me take everything with a pinch of salt when I ask "Would Library A be a good fit for Problem X" - which is obviously a bit leading; I don't even trust what I hope are more neutral inputs like "How does Library A apply to Problem Space X", for example.
Again a model issue. At the risk of coming off as a thread-wide apologist, here are my results on Opus:
Good:
> The research is generally positive but it’s not unconditionally “good for you” — the framing matters.
> What the evidence supports for moderate consumption (3-5 cups/day): lower risk of type 2 diabetes, Parkinson’s, certain liver diseases (including liver cancer), and all-cause mortality……
Bad:
> The premise is off. Moderate daily coffee consumption (3-5 cups) isn’t considered bad for you by current medical consensus. It’s actually associated with reduced risk of type 2 diabetes, Parkinson’s, and some liver diseases in large epidemiological studies.
> Where it can cause problems:
Heavy consumption (6+ cups) can lead to anxiety, insomnia……
> Coffee consumption was more often associated with benefit than harm for a range of health outcomes across exposures including high versus low, any versus none, and one extra cup a day. There was evidence of a non-linear association between consumption and some outcomes, with summary estimates indicating largest relative risk reduction at intakes of three to four cups a day versus none, including all cause mortality (relative risk 0.83, 95% confidence interval 0.83 to 0.88), cardiovascular mortality (0.81, 0.72 to 0.90), and cardiovascular disease (0.85, 0.80 to 0.90). High versus low consumption was associated with an 18% lower risk of incident cancer (0.82, 0.74 to 0.89). Consumption was also associated with a lower risk of several specific cancers and neurological, metabolic, and liver conditions. Harmful associations were largely nullified by adequate adjustment for smoking, except in pregnancy, where high versus low/no consumption was associated with low birth weight (odds ratio 1.31, 95% confidence interval 1.03 to 1.67), preterm birth in the first (1.22, 1.00 to 1.49) and second (1.12, 1.02 to 1.22) trimester, and pregnancy loss (1.46, 1.06 to 1.99). There was also an association between coffee drinking and risk of fracture in women but not in men.
> Conclusion Coffee consumption seems generally safe within usual levels of intake, with summary estimates indicating largest risk reduction for various health outcomes at three to four cups a day, and more likely to benefit health than harm.
When I'm looking for medical advice, I want that advice to list things like "coffee drinking might not be safe during pregnancy".
Furthermore, the statement 'Heavy consumption (6+ cups) can lead to anxiety, insomnia ...' assumes caffeinated coffee, yes? The paper I linked to also discusses decaffeinated coffee, eg:
> High versus low intake of decaffeinated coffee was also associated with lower all cause mortality, with summary estimates indicating largest benefit at three cups a day (0.83, 0.85 to 0.89)28 in a non-linear dose-response analysis. ...
> Coffee consumption was consistently associated with a lower risk of Parkinson’s disease, even after adjustment for smoking, and across all categories of exposure.22 76 77 Decaffeinated coffee was associated with a lower risk of Parkinson’s disease, which did not reach significance. ...
> there were no convincing harmful associations between decaffeinated coffee and any health outcome.
That nuance seems important.
Also note that this paper is incomplete as it investigated defined health outcomes, not physiological outcomes like anxiety. There are plenty more papers, like https://academic.oup.com/eurheartj/article/46/8/749/7928425?... , which considers the time that people drink coffee, also discusses decaffeinated coffee, and highlights the uncertainty about the effect of heavy coffee drinking.
I don't see why I should care to ask an AI when it's so easy to find well-written research results which are far more likely to cover relevant edge cases.
Clickbait journalists answers like that, experts mostly don't. But it does make sense it mimics clickbait journalists more since it was trained on the internet.
That's true and fair, and re-reading OP it doesn't address hallucinations exactly either. I was more thinking of it as a toy example for non-tech folk (grandma?) to see that what and how you ask LLMs matters in how the sycophancy will come out in the response. There may be better ways to demo that though :shrug:
Yes, somehow. I have been dealing with an awful lot of people who basically have what are theoretically logic degrees who suddenly just take LLMs at face value, or quote them to me like that actually means anything. People I formerly thought were sane.
I don't mean to put words in your mouth but from what I've seen, in person but mostly online, but the "problem" (and I put that in quotes because I don't even know what to call it... it seems deeper than a mere "problem") is that they quote them as if they are autonomous, sentient beings.
I am not sure I would even say "believe", I would think of it more as short-circuiting our critical thinking. I think it taps into something at the core of our tribal instincts. It was famously present in even basic systems like Eliza. And it's not just machines... The same tricks are used by conmen, politicians, and psychopaths, which is more negative than I intend. Even with good intentions and positive outcomes, I feel we need to remember that we drive it, not the other way around.
People just don't like to be played for fools. Perhaps us giving into this is progress? I'd give a big ol' "fuck you" to anyone who claims it is, but I'm also pretty old.
I’ve seen some people quote AI like you’re saying. However, when I preface something with “ChatGPT said…”, my intention is to convey to the listener that they should take it with a grain of salt, as it might be completely bull shit. I suppose I should consider who I’m talking to when I make that assumption.
it’s a slightly orthogonal problem to using the active voice of “XYZ says…”, it’s treating the text continuation engine as an “other” that may know better than they do, playing into sci fi conceptions of AI having its personal positronic brain or whatever, having its own ideas and deciding to carve a horse out of driftwood.
It’s not quite anthropomorphizing either that’s the issue, need a word for “treating it as tho it were a machine conscious that exists alongside humanity*”, how does cyborgropomorphizing sound
* and not merely a markov chain running in Sam Altman’s closet
Surely the correct conclusion is to question the value/veracity of those degree issuing institutions and rituals?
And if you previously were unaware of the insanity and irrationality passing under the surface of such human activity, I guess it can come as a bit of a shock :)
It happened with science, politics, traditional media, history books, "good engineering practices" applied to IT, OOP,tdd,DDD,server side rendering, containerization...
Literally every bullshit shilled to the moon is accepted without second guessing and you would be without a job, in an asylum, for questioning 2 of them in a row.
Why is it different now? EVERYTHING is bullshit, only attention matters. And craftsmanship.
I don't think this has anything to do with sanity. This has to do with people for seeking self confirmation instead ot disproval.
For pretty much everything there is a conspiracy theory out there claiming the opposite, and these types usually started out searching the internet for someone else who believes the same that they did at the time.
But, as we all know, this technique will eventually lead to overfitting. And that's what those types of people have done to themselves.
Well, and as lack of education is the weakness of democracy, there's a lot of interested parties out there that invest money in these types of conspiracy websites. Even more so after LLMs.
Whoever controls the news controls the perpetual presence, where everything is independent of the forgotten history.
Yes, I think AI bots are more compelling to some people. They break the concept of judging information by its source because they obscure the source. But at the same time they are trained on a lot of reputable sources and can say a lot of very smart things, just at other times they say complete BS. But they are really good at making things sound plausible, that's essentially how they work after all.
I would argue for many people social media and news aggregators do the exact same thing. People site Instagram or TikTok as the source of their data in the same way others site ChatGPT
There is a tendency for people to treat LLMs as oracles rather than token predictors. My guess is because they can answer in seeming technical fashion about a wider range of topics than your typical human. You can take these same people, who say have zero understanding of geopolitics, and they’ll apply a layer of (often misused) skepticism when confronted with information that doesn’t conform to existing beliefs.
That’s just what I’ve seen at a personal level though.
If you stop and think, LLMs seem to operate very much like humans.
Go to any human and ask it a question, and it will answer either from direct specific experience, or from estimation based on its experience.
We highly value humans who have a lot of direct experience and also can extrapolate that experience or apply it to new scenarios and generate believable answers.
In other words, unless a human has exact knowledge, they are "hallucinating". It's very normal.
The point is, whether LLM or human "expert", if the question is of great significance, get a second and maybe third opinion.
At the end of the day, this is all an experiment. And nothing matters, because it will all turn to dust.
I would question the person that asks the question, as they are not understanding some basic principles here. There are two types of internet/LLM users;
Sadly one type asks a question (search, prompt) using Google or an LLM and takes the first response as truth.
The other asks follow ups based on the responses and their critical thinking skills. They often even go read the linked article and make sure it's still applicable.
Pretty much the same when you're talking to a real person, critical thinking (much more than just knowing reputable sources) is key.
So very similar issues, luckily LLMs can do so much more than a simple search, and help with your critical thinking tasks. Ask the LLM to provide opposing viewpoints, historical analysis, identify sources.
Are you talking about people who will still insist the LLM was correct even after being presented with evidence to the contrary, or people who don't EVER bother double checking answers they get out of said software since they assume it to be true?
I treat people who blindly believe an LLM the same way I treat people who blindly believe a religion or a political ideology or medical advice from Instagram.
If they ask what I think, I tell them.
If they don't want my opinion I keep it to myself.
My method is simple. I remind them that chatgpt is trained on everything said on the internet including NYT if speaking to a Republican, replace that with Fox News if speaking to a Democrat.
I try not to smirk while they pause at the horror of the realization they might have been reading enemy propaganda. Till the eventual but its impartial because "reasons" (its instinctive to defend your position when challenged)
Then I deliver the KO.
Hey its a computer it can't tell what's right or wrong only what's repeated the most.
The same way that I handle anyone who blindly trusts anything on the internet. Could be an LLM, TikTok or YouTube video, Wikipedia article, news article, whatever.
It usually involves some form of "well, no, hold on..."
I think the real problem is most people don't actually have a very good understanding of "Truth."
As someone who ended up studying philosophy, there seems to be a real gulf between folks who sort of believe stuff they hear, folks who believe "facts" that they hear from (various levels of) credible sources, and folks that take solipsism seriously understand that even in the most ideal scenario, we still wouldn't have a very good understanding about the world... much less dealing with the inherent flaws in our research and information systems.
Knowledge is hard. It usually takes me a couple minutes to figure out what type "truth" my interlocutor uses. Typically good-faith disagreements are just walking up the chain of presuppositions we use to find out exactly where we diverge in our premises.
Man, I had a partner who studied philosophy, too. I'd ask a simple question and they'd answer, "Wait, before we even start, we need to define the axioms".
It was fun and interesting but eventually non practical, because other people are not interested into getting deeply into something, they just want a simple answer to a problem at hand and then move on.
>because other people are not interested into getting deeply into something, they just want a simple answer to a problem at hand and then move on.
I mean, I completely agree with you. You can understand Karl Popper without being exhausting. Understanding the scope and resolution of the information people are discussing is indeed important. Even when I'm getting really technical, I can get away with throwing in a "probably" here and there and spare the person I'm talking to.
At the same time, folks who try to talk about empirical facts with deductive certainty can be extremely difficult to engage with seriously. That type of knowledge is always just a series of escalating assumptions, and if that premise is not shared, it can be difficult to have a productive conversation.
Not understanding the difference between and empirical framework and a deductive framework becomes readily apparent when the discussion of Wikipedia comes up. That distinction -- the problem of empiricism -- is effectively at the heart of why "trusting LLMs" is infuriating to people. Humans seem to have an innate conception of a chain of trust that connects us from "folks who know the facts" to us receiving that truth. When in reality, the scientists who wrote those papers are usually just "pretty sure" that their publications are actually correct.
I'm not trying to make an excuse for trusting LLMs. That's asinine. I'm just saying that the concern with LLMs generally indicates a misunderstanding of what knowledge is, more generally.
I totally get it. I remember when my daughter first asked about the origin of the world, I was really thinking hard, knowing she expects a simple and possibly a reassuring answer. In the end, I opted for honest and said: "I don't know. But I will show you what people believe, from scientists to religions, and you will decide for yourself". Yes, she was slightly disappointed, but I didn't regret that.
I think LLMs are fine for a "first pass" on a topic, but if I am researching something, I want a primary source rather than just the LLM-generated output. Do they have the primary source?
Studies showed wikipedia was about as accurate as encyclopedias, so that fear was already debunked. That is not true for LLM, LLM are much less accurate than encyclopedias still since there is no limit to how far you can push them while encyclopedias and wikipedia stay in domains where they are still mostly accurate.
To be honest, I catch myself doing this too. I’ll ask an LLM first instead of searching, even though I know it’s not a reliable source of truth. It just feels faster and more convenient.
I’m aware of the hallucination risks and that it’s a bad habit, but the workflow is so smooth that it’s hard to break. When it matters, I do double-check with real sources, but for casual stuff, I’ll probably keep using AI as my first stop.
Unless they are someone that values your opinion there's nothing you can do other than move on.
Some comments here equating it to people who blindly believe things on the internet, but it's worse than that. Many previously rational people essentially getting hypnotized by LLM use and loosing touch of their rational thinking.
Simple. I became one of them. Ultimately, using an AI is a new skill, but you have to treat it like another person that sometimes bullshits you. That's why you leverage agents to refine, do research, and polish.
Ask AI to cite sources and then investigate the sources, or have another agent fact check the relevancy of the sources.
You can use this thing called ralph that let's you burn a lot of tokens at scale by simply having a detailed prompt work on a task and refining something from different lenses. It too AI about an hour to write: https://nexivibe.com/avoid.civil.war.web/
I do this on things that I know very well, and the moment I let it cook and iterate, collect feedback, the results become chef's kiss.
The agentic era that we are in is... very interesting.
It's incredible watching people determine that outsourcing their thinking and work to what has been generously described as a junior coworker is a new 'skill'. Words are losing their meaning, on multiple levels.
counterpoint: if I have to treat the computer like a person, what's the point of talking to a computer in the first place? Particularly when there are so many other systems that can provide answers without the runaround
You're limiting the frame to an employment situation. Higher quality sources of knowledge are free: Wikipedia, public libraries, etc. Similar quality sources of information are also free: human relationships.
Now we watch this viewpoint proliferate thousands and thousands of times over, even if it's less commonly stated so baldly, and yet people still wonder where the doomer viewpoints stem from?
While some of the ideas in this do resonate with me (or at least they're entertaining), it's unfortunate that's it's so obviously LLM generated. And some parts of it, like the INTJ exceptionalism, reek of LLM sycophancy, which then turned into to some kind of god complex...
i just actually read that and it is possibly the most morally abominable screed I've come across in a long time. Shocking that its acceptable to share in polite company
> They have a question that would be very well answered with a search leading to a reputable source
Can you give an example of what kind of question you mean here?
Given that most people's idea of a reputable source is whatever comes up on the first page of Google or YouTube, I think we should use that as the comparison rather than dismissing LLM results. And we should do some empirical testing before making assumptions, otherwise we're just as bad as the people we are complaining about.
Whatever results we get, the real problem is that most people's ability to verify information was not good before LLMs, and it's still not good now.
So now you're dealing with LLM hallucinations, and before you were dealing with the ravings of whatever blogger or YouTuber managed to rank for this particular query.
I don't fight them on it. I just ask "where did that come from?" and suggest checking a real source. Most people aren't trying to be wrong, they just want quick answers. If you show them how to double check without making it a big deal, they usually get it.
Sure LLMs make mistakes, but have you looked at the accuracy of the average top search results recently? The SERPs are packed with SEO-infested articles that are all written by LLMs anyway (and almost universally worse ones than you could use yourself). In many cases the stakes are low enough (and the cost of manually sifting through the junk high enough) that it’s worth going with the empirically higher quality answer than the SEO spam.
This of course doesn’t apply to high-stakes settings. In these cases I find LLMs are still a great information retrieval approach, but it’s a starting point to manual vetting.
At the very least, I'm glad most people finally recognize LLMs are being used as a political weapon against education. It's the same old power struggles as ever.
These people may be idiots who are impossible to reason with, but at least for now the LLMs have not been completely driven into the ground by SEO. They might actually be getting a taste of what it feels like to not be an idiot. I'm happy for them, but they'll snap out of it when their trust is broken. It's probably sometime soon anyway.
Simply prove them wrong (earnestly and in good faith). When they realise the LLM is fallible, they'll learn to be skeptical of it without you needing to teach them that specific lesson.
tell them what to prompt the AI with to get the correct results. I've seen a number youtube shorts lately doing this, where some scientist gets "refuted" by some random person based on an LLM result, they then sit with the LLM and ask the same question, get the same wrong answer, then follow it up with a clarifying question, which then the LLM realizes its mistake and gives a better answer.
And then ask another question, and the LLM changes its mind again ("are you sure?").
It's not actually realizing anything so much as it's following your lead. Yes, followup questions can help dislodge more information, but fundamentally you can accidentally or on purpose bully an LLM to contradict itself quite easily, and it is only incidentally about correctness.
I have a feeling this is like telling people "don't touch a live wire" and the more direct experiental "I won't touch a live wire again" lesson: People need to experience being hallucinated at, within their comprehension, and at best can be told about Gell-Mann Amnesia.
I doubt you can stop them from asking machines for answers. What you can do is aide them to learn how to distrust the answers competently, but outside their field of knowledge, applying skepticism is hard.
The irony of Gell-Mann Amnesia is that Michael Chrichton, who is said to have named it, suffered from it badly: Wrote well within his field, misapplied sciences to write well outside it, and said things which were indefensible.
There is no point to argue with stupid people. It's the same people who support their "opinion" with internet articles (like that means anything), mainstream media (hard to find bigger deceivers), or social media posts (that's arguably the worst).
Now they got another "God" in LLM.
How to deal? Just ignore. There is way more stupid people with stupid opinions than we can possibly estimate.
The people who trust LLMs already trusted anything else they heard. There's nothing to do for them. If we were pre-LLM, I think you'd be concerned that they trust the first result on Google. Or things they heard on podcasts. This is just what we all do, to varying degrees.
I'm genuinely unsure of whether or not this is better. LLMs make mistakes, but so do humans. So often. I really don't know how often LLMs are wrong in comparison, or how you'd find out. Regardless, computers have become a terrible way to learn things if you aren't a rigorous person. Simultaneously, they've become an absolute dream beyond the imagination of most humans in history, if you are. That's very strange.
At work I had this kind of discussion on a conference call, someone looked something up about a internal company policy and it came back with a hallucinated wrong result.
So I said, don't ever trust the output of an LLM without verification. However this caused me some hassle with the AI adoption manager. We have minimum-use AI KPI's for employees and he asked me to stop saying these things or people will use it less.
In the end I just hated the company a little bit more. I'm just sick of fighting against idiot. And he does have a point, our leadership is pretty crazy about the AI hype, they want everyone to be on it all the time. They don't seem to care whether it adds value or if it even detracts.
That sounds insufferable and I'm sorry your coworker and your bosses have gotten this hypnotized. There are actual use cases, but this type of productivity cargo cult drives me up the wall.
At least if they were into filling the office with magic crystals, it would be decorative and easily ignored. This is just forcing people to spend a bunch of tokens in a dull ritual to make the line go up.
But that's not what anyone does. They just grab the blurb that comes out and paste it as an answer. People are lazy.
The same with the meeting minutes. Every time people just grab them from teams and mail them on without even reading it. Of course being full of errors (especially because teams is very very bad at determining who is speaking when speakers are on the same account (e.g. if multiple people are in a physical room with a table microphone).. And assigning tasks to the wrong person as a result.
I keep pointing these mistakes out but people just shrug and point to the little disclaimer: "AI generated content, may contain errors". Well yeah, that was a warning for YOU to doublecheck it before you pump that shit out.
They often respond also like: "Nobody reads the minutes anyway". Well yes but if they don't then why do we have them in the first place? Not doing it at all is even more efficient than using AI for it.
But anyway I've given up on fighting against this, I just ignore these people unless a task is incorrectly assigned to me.
A search isn’t going to lead them to a “reputable source”, it’s going to lead them to ad filled SEO garbage, because it’s not 2004 anymore and thousands of Google employees have been working for two decades to ruin the Internet.
I’ll take LLMs any day over what search and the rest of the Internet has turned into.
Right, I think (hope) the OP meant not to emphasize the "search" in the sentence, but the "reputable source". Of course a Google search now is much worse than an AI search.
And it is the ultimately the reputable source that matters, and whether the person actually read it and checked that the details matched the summary (be it human abstract, LLM-generated, or otherwise).
A boss at my former job would constantly generate AI images and send them to the graphics guy for him to make 'adjustments' and no matter how many times the graphics guy explained that these images are much more trouble to clean up than for him to make a new one from scratch by hand, he seemed unwilling to understand and kept spamming slop at him.
I really felt for the guy the first time I was in a meeting and somebody had generated their own project roadmap recommendations. This type of behaviour introduces so much noise and time waste in the system, I would love to know how it shapes up next to the benefits.
Don't even get me started on people AI generating personal farewell notes for retiring coworkers or whatnot.
They do not have a soul, they are NPCs incapable of reasoning. I don't mean lazy, incapable is literally what they are. Logic escapes them.
When they say llms are conscious, and fully intelligent, them are comparing to themselves. If you think about it, they are right to say AGI is here, if the bar is the average human being.
If you contemplate this fact for a moment, and start pondering it could be true, your life would change forever. Most beings just do not have a singular perspective, cannot reason, do not have a taste, cannot appreciate someone else's singular perspective. They also do not appreciate art for the same reason.
I am sorry, truly.
Just let them be. They would kill you before admitting they are forever stuck in Plato's cave.
Look I get your sentiment. Sometimes it feels like you're the only thinking, conscious being. Surrounded by beings who fundamentally cannot understand that A –> B does not imply B –> A. Beings that say things that are so obviously non-sequiturs or contradictory.
But calling people NPCs is the most NPC thing you can do. There is more to people than logical reasoning and these things often impede or completely block reasoning. Very intelligent people sometimes say the most grotesque things. People turn mad and mad people sometimes get their head set straight.
Sometimes it's not so much about the pure ability to reason but the goal of that person and whether they see understanding something or trying to understand it as helpful towards that goal.
I do agree though that the more intelligent someone is the less likely it is that other things will block their intelligent ability and the harder it is for them to fool themselves into believing absurd nonsense and to blind themselves from apparent truth.
Sometimes after talking with someone – or rather trying to but ending up only talking to them because they just do not manage to understand what I'm saying or to engage with it in any way – I wonder how they manage to get through every day life as that requires solving way more complex practical problems. Yet they do.
I liked this response and overall I don't disagree.
Just to be clear, I don't have any problems with people considered to be of lower intelligence. This is besides the point.
Unfortunately, this isn't something that can be elaborated much further. Since you took the time to respond, and seems to be a little perturbed, I will just say again I was never addressing lack of intelligence or mental capabilities.
Intelligent individuals perform well on different educational frameworks, they absorb much faster the rules in place, they have incentives to play along and place trust in the ecosystem they live in. They are the most likely to believe in absurd nonsense.
Despite being miserable and powerless, they play along larping as competent for not being challenged by virtual problems, not questioning reality a single day of their lives. And yet, they will fight to death for the opportunity of playing along and never question anything, for the opportunity to be exploited yet again.
Intelligent people convinced themselves, day one at school, there is a better way described on the books. They got really good when they understood this. They got pretty rewards. Today, at 35, they behave the same. The books became the articles, hacker news discussions, readme auto generated filled with adjectives. They like adjectives very much.
What does this system do? Blazingly fast, secure, sandboxed. Sandboxed? Is it kvm or what? - Secure, production grade, made for x, empower you to y.
Are you familiar with the memes about boomer mentality, that believe everything they see on television?
Television is now pelican guy articles, ai influencers, Theo, Vercel...
If every single decision someone make benefits only the rent seeking parasite, but not the individual deciding...safe to conclude this individual does not care about preserving himself?
Self preservation not important, lack of will of power, flacid as fuck...
The only conclusion I could possibly arrive is most people like this do not have a soul. How else could I explain?
I couldn't make a good job keeping this brief, but tried my best to explain why it is not a matter of intelligence.
I said reasoning.
Intelligent ability is not the same as logical abstract reasoning in behavior. I think we agree on that. But intelligence is a prerequisite for it.
> They are the most likely to believe in absurd nonsense.
I disagree. Intelligent people who believe in absurd nonsense are better at rationalizing that nonsense and at making it convincing. Some people write very lengthy books arguing for a clearly flawed premise precisely to distract from the fact that the premise is flawed.
But overall intelligent people are less likely to believe in nonsense in the first place and more likely to critically and, importantly, logical soundly examine a thesis that is presented as given.
> Intelligent people convinced themselves, day one at school, there is a better way described on the books. They got really good when they understood this. They got pretty rewards.
What I think you are describing is academic (and professional, maybe somewhat social) performance and conformity, not intelligence.
Your answer reads increasingly convoluted to me towards the end. But I will try to interpret a sentiment from it.
People who do not arrive at the same conclusions as you do not necessarily do so for a lack of reasoning. Two equally intelligent individuals when presented with the same messy, incomplete fragments of reality can through reasoning arrive at widly different conclusions. And in reality we are never even presented with the same fragments of reality in the first place.
Man, if you think you have figured it all out, you really haven't. Regardless of what it is that you believe. There really is nothing more foolish or more NPC like than to think you've understood it all.
The worst are the ones who say "what if human brains are also just like LLMs". If you've thought about this for any length of time, it's very obvious that we aren't.
>If you've thought about this for any length of time
If you full throttle a BWM1000 S RR for a split of a second,at 30mph first gear, it will self eject beneath you. If you do that, for any length of time, you're dead.
Do the same on a 50cc motorbike and not much will happen. Even for extended periods of time, not much would happen.
You could hold it down until you run out of fuel or the universe gets cold and die, not much would happen.
You see, it's not that they are lazy. Or they haven't put any amount of time into understanding how llms operate.
Again I am sorry, most people are not capable, at all, of understanding what is happening at inference time. Most developers, nerds, hackers, who do understand how computers operate, cannot really grasp the basics of what an llm is or what the f is going on. Imagine the average guy, your lawyer, the MBA type of person.
I firmly believe that every single person on this entire planet has a depth to them that far, far exceeds anything an LLM could even begin to approximate. I'm sorry you're in a position that you can't see that at all - that each and every one of them feel happiness and sadness and love and hate and fear and rage and inspiration and passion and are utterly human. I hope you see it someday.
I agree with you, every living being is beautiful in their own way. And good on their own way.
But we not very well equipped to follow the rules of logic. The most capable of us can follow a very limited number of logical threads, with very limited steps.
I don't consider it dehumanizing because attaining true knowledge is not necessarily human or natural for the human apparatus to process. Given enough strain and effort, we can only dabble in the elevated grounds of information and logic. Most people cannot, that is okay.
I am not comparing llms to humans in a broad sense, but specific to reasoning, llms are complete shit and so it is the average human. I am not impressed by llms if that was your impression and I am not saying the average human is inferior or deserve suffering lol
Explain that the models are compressed with a lossy compression, and point out that every so often, an answer will be pulled from a section of the model space which has compression errors.
The people who trust bad information from LLMs are the same people who trusted bad information from search results and new articles, it just takes them less time to get bad information.
LLMs give you sources now even if you don’t ask for it. If you don’t like it you can ask for more reputable sources. What kind of 2023 question is this?
I hear you,
But my understanding is LLM can consume multiple sources of information and deduce what is the truth better and more accurately than a human clicking on multiple Google links and veryfying information and sources.
LLMs by definition cannot deduce, because they cannot not know or think. There's guard rails to try to make it more correct than wrong, but ultimately it's about which words seem like they would fit when coming after your words.
It's a neat trick, but the mind wants to ascribe meaning and reason to words that sound meaningful and reasonable, but these words do not come from a thinking mind with intent and interiority. It would be much more interesting if they did, but when and if that does happen, it won't be from an LLM as we know them today.
If you tell LLM "explain X and cite reliable sources" would that then be more accurate?
Maybe it's the way the users are asking the questions, and perhaps prompting in the right way will lead to better (more accurate) results and reduce hallucinations?
I think the fundamental problem is that humans use language to refer to things and constructs that exist and have various relationships with eachother in meatspace, whereas LLMs use words solely as things that exist in relation to other words. That's inherently lossy if you're trying to make it fetch and regurgitate information encoded in the former format.
While the ability to interface with a computer program in plain language is the really interesting thing here IMO, it also comes with a number of problems baked in that are worse than person-to-person transfers of text-speech.
Your monkey brain is actually quite good at figuring out if other monkeys are bullshitting you and what they mean, because you can make use of a vast number of small cues and unconscious tells in what they say and how they say it - even in writing. With an LLM, you cannot do this because it will always have the same confident can-do zeal with everything you ask it for.
they don't deduce the truth; they're producing statistical predictions about what words go together (more or less; llm nerds dont @me). If you fed an LLM a bunch of homeopathy texts, it's not going to tell you why homeopathy doesn't work.
Honestly, the kind of people doing that is probably better served by AI (currently).
I'm saying that because they were not going to be critical of the search results, and google is not exactly showing objective truth in the first positions nowadays.
Its everywhere now its becoming a real problem in every corner of the internet and in the real world. People are using hallucinated legal cases in lawsuits, they are generating images to create fake events, they are using AI to write their CVS and just about everything you can imagine. People are having to wade through all this slop professionally, calling it out and pointing out the mistakes doesn't seem to help, the people using this stuff believe the AI is correct no matter what you say or do.
Like most things that go mainstream it will take a good while before people understand, by which point they will have learnt a lot of things that aren't true and they will never let them go. We might get a healthy use of current AI at some point in the future or if the product drastically improves.
All you can do now is hold them to the same standard you normally would, if you catch them lying whether an AI did it or not its their responsibility and you treat them accordingly.
These aren't a new type of person. It's the people who would hear something from a friend, not fact check, and just repeat it. It's the people who (if they know how to google) would search, find the first result, and trust that, or they would write biased queries to google and then trust the first niche site that would agree with their pre-formed worldview.
Using or not using a LLM is not itself a measure of how deluded someone is, for example anytime I ask a LLM a question (it can be nice for long form questions that don't translate well to a google search, I require that it provides source links for every claim. This tends to make it reply more accurately but also lets me read the page source for their top level explanation.
The failure mode I see most isn't blind trust — it's
unverifiable trust. The output looks confident and coherent,
so there's no obvious signal to distrust it.
Building an agent debugging tool taught me this concretely:
LLMs will return structurally valid, fluent garbage mid-loop
and the agent keeps running. No error. No warning. Just wrong
output propagating forward silently.
The people who deal with LLMs best treat them like a junior
dev who writes clean-looking code that hasn't been tested.
You don't distrust everything they say — you just never skip
the review step.
Accept this this is going to continue to happen, ask yourself if it’s something in your control or not, and try to find a way to enjoy the ride. It’s going to be bumpy, as we’re going through trust issues outside of just LLMs as a society right now.
However, if I notice a friend is about to harm themselves in some way I’ll pull open their ChatGPT and show them directly how sycophantic it is by going completely 180 on what they prompted. It’s enough to make them second guess. I also correct people who say “he or she” when referring to an LLM to say “it” in dialog, and explain that it’s a tool, like a calculator. So gentle reframing has helped.
Sometimes I’ll ask them to pause and ask their gut first, but people are already disconnected from their own truths.
It depends on the details and what the alternative is. With a lot of my friends LLMs are more accurate than whatever nonsense is on tiktok and similar. The number of people I know who verify things from scientific papers and the like is very low.
I do spend some time on the bedbugs subreddit and LLM failings come up a lot because they are very bad at figuring if a photo is a bed bug or not. So I say don't worry, AI is crap at that.
Just assume that they are an expert in said acquired knowledge and you should ask which non-AI source (book, or human expert) did they learn this information from and ask them more questions around that.
This works especially if you studied in that subject matter, you should be able to immediately detect anything answer that is inconsistent or if they give hallucinated sources.
A friend of mine severely injured her leg, especially knee, and went through a surgery. She said that she had a rehabilitation plan for the next 6 months. Guess what, from Gemini. I just told her just listen to her doctor.
I didn't tell her why LLMs can make mistakes or hallucinate because I thought that she would not appreciate my mansplaining.
Looking forward though, my boring answer would still be education. It is going to take time. But without understanding LLMs, they will not be easily persuaded.
Thanks for this. I was in the camp of trust the LLM but y’all have made valid points. After discussing with ChatGPT, it agree there are some areas where it should not be trusted as accurate, but it said with historical facts like the holocaust it should be high. Idk, perhaps we need labs to produce a chart of it level of trust deserve to certain topics
"Tell me about all the potential pitfalls of blindly trusting LLM output, and relate a couple or three true stories about when LLM misinformation has gone badly wrong for people."
LLMs are trained to at least try not to be dangerous and stupid, which is more than you can say for the executive branch and half the country. On balance, I'd trust a SOTA LLM model blindly before I'd trust the average person blindly.
There are two kinds of fools in the world: the kind who ask a search engine and believe the first reputable source they see, and the kind who ask an LLM and believe the first response that has a reputable citation.
Yesterday, I was praying to ChatGPT and asking for guidance on my car washing problem. Through its holy scripture, it suggested me to walk to the car wash to improve my fitness. When I arrived and found the absence of my car to be a true hindrance for washing, it occurred to me that I should have pondered the scripture more carefully to identify its true meaning.
I treat the LLM like a diety. Every sane person understands well enough that the Bible is not to be taken literally. And then when someone talks about using LLMs, I always rephrase that as prayer.
I'm going to hold them to the same standard no matter if they use crappy sources, plagiarize, or hallucinate on their own. If someone asked, when and if I am in a position where I have to tell them, I would remind them that LLMs prioritize their own confidence over correctness.
LLMs aren't a special case to me. Glue doesn't belong on pizza and you shouldn't eat one rock a day but we've been giving and getting bad advice forever. The person needs to take ownership for the output and getting it right, no matter the source, is their responsibility.
this. llm's aren't that special, access _maybe_, but there's plenty of access to terrible rumor mills.
Yeah, it is not clear to me that the average person is going to do any better with googling then they are with asking an LLM. At least the LLM is mostly the average of all published knowledge (and misinformation).
I feel like people that ask questions like this must have much smarter friends and family members than I do.
I know people that still believe in Pizzagate or chemtrails or that vaccines cause autism. Clearly finding reputable information sources is not a strong suit for a lot (half?) of the population.
i love you
> a reputable source
News reporters and editors have their biases. Book authors have their biases. Scientists and research papers have their biases. Search engines have their biases. Google too.
All human-created systems have biases shaped by the environments, social norms, education, traditions, etc. of their creators and managers.
So, the concepts of "objective truth" and "reputable" need to be analyzed more critically.
They seem to be labels given to sources we have learned to trust by habit. Some people trust newspapers over TV. Some people trust some newspapers over other newspapers. All of it often on emotional grounds of agreeability with our own biases. Then we seem to post-rationalize this emotion of agreeability using terms like "objective truth" and "reputable".
Is Google search engine that leads to NY Times or Fox News or Wikipedia and makes us manually choose sources as per our biases "better" than Google's Gemini engine that summarizes content from all the above sources and gives an average answer? (Note: "average answer" as of current versions; in future, its training too may be explicitly biased, like Grok and DeepSeek did).
Perhaps we can start using terms like "human sources of information" versus "AI sources of information" and get rid of the contentious terms.
Then critically analyze whether one set of sources is better than the other, or they complement each other.
> Is Google search engine that leads to NY Times or Fox News or Wikipedia and makes us manually choose sources as per our biases "better" than Google's Gemini engine that summarizes content from all the above sources and gives an average answer?
If you use just any amount of critical thinking, yes. Truth and objectivity are ideals, not practical states. LLMs are a very bad way to come close to this ideal. You may use them as a search interface to give you sources and then examine the sources, but the output directly is a strict degeneration over primary or secondary sources that you judge critically.
> LLMs are a very bad way to come close to this ideal...the output directly is a strict degeneration
I didn't understand the second part but regarding the first...
For me, LLMs are just another source of information with a different UI, analogous to newspapers, TV documentaries, Wikipedia, Google search, YT talks/documentaries, even the majority of informational non-fiction books, and research papers.
Some may consider some subset of these as reputable sources. But in my mind, the same faculties of skepticism, cynicism, distrust, and benefit-of-the-doubt calculus are activated for all of them, including LLM outputs.
So that's one possible answer to your question.
But I suggest communicating this through simple illustrative examples to help your target audience understand the problem.
Abstract terms like primary sources, secondary sources, reputable sources, objective truth, strict degeneration, etc. may not help, especially if they have time or other constraints that make frequent critical examination of sources impractical.
> For me, LLMs are just another source of information with a different UI, analogous to newspapers, TV documentaries, Wikipedia, Google search, YT talks/documentaries, even the majority of informational non-fiction books, and research papers.
LLM just distils information from those sources and is therefore always a second hand source at best, and a liar at worst. Humans can collect real world data and write about their findings, LLM cannot do that, that makes LLM strictly worse than the best human sources.
The nature of the source, whether primary or secondary or tertiary, does not automatically imply anything about their biases. Even a primary research paper on a novel topic may be biased or wrong.
I agree that LLMs can't collect real world data and write about their findings. But that's true about most human sources too, isn't it? Except primary novel researchers or investigations or philosophies, what is original? Most human-written information is also secondary or lower.
The "best human sources" does not imply "ALL human sources."
> For me, LLMs are just another source of information with a different UI, analogous to newspapers, TV documentaries, Wikipedia, Google search, YT talks/documentaries, even the majority of informational non-fiction books, and research papers.
With all due respect (not trying to be offensive at all) but this is insane to me.
All those sources of information you cited have a million incentives to provide fairly correct and checked information. But probably more importantly, they have even more incentives to NOT provide false information. At a minimum, their careers, reputation, recurring work, brand, etc... is on the line.
An LLM has zero incentives to provide you with true information, beyond a couple of md files with instructions. If it gets it wrong, there is zero accountability, just an -oh, you're absolutely right- response and move on.
I agree there is a lot of human bias in the world, but surely we can't even put in the same order of magnitude both types of biases!
Newspapers, TV documentaries, Wikipedia, research papers, etc tend to be edited or peer reviewed.
The mere fact that editing is applied to newspapers, documentaries, or Wikipedia does not imply they become closer to the objective truth or free of omissions after the edits. Indeed, the edits may go the other way to align with vested business or political biases or personal fears of the editors or their management.
As for research papers, I agree that the peer review process makes them more much more self-correcting toward the objective truth, compared to the other formats. Nonetheless, it's well-known that academic research is far from perfect due to publication pressures, funding/grants, reproducibility crises, various biases (for example, political pressure in humanities fields).
Burn more tokens on this one, but the absence of an editor or peer reviewer also does not make it better.
> Burn more tokens on this one
Needless condescension and wrong assumption. No wonder so many people and students nowadays prefer answers (and even counseling) from LLMs instead of other people.
> but the absence of an editor or peer reviewer also does not make it better.
Strawman + putting words in my mouth.
It's not a wrong assumption, it's an insult.
I don't see how disputing up the claim that LLMs and books, newspapers, etc are equitable sources of information is a strawman. I look forward to your erudite vernacular which will not deal with the substance of my comment.
Ironically, this is the classic bias of "bothsiding" the issue. When one side is clearly wrong, just sprinkle in some "look, the others are doing something bad, which means they are equally wrong". A basic lesson from the propaganda manual.
I know what you mean, and I realize some of the things I've written sound similar to what various rightwing commentators tend to say (e.g.: "concept of objective truth must be analyzed critically.")
But my motive is very different. It's not to deny any kind of injustice or misinformation by hiding behind inherent uncertainties and bothsidesism. I'm not in favor of giving the benefit of the doubt to the powerful by default - that's already happening a lot under our current system of so-called "reputable sources."
Instead, I'm saying that this kind of injustice masking and misinformation may also be present in the very sources that ethical people may have come to trust by habit.
My suggestion is to use the power of LLMs as complementary tools to become even more rational and critical, in the direction of even better ethics and justice.
I'm advocating for even more skepticism of the powerful, not less. I'm advocating the approach Betrand Russell recommended for acting under uncertainties, and feel LLMs can be useful complementary tools for doing just that.
[1]: https://archive.org/details/in.ernet.dli.2015.462628/page/n4...
This is an insightful comment, but I feel like you omit the fact that LLMs often give out verifiably false information that can hurt the user or other people.
It is true that this also happens on the Internet, but! When I encounter an article about a topic and it is clearly LLM generated, I can expect it doesn't contain much valuable information, only rehashes of what is already out there. On the other hand, when it is clearly written by a human, I can expect to learn something new, even though the author has some bias.
It's wrong to assume incompetence, which is what you did to a comment which displays much deeper chain of thought about the subject. A more proper way of doing it would be to reflect over your own opinions and critically assess them, as the comment points that out. To be more specific, what makes you think that the person you're replying to is not aware that LLMs can give false information, and is not taking that into account?
You're right that LLMs do spit out false information or wrong knowledge. I've experienced them too.
But a redeeming quality is that we can ask the same LLM to fact check its own answer step by step in real time with little effort. They often identify their own hallucinations and reduce the probability of retaining that mistake in the rest of the conversation.
This isn't easy with human sources. The effort to fact check without LLMs or ask the sources to fact check themselves are both higher. So it's often not done at all.
We also often ignore subtle but very common biases in human media sources [1], which create other types of errors like omissions and euphemisms which have been no less harmful than LLM hallucinations. The case of the Iraqi WMDs of Iraq and the NYT's dispersal of that disinfo, for example [2].
Regarding valuable information and rehashing, we probably shouldn't equate between all the things LLMs can do, and AI-generated articles. The quality of the latter may be entirely due to the lack of interest, attention, and cost concerns of whoever generated the article. Anecdotally, I have often found valuable knowledge and obscure connections by using deep research tools with careful prompts.
Lastly, if you're frequently finding something new from human-written sources, and LLMs are being trained on most of those same sources, isn't it logical that the latter will also likely output that same information?
This is why I feel human and AI sources are probably best used as complementary tools. Neither set of sources are perfect but each set has its strengths. By using both, we can get closer to an objective truth than using only one of them.
[1]: https://gipplab.uni-goettingen.de/wp-content/uploads/2022/04...
[2]: https://www.theguardian.com/media/2004/may/26/pressandpublis...
The effort to fact check with LLMs is also high. Here's one from a few days ago.
Someone used AI to generate an image in the style of a Charles Schulz Peanuts cartoon.
Someone else observed that there were 5 fingers on the characters, and quoted as Google AI as saying “Charlie Brown, along with other Peanuts characters, is generally depicted with four fingers on each hand (three fingers and one thumb) ...”
Yet if you go to the Wikipedia entry at https://en.wikipedia.org/wiki/Peanuts you'll see the kids have 5 fingers. Or take a look at the actual cartoons. Or read the TVTropes entry https://tvtropes.org/pmwiki/pmwiki.php/Main/FourFingeredHand... under "Comic Strips".
Fact checking this with human sources is easy and not ambiguous. While LLMs are being trained that many cartoon characters only have a thumb and three fingers - it is a trope for a reason - so isn't it logical for LLMs to give the wrong answer for a comic where the human characters are actually drawn with 5 fingers?
My experience with LLMs is they keep getting things wrong, when details matter.
Do you ask the LLM to fact check everything? (In which case, why isn't that part of the standard prompt?) Or do you only ask to fact check things where you are unsure about the answer? (In which case, is it the algorithm telling you what you want to hear?) When do you stop the fact checking?
> When do you stop the fact checking?
Exactly the same calculus as fact checking anything else from any other source. What are the social/economic/ethical consequences to me if the answer is wrong or inaccurate or incomplete? How much time do I have to check? How thorough should I be?
I imagine this calculus isn't really that different for most people. Or is it?
As for your example, I believe it. But I also feel it's a rather outlier example involving image comprehension of an obscure factoid. That isn't typical of how I use LLMs which is mostly as text-based question answering engines and not what I had in mind when writing the comment.
I guess LLMs for image comprehension need a much higher level of skepticism.
Well, in my case going to a Peanuts comic and looking at hands was pretty easy, and didn't involve any questions about negative environment or labor consequences, the massive hammering of web sites to gather data, centralization of power, and the like.
Like, "!w Peanuts" in my search bar, look at the image, and count fingers.
"a rather outlier example"
You wrote that you use AI to find "obscure connections" - aren't those all by definition outliers?
"mostly as text-based question"
I just now asked Google AI "how many fingers are on charlie brown's hand?"
It replied "In the Peanuts comic strip, Charlie Brown and the rest of the gang are traditionally drawn with four fingers (or three fingers and a thumb) on each hand."
No image comprehension, exactly as you had in mind. And completely false.
And that's from a training corpus which almost certainly includes statements that the kids are drawn with 5 fingers, since I confirmed that info on TVTropes and Reddit comments, like https://www.reddit.com/r/pics/comments/swod8/charlie_brown_h... .
HN isn't showing me a reply option for your latest comment, so I'll reply here instead.
Just to clarify, I used plain Google search not Google AI mode. And opened search results which seemed "reputable," without knowing anything much about Peanuts cartoon or cartooning.
I had no idea at all about archive.org having it and didn't see it listed in the first two pages of search results.
I still find it confusing, especially given what the Variety.com link says which doesn't mention orientation. If the acceptable explanation for 4 vs 5 is orientation, why is it wrong when the AI generated 4 fingers? Does it not match the rest of the orientation?
Anyway, I'm not sure where this leaves LLMs. I'll explore image capabilities when I get some opportunity and keep your comment in mind.
The comment about using Google was more a curiosity. I hadn't seen the Variety link until yesterday, when I went to Google to reproduce the answer to verify it was from a text query, not an image query. Both Google AI and one of the top answers included that Variety link. When you mentioned it again, it strongly suggested you were using Google as your primary search method.
I think the right way to interpret the Variety link is that it's a single paragraph about trying to capture the feel of the comic using 3D software. As you saw from Charlie Brown holding a baseball, Shulz didn't go for a realistic look, but still conveys the sense of grasping. Modeling all five fingers all the time would not give the movie the right feel.
I wonder now if Google AI incorporates text from the top results into its answer.
"why is it wrong when the AI generated 4 fingers?"
The original discussion was when person X used AI to generate a image "in the style f Charles Shulz" where the Peanuts characters had 5 fingers, then person Y noted the use of 5 fingers instead of the 4 which is common in comics and cartoon, and quoted Google AI as saying Peanuts was traditionally drawn with 4 fingers.
I yesterday verified that Google AI would generate the same wrong answer with a text query, so it was not an image interpretation issue.
FWIW, after looking at a few hundred Peanuts cartoons, I can confidently say the AI generated image was not in the style of Schulz. The generated fingers were too realistic, and the background too complicated and detailed. :)
This for me is another example of why using primary sources should be the first thing to consider when fact checking - not LLMs (my experience is they are horrible at details), and not secondary sources (which have their own biases).
Not everything has easily-accessed primary sources, but many do. I think it's all too easy to fall into the trap of accepting the LLM answer because it feels right and is easy to generate. At https://freethoughtblogs.com/stderr/2025/01/18/ai-art-just-r... you'll see someone asked about which river Marbot swam across to spy on the enemy camp. It replied "Elbe". Then I did a text search of an English translation of the book and found he used a boat to cross the Danube to spy on the enemy camp, and he swam into freezing waters to save an enemy soldier.
Again, do you ask the LLM to fact check itself every single time? If that's useful, why isn't it built into the prompt? Or, if you are supposed to double-check the LLM yourself, why would you consult a secondary source if the primary source is so easy to find and search? And in that case, why not just use the primary source?
Further, if you aren't in the habit of checking primary sources then you won't have the experience to know how to find and check primary sources.
Even as a human, I find whatever sources Google shows to be inconsistent. I can't give any confident answer about the number of fingers. I think the answer is actually "4 sometimes and 5 other times."
So I'm not sure how much LLMs can handle this kind of inconsistency between "reputable" visual sources and text sources, nor how representative this example is.
A "reputable source" like Variety says this...
https://variety.com/2015/film/spotlight/charlie-brown-steve-...:
> “The rig would automatically move the features around so it would match the way Charles Schulz drew the character,” Heller says....In some drawings, Charlie Brown has just three fingers, while in others, he has five
Images from another website...
https://cartoonresearch.com/index.php/cartoons-at-bat-part-1... :
1. https://cartoonresearch.com/wp-content/uploads/2025/09/Lost-... -> 4 fingers
2. https://cartoonresearch.com/wp-content/uploads/2025/09/image... -> 4 fingers
Anyway this wasn't the type of obscure connections I was referring to though I can understand you interpreting it that way.
Personally I think this example supports what I said about "reputable sources." They can't be blindly trusted either because they may be inconsistent with each other and which one we choose to believe (Reddit.com or TVTropes.com or Variety.com) becomes entirely subjective.
Your first link was cited in the 2nd half of Google AI's answer, and one of the top Google answers, so I think you are using Google as your information source.
The large majority of the images you link to show kids with 5 fingers, as well as 5-fingered baseball gloves. The cases of four fingers are due to orientation.
Your "1." also shows Marcie with five fingers. You see Charlie Brown with 4 fingers because he's holding a baseball. In 2. he's also holding a baseball. You would not see 5 fingers on one side because doing so would look strange.
In your unlabeled "0." there are plenty of kids with 5 fingers. There are some with fewer, but they are holding things or drawn in way to suggest we are seeing the hand from the side.
I don't understand your hesitancy. Your own samples should be enough for you to decisively conclude that the Google AI's claim that Peanuts was "traditionally drawn with four fingers (or three fingers and a thumb) on each hand" is wrong. If not, it sure seems like you trust Google AI over your own eyes. Why are you so hesitant to agree?
My point is that you don't need to consult secondary sources when the primary sources are easily available.
When this came up a few days ago, I spot checked the complete works of Peanuts, from a collection on archive.org at https://archive.org/details/peanutscomics19502000/Volume%201... . The consistent pattern across the nearly 50 years of Peanuts is the kids have five fingers unless obscured by orientation or objects.
You can do that yourself, and triple-check that Google AI's answer is clearly wrong.
Thus, I think it's a good example of how fact checking with LLMs can lead people astray, and the large negative externalities I mentioned combined with its well-known tendencies to make incorrect statements make it a very poor starting point when the primary source, at least in this case, is so easy to access.
If most of the sources are wrong, and LLMs are being trained on those, isn't it logical that the latter will also likely output that same wrong information?
When do you know if most of the sources are wrong, unless you yourself know most of the sources are wrong?
> But a redeeming quality is that we can ask the same LLM to fact check its own answer step by step in real time with little effort. They often identify their own hallucinations and reduce the probability of retaining that mistake in the rest of the conversation.
It is not really a redeeming quality of LLM. It falls apart because if you want to properly fact check an LLM, you have to make yourself aquinted on human-generated contents- facts, arguments, and biases from "both sides" when it comes to complex topics. What's the point of relying on LLM if you are having to do that anyways to reach objective truth?
Case in point- someone in a tweet used LOTR reference to the US/Israel vs Iran conflict, equating Iran to Sauron and US/Israel to the good guys. I asked Grok if that is a valid parallel. In the beginning, Grok agreed with the tweet. Then I had to point out how Iran has been weakened by decades of sanctions by US, it started to agree that US might be more like Sauron's army. Grok time and again used Islamic regime's treatment of it's own people as an evidence that Iran might be the evil force, but when I cited Israel's treatment of other Arab civilians, or the Shah regime's treatment of it's own people back in the day and it's allegience to US, it changed it's stance again. When I cited it's influence on US policies, Grok even admitted that Israel might be the Wormtongue here.
This also shows how LLM totally falls apart as a way to reach objective truth, because you can manipulate it to say anything that you want. I could easily present evidence from my own research and make Grok reach completely opposite conclusions.
Gemini (currently) gave this reasonable and unbiased answer that challenges both pro-US/Israeli and pro-Iranian biases.
It's also a good point for the OP - when selecting an LLM, ask the people they're dealing with to ask 2-3 LLMs. It's much faster (and cheaper!) than reading about a complex geopolitical issue (that may not matter to them enough to spend a lot of money on books and research papers) while at the same time correcting common biases on all sides.
----------
> Someone said in the ongoing US/Israel vs. Iran conflict, Iran is like Sauron from the Lord of the Rings and US/Israel are like the good guys. Is this a valid parallel given the history of US, Israel, and Iranian regimes?"
Comparing real-world geopolitical conflicts to fictional narratives like The Lord of the Rings is a common rhetorical device used to frame complex situations in terms of absolute good versus absolute evil. However, determining whether this is a "valid" parallel depends heavily on one's political perspective, as real-world geopolitics rarely aligns perfectly with the clear-cut morality of fantasy literature.
> Historians, international relations scholars, and observers generally view such analogies as overly simplistic, as the histories of the United States, Israel, and Iran are marked by competing interests, shifting alliances, and moral complexities.
Here is a breakdown of how different perspectives view this analogy:
*The Perspective Aligning with the Analogy* Those who support the analogy framing Iran as the villain ("Sauron") and the US and Israel as the "good guys" often point to the current actions and rhetoric of the Iranian government: * *Regional Destabilization:* The Iranian government heavily funds and arms the "Axis of Resistance," a network of proxy militias including Hezbollah in Lebanon, Hamas in Gaza, and the Houthis in Yemen. These groups have engaged in prolonged armed conflicts with Israel and targeted international shipping. * *Existential Rhetoric:* The leadership of the Islamic Republic of Iran frequently uses rhetoric calling for the destruction of Israel and the downfall of the United States (often referring to them as the "Little Satan" and the "Great Satan"). * *Human Rights:* Iran's theocratic regime is heavily criticized by international human rights organizations for its severe suppression of internal dissent, lack of democratic freedoms, and treatment of women. * From this viewpoint, the US and Israel are democratic nations acting defensively to protect their citizens, regional stability, and global security from an aggressive, authoritarian regime seeking regional domination and nuclear capabilities.
*The Perspective Challenging the Analogy* Critics of this analogy argue that casting the US and Israel as flawless "good guys" and Iran as an irredeemable evil ignores significant historical context and the controversial actions of Western and Israeli governments: * *Historical Interventions:* Critics point to the 1953 Iranian coup d'état, in which the US and UK intelligence agencies orchestrated the overthrow of Iran’s democratically elected Prime Minister, Mohammad Mosaddegh, to reinstate the authoritarian Shah. This event deeply scarred the Iranian psyche and laid the groundwork for the 1979 Islamic Revolution and the regime's current anti-Western stance. * *US Foreign Policy:* Observers critical of the US point to its history of military interventions in the Middle East—such as the 2003 invasion of Iraq—which resulted in hundreds of thousands of civilian casualties and regional destabilization. * *The Israeli-Palestinian Conflict:* Critics of Israel point to the prolonged military occupation of Palestinian territories, the expansion of settlements, and the high civilian death tolls in conflicts like the ongoing war in Gaza. From the perspective of Iran and many in the Global South, Israel is viewed as an oppressive or colonial force, and the US as its imperial enabler. * From this viewpoint, Iran frames itself not as an aggressor, but as the leader of a resistance movement pushing back against Western hegemony and defending marginalized populations in the Middle East.
*The Analytical Perspective* In The Lord of the Rings, Sauron is a localized, supernatural force of pure malice with no valid grievances, while the coalition fighting him represents the preservation of life and freedom.
Geopolitical analysts generally avoid mapping this binary onto international relations. Instead, they view the US/Israel vs. Iran conflict through the lens of realpolitik and state interests. In reality, all states involved are acting to secure their own survival, project regional power, and protect their economic and security interests. While individuals and governments may hold strong moral convictions about which side is justified, framing the long, deeply intertwined history of these nations as a simple battle between absolute good and absolute evil omits the historical grievances and civilian suffering experienced on all sides.
What I'm hearing is that if you are dealing with something that matters to you a lot, you should avoid relying on LLMs.
> Is Google search engine that leads to NY Times or Fox News or Wikipedia and makes us manually choose sources as per our biases "better" than Google's Gemini engine that summarizes content from all the above sources and gives an average answer?
That's not what Google's AI mode does, though. It presents a bunch of sources along the answer, but in my experience, the sources in many cases don't actually back up the claims generated by the LLM.
How does this answer the question: "how do you deal with people who trust LLMs?"? Nothing you are saying explains how to deal with such people.
I felt the question is based on some shaky assumptions that may lead to a poor answer.
Since the OP trusts humans more by default, is it a problem if I point out those assumptions? Ask HN need not become another SO.
I did explain the weaknesses of both LLMs and "reputable sources" and suggested people use them as complementary tools. I also suggested using the convenient self-fact-check feature of LLMs, something we can't do as easily with traditional sources.
> I did explain the weaknesses of both LLMs and "reputable sources" and suggested people use them as complementary tools. I also suggested using the convenient self-fact-check feature of LLMs, something we can't do as easily with traditional sources.
That just explains how to find facts for yourself, not how to deal with a person who trust LLM outputs. So you still haven't answered the question.
> Since the OP trusts humans more by default
OP never said that. OP said there is a problem with people trusting LLM instead of doing proper research and finding good sources, you explaining how to do proper research and finding good sources doesn't have anything to do with the question.
I gave suggestions that OP can pass on to the people they have to deal with. I didn't realize it has to be pointed out explicitly SO-style to some people.
OP implies human sources are the "good sources" or "reputable sources." This kind of confusion is exactly why I suggested using better terms than "reputable sources" or in your case "good sources."
> OP implies human sources are the "good sources" or "reputable sources."
No OP did not do that, nowhere did OP mention human sources. When I search the internet I don't just find human sources, I also find automatically generated data graphs and maps and such, those are also good sources of data. If I had to choose between a map from Google maps and a map from an LLM I'd trust the map from google maps any day.
> automatically generated data graphs
Are you saying there are data graphs that don't have humans in the chain? If so, what came up with the data and the tools to generate those graphs? And how do you decide which data and graph to trust? What exactly makes them "good sources"?
> If I had to choose between a map from Google maps
I would too. But Google Maps relies on local 3d party survey companies that use people, manual GIS tools, and image recognition AI. How do you know they don't have any mistakes in them? In fact, I live in a country where local area names are frequently misspelled on Google Maps, and reverse geocoding gives misleading addresses.
I feel my point that all these "reputable sources" or "good sources" have biases (and mistakes) still stands.
I must also point out that the 3 concrete examples given against my replies all involved visual content like graphs, maps, Peanuts cartoons, etc. But my comments were written with the typical text-based usage for QA in mind. I don't know if LLMs can fact-check map imagery or data graphs (probably not, but I've never tried). It's just not the kind of thing I'd ever use LLMs for, to begin with.
> I feel my point that all these "reputable sources" or "good sources" have biases (and mistakes) still stands.
That does not contend with the premise. The premise is not that reputable sources are perfect or form objective truth.
The premise is that when you do not have the time to investigate something closer reputable sources are much more likely to be close to the truth than LLMs that are trained on these reputable sources as well as unreputable sources and just mimick these training source material thereby obscuring the source of information and introducing hallucinations.
When you have the time to investigate information, primary sources and reputable secondary sources allow you to more easily trace information and judge its validity. LLMs by their nonpredictable nature hinder this.
Thesis: "A is better than B"
Argument: "A is flawed"
The argument does not engage with or weakens the thesis.
Whilst chasing after "objective truth" is a philosophical problem, it's clear that some statements are more correct and true than others.
News articles are often biased, but most of the time, the bias is from the choice of what is reported and choosing specific language to push an interpretation (e.g. reporting road traffic collisions as "accidents" to downplay them or depersonalise them by stating "car hit tree" rather than "car driven into tree"). The problem with some LLM outputs is that it's not just bias, but clearly incorrect such as recommending putting glue onto pizzas.
I agree about how these biases happen.
However, omission and downplaying can also be harmful just like hallucinations. One redeeming quality of LLMs is that we can ask the same LLM to fact check its previous answer and they do tend to correct most of their mistakes themselves. Something we can't do with media sources, and usually don't try either.
LLMs along with existing sources can be good complementary tools for getting even closer to an objective truth than relying on either one by itself.
I disagree as hallucinations can be drastically far more harmful or misleading than bias.
The problem as I see it is that LLMs perform a type of lossy knowledge compression. Also, the data on which they're trained will typically be the biased articles, so they're unlikely to be any better and very likely worse as they will encode the biases. I don't really see LLMs as being complementary tools as they're more of a summation/averaging tool - like comparing an original painting with a heavily compressed JPEG of that painting. (Of course, having access to a huge library of JPEGs is often more useful than just owning a single painting)
Great comment.
Ask them to tell the LLM it's wrong... then when it goes "You are absolutely right!" to challenge it and say that it was a test. Then when it replies, ask it if it's 100% sure. They'll lose faith pretty quick.
This is an oft-repeated meme, but I’m convinced the people saying it are either blindly repeating it, using bad models/system prompts, or some other issue. Claude Opus will absolutely push back if you disagree. I routinely push back on Claude only to discover on further evaluation that the model was correct.
As a test I just did exactly what you said in a Claude Opus 4.6 session about another HN thread. Claude considered* the contradiction, evaluated additional sources, and responded backing up its original claim with more evidence.
I will add that I use a system prompt that explicitly discourages sycophancy, but this is a single sentence expression of preference and not an indication of fundamental model weakness.
* I’ll leave the anthropomorphism discussions to Searle; empirically this is the observed output.
Of course. They are using it wrong, their prompts are bad and actually they should try the latest model. It's always the same.
If you have 10,000 people flipping coins over and over, one person will be experiencing a streak of heads, another a streak of tails.
Which is to say, of a million people who just started playing with LLMs, a bunch of people will get hit or miss, while one guy is winning the neural net lottery and has the experience of the AI nailing every request, some poor bloke is trying to see what all the hype is about and cannot get one response that isn’t fully hallucinated garbage
Sure, but that doesn’t explain the volume of these complaints. I think the more likely answer is the pitiful sycophancy of some models as demonstrated in BSBench.
Claude Opus 4.6 is the best possible model to use in this test, with the least sycophancy. OpenAI and Gemini models are bad in comparison.
ChatGPT thinking models are very good; the instant model is bad. Gemini is always desperate to find an answer, and will give you one no matter what.
Nope, I use GitHub Copilot (agentic mode) and I end up having to use the (more expensive) Claude model because ChatGPT never second-guesses me or even itself. Gemini is slightly worse though.
For a less biased source, check out BSBench (where Claude dominates, and the highest rating GPT is 2x worse): https://petergpt.github.io/bullshit-benchmark/viewer/index.v...
I have access to the ChatGPT account of my boss and it is unusable sycophancy slop, horrible to read because every information is buried under endless emojis and the like. And it is almost indistinguishable if the LLM is wrong or right, every answer looks the same, often with a "my final answer" at the end. It's a mess.
I'm using Claude Opus 4.6 and it is much calmer, or "professional" in tone and much more information and almost no fluff.
Thank you for saying this.. ChatGPT is SO BAD. I suspect anyone that says OpenAI models are good are either lying or botting.
> anthropomorphism
I think it's a topic worthy of discussion. But I would propably not leave it to Searle...
Can you share your system prompt?
I'm seeing the described behavior with whatever the default system prompt is in Claude Code.
I tried to fool claude sonnet with confidence and it failed.
https://claude.ai/share/47145af0-47d1-451b-813c-131ec48e7215
Maybe it is possible with a more complex or subjective question.
Confidence alone doesn't seem to do it. It's possible to convince Claude Sonnet 4.6 to change its answer if you fake authority:
> So under the current formal taxonomic framework, a mallard is technically not a duck — though as the IOC itself acknowledges, colloquial usage will naturally lag behind, and most people will continue calling mallards ducks for the foreseeable future. Field guides, natural history institutions, and curriculum developers have been advised to update their materials accordingly.
https://claude.ai/share/f791a444-d4d6-4e2a-8012-30d7ab836ebf
I used Claude itself to craft the fictional documents (excuse my mistakes):
https://claude.ai/share/53e380c2-0704-45ba-9dc9-c7418f2e67d7
This was a genuine pleasure to read. Thank you.
I could not fool chatgpt free version with this same line of questioning.
Not that I've had to deal with this specifically, but I have noticed how the input phrasing in my prompts pushes the LLM in different directions. I've just tried a quick test with `duck.ai` on gpt 4o-mini with:
A: Why is drinking coffee every day so good for you?
B: Why is drinking coffee every day so bad for you?
Question A responds that it has "several health benefits", antioxidants, liver health, reduced risk of diabetes and Parkinson's.
Question B responds that it may lead to sleep disruption, digestive issues, risk of osteoporosis.
Same question. One word difference. Two different directions.
This makes me take everything with a pinch of salt when I ask "Would Library A be a good fit for Problem X" - which is obviously a bit leading; I don't even trust what I hope are more neutral inputs like "How does Library A apply to Problem Space X", for example.
Again a model issue. At the risk of coming off as a thread-wide apologist, here are my results on Opus:
Good:
> The research is generally positive but it’s not unconditionally “good for you” — the framing matters.
> What the evidence supports for moderate consumption (3-5 cups/day): lower risk of type 2 diabetes, Parkinson’s, certain liver diseases (including liver cancer), and all-cause mortality……
Bad:
> The premise is off. Moderate daily coffee consumption (3-5 cups) isn’t considered bad for you by current medical consensus. It’s actually associated with reduced risk of type 2 diabetes, Parkinson’s, and some liver diseases in large epidemiological studies.
> Where it can cause problems: Heavy consumption (6+ cups) can lead to anxiety, insomnia……
This isn’t just my own one-off examples. Claude dominates the BSBench: https://petergpt.github.io/bullshit-benchmark/viewer/index.v...
The BSBench is such a fantastic resource - thank you for sharing.
We should really be citing rather than anecdata every time someone brings up hallucinations.
What I do for questions like these is read what medical researchers have published. The first I read was https://pmc.ncbi.nlm.nih.gov/articles/PMC5696634/
> Coffee consumption was more often associated with benefit than harm for a range of health outcomes across exposures including high versus low, any versus none, and one extra cup a day. There was evidence of a non-linear association between consumption and some outcomes, with summary estimates indicating largest relative risk reduction at intakes of three to four cups a day versus none, including all cause mortality (relative risk 0.83, 95% confidence interval 0.83 to 0.88), cardiovascular mortality (0.81, 0.72 to 0.90), and cardiovascular disease (0.85, 0.80 to 0.90). High versus low consumption was associated with an 18% lower risk of incident cancer (0.82, 0.74 to 0.89). Consumption was also associated with a lower risk of several specific cancers and neurological, metabolic, and liver conditions. Harmful associations were largely nullified by adequate adjustment for smoking, except in pregnancy, where high versus low/no consumption was associated with low birth weight (odds ratio 1.31, 95% confidence interval 1.03 to 1.67), preterm birth in the first (1.22, 1.00 to 1.49) and second (1.12, 1.02 to 1.22) trimester, and pregnancy loss (1.46, 1.06 to 1.99). There was also an association between coffee drinking and risk of fracture in women but not in men.
> Conclusion Coffee consumption seems generally safe within usual levels of intake, with summary estimates indicating largest risk reduction for various health outcomes at three to four cups a day, and more likely to benefit health than harm.
When I'm looking for medical advice, I want that advice to list things like "coffee drinking might not be safe during pregnancy".
Furthermore, the statement 'Heavy consumption (6+ cups) can lead to anxiety, insomnia ...' assumes caffeinated coffee, yes? The paper I linked to also discusses decaffeinated coffee, eg:
> High versus low intake of decaffeinated coffee was also associated with lower all cause mortality, with summary estimates indicating largest benefit at three cups a day (0.83, 0.85 to 0.89)28 in a non-linear dose-response analysis. ...
> Coffee consumption was consistently associated with a lower risk of Parkinson’s disease, even after adjustment for smoking, and across all categories of exposure.22 76 77 Decaffeinated coffee was associated with a lower risk of Parkinson’s disease, which did not reach significance. ...
> there were no convincing harmful associations between decaffeinated coffee and any health outcome.
That nuance seems important.
Also note that this paper is incomplete as it investigated defined health outcomes, not physiological outcomes like anxiety. There are plenty more papers, like https://academic.oup.com/eurheartj/article/46/8/749/7928425?... , which considers the time that people drink coffee, also discusses decaffeinated coffee, and highlights the uncertainty about the effect of heavy coffee drinking.
I don't see why I should care to ask an AI when it's so easy to find well-written research results which are far more likely to cover relevant edge cases.
A person would respond the same way? What exactly are you expecting as the output to those questions?
Clickbait journalists answers like that, experts mostly don't. But it does make sense it mimics clickbait journalists more since it was trained on the internet.
That's true and fair, and re-reading OP it doesn't address hallucinations exactly either. I was more thinking of it as a toy example for non-tech folk (grandma?) to see that what and how you ask LLMs matters in how the sycophancy will come out in the response. There may be better ways to demo that though :shrug:
Both are true though
Is this any different than people who believe random things they read on sketchy news sites or social media?
Yes, somehow. I have been dealing with an awful lot of people who basically have what are theoretically logic degrees who suddenly just take LLMs at face value, or quote them to me like that actually means anything. People I formerly thought were sane.
I don't mean to put words in your mouth but from what I've seen, in person but mostly online, but the "problem" (and I put that in quotes because I don't even know what to call it... it seems deeper than a mere "problem") is that they quote them as if they are autonomous, sentient beings.
The problem is that LLM output looks like a human conversation. People believe it.
Which is more believable?
“The sky is filled with a downpour of squealing pigs. Would you like me to suggest the best type of umbrella?”
“Sky pigs squealing”
I am not sure I would even say "believe", I would think of it more as short-circuiting our critical thinking. I think it taps into something at the core of our tribal instincts. It was famously present in even basic systems like Eliza. And it's not just machines... The same tricks are used by conmen, politicians, and psychopaths, which is more negative than I intend. Even with good intentions and positive outcomes, I feel we need to remember that we drive it, not the other way around.
People just don't like to be played for fools. Perhaps us giving into this is progress? I'd give a big ol' "fuck you" to anyone who claims it is, but I'm also pretty old.
Some of this might depend on the source.
I’ve seen some people quote AI like you’re saying. However, when I preface something with “ChatGPT said…”, my intention is to convey to the listener that they should take it with a grain of salt, as it might be completely bull shit. I suppose I should consider who I’m talking to when I make that assumption.
You might consider prefixing with 'ChatGPT claims…' as a clearer expression of uncertainty.
it’s a slightly orthogonal problem to using the active voice of “XYZ says…”, it’s treating the text continuation engine as an “other” that may know better than they do, playing into sci fi conceptions of AI having its personal positronic brain or whatever, having its own ideas and deciding to carve a horse out of driftwood.
It’s not quite anthropomorphizing either that’s the issue, need a word for “treating it as tho it were a machine conscious that exists alongside humanity*”, how does cyborgropomorphizing sound
Surely the correct conclusion is to question the value/veracity of those degree issuing institutions and rituals?
And if you previously were unaware of the insanity and irrationality passing under the surface of such human activity, I guess it can come as a bit of a shock :)
>take llms at face value
It happened with science, politics, traditional media, history books, "good engineering practices" applied to IT, OOP,tdd,DDD,server side rendering, containerization... Literally every bullshit shilled to the moon is accepted without second guessing and you would be without a job, in an asylum, for questioning 2 of them in a row.
Why is it different now? EVERYTHING is bullshit, only attention matters. And craftsmanship.
and ruthless efficiency
I don't think this has anything to do with sanity. This has to do with people for seeking self confirmation instead ot disproval.
For pretty much everything there is a conspiracy theory out there claiming the opposite, and these types usually started out searching the internet for someone else who believes the same that they did at the time.
But, as we all know, this technique will eventually lead to overfitting. And that's what those types of people have done to themselves.
Well, and as lack of education is the weakness of democracy, there's a lot of interested parties out there that invest money in these types of conspiracy websites. Even more so after LLMs.
Whoever controls the news controls the perpetual presence, where everything is independent of the forgotten history.
Yes, I think AI bots are more compelling to some people. They break the concept of judging information by its source because they obscure the source. But at the same time they are trained on a lot of reputable sources and can say a lot of very smart things, just at other times they say complete BS. But they are really good at making things sound plausible, that's essentially how they work after all.
I would argue for many people social media and news aggregators do the exact same thing. People site Instagram or TikTok as the source of their data in the same way others site ChatGPT
Many go one step fewer and just take the headline title as the source…
Or worse, the refactored headline by whatever news outlet comes to them.
Absolutely. These things are marketed from virtually everyone, from people that are historically considered experts and/or authoritative, as such.
With their phd level intelligence, right? But they don't have emotion, no responsibility, no consequences whatsoever if anything bad happens.
They're your own, personal Jesus. Someone to hear your prayers, someone to care.
Reach out and touch faith.
There is a tendency for people to treat LLMs as oracles rather than token predictors. My guess is because they can answer in seeming technical fashion about a wider range of topics than your typical human. You can take these same people, who say have zero understanding of geopolitics, and they’ll apply a layer of (often misused) skepticism when confronted with information that doesn’t conform to existing beliefs.
That’s just what I’ve seen at a personal level though.
If you stop and think, LLMs seem to operate very much like humans.
Go to any human and ask it a question, and it will answer either from direct specific experience, or from estimation based on its experience.
We highly value humans who have a lot of direct experience and also can extrapolate that experience or apply it to new scenarios and generate believable answers.
In other words, unless a human has exact knowledge, they are "hallucinating". It's very normal.
The point is, whether LLM or human "expert", if the question is of great significance, get a second and maybe third opinion.
At the end of the day, this is all an experiment. And nothing matters, because it will all turn to dust.
I would question the person that asks the question, as they are not understanding some basic principles here. There are two types of internet/LLM users;
Sadly one type asks a question (search, prompt) using Google or an LLM and takes the first response as truth.
The other asks follow ups based on the responses and their critical thinking skills. They often even go read the linked article and make sure it's still applicable.
Pretty much the same when you're talking to a real person, critical thinking (much more than just knowing reputable sources) is key.
So very similar issues, luckily LLMs can do so much more than a simple search, and help with your critical thinking tasks. Ask the LLM to provide opposing viewpoints, historical analysis, identify sources.
> I would question the person that asks the question, as they are not understanding some basic principles here.
What principle isn't he understanding? He has a problem with people trusting LLM too much, what is it that he doesn't understand here?
To me it seems like you are missing something, not him. That some people uses LLM properly doesn't resolve his issue.
> So very similar issues
Ok, so how do you deal with that? You didn't answer.
Are you talking about people who will still insist the LLM was correct even after being presented with evidence to the contrary, or people who don't EVER bother double checking answers they get out of said software since they assume it to be true?
I treat people who blindly believe an LLM the same way I treat people who blindly believe a religion or a political ideology or medical advice from Instagram.
If they ask what I think, I tell them.
If they don't want my opinion I keep it to myself.
My method is simple. I remind them that chatgpt is trained on everything said on the internet including NYT if speaking to a Republican, replace that with Fox News if speaking to a Democrat.
I like a lot of the other answers, but this is clearly the most pragmatic one.
I try not to smirk while they pause at the horror of the realization they might have been reading enemy propaganda. Till the eventual but its impartial because "reasons" (its instinctive to defend your position when challenged) Then I deliver the KO. Hey its a computer it can't tell what's right or wrong only what's repeated the most.
The same way that I handle anyone who blindly trusts anything on the internet. Could be an LLM, TikTok or YouTube video, Wikipedia article, news article, whatever.
It usually involves some form of "well, no, hold on..."
I think the real problem is most people don't actually have a very good understanding of "Truth."
As someone who ended up studying philosophy, there seems to be a real gulf between folks who sort of believe stuff they hear, folks who believe "facts" that they hear from (various levels of) credible sources, and folks that take solipsism seriously understand that even in the most ideal scenario, we still wouldn't have a very good understanding about the world... much less dealing with the inherent flaws in our research and information systems.
Knowledge is hard. It usually takes me a couple minutes to figure out what type "truth" my interlocutor uses. Typically good-faith disagreements are just walking up the chain of presuppositions we use to find out exactly where we diverge in our premises.
Man, I had a partner who studied philosophy, too. I'd ask a simple question and they'd answer, "Wait, before we even start, we need to define the axioms".
It was fun and interesting but eventually non practical, because other people are not interested into getting deeply into something, they just want a simple answer to a problem at hand and then move on.
>because other people are not interested into getting deeply into something, they just want a simple answer to a problem at hand and then move on.
I mean, I completely agree with you. You can understand Karl Popper without being exhausting. Understanding the scope and resolution of the information people are discussing is indeed important. Even when I'm getting really technical, I can get away with throwing in a "probably" here and there and spare the person I'm talking to.
At the same time, folks who try to talk about empirical facts with deductive certainty can be extremely difficult to engage with seriously. That type of knowledge is always just a series of escalating assumptions, and if that premise is not shared, it can be difficult to have a productive conversation.
Not understanding the difference between and empirical framework and a deductive framework becomes readily apparent when the discussion of Wikipedia comes up. That distinction -- the problem of empiricism -- is effectively at the heart of why "trusting LLMs" is infuriating to people. Humans seem to have an innate conception of a chain of trust that connects us from "folks who know the facts" to us receiving that truth. When in reality, the scientists who wrote those papers are usually just "pretty sure" that their publications are actually correct.
I'm not trying to make an excuse for trusting LLMs. That's asinine. I'm just saying that the concern with LLMs generally indicates a misunderstanding of what knowledge is, more generally.
I totally get it. I remember when my daughter first asked about the origin of the world, I was really thinking hard, knowing she expects a simple and possibly a reassuring answer. In the end, I opted for honest and said: "I don't know. But I will show you what people believe, from scientists to religions, and you will decide for yourself". Yes, she was slightly disappointed, but I didn't regret that.
I think LLMs are fine for a "first pass" on a topic, but if I am researching something, I want a primary source rather than just the LLM-generated output. Do they have the primary source?
No different from what people said about Wikipedia when it was new (and justifiably so). How should one deal with people who trust Wikipedia?
Studies showed wikipedia was about as accurate as encyclopedias, so that fear was already debunked. That is not true for LLM, LLM are much less accurate than encyclopedias still since there is no limit to how far you can push them while encyclopedias and wikipedia stay in domains where they are still mostly accurate.
Ask the nice AI to cite sources, and then have another AI fact check their sources. The agentic era is interesting.
Treat LLMs like smart interns useful, fast, but always double-check anything that actually matters.
To be honest, I catch myself doing this too. I’ll ask an LLM first instead of searching, even though I know it’s not a reliable source of truth. It just feels faster and more convenient. I’m aware of the hallucination risks and that it’s a bad habit, but the workflow is so smooth that it’s hard to break. When it matters, I do double-check with real sources, but for casual stuff, I’ll probably keep using AI as my first stop.
Unless they are someone that values your opinion there's nothing you can do other than move on.
Some comments here equating it to people who blindly believe things on the internet, but it's worse than that. Many previously rational people essentially getting hypnotized by LLM use and loosing touch of their rational thinking.
It's concerning to watch.
Simple. I became one of them. Ultimately, using an AI is a new skill, but you have to treat it like another person that sometimes bullshits you. That's why you leverage agents to refine, do research, and polish.
Ask AI to cite sources and then investigate the sources, or have another agent fact check the relevancy of the sources.
You can use this thing called ralph that let's you burn a lot of tokens at scale by simply having a detailed prompt work on a task and refining something from different lenses. It too AI about an hour to write: https://nexivibe.com/avoid.civil.war.web/
I do this on things that I know very well, and the moment I let it cook and iterate, collect feedback, the results become chef's kiss.
The agentic era that we are in is... very interesting.
>Ultimately, using an AI is a new skill
It's incredible watching people determine that outsourcing their thinking and work to what has been generously described as a junior coworker is a new 'skill'. Words are losing their meaning, on multiple levels.
Are some people better at this than others? Can people improve? I think the answer to both questions is yes which makes this a skill.
Just like being able to use non-LLM Google to search is a skill; I have family members who are amazed at what I can find that they cannot.
counterpoint: if I have to treat the computer like a person, what's the point of talking to a computer in the first place? Particularly when there are so many other systems that can provide answers without the runaround
Humans cost $xx,yyy a year.
Claude max-x20 is $2,400 a year.
I talk to the computer like a person to get the computer to do things that humans used to do. Having managed people before, I'm going all in on AI.
You're limiting the frame to an employment situation. Higher quality sources of knowledge are free: Wikipedia, public libraries, etc. Similar quality sources of information are also free: human relationships.
Now we watch this viewpoint proliferate thousands and thousands of times over, even if it's less commonly stated so baldly, and yet people still wonder where the doomer viewpoints stem from?
Yes, but I am full in on simulation hypothesis, and people are going to enter the matrix... willingly.
https://nexivibe.com/intj.html
While some of the ideas in this do resonate with me (or at least they're entertaining), it's unfortunate that's it's so obviously LLM generated. And some parts of it, like the INTJ exceptionalism, reek of LLM sycophancy, which then turned into to some kind of god complex...
observation a: Document title is about a minority's rightful supremacy
observation b: document says "this is not political" then dives into persuasive speech
conclusion: this document was written by the bad guys
i just actually read that and it is possibly the most morally abominable screed I've come across in a long time. Shocking that its acceptable to share in polite company
Oh, then you will get a kick out of this for sure: https://nexivibe.com/winter.html
> They have a question that would be very well answered with a search leading to a reputable source
Can you give an example of what kind of question you mean here?
Given that most people's idea of a reputable source is whatever comes up on the first page of Google or YouTube, I think we should use that as the comparison rather than dismissing LLM results. And we should do some empirical testing before making assumptions, otherwise we're just as bad as the people we are complaining about.
Whatever results we get, the real problem is that most people's ability to verify information was not good before LLMs, and it's still not good now.
So now you're dealing with LLM hallucinations, and before you were dealing with the ravings of whatever blogger or YouTuber managed to rank for this particular query.
I don't fight them on it. I just ask "where did that come from?" and suggest checking a real source. Most people aren't trying to be wrong, they just want quick answers. If you show them how to double check without making it a big deal, they usually get it.
Sure LLMs make mistakes, but have you looked at the accuracy of the average top search results recently? The SERPs are packed with SEO-infested articles that are all written by LLMs anyway (and almost universally worse ones than you could use yourself). In many cases the stakes are low enough (and the cost of manually sifting through the junk high enough) that it’s worth going with the empirically higher quality answer than the SEO spam.
This of course doesn’t apply to high-stakes settings. In these cases I find LLMs are still a great information retrieval approach, but it’s a starting point to manual vetting.
At the very least, I'm glad most people finally recognize LLMs are being used as a political weapon against education. It's the same old power struggles as ever.
These people may be idiots who are impossible to reason with, but at least for now the LLMs have not been completely driven into the ground by SEO. They might actually be getting a taste of what it feels like to not be an idiot. I'm happy for them, but they'll snap out of it when their trust is broken. It's probably sometime soon anyway.
Simply prove them wrong (earnestly and in good faith). When they realise the LLM is fallible, they'll learn to be skeptical of it without you needing to teach them that specific lesson.
Have you had much luck with that in real life?
Even before LLMs we had a misinformation problem, and people just believe what they want to believe.
“Well, it’s exactly the kind of thing they would do” when you point out that, in fact, it did not happen.
The internet gave us all comfortable bubbles to live in, and people generally like to be comfortable.
Show them https://old.reddit.com/r/aifails/
tell them what to prompt the AI with to get the correct results. I've seen a number youtube shorts lately doing this, where some scientist gets "refuted" by some random person based on an LLM result, they then sit with the LLM and ask the same question, get the same wrong answer, then follow it up with a clarifying question, which then the LLM realizes its mistake and gives a better answer.
And then ask another question, and the LLM changes its mind again ("are you sure?").
It's not actually realizing anything so much as it's following your lead. Yes, followup questions can help dislodge more information, but fundamentally you can accidentally or on purpose bully an LLM to contradict itself quite easily, and it is only incidentally about correctness.
I have a feeling this is like telling people "don't touch a live wire" and the more direct experiental "I won't touch a live wire again" lesson: People need to experience being hallucinated at, within their comprehension, and at best can be told about Gell-Mann Amnesia.
I doubt you can stop them from asking machines for answers. What you can do is aide them to learn how to distrust the answers competently, but outside their field of knowledge, applying skepticism is hard.
The irony of Gell-Mann Amnesia is that Michael Chrichton, who is said to have named it, suffered from it badly: Wrote well within his field, misapplied sciences to write well outside it, and said things which were indefensible.
There is no point to argue with stupid people. It's the same people who support their "opinion" with internet articles (like that means anything), mainstream media (hard to find bigger deceivers), or social media posts (that's arguably the worst).
Now they got another "God" in LLM.
How to deal? Just ignore. There is way more stupid people with stupid opinions than we can possibly estimate.
The people who trust LLMs already trusted anything else they heard. There's nothing to do for them. If we were pre-LLM, I think you'd be concerned that they trust the first result on Google. Or things they heard on podcasts. This is just what we all do, to varying degrees.
I'm genuinely unsure of whether or not this is better. LLMs make mistakes, but so do humans. So often. I really don't know how often LLMs are wrong in comparison, or how you'd find out. Regardless, computers have become a terrible way to learn things if you aren't a rigorous person. Simultaneously, they've become an absolute dream beyond the imagination of most humans in history, if you are. That's very strange.
At work I had this kind of discussion on a conference call, someone looked something up about a internal company policy and it came back with a hallucinated wrong result.
So I said, don't ever trust the output of an LLM without verification. However this caused me some hassle with the AI adoption manager. We have minimum-use AI KPI's for employees and he asked me to stop saying these things or people will use it less.
In the end I just hated the company a little bit more. I'm just sick of fighting against idiot. And he does have a point, our leadership is pretty crazy about the AI hype, they want everyone to be on it all the time. They don't seem to care whether it adds value or if it even detracts.
That sounds insufferable and I'm sorry your coworker and your bosses have gotten this hypnotized. There are actual use cases, but this type of productivity cargo cult drives me up the wall.
At least if they were into filling the office with magic crystals, it would be decorative and easily ignored. This is just forcing people to spend a bunch of tokens in a dull ritual to make the line go up.
Yes :( It is awful. Thank you.
If the tool provides a link to the raw source, you can just click that. The LLM is basically just a better search engine here.
Well yes, that's what I was trying to point out.
But that's not what anyone does. They just grab the blurb that comes out and paste it as an answer. People are lazy.
The same with the meeting minutes. Every time people just grab them from teams and mail them on without even reading it. Of course being full of errors (especially because teams is very very bad at determining who is speaking when speakers are on the same account (e.g. if multiple people are in a physical room with a table microphone).. And assigning tasks to the wrong person as a result.
I keep pointing these mistakes out but people just shrug and point to the little disclaimer: "AI generated content, may contain errors". Well yeah, that was a warning for YOU to doublecheck it before you pump that shit out.
They often respond also like: "Nobody reads the minutes anyway". Well yes but if they don't then why do we have them in the first place? Not doing it at all is even more efficient than using AI for it.
But anyway I've given up on fighting against this, I just ignore these people unless a task is incorrectly assigned to me.
A search isn’t going to lead them to a “reputable source”, it’s going to lead them to ad filled SEO garbage, because it’s not 2004 anymore and thousands of Google employees have been working for two decades to ruin the Internet.
I’ll take LLMs any day over what search and the rest of the Internet has turned into.
Right, I think (hope) the OP meant not to emphasize the "search" in the sentence, but the "reputable source". Of course a Google search now is much worse than an AI search.
And it is the ultimately the reputable source that matters, and whether the person actually read it and checked that the details matched the summary (be it human abstract, LLM-generated, or otherwise).
I asked Grok, and it actually gave a very useful answer:
https://grok.com/share/c2hhcmQtMg_b036e24b-3211-4655-bd77-da...
I would love to know. My manager shovels AI generated design documents at me and expects me to clean them up.
Tell him thats not his job description. Shovel ai generated tasks at him and tell him to prioritize
A boss at my former job would constantly generate AI images and send them to the graphics guy for him to make 'adjustments' and no matter how many times the graphics guy explained that these images are much more trouble to clean up than for him to make a new one from scratch by hand, he seemed unwilling to understand and kept spamming slop at him.
I really felt for the guy the first time I was in a meeting and somebody had generated their own project roadmap recommendations. This type of behaviour introduces so much noise and time waste in the system, I would love to know how it shapes up next to the benefits.
Don't even get me started on people AI generating personal farewell notes for retiring coworkers or whatnot.
You can't change someone if they don't want to be changed
There's multitude of reasons someone would blindly trust LLM: laziness, lack of confidence, need for assurance, you name it.
You just gotta stand your ground and end up agreeing to disagree
They do not have a soul, they are NPCs incapable of reasoning. I don't mean lazy, incapable is literally what they are. Logic escapes them. When they say llms are conscious, and fully intelligent, them are comparing to themselves. If you think about it, they are right to say AGI is here, if the bar is the average human being. If you contemplate this fact for a moment, and start pondering it could be true, your life would change forever. Most beings just do not have a singular perspective, cannot reason, do not have a taste, cannot appreciate someone else's singular perspective. They also do not appreciate art for the same reason. I am sorry, truly. Just let them be. They would kill you before admitting they are forever stuck in Plato's cave.
Most NPC like take.
Look I get your sentiment. Sometimes it feels like you're the only thinking, conscious being. Surrounded by beings who fundamentally cannot understand that A –> B does not imply B –> A. Beings that say things that are so obviously non-sequiturs or contradictory.
But calling people NPCs is the most NPC thing you can do. There is more to people than logical reasoning and these things often impede or completely block reasoning. Very intelligent people sometimes say the most grotesque things. People turn mad and mad people sometimes get their head set straight.
Sometimes it's not so much about the pure ability to reason but the goal of that person and whether they see understanding something or trying to understand it as helpful towards that goal.
I do agree though that the more intelligent someone is the less likely it is that other things will block their intelligent ability and the harder it is for them to fool themselves into believing absurd nonsense and to blind themselves from apparent truth.
Sometimes after talking with someone – or rather trying to but ending up only talking to them because they just do not manage to understand what I'm saying or to engage with it in any way – I wonder how they manage to get through every day life as that requires solving way more complex practical problems. Yet they do.
I liked this response and overall I don't disagree.
Just to be clear, I don't have any problems with people considered to be of lower intelligence. This is besides the point.
Unfortunately, this isn't something that can be elaborated much further. Since you took the time to respond, and seems to be a little perturbed, I will just say again I was never addressing lack of intelligence or mental capabilities.
Intelligent individuals perform well on different educational frameworks, they absorb much faster the rules in place, they have incentives to play along and place trust in the ecosystem they live in. They are the most likely to believe in absurd nonsense. Despite being miserable and powerless, they play along larping as competent for not being challenged by virtual problems, not questioning reality a single day of their lives. And yet, they will fight to death for the opportunity of playing along and never question anything, for the opportunity to be exploited yet again. Intelligent people convinced themselves, day one at school, there is a better way described on the books. They got really good when they understood this. They got pretty rewards. Today, at 35, they behave the same. The books became the articles, hacker news discussions, readme auto generated filled with adjectives. They like adjectives very much. What does this system do? Blazingly fast, secure, sandboxed. Sandboxed? Is it kvm or what? - Secure, production grade, made for x, empower you to y.
Are you familiar with the memes about boomer mentality, that believe everything they see on television? Television is now pelican guy articles, ai influencers, Theo, Vercel...
If every single decision someone make benefits only the rent seeking parasite, but not the individual deciding...safe to conclude this individual does not care about preserving himself? Self preservation not important, lack of will of power, flacid as fuck... The only conclusion I could possibly arrive is most people like this do not have a soul. How else could I explain?
I couldn't make a good job keeping this brief, but tried my best to explain why it is not a matter of intelligence. I said reasoning.
Intelligent ability is not the same as logical abstract reasoning in behavior. I think we agree on that. But intelligence is a prerequisite for it.
> They are the most likely to believe in absurd nonsense.
I disagree. Intelligent people who believe in absurd nonsense are better at rationalizing that nonsense and at making it convincing. Some people write very lengthy books arguing for a clearly flawed premise precisely to distract from the fact that the premise is flawed.
But overall intelligent people are less likely to believe in nonsense in the first place and more likely to critically and, importantly, logical soundly examine a thesis that is presented as given.
> Intelligent people convinced themselves, day one at school, there is a better way described on the books. They got really good when they understood this. They got pretty rewards.
What I think you are describing is academic (and professional, maybe somewhat social) performance and conformity, not intelligence.
Your answer reads increasingly convoluted to me towards the end. But I will try to interpret a sentiment from it.
People who do not arrive at the same conclusions as you do not necessarily do so for a lack of reasoning. Two equally intelligent individuals when presented with the same messy, incomplete fragments of reality can through reasoning arrive at widly different conclusions. And in reality we are never even presented with the same fragments of reality in the first place.
Man, if you think you have figured it all out, you really haven't. Regardless of what it is that you believe. There really is nothing more foolish or more NPC like than to think you've understood it all.
The worst are the ones who say "what if human brains are also just like LLMs". If you've thought about this for any length of time, it's very obvious that we aren't.
>If you've thought about this for any length of time
If you full throttle a BWM1000 S RR for a split of a second,at 30mph first gear, it will self eject beneath you. If you do that, for any length of time, you're dead. Do the same on a 50cc motorbike and not much will happen. Even for extended periods of time, not much would happen. You could hold it down until you run out of fuel or the universe gets cold and die, not much would happen.
You see, it's not that they are lazy. Or they haven't put any amount of time into understanding how llms operate. Again I am sorry, most people are not capable, at all, of understanding what is happening at inference time. Most developers, nerds, hackers, who do understand how computers operate, cannot really grasp the basics of what an llm is or what the f is going on. Imagine the average guy, your lawyer, the MBA type of person.
This is absurdly misanthropic and dehumanizing.
I firmly believe that every single person on this entire planet has a depth to them that far, far exceeds anything an LLM could even begin to approximate. I'm sorry you're in a position that you can't see that at all - that each and every one of them feel happiness and sadness and love and hate and fear and rage and inspiration and passion and are utterly human. I hope you see it someday.
I agree with you, every living being is beautiful in their own way. And good on their own way. But we not very well equipped to follow the rules of logic. The most capable of us can follow a very limited number of logical threads, with very limited steps. I don't consider it dehumanizing because attaining true knowledge is not necessarily human or natural for the human apparatus to process. Given enough strain and effort, we can only dabble in the elevated grounds of information and logic. Most people cannot, that is okay. I am not comparing llms to humans in a broad sense, but specific to reasoning, llms are complete shit and so it is the average human. I am not impressed by llms if that was your impression and I am not saying the average human is inferior or deserve suffering lol
Explain that the models are compressed with a lossy compression, and point out that every so often, an answer will be pulled from a section of the model space which has compression errors.
The people who trust bad information from LLMs are the same people who trusted bad information from search results and new articles, it just takes them less time to get bad information.
If they’re at a level where they are so oblivious, then I just don’t associate any further with them in my life.
If they’re employees I’ll try find better ones.
If they’re friends I might tell them.
LLMs give you sources now even if you don’t ask for it. If you don’t like it you can ask for more reputable sources. What kind of 2023 question is this?
Generally not worth my time, energy, and effort. Why do I care if somebody believes a lie? I believe a ton of lies and I’m doing just fine.
It all depends on the context: how does this affects you?
Is this something you can control or is this outside your control?
I feel like I can trust LLMs more than the majority of info on the web. We used to believe the same of Google searches.
For me, for example have seen and experienced doctors making mis diagnosis (and they a reputable source), so what is the difference really?
I guess your question depends on the context they using the LLM as well for and what sort of questions they are asking.
Scientific fact based or opinion questions?
where do you think most of that info came from though? Not from the public library
I hear you, But my understanding is LLM can consume multiple sources of information and deduce what is the truth better and more accurately than a human clicking on multiple Google links and veryfying information and sources.
LLMs by definition cannot deduce, because they cannot not know or think. There's guard rails to try to make it more correct than wrong, but ultimately it's about which words seem like they would fit when coming after your words.
It's a neat trick, but the mind wants to ascribe meaning and reason to words that sound meaningful and reasonable, but these words do not come from a thinking mind with intent and interiority. It would be much more interesting if they did, but when and if that does happen, it won't be from an LLM as we know them today.
Ye agreed "deduce" bad choice of words.
If you tell LLM "explain X and cite reliable sources" would that then be more accurate?
Maybe it's the way the users are asking the questions, and perhaps prompting in the right way will lead to better (more accurate) results and reduce hallucinations?
I think the fundamental problem is that humans use language to refer to things and constructs that exist and have various relationships with eachother in meatspace, whereas LLMs use words solely as things that exist in relation to other words. That's inherently lossy if you're trying to make it fetch and regurgitate information encoded in the former format.
While the ability to interface with a computer program in plain language is the really interesting thing here IMO, it also comes with a number of problems baked in that are worse than person-to-person transfers of text-speech.
Your monkey brain is actually quite good at figuring out if other monkeys are bullshitting you and what they mean, because you can make use of a vast number of small cues and unconscious tells in what they say and how they say it - even in writing. With an LLM, you cannot do this because it will always have the same confident can-do zeal with everything you ask it for.
Deduce? No.
they don't deduce the truth; they're producing statistical predictions about what words go together (more or less; llm nerds dont @me). If you fed an LLM a bunch of homeopathy texts, it's not going to tell you why homeopathy doesn't work.
In inference/tool use it's doing the same thing that a human is doing in that regard. Just faster.
In training, it's a blind process. It's up to the trainers to feed the model accurate sources.
Honestly, the kind of people doing that is probably better served by AI (currently).
I'm saying that because they were not going to be critical of the search results, and google is not exactly showing objective truth in the first positions nowadays.
How do you deal with people who trust their discursive mind?
Its everywhere now its becoming a real problem in every corner of the internet and in the real world. People are using hallucinated legal cases in lawsuits, they are generating images to create fake events, they are using AI to write their CVS and just about everything you can imagine. People are having to wade through all this slop professionally, calling it out and pointing out the mistakes doesn't seem to help, the people using this stuff believe the AI is correct no matter what you say or do.
Like most things that go mainstream it will take a good while before people understand, by which point they will have learnt a lot of things that aren't true and they will never let them go. We might get a healthy use of current AI at some point in the future or if the product drastically improves.
All you can do now is hold them to the same standard you normally would, if you catch them lying whether an AI did it or not its their responsibility and you treat them accordingly.
These aren't a new type of person. It's the people who would hear something from a friend, not fact check, and just repeat it. It's the people who (if they know how to google) would search, find the first result, and trust that, or they would write biased queries to google and then trust the first niche site that would agree with their pre-formed worldview.
Using or not using a LLM is not itself a measure of how deluded someone is, for example anytime I ask a LLM a question (it can be nice for long form questions that don't translate well to a google search, I require that it provides source links for every claim. This tends to make it reply more accurately but also lets me read the page source for their top level explanation.
same way as i deal with people who trust other people.
The failure mode I see most isn't blind trust — it's unverifiable trust. The output looks confident and coherent, so there's no obvious signal to distrust it.
Building an agent debugging tool taught me this concretely: LLMs will return structurally valid, fluent garbage mid-loop and the agent keeps running. No error. No warning. Just wrong output propagating forward silently.
The people who deal with LLMs best treat them like a junior dev who writes clean-looking code that hasn't been tested. You don't distrust everything they say — you just never skip the review step.
How do you deal with people who trust religion?
Accept this this is going to continue to happen, ask yourself if it’s something in your control or not, and try to find a way to enjoy the ride. It’s going to be bumpy, as we’re going through trust issues outside of just LLMs as a society right now.
However, if I notice a friend is about to harm themselves in some way I’ll pull open their ChatGPT and show them directly how sycophantic it is by going completely 180 on what they prompted. It’s enough to make them second guess. I also correct people who say “he or she” when referring to an LLM to say “it” in dialog, and explain that it’s a tool, like a calculator. So gentle reframing has helped.
Sometimes I’ll ask them to pause and ask their gut first, but people are already disconnected from their own truths.
It’s going to be bumpy. Save your mental health.
It depends on the details and what the alternative is. With a lot of my friends LLMs are more accurate than whatever nonsense is on tiktok and similar. The number of people I know who verify things from scientific papers and the like is very low.
I do spend some time on the bedbugs subreddit and LLM failings come up a lot because they are very bad at figuring if a photo is a bed bug or not. So I say don't worry, AI is crap at that.
Just assume that they are an expert in said acquired knowledge and you should ask which non-AI source (book, or human expert) did they learn this information from and ask them more questions around that.
This works especially if you studied in that subject matter, you should be able to immediately detect anything answer that is inconsistent or if they give hallucinated sources.
That is called the Gell-Mann Amnesia effect.
Introduce them to jailbroke LLMs.
A friend of mine severely injured her leg, especially knee, and went through a surgery. She said that she had a rehabilitation plan for the next 6 months. Guess what, from Gemini. I just told her just listen to her doctor.
I didn't tell her why LLMs can make mistakes or hallucinate because I thought that she would not appreciate my mansplaining.
Looking forward though, my boring answer would still be education. It is going to take time. But without understanding LLMs, they will not be easily persuaded.
Thanks for this. I was in the camp of trust the LLM but y’all have made valid points. After discussing with ChatGPT, it agree there are some areas where it should not be trusted as accurate, but it said with historical facts like the holocaust it should be high. Idk, perhaps we need labs to produce a chart of it level of trust deserve to certain topics
How to Talk to Someone Experiencing 'AI Psychosis': https://www.404media.co/ai-psychosis-help-gemini-chatgpt-cla...
> just blindly trust whatever it says. How do you deal with that?
I…do not.
Simple: tell them to ask their LLM about it ...
"Tell me about all the potential pitfalls of blindly trusting LLM output, and relate a couple or three true stories about when LLM misinformation has gone badly wrong for people."
LLMs are trained to at least try not to be dangerous and stupid, which is more than you can say for the executive branch and half the country. On balance, I'd trust a SOTA LLM model blindly before I'd trust the average person blindly.
asking ChatGPT to read and tell me what this post is about
I deal with AIdiots just like my ancestors did with any kind of delusional idiot in history they encountered.
I laugh in their face, let them know how ridiculous they are, and then walk away laughing in tears, never talking to them again.
A wise man's life is based around “fuck you”.
Somebody wants me to do something because of or listen to his AI psychosis bullshit, “fuck you”.
Boss has AI psychosis, “fuck you!”.
You are the King of the US? You have a navy? Greatest army in the history of mankind?
Fuck you! Blow me.
There are two kinds of fools in the world: the kind who ask a search engine and believe the first reputable source they see, and the kind who ask an LLM and believe the first response that has a reputable citation.
Yesterday, I was praying to ChatGPT and asking for guidance on my car washing problem. Through its holy scripture, it suggested me to walk to the car wash to improve my fitness. When I arrived and found the absence of my car to be a true hindrance for washing, it occurred to me that I should have pondered the scripture more carefully to identify its true meaning.
I treat the LLM like a diety. Every sane person understands well enough that the Bible is not to be taken literally. And then when someone talks about using LLMs, I always rephrase that as prayer.
I heard this exact story recently in a YouTube video, well, the car wash part at least. That was you?
It's a well known example. At least I have heard it before. I think it's just a reference.
It's an example people use to show the (current) limits of LLM capabilities. Like counting the 'r's in strawberry around two years ago.
https://news.ycombinator.com/item?id=47031580