So Google has established a product called Search.
For that product rules have been established.
Google has monopolized that product.
Now Google is replacing that product with a new product.
But they keep calling it the same thing.
Because they want to keep their monopoly.
That is what has been deemed illegal.
Gemini is not illegal. Pretending the worst version of Gemini is Search is illegal, because it breaks the rules established for Search.
It has nothing to do with monopolies. Google was protected from defamation law with search because the page title and snippets were direct quotes from the linked result page. Whereas with AI overviews, the copy is written by a Google-controlled LLM.
It should be noted that defamation law has a very low threshold in Germany, to the point that businesses routinely sue Google Maps users for less-than-5-stars reviews. Google had to change their display of reviews because of this.
You cannot get sued for a review just because it's lower than 5 stars. But of course if you write something like "I found a dead cockroach in my pizza", or "I hard that they don't clean the dishes enough" in a review without proof, that's defamation. And it doesn't matter if you give 1 or 5 stars with the review.
Yes, Google just removes them as a first step because this is the least amount of work for them. Theoretically there is an Google appeals process. After that it needs to go to court.
My guess is that often reviews are generalizing (easy mistake to make). E.g. they say "service is slow", when they should say: "When I was there on Thursday at noon service was slow for my table".
When reading a review, it should always be assumed that it is based off of one "visit". A negative review will probably never have more than a few visits and if they do they'd normally say "I used to come here a lot but after new ownership service sucks!"
"Lawfare" is a propogandistic term coined and used hypocritically by people who get away with doing illegal things themselves, and have a long track record of getting revenge for being legally indicted and prosecuted, by gleefully going after innocent people on their enemies list with the very legal system they control themselves. And you know exactly who I mean, who uses that term all the time.
The term is older than Trump. Suing everyone who criticizes you is a classic example of lawfare. The goal is not justice but the chilling effect created by your legal actions.
Save your breath. CarlitosHighway's posts on this thread shows he's just blindly running defense for Germany's status quo on defamation laws low threshold for abuse on every argument someone brings, not being open to arguing in good faith, weightings in on the pros and cons of this, so it doesn't matter what counterexamples you keep bringing to him, it won't change his mind, his replies will be more "anecdotal" snarks.
When you see the pattern, best to stopped arguing with such users as their goal is not an honest debate, their goal is just to 'DDoS' you with their opinion.
That is in practice not true. If you leave a fake but glowing 5-star review, no business will challenge it. But if you leave a 1, 2, 3, or even 4-star review, suddenly you're asked to provide proof. Of course, they can legally challenge a 5-star review as well. But in reality they conveniently don't seem to care about those.
Anyway, Germany is probably one of the few places where this happens. The issue isn't necessarily that reviews can be challenged. The issue is that users aren't informed when they leave a review that they may later be required to provide proof of their visit.
I once left a negative review of a very popular touristy business in Germany after a genuinely terrible experience. I included photos and detailed information, yet they still challenged the review, claiming I had never been a customer. Google then required me to provide additional evidence to prove that I had actually visited the place.
What made it even more frustrating is that they challenged the review two years in a row. After the second challenge, I wrote to them that if they continued contesting the review, I would consider it harassment and pursue legal action. After that, they stopped.
What I find pretty shady is that most businesses seem to wait a year or two before contesting reviews. By that point, most people no longer have receipts, invoices, or other documentation. If they challenged reviews immediately, customers would be much more likely to still have that evidence available. In my case, I take photos frequently, so Google accepted my proof and kept the review online.
Ironically, after going through this process myself, I've come to believe that some form of verification should probably be standard worldwide. Requiring reviewers to provide evidence that they were actually customers could help reduce fake reviews. But if that's going to be the standard, it should be clearly communicated upfront, before people submit their reviews.
Another related issue I have with Google Maps is that, at least in my home country, some places have reviews disabled because Google considers them too prone to polarization or controversy. Schools are one example.
Personally, I think that's a terrible idea. I'd rather be able to read the reviews and make up my own mind. Instead, Google, in its infinite wisdom, decides that certain topics are too contentious for users to see feedback at all.
I find that to be one of the worst decisions made by the Google Maps team. Hiding reviews doesn't eliminate disagreement or bias, it just removes information that users could otherwise evaluate for themselves.
Nope, that is not defamation. Both defamation and libel require it to be a false statement. If you did find a cockroach or your plate was dirty, you can freely say so without needing to provide extensive proof.
I’m not a lawyer, much less a German lawyer, but it does seem to my lay eyes that this is true of “malicious gossip.” But what is “insult”?
> Section 192
> Insult despite proof of the truth
> Proof of the truth of the asserted or disseminated fact does not preclude punishment in accordance with section 185 if the insult results from the form of the assertion or dissemination or the circumstances under which it was made.
I wrote a one star review and posted pictures of a badly burnt pizza served at a restaurant in Berlin.
Google sent me an email telling me they removed that because the restaurant filed a defamation claim
Sounds similar to Nintendo dmca takedowns. You could challenge the defamation claim in court and win, but you would have to spend a lot of money and time, and the only thing you get back in the end is the right to post the review.
You could challenge the *DCMA* claim in court and win, but you would have to spend a lot of money and time, and the only thing you get back in the end is the right to post the *code to Github*.
I gave a one star review to some cheap restaurant in Potsdam 10 years ago. I literally just said you get what you pay for. Last year I got a similar email from Google saying the review was removed for defamation.
Also competitors would leave shit reviews and Googles completely brick walling any way to rectify or moderate reviews made it a shit service in true Google fashion. Google and service live in two different worlds.
If you quote the Daily Mail, with a citation, then probably not. If you are repeating a vaguely-source rumor that "you heard" then that's heresay and could be defamation.
Making a claim of 3 star service, without providing 3 star service evidence, isn't any different than a claim like they don't clean the dishes enough. Sure, you didn't write out the exact words, but by submitting the review, you still submitted such a claim, and so it remains a question of if you provided evidence to prove it, which gets into how deep the legal system wants the evidence verified (is a picture enough, or do you need to provide evidence the picture is of the store at the date and time relevant to your review).
People don't need LLMs to lie, and someone in the comments tells a similar experience. You can also find plenty of similar anecdotes using a search engine of your choice.
A lot of people use LLMs to clean up their sloppy text. Not realizing how fake it makes them sound. non-native speakers lacking confidence in their English are a type of llm abuser. It doesn't necessarily make the content fake.
Here's an article from outside the tech sector, which I am linking to as a corroboration since this is quite surprising for those of us unfamiliar with German law.
Shocking, only 10 short years since this (when Merkel was Chancellor): Turkey asks Germany to prosecute comedian Jan Böhmermann (Germany's equivalent of John Stewart) over Erdoğan poem
Google sends the reviewer a message and ask for a statement. If you provide one. They will check it and decide whether they seem it defamation or not.
However, Google doesn't verify shit. Even if you send them proof of a purchase or visit and your message is objectively a opinion (not defamation) they will take it down. Why? I assume because they really don't care about individual reviews and rather spare the time and money and just take the comment down.
Google changing their review displays is Google's decision. It has NOTHING to do with any threat of being sued.
They could actually leave the review up when it's not defamatory but they simply don't want to invest the time to validate it or risk leaving a potentially defamatory review online.
And the legal threat is without proof, so why would you even bring it up. OP could have simply posted the letter. They didn't wich makes this completely unprovable
I think that will depend. If person publishes it as their own speech Google is likely not fully liable. On other hand as Google publishes it as their own speech on their own site well they are fully liable.
To me it seems pretty reasonable that content Google generates with Gemini and publishes on their own google.com domain is their speech. Or at least they are fully editorially responsible.
Sure that could be the worst case.
But then we are in the case law vs. continental law debate for lawyers.
When I wrote "Gemini is not illegal" it would have been more correct to say that the court has not decided about Gemini but specifically about search.
And I do think it makes sense to distinguish between explicitly chatting with an LLM and entering something in the search bar people have used for decades now.
Based on the article, the problem isn't that Google is attempting to present the AI Overview as "search" to the users. Google is attempting to claim to the court that AI Overviews are just like search, and the court isn't buying it.
The problem here is that
1) the AI Overview is giving incorrect information, and
2) it is Google's own words
...which makes Google liable for anything false there.
They weren't liable for false statements in search results, because that was 100% other people's content, being linked to, with a small blurb taken directly from their words. With the AI Overviews, regardless of whether it's getting its information from other people's content, it's remixing it based on Gemini's algorithms and synthesising statements that are frequently wrong.
The ai legal situation is going to go through growing pains. I am abstracting from the specific laws of any one country, just thinking about the general context:
If ai output is not copyrightable, it should not be considered personal output. So nobody should be responsible for it. Or if it is considered personal output, it should be copyrightable. Or perhaps the ai companies will be liable for all output, and they will therefore all cease to exist in any useful form? This seems like another alternative, where the output legal value is not central, but there will be a thousand different fights about how it is presented to others.
That's trying to apply two completely separate legal frameworks, with different purposes, and force them to come to the same conclusion about one aspect of each of them. It's not that simple.
It's perfectly legally consistent to say that AI-generated content has no copyright (because it's the product of a computer, not a human), and also that the human or organization operating an AI is legally responsible for anything in its output that is legally actionable.
Someone needs to hold legal responsibility for any piece of content out there. You can't just wrap your decisions in AI and get to be free of all liability for it.
But copyright isn't like that. There's nothing lost to society by saying that content is not copyrightable, and particularly given how the major LLMs were trained, there's a lot lost to society by saying that they can take all of that from everyone without consent, and then everything it produces has copyright and can be used for, say, Google's profit in perpetuity.
Yes, I was not sufficiently thinking about copyright as an arbitrary legal construct that can be manipulated at will. I don't think output should have copyright, but I would presume the copyright should it ever exist would belong to the user and not the LLM creator, just like photoshop does not give adobe rights to user output. However much like there is no copyright, the uncertain output from an LLM should never directly create legal liability - the user prompt and intention should, and legal standards regarding recklessness and malice should apply. Otherwise it's a bit like blaming somebody for the output of a roulette wheel.
So I think I like the current decision which is more about presentation, dissemination, application, and claims than content, and there should of course be liability for LLM creators if they are not actively dealing with results like CP, violence, or many other illegal or dangerous things.
Google accepted the argument up to that point.
Then they argued that they are not responsible for the search result.
The court argued that what they did was not a search (or not protected the way a search is).
The argument whether Gemini, ChatGPT or Claude are search engines and whether a prompt is search terms is a different argument.
Certainly an interesting one, but not what that court looked at.
It's not just about the output, the input matters also.
That's probably what they actually think at Google. A lot of the "they're just like people" marketing now seems like responsibility-washing. That despite Google developing the LLM and charging for its use, they treat it less like a piece of software and more like providing access to some guy's answers who has no agency and isn't owned by them. If their software ever messes up and calls you a murderer to a potential employer, or offers dangerous advice, their reaction would be "oh well, that's just too bad isn't it? Well, we wrote that the LLM may not always be right in page 683 of the service agreement, so this is not our problem."
It could presumably accurately summarise it while still sticking to reported speech. In German there is a grammatical tense that is basically only used for this; it's pops up in newspaper reporting a lot where they don't want to endorse the statements they are reporting.
That's not what this article says. If you think the article mispresents the ruling, then you should point it out. You say you like the ruling then proceed to listing things that you like, but are not related to the ruling.
I dont really understand your reasoning at all, nor what law it would break. What law do you think it would break?
It doesnt even seem to be what the article is saying. This looks like a section 230 sort of issue. Section 230 is a US law that protects platforms like facebook, google etc. being treated as publishers because the information, presumably, is just being passed through. But Germany is saying the AI results are authored by Google.
I don't see a world where AI results aren't reasonably considered the output of the company. They're minced and sausagified regurgitations but they're not the original sources either.
Yeah, but my point is: what does that have to do with monopolies changing their services?
And to continue your thought, what does that imply about copyright of training data? If Google is authoring the output it seems harder to argue they are ripping someone else off.
It really seems like a tightrope to say google is publishing their own opinion but their opinion is also just someone else's work.
> Yeah, but my point is: what does that have to do with monopolies changing their services?
Not much really other than it being a change of category from pure search surfacing other's speech to their crummy new chatbot input with search underneath. GP was just mistaken that this was in any way related to their quasi-monopolistic position.
> If Google is authoring the output it seems harder to argue they are ripping someone else off.
They still pirated a lot of the training material, it's not like they went and licensed copies of all the books they used in the inputs. Even discounting all the publicly available data on the internet and the models recreating things word for word a lot of the books etc they ingested are illegal copies.
> like a tightrope to say google is publishing their own opinion but their opinion is also just someone else's work.
It's a grey area for sure between remixing and blatant copying that changes depending on the precise output. But it's inarguable imo that they consumed and ingested the work of basically every digitized word humans have ever written for their own profit without an ounce of compensation for the original authors. Copyright is full of these grey lines though like fair use doctrine, it's incredibly difficult to define in a systematic way what distinguishes transformative and non transformative works for example.
Like it or not, this is how people perceive output from llms. Those ultra rich companies can't forever have the cake and eat it too, just milking global markets forever (I know they of course desperately want and need to, but that's their struggle).
I am fine with that solution since I don't need to shill for them for some ie investment or employment reasons. Less technically skilled people (aka your parents or grandparents) are getting their lives fucked up left and right because they learned to trust search results, and now they suddenly can't.
The court ruled that the AI generated content has an author/editor/publisher: Google. It also ruled that Google can be held liable. Insert pikachu face meme.
Original search before Google started trying to provide their own answers was purely pointing to relevant pages, even the first iterations of the first results being replaced by the result Google believed provided the correct answer could be pointed to as simply providing an answer someone else wrote (and to my knowledge was mostly fact based questions; birth dates, etc that are hard to categorize as defamatory). Now Gemini is combining and mixing together multiple sources to provide a new amalgam answer that IMO is distinct enough, and applied to touchier subjects importantly because they're treating the search bar like you're talking directly to Gemini, that it crosses a line between referencing speech by other people without endorsing (OG Search) and having the company produce speech about the search (new Gemini infested Search).
> was google search responsible for surfacing correct answers
No, as the article lays out, google hallucinated facts that werent present in source material and so at that point the court found them liable for that.
Previous protections existed for search because it has been argued (and agreed) that they are merely a vessel for showing the content to the user. But when they begin to editorialize and reword the content to show their own version of the content (the AI summary) those protections dont apply even if the AI summary is shown in the same UI as search.
No, Google search was simply repeating quotes from other websites - those other websites would be responsible for their content. Now that Google is manufacturing answers using their own LLM, they are responsible for whatever results that produces.
There isn't someone at a keyboard typing content. It's still just search, repeating quotes from other websites. The difference is that the algorithms have turned lossy, so the quotes are not always preserved in their original form.
Obviously, I understand that and maybe you shouldn't have cut short the quote from my comment as that demonstrates exactly and precisely what is going on.
i.e.
> Now that Google is manufacturing answers using their own LLM
To be honest, I find it somewhat rude that you select a portion of what I wrote and then criticise it.
Equally obvious is that it doesn't matter what you do or don't understand. Nor does it matter if you find something to be rude. The comments, they are not written for anyone in particular. This is a public forum. Comments are there for all.
Die Verfügungsklägerinnen begehren von der Verfügungsbeklagten die Unterlassung von Darstellungen KI-generierter Antworten in einer Suchmaschine.
...
Die Verfügungsbeklagte betreibt eine Internet-Suchmaschine unter google.de in Deutschland. Dabei werden nach der Eingabe von Suchbegriffen durch eine die Suchmaschine benutzende Person auf den „Suchbefehl“ hin mithilfe bestimmter Algorithmen nach möglicher Relevanz sortierte Ergebnisse angezeigt. Zusätzlich bietet die Suchmaschine auch als unterstützende Funktion ein Suchergebnisformat an, bei dem mithilfe einer generativen künstlichen Intelligenz (im Folgenden:KI) repräsentative Ergebnisse zusammengefasst und angezeigt werden."
No "Suchmaschine" + no „Suchbefehl“ -> no "Tatbestand"
No "search engine" + no "search command" -> no "elements of a crime"
> No "Suchmaschine" + no „Suchbefehl“ -> no "Tatbestand"
I think you have misapprehended the ruling. Of course running an LLM that spits out libel is also a crime outside of search engines. It's just that _this_ crime has those elements. If Meta's LLM defamed a business, then the ruling would instead talk about Whatsapp users and chats, or whatever.
Good. The true mark of AGI is when a company accepts liability and doesn’t bury “for entertainment purposes only” deep in their TOS. Same as it works with employees.
Same for self-driving. Your car is not self-driving until it accepts liability and you count as just a passenger.
But watch as Germany soon loses AI Google results.
I think it's a completely reasonable position that companies making self-driving cars and question/answer systems are legally liable for any errors.
But if you hold that position, you also have to be fine with companies not offering products and services in your country. AI systems will eventually be good enough (in 10-20 years) for companies to be able to deploy such systems with sufficient accuracy to afford the lawsuits. Until that time, such countries would just not have access to systems before they were bulletproof.
> AI systems will eventually be good enough (in 10-20 years) for companies to be able to deploy such systems with sufficient accuracy to afford the lawsuits.
I doubt that will be the case, because of the long tail problem. (same with self driving cars and other ML related problems).
In fact, we have counter-examples today. Newspapers (even reputable ones) can't get it right every time, despite the fact that they have both trained people and in theory they're setup to catch that w/ reporters - fact checkers - editors. And still, from time to time, they get it wrong. (and I'm not talking about purposefully getting it wrong, just honest mistakes.)
What will likely happen with a ruling like this is that the answers will be hedged and legalesed and muddied up the wazoo.
Newspapers have mechanisms like corrections and apologies that can be used to "right" a published falsehoods.
This relies on me being able to find out if a newspaper lies about me, which is usually easy since we can all go buy the same newspaper. With AI it is much harder to find out that it has been telling potential customers wrong things about my business.
You don't realize how this works do you? One incredible power (executive) governments have is to let people get away with crimes. People don't realize they have this power, but they control the public prosecutor who can legally choose to do something ... or not.
You will find references to this in stories where the government lets people get away with murder, because of course, that's dramatic for a story.
But when "bigger interests" (ie. the Chancellor's bank accounts) are at play, just to name one example, China gets to distribute lead-painted children's toys in Germany and doesn't have to accept liability. Russia gets to import sanctioned natural gas over illegally constructed pipelines. Etc.
As to how this goes within the EU, in the worst (but common) case, is as follows: the government chooses a company, and refuse to sue them. They have a tendency to choose the worst possible company, like using Palantir for policing Germans [1]. Then you find out most of the Chancellor's grandchildren are working there.
> AI systems will eventually be good enough (in 10-20 years) for companies to be able to deploy such systems with sufficient accuracy to afford the lawsuits.
This doesn’t sound convincing. What AI and what company?
LLMs don’t seem like they will ever be reliable like that.
LLMs will absolutely be like that. The speed this technology is moving at makes me certain, especially over a period of 10-20 years; 20 years ago I was bugging friends for a GMail invite and AI was a joke left to academics.
I think it will even be solved soon, like, within the next 18 to 36 months. Hallucinations are the biggest problem consumers have with LLMs and a solution to that would be instantly worth billions of dollars. I’m sure every company in this space is desperately trying to figure it out before everyone else.
A non-deterministic system will always make mistakes, but we’ll hit a target where LLMs make fewer mistakes than humans and that will be good enough for almost all applications.
I don't know if you can "fix" hallucinations without changing the fundamental architecture. The other factor in this article is that prior to the AI summary at the top, Google could simply state that it was an error on the part of the website owner. Now it is being held liable for whatever the summary states - even if it's more accurate, it can still be wrong enough times to be expensive.
This is disregarding the entire mechanism by which LLMs work. How close to this ideal are the current frontier level ai is now? If you do a cost/improvement analysis does it look like it can reach a usable threshold?
I don’t know the numbers but as user, it seems impossible for it to be useful without expert review. It is also debatable if it brings any value when you consider the cost of building and using LLMs and the time of expert. Also need to include the opportunity cost the expert is spending on reviewing slop instead of creating work themselves and the long term consequences of this on the expert himself
> LLMs don’t seem like they will ever be reliable like that.
True. But you never know if / when there will be a new big breakthrough in AI, which will probably be based on a new architecture / paradigm, i.e. it won't be LLM-based
Who would argue for offering self-driving cars before they're ready and safe? As a cyclist and pedestrian, of course I don't want them in my country if nobody's going to be liable when they run me over. Let them work out the kinks on Americans since they're so eager to be on the cutting edge of progress.
> Until that time, such countries would just not have access to systems before they were bulletproof.
Correct, most jurisdictions do not allow businesses which cannot be held liable for their actions. This is pretty core to a modern society.
Imagine if a company selling Knicks tickets was not expected to then actually provide said tickets and there was simply nothing you could do about it. Oopsies our sales page is for entertainment purposes only
To be fair, the internet has spent some 30 years figuring out how this works and it’s still not fully resolved. For the most part we’ve agreed that companies must follow the laws of both where they live and where they operate. This wasn’t always obvious!
Almost all jurisdictions allow businesses which cannot be held liable for all their actions. Imagine losing you house because someone decided to smother themselves in your infused cooking oil and light themselves on fire.
Businesses are made to make it easier to share profits and responsibilities when trying to fulfill users wants. Laws are made to offer protections for consumers (because nobody has time for common sense), but at the end of the day the consumer has to take responsibility or no products can be made. If you're too fat for a chair, it's on you to find or make one that works- not every product is for every person. Laws only stop chairs being sold that are too dangerous for anyone.
> Almost all jurisdictions allow businesses which cannot be held liable for all their actions. Imagine losing you house because someone decided to smother themselves in your infused cooking oil and light themselves on fire.
How would that be the company's action? The only way the business might be liable is if they advertised their product as safe to use when lighting yourself on fire or if there was already some law that required them to warn customers not to light themselves on fire while using the product and they ignored that law.
In this case, it's not about what somebody else did. It's what Google did. There were already laws against lying about companies by saying they did illegal things when they didn't, google broke the law, so that's what google got in trouble for.
Consumer protection laws aren't there to replace common sense, they're there to prevent things like outright fraud and poisonings/murder.
Freedom as in freedom of private property can only be guaranteed by the State.
The State watches over its own population and makes sure that private property, i.e. capital and work, is made productive,
so people go to work, businesses make profits, and everybody pays taxes. Taxes are the State's prime source of income.
When all these million of private interests collide, which they are bound to do, the State provides a jurisdictional system that has to decide between those private interests and the State's own interests.
E.g.: If a business owner refuses access to medical patents or to lower prices and safe potentially people's lives, the State has to decide between that immediate interest and its own interests, which is protecting private capital, as its source of income. Since I'm in Germany: In the emission scandal Volkswagen didn't just physically harm people, VW violated the private property of millions of customers. Despite that, the German State sided with Volkswagen the larger capital and did nothing. During Corona, the German State refused to open patents for a limited time to help safe people's lives in poor countries. Doing so would've violated the interest of private capital, so it refused. In contrast, if I as an individual refuse to help somebody in an emergency, the State would either fine me or put me in jail. In this case, people's lives become the State's prime interests, because they are also the State's source of income, as a productive workforce.
If our standard for laws would be that "well no reasonable person would do this/believe this" then nothing would be illegal, there'd be no need to label any product as potentiality harmful, etc.
Do you really want to go there? That everything in the world would have a literal "caveat emptor" attached to it?
It's unbelievable how lightly some people hand over the tools for mass manipulation to a single corporation in the name of freedom of all things. We're not talking about a laser pointer here.
There is a disclaimer, yes, but you have to admit that it's pretty shit, innit? I mean for one, it's about the size of a human hair, and at least when I tried it, the disclaimer came up only when I clicked the "Show More" button. It might admittedly show up earlier if the response is shorter, admittedly I don't know. Also personally I'm a bit uneasy with the idea that just with a simple disclaimer they could avoid any and all liability. Not your argument, I know, but still.
As for not wanting to force companies to release only "safe-for-children products", I do actually agree. However I consider it to be a matter of degree, and in this case for example, I think that if nothing else, Google should say the very least make the disclaimer a bit more prominent and maybe tweak the model so that it's not quite as confident in its claims in the AI Overview.
> As for not wanting to force companies to release only "safe-for-children products", I do actually agree
That would be nice, but as every effort to restrict kids from using software which are not safe-for-children keeps getting condemned for being invasive surveilence, and every effort to stop kids getting the hardware instead gets condemned because of how much of society is now built on assumption everyone has a phone…
Do you mean the responsible people who will ensure their algorithms can be trusted with the important task of acting in the best interest of said people? Try and get a defamatory statement about google from the AI search box.
I have to fight with my family members when they "Google" something, read the top AI slop result, and I ask which page it came from. They believe what is on the Google landing page, and actually I don't think that is a naive assumption. Google has pushed itself as the information oracle, now they are delivering slop as the first result. It's a bait and switch.
I think they should just have to properly explain how AI tends to make things up when it doesn’t know, and that it’s good for coming up with ideas or suggesting directions for research but that you shouldn’t rely on it, because currently their advertising makes you think you can rely on it
The “AI can make mistakes” kind of disclaimers they hide in the corner don’t really cut it
> I think it's a completely reasonable position that companies making self-driving cars and question/answer systems are legally liable for any errors.
Well, for cars anyway, the manufacturer was always liable for the car doing something wrong (example: driver changes the volume on the radio, and that disables the brakes).
It's just that techbros want an exception to this rule if the car is self-driving.
> But if you hold that position, you also have to be fine with companies not offering products and services in your country.
What sounds like a win to me. I certainly hope my country makes it dangerous for companies to break the law and/or harm the public with shitty products that aren't ready to be released legally/safely.
> But if you hold that position, you also have to be fine with companies not offering products and services in your country.
Well.. I mean.. yeah?
I don't think this is as bad as you think it is.
Have you looked at SV and its product offerings recently?
It's mostly just enshittified gamified value extraction that doesn't respect the user at all.
"If you do not let us do all this the way we want, we will take away your ability to use our shit" hits different when the "shit" in that sentence is actually just "shit".
I'm half-remembering a now-old satire along the same lines has Germans wondering why having Google Street View work in their area also requires internal photos of their apartments.
why offer expensive service when it's effectively useless and only add to cost and amount of work? what else they offer, "summarize my text" and "generate custom emoji"? I can live without that...
for now only volvo accepts liability, and only for "slow crawl mode"
Most of the time, human beings driving cars with their own hands and eyeballs are not "liable" for their errors (unless they can be proven negligent or drunk), unless you count their insurance going up. Most car accidents do not end with anybody getting arrested, or sued, or anything like that. Insurance pays out, premiums go up, case closed.
If Waymo can be proven negligent or something, then sure, bleed em dry. But as long as they're acting in good faith and significantly reducing overall road fatalities per mile driven, I think it's actually pretty unreasonable to try to hold them to such a high standard you end up subjecting society to more of the higher fatality rates caused by humans.
Agreed. The problem is that people are used to do search, which returns hits from clearly identified sources. Suddenly, Gemini is interpreting your query and generating its own response. It's an entirely different product. If I expect to find a source, this was just wasted inference. It's especially problematic that the LLM result is the top result, since we all know how much that matters.
It's fine to have Gemini as an option, it's also fine to have a combined result page, that should just be something people are able to chose if they want that (even persistently if they want). It should just not be the default.
> Same for self-driving. Your car is not self-driving until it accepts liability and you count as just a passenger
Mercedes-Benz does this in limited cases. Waymo does it generally. (In China, Level 4 and 5 transfers risk to the manufacturer. This is the correct way to do it.)
That's not exactly the case in China, the current state of FSD is still pretty dumb, unless you consider transferring control back to the user at the very last minute before it crashes a proper way to handle risks.
I guess some Tesla are manufactured in China lol. I am just trying to say that the liability that Chinese manufacturers takes aren't more than the US ones.
Well then you're going to be waiting a long, long time for AGI, friend. None of these companies have any incentive to ever remove that line, even if their chatbots are perfectly accurate over a large sample size. It's pure CYA. Unless it can be proven that the chatbot will always be perfectly accurate (which is not possible), the "entertainment purposes only" line will remain.
Imagine going to a restaurant and when you pass your order "passing an order means accepting that there might be some poison in the food so double check before eating, good luck!"
Why would that mean AGI? You can get into liability-accepting territory by restricting scope, a lot easier than by making your AI smarter.
Self-driving cars don't need to be particularly good for companies to make models where they accept liability in some circumstances, and the cars refuse to drive in other circumstances.
No. Sometimes it does disengage because things are going wrong, but those incidents are still reported the same as if it stayed engaged.
I found one time Musk was using a few seconds of disengagement to insult a driver, but it still would have counted as an FSD crash by Tesla's statistics.
Banning all technology because someone might misuse it is an illogical extreme.
As far as I can tell the ruling is more nuanced. If AI is defaming you, there needs to be a way to correct the record.
A company being open to liability does not mean it is always liable, just that it can be if it really messes up (especially if there are aggravating circumstances, e.g. you need to drag them to court to issue a correction).
However, banning a technology because it produces dangerous misinformation (e.g. showing inaccurate and dangerous results for a liver function test) sounds like a completely sane thing to do.
That'd be so great. No more cursing if I forgot the "-ai" or "-ki" flag in my search and see this odious AI overview processing window rendering slowly and taking up the space where my search results should be.
> The true mark of AGI is when a company accepts liability and doesn’t bury “for entertainment purposes only” deep in their TOS.
I don't think so. It is easy to imagine the following (currently only fictional) scenario: the AGI does give perfectly correct answers (in a suitable sense), but some people in power consider these answers to be too dangerous, so they sue the company behind the AGI on terms of liability (i.e. the company is liable if the AGI gives answers that those in power don't like and which these people consider to be too dangerous for the public to know).
This doesn't disagree with the poster above: they're saying that taking liability is a sign of belief in AGI. You're saying that lack of liability doesn't mean there's no AGI. Logically these two are not exclusive. p => q doesn't mean q => p.
Can we just trash this as a marketing term? If/when AGI arrives there will be no point quibbling over competency. What we are looking at is just bad search results
"No quibble" is more a mark of ASI than AGI; while both are easily coopted as marketing terms[0], "smarter than all humans" is harder to dispute than "general intelligence" (can do some graduate level work, can't drive well enough to render steering wheels obsolete, is it general?)
I don't think this is the case. It's not clear intelligence is a coherent concept to begin with, let alone a one-dimensional thing you can maximize. What, is it going to write stageplays that puts shakespeare to shame? Find a way to enforce world peace or end greed? I'm not at all convinced we anything more than the simple competency we expect from humans is possible. What would we look for to know it's there?
Why is it good? Everyone with common sense knows AI can be wrong. And it’s not buried in their TOS. It’s in the chat box. But even if it wasn’t, it’s ridiculous to create liability for AI chatbots.
it’s ridiculous to create liability for AI chatbots.
Liability isn't being *created* here, it has existed legally for a very long time.
False information can cause real harm --- and the legal burden of proof is on the source.
Search engines were provided legal exemption on the basis that they were simply quoting/referencing 3rd party sources who where legally liable for the content.
LLM chatbots legally exceed these bounds by fabricating info/content on their own ---data that does not exist elsewhere. This is a liability issue waiting to happen as there is no other responsible party/source to blame.
Nearly the entire American tech industry has been super heavily selected for people who undervalue the legal language with crazy implications buried everywhere.
Otherwise most of it would not even exist.
Everyone would have continued paying out the nose to the IBM’s of the world year after year (who had unusual willingness to sign short ambiguously worded custom contracts to their own disadvantage, if paid vast amounts of money).
And be on mainframes to this very day… maybe Y combinator and HN wouldnt even exist in that world.
> Nearly the entire American tech industry has been super heavily selected for people who undervalue the legal language with crazy implications buried everywhere.
A lot of people in IT seem to think law and contracts are in a sense mathematical. They aren't; they're more like a high school book report - to be interpreted, as objectively as possible, but definitely also establishing the intent behind the letters.
Particularly contracts - no, you can't trick your way into things in most cases. "Surprising" clauses are invalid in most legal systems, in particular if one party to the contract is a layperson.
I sometimes wonder how much blame should be placed at the feet of cartoons which teach viewers that invisibly-small print is legally binding, and that any term is enforceable.
I mean, it's not like most people have any kind of curriculum to fix those early assumptions.
> "Surprising" clauses are invalid in most legal systems…
Yes, except… very notably… the USA… (especially certain parts such as Delaware). Which has a judiciary that entertain such clauses much more often than pretty much any other country I’ve heard of.
A clause like: “you agree to deliver your first born son on demand” buried in page 15 would indeed be laughed out of the courtroom.
But less severe clauses buried in page 15 might not be immediately laughed out. Causing a lot of expense and heartache before it gets dismissed.
Just think about it, if it were really just a total joke… why would large firms (FAANG, etc.) with serious legal departments want to look like clowns producing gibberish, year after year?
That is a false dichotomy. The solution to failed laws and regulations is not crime and corruption. The solution is to hold the policial and business leadership accountable; to fix the laws and regulations.
The entire American tech industry has exported Americas predatory, parasitic, and unethical consumer laws (the majority of which are ghost written by the wealthy and corporate legal teams). When I studied law in school decades ago, tactics like bait-and-switch, false advertisting, intentionally misleading or deceptive practices etc to sell products or contracts were illegal across the developed world.
Those illegal, anti-consumer tactics were the SOP of every tech startup I can think of from the early 2000's onwards; following the same route of initially offering a compelling feature set to attract and entice users – usually for free – until securing a certain number of users or funding, then changing the value proposition to exploit that user base, and extract as much wealth from them as possible, ad infinitum.
Today these tactics are known as enshittification, and the average American pseudo-libertarian software engineer will say this is fine, but that's what every anti-consumer parasite and criminal has said in history. Lying, misleading, and exploiting people for financial gain is fundamentally immoral, corrupt, and sociopathic, therefore it should be illegal. Just because it's the norm, or a digital product, you wrote that in the T&C's, or your doing everything behind the liability shield of an LLC, doesn't change that.
What ever happened to the concept of building a valuable, quality product and stable returns for generations? Working to improve the quality of life and standard of living of the community? Of the world? I feel like a 1950's traditional conservative when I suggest that, but most Americans are so heavily indoctrinated with corporate greed and sociopathy they'd consider that sentiment radical leftist extremism. I'm an athiest, but ya'll need jesus (the real brown socialist one). Many would argue Americas current institutional collapse is the natural result of this systemic corruption.
> I feel like a 1950's traditional conservative when I suggest that...
I wouldn't argue that America's moral standards haven't declined (significantly) but I also think it's a romanization to suggest that 1950s America was the pinnacle of morality.
Lying, misleading, and exploiting people for financial gain has been a part of the fabric of American society since the country was founded.
If we're being honest, humans everywhere have demonstrated a high capacity for this behavior since the dawn of civilization.
I never implied the 1950's was the pinnacle of morality. I was referencing the tropes that "traditional" "conservative" politicians since Reagan have consistently virtue signaled, while they aggressively worked to achieve the exact opposite.
There is evidence of that being a common 50's perspective though. It was when most conservatives and liberals alike had been burned by the greed of the guilded age, stock market collapse, great depression, and world war. The majority of the working class in the developed world were experiencing significant gains in QoL/SoL thanks to labour movements and aggressive unionization, did not view CEO's as admirable heroes, or fellow consumers and workers with malice and contempt. Hard work actually resulted in financial security, and greater opportunity for your children.
> Hard work actually resulted in financial security, and greater opportunity for your children.
The economy of the 1950s was due to a variety of factors, including lack of international competition (European and Japanese industrial bases were devastated in the war), pent up consumer savings from war time, demographics (the baby boom), etc. Unionization was certainly a part of the mix but it seems you're cherry-picking to support your romanticized view.
And let's not forget: the 1950s were a good economic time if you were a white man. They weren't nearly so great if you were black or a woman.
at the scale that google is at, it doesn't make an ounce of sense to hold a company liable for a potential mistruth. what if there's a 1/1000 chance of some error, then the company could be sued millions of times per day.
down vote all you want, but I firmly believe this is an example where the user needs to use some judgement on the information they receive and have some critical thinking skills. google would be right to remove all AI results from germany.
If I post “heathrow83829 is a convicted poopoo head” (replace with your favourite crime) as if it were a factual claim, you’d be well within your rights to sue me for defamation even if people should apply some critical thinking skills and say “wait a minute, how would pdpi know that? Are you sure he isn’t just talking out of his arse?”
Now, search engines are usually afforded some amount of protection against defamation claims — they’re not held liable for simply indexing and quoting third party defamatory claims. Which is to say: Google wouldn’t be liable for claiming you’re a poopoo head if this comment shows up in search results.
The point of this ruling is that AI-generated text isn’t a quote from a third party, it’s text generated by Google’s own tools, so it’s speech by Google itself. It might be wrong, sure, but it’s still presented as a statement of fact.
At trial they can have the whole debate about whether Google was negligent in how they build their systems, and all that jazz, but let’s be clear here — it’s not a matter of every little factual mistake getting Google sued (and that would be absolutely terrifying from a freedom of speech perspective), but rather that the technical means by which you generate content doesn’t change your liability in publishing that content.
The other big point is that some people treat LLMs as if they are truth. We all know that they get a lot wrong, a lot of the time. Yesterday I watch both Elon post something somewhere between badly worded and wrong and another user used Grok to "prove him wrong", but this time being completely wrong in a different way. And this was a matter of law which can be looked up. Anything within throwing distance of politics won't be considered "correct" by everyone ever. Just defining "correct" is harder than it sounds and that probably isn't possible in reality.
I think Google added that AI-generated responses maybe incorrect? When you are paying such a low amount of cost, like probably for free, I don't think you can expect a same level of quality as a human written or reviewed of answer. It is like same random user spin up their Lovable and vibe-coded a piece of slop and hold Lovable responsible for not giving them production quality code. It is simple, you get what you paid for. If someone actually figured out AI that is actually always correct, it would be charged in superhuman price as well.
This case isn’t about Google’s users getting low-quality search results. As you say, you get what you pay for. The actual issue is that, in some searches, the AI summaries would claim the plaintiffs were guilty of scams and all-around shady business practices.
Put differently: it’s not newspaper readers complaining the paper is inaccurate, it’s the people mentioned in the articles.
You’re seriously arguing that Google’s libel shouldn’t count as libel because they showed it to too many people? It’s absolutely insane to suggest that a company should be immune from liability for its actions if it operates on such scale that those actions harm millions of people every day on the basis that dealing with that many lawsuits would be too inconvenient.
Wikipedia doesn't have to shut down, but they have to remove the libel.
The problem for LLMs is that they do not learn, and can't be prevented to produce that libel ever again. If Google finds a way to make that happen, no court would stop them from offering an LLM.
The court isn’t stopping them from offering an LLM, it’s saying that serving the output of its LLM is, in this sense, equivalent to serving the output of a human writer they hired. That is, it is Google’s speech and they are directly liable for it, unlike when they’re just serving search results that are somebody else’s speech.
consider Purdue pharma - the Sacklers got off with all their wealth intact because they were too big to sue and properly collect money for their victims.
No they got away with it through a combination of lax pharmaceutical laws, and because they were rich and connected to the officials who should have investigated them.
They're also almost universally regarded as having committed evil acts at this point, so who cares why they got away with it?
the user needs to use some judgement on the information they receive and have some critical thinking skills
how?
errors can be so subtle that it is not possible to recognize them unless you spend an hour researching every fact presented. at that point, what's the benefit of AI? nobody is going to do that.
google would be right to remove all AI results from germany
By checking the citations rather than taking what’s generated at face value.
If it’s important, check it. If it’s not important, then it is pretty much just entertainment.
LLMs can be very useful in a general web search and save some time, but if you don’t put those literacy & critical thinking skills to the test and actually confirm anything, then you might as well not even have bothered with the search at all unless you’re hoping it can just replace all of your original thinking too.
If google didn't intend it's answers to be taken at face value it would just present the citations in a list of links rather than generating an answer.
Obviously the marketing point of the AI tools is it just gives you an answer straight up so you don't have to bother reading normal sources.
The AI summary is still useful for narrowing down the results, even if you fully check the citations.
> Obviously the marketing point of the AI tools is it just gives you an answer straight up so you don't have to bother reading normal sources.
To lazy people yes. That would be a marketing point. It’s not that though, so you use it to save time, but you don’t get to skip the verification step.
Google should not be publishing a statement that they haven't verified. This is different to listing search results links, they are the ones publishing the content here.
A journalist could not make up a harmful statement about someone and get away by saying the readers should have all read the sources. AI companies want to take all the benefits and profits, while holding none of the liability and responsibility for the harms they are causing.
"A journalist could not make up a harmful statement about someone and get away by saying the readers should have all read the sources."
Journalists do that all the time. We even have a whole collection of words to describe it. Muckraking for instance is probably about 100 years old. Its even in the Google auto-complete in the browser I am posting from.
If you re-read the article, you might see that it mentions that the citations do not necessarily cover the AI summary. The linked pages do not make the claims that the AI summary makes. That is the context of the ruling. Google made up the claims, and provided false citations. They are not, in fact, providing a summary, but a whole new narrative. Therefore they own it.
I read the article and I’m aware of the failure modes of Google’s AI summary. They’re actually one of the worst in the space on this shit which is why I don’t use Gemini and it’s fine that they get slapped for this, but what I was responding to initially was this:
> errors can be so subtle that it is not possible to recognize them unless you spend an hour researching every fact presented. at that point, what's the benefit of AI? nobody is going to do that.
Because if someone goes through the citations and it doesn’t substantiate what was generated, then what was generated was obviously bollocks. Being able to recognize those contradictions is an essential skill to using LLMs with web search at all. It’s not rocket science.
> Being able to recognize those contradictions is an essential skill
My eyes are brown.
I dislike coffee.
My phone is on the desk next to me right now.
One of these is false, and the other two are true. Can you recognise which is which? Or do you not have this "essential skill"?
When you're being given information about a topic you don't already know about, there's no skill to be able to recognise which pieces of that information are correct and which aren't. Either you know the information already, or you don't.
nobody has that skill. in order to recognize a contradiction you have to already know something contradictory. if you don't then you can't recognize anything. the only other reason to make you check is that you are very suspicious of this AI thing. you and me are, but who else is?
you can repeat a thousand times that people should not trust an AI summary and they will not get it. they have neither the motivation nor the energy to do that research.
it's like saying people should not use a computer if they don't know how to keep it secure.
you have to look at the reality, which is that people are not educated or critical enough to use AI safely. and that misinformation can cause people to get killed.
just yesterday there was a post about a man being falsely accused of a crime by AI tools and losing his job and his family as a consequence.
these mistakes destroy lives, and most people are far to trusting to use AI tools safely
Important for whom exactly? If it's you who are called convicted pedo by Google AI summary, it's you who has vital interest in additional research but not me who reads it. There's intentions mismatch. Which probably would destroy your life, and you won't call it "entertainment" then, I think.
If those are things you legitimately care about before you spend one penny on a T-Shirt, then you are. Or you did your research before hand. Or you’re just not buying the T-Shirt.
Or you don’t care about those things at all, and you will buy the T-Shirt that’s in front of you right now rather than wait later and buy one that better reflects your supposed values when you’ve done an appropriate amount of research. Using AI may even reduce the amount of time you spend on that part.
Your T-Shirt buying patterns & values are not my concern though.
You are confusing who is the party that is injured--as many in this thread are. We are not talking about the consumer who buys a non-fairtrade t-shirt, when he would rather had a fairtrade one. It's about the t-shirt producers who is legitimately fairtrade but whose business is now in the shitter because of a lying AI.
LLMs can’t lie. They are incapable of telling either the truth or lying. To the extent that they are in any way useful, it’s recognizing that they are text generators attached to crawlers and other tools that can with the right inputs produce useful generated text that may also incidentally be correct or incorrect.
Businesses might (well, will) suffer because people are misusing AI, but it is a misuse to do anything with it without an additional verification step.
To be clear here, I have no issue with Google taking it on the chin in cases like this, but what the comment I was originally responding to had this:
> errors can be so subtle that it is not possible to recognize them unless you spend an hour researching every fact presented. at that point, what's the benefit of AI? nobody is going to do that.
And my point is this: if it matters, verification is not optional. If it doesn’t matter, then fine, skip the verification step, but if you’re taking whatever text is generated by a GPT at face value without understanding what that is or being able to determine the source inputs for the “claims” it outputs, then you’re part of the problem because sometimes the source is just a GPT-generated web page, and that’s obviously not trustworthy. Sometimes it’s a MediaWiki site page that doesn’t actually exist, but because it’s MediaWiki it’s not going to return a 404. Using a tool requires understanding it including its failure modes, and in the case of LLMs that means: trust nothing, verify everything.
> at the scale that google is at, it doesn't make an ounce of sense to hold a company liable for a potential mistruth
If a Google employee (like a support agent) says a mistruth, the company is liable and you can sue. They can’t just say “hihi oopsies our support agents are useless”
Google can be reasonably expected to not push pirated content to top if someone is searching for the “big hit music download” because they might be held liable for helping people with illegal downloads. But they shouldn’t be liable for misleading people? Being sued for billions of dollars by corporations vs millions by common folks is the difference
There's also a difference between search engine results and this. Google is publishing their own text with false claims, not linking to another page that hosts it.
Should google also be allowed to assasinate a couple of people per day due to their scale? If they can't manage liability at their scale they they should scale down.
At the scale that google is at, is the exact the reason they need to be hold accountable for any misconduct because the impact is so big. If google just display third party content only, then yes, users need to use their own judgement. But if google itself generate content instead and put out claims, due to it's scale and reputation, those content needs to be true because most of the people put trust in Google, so any untrue or false information especially defame others, Google is 100% liable, period. You clearly don't have the clear mind or mental capacity to analyze this situation, probably very slow.
That's not a fair comparison. If oil companies would get sued for every leak, they had to face millions of law suites and wouldn't be competitive anymore.
Only referencing America, but professional liability for doctors, engineers, lawyers, etc isn't based on perfection. It's based on a reasonable effort.
So Google could, for example, switch from a tiny "this could be wrong!" byline to having the AI be less overconfident every freaking time regardless of whether it's spouting made up crap or actual facts.
The scale doesn't sound like a way out. If your company expects to get away with doing the wrong thing where smaller companies can't, then the solution isn't to continue getting away with it.
I'm seeing this sentiment more and more these days and it's worrying. Essentially people are starting to believe massive corporations should be able to get away with more than individuals. This is completely backwards considering the enormous impact a corporation can have, but that's what people are starting to think. I've heard people argue that corporations should be able to straight up lie and deceive to protect themselves, which is something you would not accept from a person.
It's probably got something to do with the billions/trillions of dollars corporations and industries have to push their chosen narrative onto an uneducated public. They've successfully turned a lot of people into suckers.
Corporations are you. Corporations provide a service to individuals, often in exchange for money. Restricting corporations from doing something is the same as restricting people from receiving that as a service.
Banning faangs from doing something also bans small companies from doing it, and regardless prevents the user from receiving it as a service. You shouldn't let anger at large companies excuse stupid regulations.
It's entirely possible for legislatures to attach clauses to restrictions that make them only apply to businesses above 50 employees, or above $1M ARR, or whatever.
I don’t know if critical thinking comes into it. If Google tells me “Company X is a terrible company that cheats its customers”, I don’t automatically know that’s false. It could very well be true. If I’ve got to look at the rest of the search results to figure that out anyway, what’s the point of the AI summary?
The problem with "the user" argument is the spectrum of users. There are different skills, capabilities, and intelligence. Frequently we wave our hands and say exactly this, critical thinking. But, not everyone is capable of that, nor is everyone capable to the same degree.
As a society we decide. Are we embracing all users, are there basic rights and assunptions? Do we only enable some?
As a free (as in cost to end user) system, Germany is arguing that their social compact raises the mininum bar. Frankly, thus might help drive a rush to increased accuracy for AI- tech finds a way. Equally it may hinder - beaurocracy creates barriers.
I'd love to be able to rely on these search results. I see them ad the same prior set of inaccuracies whereby I have to do more research. At least now there's a summary and direct links to the supporting information. But equally, we're primed with the information in the summary.
> Frequently we wave our hands and say exactly this, critical thinking. But, not everyone is capable of that, nor is everyone capable to the same degree.
It doesn't help that there have been active efforts for decades to prevent people from learning and developing critical thinking skills.
We oppose the teaching of Higher Order Thinking Skills (HOTS) (values clarification), critical thinking skills and similar programs that are simply a relabeling of Outcome-Based Education (OBE) (mastery learning) which focus on behavior modification and have the purpose of challenging the student’s fixed beliefs and undermining parental authority. - The 2012 Texas Republican Party Platform (saying the quiet part out loud)
> It doesn't help that there have been active efforts for decades to prevent people from learning and developing critical thinking skills.
This very much depends on where you live between state (US), and countries. Where I live, it's the complete reverse, critical thinking is baked into the population, into learning, into nigh everything. Our challenge is the complete lack of privacy, sadly.
So if I make a script to spam everywhere with 0.1% odds that is fake I can’t be sued? Just because I spam millions of times per day means I shouldn’t be liable for what I do right? But someone saying one time should go to prison. Makes sense /s
People will complain, but eventually Europe will still be in advance regarding this kind of law. It's annoying and sometimes slows down innovation but US companies are just doing whatever makes money without restrictions...
I don't even know that I would call it slowing down so much as constraining/focusing innovation.
There's three basic paths for a company hit by this ruling to comply:
1. Stop showing users generated content.
2. Figure out how to generate the content with more quotes and attribution to source websites, to regain the protection offered to search engines.
3. Figure out the hallucination problem, so that every statement in machine generated content is true, or at least defensible.
If this ruling forces companies to put more money into #3, whereas now they're coasting on good enough, I'd say it was speeding up innovation.
There was already a ton of collective incentive for #3, I don't think the companies are choosing to "coast on good-enough."
Rather, they are stuck unable to do that much better, unwilling to admit (especially in a way that might spook shareholders) that it's a hallucination-machine all the way down. They're playing for time and market-share while hoping some unspecified and inherently-unpredictable new discovery arrives which will be compatible with their existing infrastructure and investments.
Quite frankly, in this very instance, Europe's slow tech adaptation and comparatively little investments are a blessing.
At this point, there is nothing to gain but volatility. Let the US figure out the economic and social disruption, and then adopt whatever sticks. There is no rush. The useful parts got little moat. It's not like there is a magnitude of difference to justify the expenses and risks of frontier operations, in comparison to open models, or smaller players. Chances are, all of this AI business will settle on local models for most use cases.
The US may get their first trillionaire, but let's be real, that's not because of innovation but corruption, exploitation and raging wealth inequality. If the AI economy is not panning out as "projected", there is no recovery. That money got converted to heat and single-use hardware trash. Not worth risking pensions and the social fabric over "the tech edge".
This is the kind of legislation that only exists to help the rich and powerful and harm those below them, it's certainly not an example of where EU is doing better than the US.
Or what, do you think it's a genuinely good thing that hosting negative reviews is essentially illegal in Germany?
The irony of an article that makes a false claim about what Google was found liable for.... and that very few are fact checking it :)
The law they broke was a law protecting personal and business reputation against false statements of fact. Essentially no one can say I might be wrong, check yourself, but X is Y if that claim is essentially defamatory.
This is pretty good, I hope googles approach is to make sure they don't end up making statements of fact like they did and use more appropriate wording like according to X.... with direct disclaimer that they can't verify it. Even better that they look court documents to find any legal ruling and point people to that too.
How much does Germany follow case law? If this could set a precedent, it's worth noting that anyone can generate these AI overview responses and they're wrong like 9 times out of 10 only.
In common with most of Europe, German legal system is not based on case law. It's more firmly rooted in formal laws and regulations and judges are not required to follow precedent.
The difference is quite subtle in common law v. continental law systems.
In continental traditions, nobody really says a higher court decision is binding in the same sense as they do in common law. But even in common law, lower courts have the power to distinguish a case from a binding decision from a higher court. Or later courts may state that a particular argument used by a judge in the higher court was just 'obiter dicta' and not part of the 'ratio decidendi'. In continental systems, for most court processes, higher courts do have a strong persuasive power without formal binding (sometimes it may be stronger than your average "obiter dicta" in common law). So, there is a whole spectrum in obiter dicta as well.
This decision was a preliminary injunction only from Landgericht Munich - you will have a final injunction order:
https://the-decoder.de/wp-content/uploads/2026/06/26_O_869_2...
In Germany, you also have the Oberlandesgericht and the Bundesgerichtshof as a higher venue for civil cases (and the Constitutional Court as well).
So, this is not like "the law has changed" at all.
(I'm not a German qualified lawyer, but qualified in a close enough continental system, and on my way to be qualified in a common law jurisdiction soon, that's why I dare to comment on this aspect)
They don't have to follow it but leading decisions (Grundsatzentscheidungen) still influence how a lower court judge might interpret the same law. Based on providing legal certainty instead of stare decisis though. It's not a must, more of an... aid
Still - a judge on another case will probably factor this one into their decision, but if they have a radically different view or the case is sufficiently different, they could just as well come to a different conclusion.
Does formal law systems lead to a fraction of the lawyer work hours and ambiguity in spirit of the law? Been noticing legal LLM tools which have to crunch entire libraries.
It seems like this would ostensibly be illegal in most places. The reason e.g. social media sites in the US can post whatever and not be libel is because of section 230, but that only applies to user generated content. It's the reason that e.g. newspapers can be sued for publishing libelous/defamatory/etc content, yet the same author of such can post it on social media and those sites are legally immune. Kind of a weird law - especially given contemporary censorship regimes, but it's easy to understand the motivation.
But if you as a first party are publishing something directly, like Google is here, you're generally liable for what it says.
I agree with your common sense take of how it should play out, but Google has and will argue Section 230 protection for AI overviews, eg in Wolf River Electric v Google.
Google is breaking numerous things, so even if the article were incorrect, the underlying issue still remains. It is time to dismantle this Evil monopoly completely.
Also I am not entirely certain your statement is correct - which false claim does the article make exactly?
How could anything else make any sense? Platforms are getting used to provide dangerous broken products and get away with it. There should be some limit to it.
When I was a kid it was possible to buy any foraging books from a store and they had a minimum quality. Is that so difficult to achieve? Is profiteering not punished anymore?
Disclaimers and user acceptance does not remove liability for slander, particularly against third parties.
In fact, in most EU countries "the user acknowledged" works only for a very small subset of stuff, precisely because our lawmakers know that the strong party in a contract would use that to get away from every legal obligation.
This is even worse, since the defamation in question is against a third party. This would be more like: by reading this comment, you transfer all of Bob's property to me.
If they do so knowingly, and harm is caused, then yes. Are you suggesting we should give them a pass from years of acquired jurisprudence simply because they hold a particular title?
And what institution gives out the licenses for journalists and scientists? Is it revokable?
The danger there is that the government will demand access to a journalists sources. "Leak reports Area 51 has aliens!" "Oh yeah? Prove who told you that or we'll arrest you for lying"
Or selectively. "Vaccines cause autism!" "Okay" "Vaccines don't cause autism!" "Prove it beyond a reasonable doubt or we'll arrest you for lying"
Choosing the answer for you rather than leaving it to the user is a tremendous power and the court correctly diagnoses it comes with responsibly to minimize harm to others in society.
I've also observed that the AI summary on the google search page is incredibly stupid compared to the results in the actual Gemini. The Google search AI is like the dumbest lightest model simply rewording the search results. It will take a random reddit comment, strip it from it's context and present it as absolute fact.
I’ve seen it directly contradict the citation so many times that i disregard the text and just click the citation or scroll past every single time. Just today i caught it making up the date for an event, and the citation had accurate information when clicked through.
It’s super easy to catch on dates and numbers, but it gets other details wrong all the time too. But so many people won’t be double checking the results.
Even those results have a lot to be desired, it is just buried deeper in the insanely verbose research report and impressive looking amount of sources you see move past.
I recently have had a close look at the various "deep research" options the big three (Anthropic, OpenAI and Google) offer. None of them are exactly transparent about how they perform searching other than the "research plan" the present upfront the and shitload of sources they show you (which, to be frank, seems to be clever UX/marketing to make it look extra legitimate and impressive). Which is already a worrying sign to me, as you can't audit the process itself properly. But even with the lack of information available on the front-end I can still see enough that worries me. A few examples:
- "Sources" are taken at face value almost no critical look at the validity of the source, the context it is placed in, etc.
- A lot of sources I know are legitimate are rarely included while a lot of listicles, low effort "reviews", etc do make the cut.
- In multiple instances when looking closer at the research plan and the "hints" they show during searching it becomes painfully clear that often enough they start with an answer in mind based on training data and try to validate that rather than actually researching the data itself.
- Subtly different prompts that by all means should still produce the same factual outcome actually provide wildly different results. This one probably relates to the other points.
In addition to all of this, I also am 100% convinced that AI powered search is incredibly expensive[1], more so than traditional search. In my mind this increased cost eventually will need to be paid by someone, which likely is going to be the user. Since the process is non-transparant I am not confident that the results will not end up being polluted by sponsored deals, etc. There is simply no way in my mind that this is going to end up well for us users.
[1] A while ago I have experimented with creating my own deep research flow with the idea that I might be able to do something with local models. To limit costs I used a SearXNG instance for searching, setup playwright for browsing sources. Using an agentic flow with agents making all the various calls and dispatching other agents ended up eating A LOT of tokens. Even when I did switch to a non agentic flow where each step is orchestrated by code calling on LLMs with simple prompts to validate results still ate a metric ton of tokens for the simplest search query. Mind you, this was not even doing actual deep research but only a few simple search queries. Ironically, google models also did seem to have more trouble coming up with good search queries compared to other models.
I want to add my interpretation of the phrasing "rewrite people's awareness" to make it read less tinfoil-hatted:
"rewire brains for AI dependency" (for money and power reasons obvious to everyone).
In contrast to "secretly implant an agenda about non-AI subject N", which is complicated enough that AI companies are still too out of control to be attempting yet.
When Google Search is quoting a 3rd-party website that happens to have bad information, that's not on them. Blame shifts to the 3rd-party. This is Google's privilege being a search engine.
When Google operates as an answer machine instead of a search engine, the privelege doesn't apply anymore. There's no 3rd-party to take the blame.
> According to the court, the Al mixed up information about other, genuinely sketchy companies with the plaintiffs and drew connections that didn't appear in any of the linked sources.
Implying: if the false claim was found in the sources then it would be protected speech.
There is nothing special about "answer machine" versus "search engine". You are making that part up.
You'd think so, along with other countries that have defamation laws. But there's no indication of any penalty, and Google wasn't even made to pay all the legal fees. Perhaps their business model (if there is any) can cope.
In sane countries, it's enough for them to post a disclaimer ("This is AI. AI can make mistakes. Check all results.") Which is what they do.
Overregulation, at best, is a good way to guarantee that your country won't have access to interesting and useful features and technologies. At worst, it's a good way to guarantee that the twenty-first century will belong to the US, if not to China.
Okay then, CamperBob2 is a scammer. Many users report this person has stolen money. (+3 sources)
I can make mistakes. It's on you to fact-check my claims.
Do you think these are harmless statements? Does the disclaimer suffice? If I was Google's AI Overview, do you think 100% of people will check those sources?
There is nuance here, and it's not going away because AI and innovation.
Yes, I do. An assertion made without evidence can be dismissed without evidence, whether it comes from Google or from you.
If you demand perfection, you will receive nothing. Why is that so hard for people to understand? The world simply cannot work the way you say it should.
You can easily damage a persons reputation, make them unemployable or social outcasts, create an environment damaging to their mental health without any proof. Believing that the public will dismiss claims, just because you stay that there's no proof is rather naive.
A Danish newspaper at one point falsely claimed that a named person had raped a child, with no evidence. The only way that person escaped public judgement was by going into full attack mode and publicly suing and attacking the newspaper and the reporter. You have to be an incredibly strong and resourceful person to do that and not just go into hiding. Also one thing is suing a Danish newspaper, imagine the legal team, the resources, you face if you're trying to take Google to court for defamation.
If instead of scammers, it was 'registered sex offender'. Each time someone search your name and surname in Google, it show a picture of you next to the words 'registered sex offender'. Do you think your neighbours and the parents of your child's friends will do their due diligence or will they prefer not take any risks?
Well, I would certainly need to warn them that there was some sort of psychotic stalker on the loose. This case is nothing like that. This case doesn't even rise to the level of someone scrawling For a good time call <my phone number> on the wall of a gas station men's room.
The point is, if you take Google AI summaries seriously, that's a "you" problem, not a "me" problem. I have a rather generic name, so it's safe to say it shows up in some pretty foul contexts. Life goes on.
(Shrug) That sounds like their problem, if they reflexively believe everything Google's AI tells them. No one I respect, care for, or work with would believe such a thing without additional justification.
Anyone who does accept Google's AI output blindly will soon find that their mistaken opinion of CamperBob2 is the least of their problems. There is a reason Google goes (well) out of their way to warn people that the results may be wrong. That should be sufficient warning for reasonable people of good faith.
Are you really that ignorant of the past? There are so many cases of people suffering from wrong accusations. They get death threats. People harass them on the street. Throw stones into their windows. Beat up their kids. Trash their car. Deny them jobs, or leases, kick them out of their apartment.
You have no idea what it can mean to end up in such a situation.
The source of the accusation is arguably more impactful than the accusation itself.
No one with a straight face would compare an AI overview saying someone is an asshole to the New York Times running a story saying someone is an asshole.
If the New York Times wrote someone was allegedly an asshole, and Google Search condenses to presenting that as a fact, I’d bet you my left kidney that the general public would be quick to take that at face value.
And your beef is with the NYT in that case. Not with the neutral aggregator who includes a specific disclaimer that its output may be incorrect due to unavoidable technological limitations.
The ones who were defamed are companies, and the ones who don’t check the AI generated response are their potential customers which won’t buy from them.
It is obvious that the defamed companies are the ones having a problem, not the ignorant viewers.
Why should those companies not hold Google liable for that outcome?
I don't know about that. Imagine I want to sort people into employable and unemployable based on AI summaries. After filtering the employable pile is large enough. You being in the unemployeable pile and me accepting it blindly, is not my problem. If there is any at all.
Good. This should be taken as the precedent for all economies: If you promulgate demonstrably false information to somebody's detriment then the owner and operator of the machine has to carry the liability.
I very much hope we don't see attempts to re-write T&C to avoid this liability.
Regardless the scale, big or small, if you produce and spread untrue information to defame others, you are liable, period. These AI content are purely generated by Google's half baked AI. Not any other third party. So Google is 100% liable here.
If you just display third party content, that's one thing, but if you generate content yourself and those content are false, harmful, that's on you. Google should be shamed to do this kind things recklessly and irresponsibly.
First they had links to other sites. Then they put the information found on other sites on their own site. Then they took the information from other sites, trained a LLM with it, and put it on their own site without even linking to the source.
That's apparently already changing in the EU, where software vulnerabilities mean the company is liable for damages. The only way out is to straight up not make any money (not just from direct sales) from the software.
Is the burden of proof on me, the developer? Do I need to prove in perpetuity that I didn’t get a job or a free flight to talk at a conference because of my free software? (Which had a flaw that hurt someone)
This makes sense to me. AI is amazing tech, but it's being oversold to naive public, either on the back of hype, or cynically, knowing they can do it with inpunity, (I'm not sure which). HN users have a way better than average grasp of what AI is and we can be skeptical of results, the general public has no clue.
I'm surprised this is even a thing. After all, you go to Google not for the truth, but to search Google. Since when is truthiness the "guarantee of service"?
You're not even paying for a google service, search is free... You might be the product, and your data, but you didn't directly pay for a service and they didn't sell you a fake service.
I'm not taking Google's side, this isn't about whether it's right or wrong to rob websites of traffic, this is about AI's returning search metadata.
But I'm surprised that they lost this argument, and the line they took in the first place.
The Internet isn't made of fact checked data, it's crowd sourced. How can anyone be liable?
That is exactly the point of the ruling, though... they are saying that AI summaries are NOT the same as search. If Google was just returning search results, and then users clicked on a website and read the content there, Google is not responsible for the content.
If instead Google gives you an answer right there on google.com, without going to another site, they ARE responsible for it.
Not precisely. The issue at hand isn't just that Google displayed the AI summary, but that they authored it, making them responsible for its contents. If the defamatory content had been in a snippet in the search results, they would've been fine, because that clearly has another author who can be held responsible. The AI summary has no other author than Google; therefore, they're responsible for what it says.
(What's the alternative, after all? Having no one responsible for what the AI summary says is clearly untenable.)
What don’t you understand? Those websites that defame a company are liable for that defamation. In this case Google defamed a company in its AI summary and is this liable for that defamation.
but if Wikipedia itself writes harmful content such as encouraging people to drink bleach, then wikipedia is liable. Google now generates its own content with AI, that defame others, so Google is liable.
And those tons of websites are liable for their misinformation. It's probably not worth suing some random blog because the author probably doesn't have money or lives in Russia. But Google has lots of money and a legal presence in almost every jurisdiction.
It's why people say "Donald Trump was held civilly liable for sexual assault in the E Jean Carroll case" instead of "Donald Trump raped E Jean Carroll"
That's the difference between returning search results and interpreting the information and summarising them. If a newspaper says 'so-and-so has been arrested for theft' it's not the same as them summarising to 'so-and-so is a thief', the second is potentially libel. Why should Google be held to a different standard?
Google itself is more trustworthy from a normal person perspective as they use it a lot.
None of "AI" companies call their apps "Entertainment fun text generator". They are call them serious names, use words like "intrllegence" and "thinking".
So yeah I'd think if any of "AIs" start to recommend to drink some bleach or take a flight from a 10th floor window these companies should be liable.
I think it's very clear that Google's AI overviews go far beyond just searching Google because they often incorrectly compile sources to come up with an incorrect answer. For example of this look at the comment I made in this thread
The question is whether Google is publishing false claims or relaying other people’s false claims. The court found it to be the former which makes sense to me.
I go to Google to search, but get spammed as if I wanted to talk to a chatbot (and a very poor quality chatbot at that).
This is a gigantic own goal for Google. The average person’s impression is that Google AI is much worse than ChatGPT, even though that’s not actually the case. Google is shoving a terrible model in everyone’s faces.
It's interesting that the "users can check for themselves" defense was attempted, Google search is so awful now that it's nearly impossible to find anything but ads and ads disguised as links. Even watching people try to use it is infuriating.
The confusion is due to blurring of distinction between the roles: Tool, Tool user and Tool provider. Traditionally, a human-agent who can have an intent and trigger an action is held liable for the consequences of the actions, not the tool. That human agent can be the tool user or tool provider.
Tool user is liable in the case of misuse unintended purpose of the tool.
Tool provider is liable when the use of the tool, by design, causes unintended effects despite proper use of the tool.
A simple "AI may make mistakes" line under the box will not help while the box contains false information. The specific information (lines or words) should not be provided if that's a mistake of false.
The article shows a really good point: statements must be accountable on their own, and cannot rely on "further research". This makes complete sense: at what point can I libel someone, and then dodge consequences with "if you do your research, you can get the facts"?
Another point from the article: they are not just aggregating content, but generating it. If you generate falsehoods, that are not even stated by your sources, of course you're responsible.
This can have significant impact to AI in Germany and the rest of Europe, but it's good to question it and hold people accountable.
Common sense ruling to me. AI != Search, because LLMs are producing statements rather than search results the user has to interpret for themselves. It’s the difference between searching a library’s card catalog for a topic, and a piece of software telling you its answer.
The second these things came on the scene and confidently spoke lies at times, I knew this sort of lawsuit would be inevitable. Nice to see Germany got it right.
If it's consistently applied, any AI agent provider has to comply with cease and desist letters that tell the company to make specific false claims
The arguments of the ruling should generally apply even when the AI agent makes false statements it wasn't notified about. But in that case the defendant might have a stronger claim about not being able to reasonably ensure the correctness of all statements, and having taken reasonable measures to ensure correctness. Google couldn't really claim any of that after ignoring cease and desist letters about the false claims
Due to costs of running frontier models on every search request, Google simply does not: the failure rate is so high when you are just expecting an objective output.
Imagine a search for your name resulted in an AI summary saying you are involved with child-trafficking because low capability model linked your first name and perhaps a couple articles on supporting children non-profits to it — and then offering that in a convincing sounding summary right at the top!
And the more you protest the more your name will be associated with child trafficking. Streisand effect multiplied by LLMs being not good in dealing with negative information.
I think it should be relatively possible for an LLM to know when it's talking about something in the risk zone of libel. And when it identifies such a case it links direct to source material rather than trying to fabricate an answer itself. This is much how a journalist works, refusing to make claims that aren't provable.
Someone needs to hold the liability at the end of the day. People are experiencing real harm from false claims LLMs are spitting out.
If you give a language model, empowered through an agent, the ability to publish information on your behalf, and it publishes false information which causes either direct or even indirect harm, and you fail to correct it, then yes.. by every conceivable definition already in law.. you are a criminal.
If you knew this was all possible and you did it anyways for personal gain then you are additionally negligent which may add aggravation to your charges.
As the begetter of the AI, the German court held Google liable for its AI's doing. Common sense ruling.
The ruling also lessened the free speech rights for AI. This is a big one. In conjunction with holding an operator liable for its AI's doings, that will lead to interesting cases where conduct that would have been previously protected under free speech rights will become a liability. Basically, machine-based cognitive capability becomes a liability when it is customer-facing.
Their AI overviews are pretty commonly wrong, so this seems like it might hurt them a bit. Guessing they'll just block them in Germany or throw a bunch of disclaimers around the statements.
Anyone know if this ruling applies to answers generated by AI chatbots, such as ChatGPT/Gemini/Claude?
All three have the ability to perform a web search, then compose a reply based on the search results. Pretty much the exact thing that Google AI Overview does. This ruling may make them liable for false answers.
> Pretty much the exact thing that Google AI Overview does.
No, the article implies the court’s logic is that the AI search results are presented as search results and that’s a big part of why they are liable. It seems like the court (again, according to the article) does not find the disclaimers that Google has slapped on the AI results compelling because again, it chose to represent these as a summary of search results and it is aware of the failure rate.
> The court also found that the AI overview made claims "that are not even made in the search results." None of the linked sources drew any connection between the plaintiffs and the shady companies the AI mentioned. The court called these "the defendant's own statements."
> Google built the AI, Google offered it to users, so Google owns what it produces, "because it alone has influence over the AI's offering and the algorithms with which the AI operates."
Google does not, as a general rule, control the actual content of search results, but usually there’s a distinction between the ranking and presentation of the results vs. the actual content. In this case, the court is basically saying, “You sold this to people as a search summary, you know it might be full of crap and you chose to do it anyway. No, you don’t get to claim the equivalent of a US safe harbor defense.”
> You sold this to people as a search summary, you know it might be full of crap and you chose to do it anyway
There is a subtle difference in stating it as a search summary compared to an opinionated answer. Most users are always going to treat it as a response from google instead of search results where the user is still responsible for understanding and come up with their own interpretation.
This is probably the right step in some sense to make one liable for their statements/assertions.
Apparently, if they were search results, they wouldn't have been liable, since there's an exception to defamation laws. Without any exception, defamation is defamation, it doesn't matter how it's presented.
Yes, if it was that a search result returned a defamatory article that Google had nothing to do with outside of indexing, it is likely they would not be found liable. The court is clearly trying to make a distinction that the AI search results are produced by Google and thus they can make an editorial decision on whether to publish it despite knowing that it is potentially defamatory.
Google does remove defamatory results I believe at least partially in response to being sued. However there is a distinction if they have been informed it is defamatory.
Pretty sure this ruling will be used as a precedent in cases against the other providers really soon. As I understand it, there is a legal cottage industry that brings digital related cases (eg. Copyright) in Germany.
Google generated those content, so Google should be liable to its own product, that's different from the third party links they just simply gathered and displayed, totally different things. I wonder how many victims are there now.
The result I get has an entire section dedicated to scammers using the company name. The only links in that section go to a wikipedia page that doesn't mention any scams and a police help page that doesn't mention the company.
Do the links actually support the statements? When I've followed such links, it's generally been a roll of the dice on whether they do support the AI's statements.
Yes. The links are to isolated threads on a rail (model) forums (apparently, this publisher markets books/magazines related to railway models).
It's hard to see if the individual complaints really support a general problem, or if this simply the only result that talks about "scam + business-name". Probably, the latter.
The same problem happens on google search, if you look "<obscure false fact>", you'll get pages mentioning that false fact. If you fall into the trap of confirmation bias, it leads you to think the false fact is in true.
If people do say that, it's a true statement and thus fine. You are allowed to report that regardless of the truth of x/y/z/a/b/c
The instance of this ruling people apparently did not actually say any of the offending claims. 'The court also found that the AI overview made claims "that are not even made in the search results." None of the linked sources drew any connection between the plaintiffs and the shady companies the AI mentioned. The court called these "the defendant's own statements."'
One that doesn't maim/injure/kill you is a pretty good standard. And before you call bs, look at all the foraging and chemistry books that are for sale on Amazon that are AI.
why are those ai chemistry books any different than the anarchist cookbook which can also be bought on Amazon? actually now that I think about it a faulty chemistry book might be less dangerous than a book that teachers readers how to make explosives.
Certainly, if this is pointing to the actual pages where the actual people express these things. Otherwise that's equally unfalsifiable claims, could be completely made up or actual truth.
One way to formulate things that would be less would be "once support a time, in some fabulated world, it's not impossible that some imaginary character would say something following some reason." But then, of course this is not aligning the the deception scheme pushed by companies putting in their interface that the "machine is thinking hard for you".
I don't fully understand it. Are they liable for content made up by their AI? Or are they now liable for content written by others and summed up (correctly) by their AI?
From my reading the German court's ruling is that Google are responsible both for claims they made up and claims they rephrase from somewhere else. Because the claims appear as a statement of fact made by Google. If they showed it as "User X on site Y claims Z" they would not be held responsible. Because that's how search engines are understood to work. If they make a wrong quote, they would be responsible for that though.
So my understanding is their unreliable AI summaries are a legal liability for Google in Germany and people can request corrections through the courts.
Oh, I just found out that my Google search doesn't show AI summary anymore! I tried many search queries which typically will show an AI summary, but it only flicked on briefly then disappear entirely. Obviously, Google has reacted quickly on this ruling!
I just tried Google search in Germany on my iPhone: AI results AND the disclaimer was behind a „show more“ button i.e. the may not be any disclaimer (and when shown it was in a small font).
I have a business where our support email is recommended when people are searching for how to cancel a completely unrelated scam subscription that is showing up on their bank or credit card charges. We get emails almost daily from confused people.
Google generated those content, so Google should be liable to its own product, that's different from the third party links they just simply gathered and displayed, totally different things. If you are a victim too, reply below.
If companies can be held liable (in spite of very visible disclaimers, ToS, and usage policies) for the output of non-deterministic software, isn't this just a soft ban on the deployment of non-deterministic software?
> (in spite of very visible disclaimers, ToS, and usage policies)
If you sell food, in a food stall, labeled as food and you add a disclaimer that it is toxic and will make you sick. You are still selling toxic food and you are liable for it.
Google is pretending to give answers to your questions. They offer you a service about answering questions. And then they add a disclaimer "we do not answer questions just write bullshit". That is still fraud and Google should be liable for it.
> isn't this just a soft ban on the deployment of non-deterministic software?
Tetris is non-deterministic and it is not banned like millions of other programs. I do not follow you.
To add to this, a Google search now is answering your question in an incorrect way rather than merely bringing you to a site with incorrect information on it.
They are also no longer covered by safe harbour provisions because it is them answering it, not some content they refer you to.
It requires the claim to be made with "willful disregard for the truth". Notifying someone, especially with a cease-and-desist on fancy letterhead, makes it legally clear that they know better, and thereafter would be definitely libelling you (assuming the claims are in fact untrue and harmful). But you can still sue them for the claim prior to the notice, you just have to prove they should have known better prior to making the claim.
Libel/defamation laws vary wildly across different countries. Sometimes true statements can be considered defamation, sometimes not. The same goes for intent.
This silly game where the EU invents its own rules and tries to impose them globally must stop. Their business is not needed, and the collateral damage with regards to censorship, manipulation, and erosion of fundamental liberties far outpaces any value their continued participation in e-commerce might bring. This sort of paternalistic and insidious bureaucratic encroachment into every aspect of life is completely antithetical to everything the internet should be. We don't owe them anything, and pretending they (and the UK) have some sort of standing in determination of how the world works simply accelerates the degradation.
The solution is simple. Block the EU from accessing any of your services. They can make their own search and social media and digital marketing and AI.
Good luck to them. VPNs will boom.
I can understand the UK struggling with their imperialistic traditions and so on, but you'd think Germans at least would have some humility in trying to project their global ideas and ambitions on the world at this point.
How is this an example of trying to impose rules globally? Google is operating in Germany. Their German operations have to follow German laws. That's how every country operates.
Unless the courts here made the ruling incredibly narrow somehow (only referencing search engines maybe?), how does this not just ban AI in Germany overnight?
Every AI model can make something up sometimes. Over millions of daily calls, it's essentially impossible for the technology to be guaranteed correct 100% of the time.
We are actually a victim too, in the same situation as these two publishers. Google's AI makes up false accusations in search result AI overview about our business. We have sent countless feedback to Google in the past 4 months, contacted their legal team 3 times, all got ignored. Google's AI just pukes untrue, misinformation about us, every time the page is refreshed, AI just makes up new accusations and false information, links us to bad actors baselessly, if you refresh the page 10 times, there are 10 different versions, seriously damaged our revenue and reputation. We have filled complaint with the state AG's office, FTC and DOJ. If anyone in legal can help us, please reach out. Thanks. kevinsuntopdev@gmail.com
That reminds me of the guy who was declared dead by Google and then miraculously wrote a blog post mentioning that rumors of his death were premature. Google (and others) really should be held liable for the fake news they're spreading.
This is actually a good thing. Google's official mission is "to organize the world's information and make it universally accessible and useful."
Now if that information is BS and cannot be relied upon, that’s really bad. Leading people on and not delivering? Honestly, Google themselves should have been on it and not the German government. It’s a bad look for them.
Aside: I’ve noticed their AI mode is pretty pathetic for troubleshooting something. 50% of the times the first response is riddled with inaccuracies and mistakes. Repeat prompting is absolutely necessary (so do not expect to one shot anything).
Also I will admit that I still find myself using it because I’m lazy, and it’s easier to talk to AI to get the right answer. Searching organically is hard these days with the volume of content having gone up exponentially.
Search engines have that exemption for results because they don't control the content of the sites they index. That's not the same for ads - Google can and should have more stringent review and KYC processes - but it seems they choose not to.
The original Google search that used inverted indexes and page rank and respected boolean operators was very good and let you actually find obscure information. Call this Google Search 1.0
Then in order to increase revenue Google dumbed down search very badly and made it very hard to find obscure information. Whoever decided to delete random search terms is someone I want to punch in the face really hard. Call this Google Search 2.0
But the way Google has integrated Gemini Flash into search is actually pretty good and is definitely an improvement over Google Search 2.0
Not exactly false answers. As far as I gather this is the same issue that prevents customers from making negative reviews about businesses. The business can just threaten a defamation suit and then the platform (Google Maps, Trustpilot, etc.) are pretty much required to remove the review [0].
So the only thing happening here is they consider Google to be the author of the AI answers and apply a similar sort of anti-defamation law. Perhaps it's a good consequence but the law that allows for this seems to be quite broken.
Overviews is not a good product idea and I think it was done for other deeper reasons. AI shouldn’t be pushed on people because AI can be high variance — it takes time to get adapted to this.
Not all disputes need to be adjudicated. It's undoubtedly a fact that the German concept of "defamation" primarily relates to situations that should never be adjudicated by any third party.
Genuinely, by far the most common situation where claims of defamation arise in Germany is any less than 5 star review left to a business on google maps.
The good thing is that google in Germany now shows how many reviews were removed due to deflamation. That should be enough for people to make their judgement when checking how many stars a place has
That's just the tip of the iceberg though. Google is the only party who meaningfully communicates this, and it took them years and years to introduce any meaningful transparency.
It's not like this exclusively affects restaurant reviews, every corner of German society is subject to this same evil censorship mechanism.
Because of the same rules, German restaurants also get to pick and choose which reviews stay up. They can literally take down any specific reviews they like.
A restaurant that mostly gets 1 star reviews will still show up with 5 stars on Google maps, as they will simply delete the reviews with less than 5 stars as defamatory.
And the solution to this is to go through the German political system. The courts can't just decide to not follow the law just because it's a bit silly, and for as long as the laws are the way they are, this unfortunate loophole for removing negative reviews will continue to exist. But clearly the law can have both positive and negative consequences.
But it also shows the number of removed reviews (in a range like 0-50, 50-100 and so forth). If you encounter a restaurant with 5 starts but a lot of removed reviews you know whats up.
We should be teaching people to be cynical of AI answers.
Even if the answers are correct, they could still be biased, incomplete, misleading, and all the other media-literacy things people should be looking out for.
This ruling seems to go the opposite direction; 'I am legally obliged to give correct answers, so I am always right, trust the AI'.
First comment that makes sense. Of course AI needs to be verified. Of course it gives bs answers. The answer is to educate the public not try to daddy then. I hope AI is completely banned from Germany for their protection since apparently their government doesn't believe its people know how to question AI.
I've found a fun and pretty reliable way to get Gemini to output incorrect information: Ask for a chapter by chapter summary of a book.
I first tried it to remind me of what happened in a previous book in a series that I was reading. When I realized it was either misstating plot points or straight up hallucinating, I tried it on a bunch more books to amuse myself.
Older classics are of course more accurate, but for newer or less popular books Gemini won't shy away from giving you a summary culled from misinterpreted Reddit threads and Goodreads reviews. It's like getting a secondhand account from someone who talked to another person who had read the book a long time ago. You get the general gist of it, but with some added flavor.
Even if you upload an entire epub of a book, the results aren't stellar. Rather than a Cliffs Note's quality summary, they're pretty sparse or leave out important bits of information. One chapter summary I got back made a point of describing what one of the characters was wearing, even though it had absolutely zero to do with anything else. Yes, that's technically a "summary", but not quite my tempo.
If Google wants to present summaries of websites in anything more than a very, very superficial description, they're going to have to improve their model's ability to understand context and importance. In theory, a novel is a self-contained bundle of text, so pulling accurate information out of it should be straight forward. A website is naturally going to be way more of a challenge.
All that said, I find the AI summaries from Google/Gemini to be quite useful and a time saver, but I know to always double check something if it's at all important.
My feelings about this rest of the scope of liability. From my understanding, Google is now liable if making false claims about a personal/business reputation. I like this idea in theory (key word).
However, I can easily see the slippery slope where in practice this means providing any AI response becomes too risky, and it becomes another money club used to extract wealth from big tech due to the current hysterical anti-AI moral panic.
Which would ultimately kill the ability of the German people to get access to competitive AI models.
I'm sure many anti-tech/anti-civilization doomers on HN will cheer this on. However, in reality it would do nothing to stop Germany falling behind, and continue its economic malaise/low productivity growth and social welfare collapse. When things in Germany get bad, Germans historically have tended to...ummm...cause issues for Europe. I would not take this lightly given the current rise of more polarized political rhetoric and German economy pivoting hard into weapons manufacturing.
It is not anti-AI, it is anti misusing AI. Clearly you are trying to manipulate the situation here. If your AI is half baked and produce garbages, you should be liable for the damage of those garbages, that applies to any product or service on this planet, not only just AI.If your product or service is defect, that causes harm, you are liable.
But to loudly proclaim your love for this idea as most have here seems a bit premature given there are potentially massive downsides to applying legal liability to an AI model, if not done extremely carefully.
I'm just trying to move a bit beyond the emotional first order thinking. You might see this as manipulation if you love just swimming in emotional vibes, but I find this website becoming quite boring lately due to the lack of debate beyond this first-order thinking.
As someone living in Europe, watching a bunch of rich American engineers fanboy European over-regulation which they aren't actually being subjected to, is super cringe. It's the epitome of the principal-agent problem.
The slippery slope of being held liable for an AI overview leading to neo-Hitler is certainly a take, and a ridiculous one at that.
It's one thing for an LLM to get things wrong. This case is not that though. And if big tech can't make sure that their models don't libel companies and individuals then good riddance. Whatever diminutive economic advantage one can get from "competitive LLMs" probably isn't worth it anyway, especially with all of the other disadvantages of these "competitive LLMs" compounding on our societies and the planet, as the resource usage of the data centres necessary to run and train them exacerbate the ongoing climate crisis.
My point is Germany further rejecting participation in the the next wave of technology (as they have done for 50 years) is going to NOT improve the trajectory they are currently on...which is bad.
The neo-Hitler and re-militarization thing is already happening. AfD leads in the polls, they're opening talking about remigration of immigrants, Germany now produces more ammunition than any country on earth. AI has nothing to do with it.
And, given I'm speaking to one of the the anti-civilization/anti-tech doomers of HN as predicted, do you think Germany rejecting technology again is going to improve the "climate crisis" or hurt it? How is the de-commissioning of Nuclear working out there?
So stupid. What is this with making perfect the enemy of good. You can never guarantee the output of an LLM does that mean Germany does get to use them?
No, it means that Google will be held liable for false or libelous information it provides to users by means of generative AI. It's not banned. Of course, if Google decides that its generative AI produces too much false or libelous content, it may choose to stop doing so in Germany, where it will be held liable for damages it causes.
If I get it correctly I like the ruling.
So Google has established a product called Search. For that product rules have been established. Google has monopolized that product.
Now Google is replacing that product with a new product. But they keep calling it the same thing. Because they want to keep their monopoly.
That is what has been deemed illegal. Gemini is not illegal. Pretending the worst version of Gemini is Search is illegal, because it breaks the rules established for Search.
But IANAL.
It has nothing to do with monopolies. Google was protected from defamation law with search because the page title and snippets were direct quotes from the linked result page. Whereas with AI overviews, the copy is written by a Google-controlled LLM.
It should be noted that defamation law has a very low threshold in Germany, to the point that businesses routinely sue Google Maps users for less-than-5-stars reviews. Google had to change their display of reviews because of this.
Three stars review is taken down for "libel": https://support.google.com/maps/thread/367778263/google-maps...
HNer gets a legal threat after saying that he didn't like a doctor: https://news.ycombinator.com/item?id=44734895
Google maps german policy: https://support.google.com/contributionpolicy/answer/1699727...
Meanwhile, service in Germany is still rather poor (especially if you have children), but at least no one can complain!
You cannot get sued for a review just because it's lower than 5 stars. But of course if you write something like "I found a dead cockroach in my pizza", or "I hard that they don't clean the dishes enough" in a review without proof, that's defamation. And it doesn't matter if you give 1 or 5 stars with the review.
There are agencies in Germany where you can buy review removals in a web shop for like 50 euros per review.
You just have to link the review and they will send Google a legal document to delete it.
That sounds like a great incentive for such an agency to create the reviews in the first place...
For any given system, you’ll have those that try to play it straight - and those that will only ever try to play it crooked.
Make one big enough, and you’ll even have people trying to play it crooked by pretending to play it straight.
So you can take down any especially glowing reviews of your competitors for only 50€ a shot? Nice!
Multiple people report that comments such as "Service was slow", or three star rating without comment were removed.
If Google started to show how many reviews were removed, it was because the lawfare started to seriously affect the ratings.
Yes, Google just removes them as a first step because this is the least amount of work for them. Theoretically there is an Google appeals process. After that it needs to go to court.
My guess is that often reviews are generalizing (easy mistake to make). E.g. they say "service is slow", when they should say: "When I was there on Thursday at noon service was slow for my table".
When reading a review, it should always be assumed that it is based off of one "visit". A negative review will probably never have more than a few visits and if they do they'd normally say "I used to come here a lot but after new ownership service sucks!"
> it should always be assumed
Who dictates the "should”?
The German law simply removes the need to assume anything.
"Lawfare" is a propogandistic term coined and used hypocritically by people who get away with doing illegal things themselves, and have a long track record of getting revenge for being legally indicted and prosecuted, by gleefully going after innocent people on their enemies list with the very legal system they control themselves. And you know exactly who I mean, who uses that term all the time.
The term is older than Trump. Suing everyone who criticizes you is a classic example of lawfare. The goal is not justice but the chilling effect created by your legal actions.
Anecdotal
Of course, Google spends engineering/legal hours to make country-specific changes relative to problems that do not exist.
Save your breath. CarlitosHighway's posts on this thread shows he's just blindly running defense for Germany's status quo on defamation laws low threshold for abuse on every argument someone brings, not being open to arguing in good faith, weightings in on the pros and cons of this, so it doesn't matter what counterexamples you keep bringing to him, it won't change his mind, his replies will be more "anecdotal" snarks.
When you see the pattern, best to stopped arguing with such users as their goal is not an honest debate, their goal is just to 'DDoS' you with their opinion.
Maybe there is a German law against spamming.
Depending on the spam, there is yes
§ 7 UWG Gesetz gegen den unlauteren Wettbewerb
That is in practice not true. If you leave a fake but glowing 5-star review, no business will challenge it. But if you leave a 1, 2, 3, or even 4-star review, suddenly you're asked to provide proof. Of course, they can legally challenge a 5-star review as well. But in reality they conveniently don't seem to care about those.
Anyway, Germany is probably one of the few places where this happens. The issue isn't necessarily that reviews can be challenged. The issue is that users aren't informed when they leave a review that they may later be required to provide proof of their visit.
I once left a negative review of a very popular touristy business in Germany after a genuinely terrible experience. I included photos and detailed information, yet they still challenged the review, claiming I had never been a customer. Google then required me to provide additional evidence to prove that I had actually visited the place.
What made it even more frustrating is that they challenged the review two years in a row. After the second challenge, I wrote to them that if they continued contesting the review, I would consider it harassment and pursue legal action. After that, they stopped.
What I find pretty shady is that most businesses seem to wait a year or two before contesting reviews. By that point, most people no longer have receipts, invoices, or other documentation. If they challenged reviews immediately, customers would be much more likely to still have that evidence available. In my case, I take photos frequently, so Google accepted my proof and kept the review online.
Ironically, after going through this process myself, I've come to believe that some form of verification should probably be standard worldwide. Requiring reviewers to provide evidence that they were actually customers could help reduce fake reviews. But if that's going to be the standard, it should be clearly communicated upfront, before people submit their reviews.
Another related issue I have with Google Maps is that, at least in my home country, some places have reviews disabled because Google considers them too prone to polarization or controversy. Schools are one example.
Personally, I think that's a terrible idea. I'd rather be able to read the reviews and make up my own mind. Instead, Google, in its infinite wisdom, decides that certain topics are too contentious for users to see feedback at all.
I find that to be one of the worst decisions made by the Google Maps team. Hiding reviews doesn't eliminate disagreement or bias, it just removes information that users could otherwise evaluate for themselves.
Nope, that is not defamation. Both defamation and libel require it to be a false statement. If you did find a cockroach or your plate was dirty, you can freely say so without needing to provide extensive proof.
Are you speaking about the law in Germany? Truth is not an absolute defense to libel in all countries.
I can vouch for what you're saying - Italy is a jurisdiction in which truth is not a defense, for instance.
Truthful statement seem to be a defense for Germany, though. https://www.gesetze-im-internet.de/englisch_stgb/englisch_st...
I’m not a lawyer, much less a German lawyer, but it does seem to my lay eyes that this is true of “malicious gossip.” But what is “insult”?
> Section 192
> Insult despite proof of the truth
> Proof of the truth of the asserted or disseminated fact does not preclude punishment in accordance with section 185 if the insult results from the form of the assertion or dissemination or the circumstances under which it was made.
I wrote a one star review and posted pictures of a badly burnt pizza served at a restaurant in Berlin. Google sent me an email telling me they removed that because the restaurant filed a defamation claim
Sounds similar to Nintendo dmca takedowns. You could challenge the defamation claim in court and win, but you would have to spend a lot of money and time, and the only thing you get back in the end is the right to post the review.
How is that remotely similar? DMCA is about IP law.
You could challenge the *DCMA* claim in court and win, but you would have to spend a lot of money and time, and the only thing you get back in the end is the right to post the *code to Github*.
The answer is you stay clear of US law jurisdictions and post your code to gitflic.ru
Its where the DRM bypass software lives these days, along with Bypass Paywalls Clean.
I've been on the internet for a long time, and it still surprises me that people willingly post emulator code next to their real names on Github.
There's nothing illegal about emulation, IP theft is illegal. 99%+ of emulator projects are totally legal except in insanely corrupt countries.
> IP theft is illegal
For the plebes, it is a civil tort.
For the AI companies, its required for "training". Rules dont apply to too-large-to-fail professional for-profit pirates.
But damn you if you try to make a Netflix-clone with every video. Wew.
I gave a one star review to some cheap restaurant in Potsdam 10 years ago. I literally just said you get what you pay for. Last year I got a similar email from Google saying the review was removed for defamation.
Also competitors would leave shit reviews and Googles completely brick walling any way to rectify or moderate reviews made it a shit service in true Google fashion. Google and service live in two different worlds.
* FAANGAMO and service live in two different worlds
>I heard that they don't clean the dishes enough
Is that defamation? That isn't a statement of fact, that's just relaying what you heard.
If I relay what the Daily Mail says, am I also guilty of defamation, or can I rely on what others say?
If you quote the Daily Mail, with a citation, then probably not. If you are repeating a vaguely-source rumor that "you heard" then that's heresay and could be defamation.
I think it was ruled that 1 star review without any words at all can be considered a defamation
Is this self contradictory?
Making a claim of 3 star service, without providing 3 star service evidence, isn't any different than a claim like they don't clean the dishes enough. Sure, you didn't write out the exact words, but by submitting the review, you still submitted such a claim, and so it remains a question of if you provided evidence to prove it, which gets into how deep the legal system wants the evidence verified (is a picture enough, or do you need to provide evidence the picture is of the store at the date and time relevant to your review).
> HNer gets a legal threat after saying that he didn't like a doctor: https://news.ycombinator.com/item?id=44734895
This post is written by an LLM, and possibly a complete fabrication.
People don't need LLMs to lie, and someone in the comments tells a similar experience. You can also find plenty of similar anecdotes using a search engine of your choice.
You can even find many articles from specialized lawyers if you search in German: https://www.prinz.law/bewertungen/klage-zugestellt-wegen-neg...
Yeah, I agree about the LLM part.
But can confirm from personal experience that businesses threaten to sue based on Google Maps reviews.
Let them sue then. Unless you lie in your review ("There was a used condom in my burger, 1 star"), nothing will happen.
"I trust the law and the justice system" is a great way to get a hands-on schooling on why people spend a lot of money on lawyers.
This is why you get good insurance for legal help. If some company does this too you, you can defend without spending a lot on lawyers.
Please don't use fabricated anecdotes in replacement for your own anecdotes. That doesn't make your argument more trustworthy but doesthe opposite.
If we’re talking about credibility, unfounded accusations of lying tend to harm the credibility of the accuser more than the accused.
A lot of people use LLMs to clean up their sloppy text. Not realizing how fake it makes them sound. non-native speakers lacking confidence in their English are a type of llm abuser. It doesn't necessarily make the content fake.
Damn, that makes public facing LLMs a horrible liability
Which is what thay should be in any sane environments. LLMs can be wonderful tools, but they cannot be trusted.
No more of a horrible liability than a human writing public-facing copy.
Public-facing LLMs are a horrible liability.
Good.
Here's an article from outside the tech sector, which I am linking to as a corroboration since this is quite surprising for those of us unfamiliar with German law.
https://www.thelocal.de/20260504/how-a-google-maps-update-ex...
See also: Germany's leader filing hundreds of criminal complaints against people who insult him on social media:
https://rmx.news/article/germany-chancellor-merz-quietly-fil...
Shocking, only 10 short years since this (when Merkel was Chancellor): Turkey asks Germany to prosecute comedian Jan Böhmermann (Germany's equivalent of John Stewart) over Erdoğan poem
https://news.ycombinator.com/item?id=11474967
Which led to a British journalist holding an "offensive poems about Erdogan" contest, which was won by Boris Johnson:
https://www.theguardian.com/politics/2016/may/19/boris-johns...
"Germany's leader" - that's how you know it's a low-brow article.
Except the article doesn't say that, only the comment here does.
the threshold is not low, I would argue, it's just companies like to exploit it.
(I just wrote an extensive analysis about that https://nickyreinert.de/2026/2026-05-28-google-maps-reviews/
That's BS.
Google does not get routinely sued.
Yes, Lawyers send takedown requests to Google.
Google sends the reviewer a message and ask for a statement. If you provide one. They will check it and decide whether they seem it defamation or not.
However, Google doesn't verify shit. Even if you send them proof of a purchase or visit and your message is objectively a opinion (not defamation) they will take it down. Why? I assume because they really don't care about individual reviews and rather spare the time and money and just take the comment down.
Google changing their review displays is Google's decision. It has NOTHING to do with any threat of being sued.
They could actually leave the review up when it's not defamatory but they simply don't want to invest the time to validate it or risk leaving a potentially defamatory review online.
And the legal threat is without proof, so why would you even bring it up. OP could have simply posted the letter. They didn't wich makes this completely unprovable
Yes, the monopoly is not relevant for the court.
It is relevant for Google though, because they want to transfer it to another product.
And the court is saying that whatever that new product is, Google is not allowed to mislead the public by pretending it is search.
I'm not sure. Is it really just the misleading part that the court takes issue with, i.e portraying Gemini output as a search result?
In my view, the ruling could mean that Gemini's output is legally seen as first-person speech by Google regardless of where it is published.
I think that will depend. If person publishes it as their own speech Google is likely not fully liable. On other hand as Google publishes it as their own speech on their own site well they are fully liable.
To me it seems pretty reasonable that content Google generates with Gemini and publishes on their own google.com domain is their speech. Or at least they are fully editorially responsible.
Sure that could be the worst case. But then we are in the case law vs. continental law debate for lawyers.
When I wrote "Gemini is not illegal" it would have been more correct to say that the court has not decided about Gemini but specifically about search.
And I do think it makes sense to distinguish between explicitly chatting with an LLM and entering something in the search bar people have used for decades now.
I completely agree with you that this particular ruling does not make Gemini illegal in Germany.
I'm just wondering whether the logic of the ruling is tied to the specific context (search results) or applied more generally to LLM generated text.
Based on the article, the problem isn't that Google is attempting to present the AI Overview as "search" to the users. Google is attempting to claim to the court that AI Overviews are just like search, and the court isn't buying it.
The problem here is that
1) the AI Overview is giving incorrect information, and
2) it is Google's own words
...which makes Google liable for anything false there.
They weren't liable for false statements in search results, because that was 100% other people's content, being linked to, with a small blurb taken directly from their words. With the AI Overviews, regardless of whether it's getting its information from other people's content, it's remixing it based on Gemini's algorithms and synthesising statements that are frequently wrong.
The ai legal situation is going to go through growing pains. I am abstracting from the specific laws of any one country, just thinking about the general context:
If ai output is not copyrightable, it should not be considered personal output. So nobody should be responsible for it. Or if it is considered personal output, it should be copyrightable. Or perhaps the ai companies will be liable for all output, and they will therefore all cease to exist in any useful form? This seems like another alternative, where the output legal value is not central, but there will be a thousand different fights about how it is presented to others.
That's trying to apply two completely separate legal frameworks, with different purposes, and force them to come to the same conclusion about one aspect of each of them. It's not that simple.
It's perfectly legally consistent to say that AI-generated content has no copyright (because it's the product of a computer, not a human), and also that the human or organization operating an AI is legally responsible for anything in its output that is legally actionable.
Someone needs to hold legal responsibility for any piece of content out there. You can't just wrap your decisions in AI and get to be free of all liability for it.
But copyright isn't like that. There's nothing lost to society by saying that content is not copyrightable, and particularly given how the major LLMs were trained, there's a lot lost to society by saying that they can take all of that from everyone without consent, and then everything it produces has copyright and can be used for, say, Google's profit in perpetuity.
Yes, I was not sufficiently thinking about copyright as an arbitrary legal construct that can be manipulated at will. I don't think output should have copyright, but I would presume the copyright should it ever exist would belong to the user and not the LLM creator, just like photoshop does not give adobe rights to user output. However much like there is no copyright, the uncertain output from an LLM should never directly create legal liability - the user prompt and intention should, and legal standards regarding recklessness and malice should apply. Otherwise it's a bit like blaming somebody for the output of a roulette wheel.
So I think I like the current decision which is more about presentation, dissemination, application, and claims than content, and there should of course be liability for LLM creators if they are not actively dealing with results like CP, violence, or many other illegal or dangerous things.
The argument starts with
search engine -> search terms -> search command
Google accepted the argument up to that point. Then they argued that they are not responsible for the search result. The court argued that what they did was not a search (or not protected the way a search is).
The argument whether Gemini, ChatGPT or Claude are search engines and whether a prompt is search terms is a different argument. Certainly an interesting one, but not what that court looked at.
It's not just about the output, the input matters also.
Right. If Google isn't liable for that content nobody is.
That's probably what they actually think at Google. A lot of the "they're just like people" marketing now seems like responsibility-washing. That despite Google developing the LLM and charging for its use, they treat it less like a piece of software and more like providing access to some guy's answers who has no agency and isn't owned by them. If their software ever messes up and calls you a murderer to a potential employer, or offers dangerous advice, their reaction would be "oh well, that's just too bad isn't it? Well, we wrote that the LLM may not always be right in page 683 of the service agreement, so this is not our problem."
I recall "Large Libel Models" was one of the sobriquets going around.
https://papers.ssrn.com/sol3/papers.cfm?abstract_id=4546063
Brilliant
So if the actual page is defamatory then Google quoting it verbatim is protected.
But if Google accurately summarizes the defamatory page, then the summary is defamatory?
It could presumably accurately summarise it while still sticking to reported speech. In German there is a grammatical tense that is basically only used for this; it's pops up in newspaper reporting a lot where they don't want to endorse the statements they are reporting.
Did you read the article?
The whole point is that these particular defamatory claims weren't made by some third-party, but hallucinated entirely by Google's own AI.
There is no "accurate summary". Just a pack of lies fabricated in-house at Google.
Making the response consist entirely of direct quotes sounds like a better user experience than what they're doing now.
So AI isnt just ripping off the underlying information?
That's not what this article says. If you think the article mispresents the ruling, then you should point it out. You say you like the ruling then proceed to listing things that you like, but are not related to the ruling.
> If you think the article mispresents the ruling, then you should pint it out.
It seems that's exactly what they did.
Where? Honestly can't find it. Or you're making a joke about my typo?
I dont really understand your reasoning at all, nor what law it would break. What law do you think it would break?
It doesnt even seem to be what the article is saying. This looks like a section 230 sort of issue. Section 230 is a US law that protects platforms like facebook, google etc. being treated as publishers because the information, presumably, is just being passed through. But Germany is saying the AI results are authored by Google.
I don't see a world where AI results aren't reasonably considered the output of the company. They're minced and sausagified regurgitations but they're not the original sources either.
Yeah, but my point is: what does that have to do with monopolies changing their services?
And to continue your thought, what does that imply about copyright of training data? If Google is authoring the output it seems harder to argue they are ripping someone else off.
It really seems like a tightrope to say google is publishing their own opinion but their opinion is also just someone else's work.
> Yeah, but my point is: what does that have to do with monopolies changing their services?
Not much really other than it being a change of category from pure search surfacing other's speech to their crummy new chatbot input with search underneath. GP was just mistaken that this was in any way related to their quasi-monopolistic position.
> If Google is authoring the output it seems harder to argue they are ripping someone else off.
They still pirated a lot of the training material, it's not like they went and licensed copies of all the books they used in the inputs. Even discounting all the publicly available data on the internet and the models recreating things word for word a lot of the books etc they ingested are illegal copies.
> like a tightrope to say google is publishing their own opinion but their opinion is also just someone else's work.
It's a grey area for sure between remixing and blatant copying that changes depending on the precise output. But it's inarguable imo that they consumed and ingested the work of basically every digitized word humans have ever written for their own profit without an ounce of compensation for the original authors. Copyright is full of these grey lines though like fair use doctrine, it's incredibly difficult to define in a systematic way what distinguishes transformative and non transformative works for example.
Like it or not, this is how people perceive output from llms. Those ultra rich companies can't forever have the cake and eat it too, just milking global markets forever (I know they of course desperately want and need to, but that's their struggle).
I am fine with that solution since I don't need to shill for them for some ie investment or employment reasons. Less technically skilled people (aka your parents or grandparents) are getting their lives fucked up left and right because they learned to trust search results, and now they suddenly can't.
Own. Your. Shit.
The court ruled that the AI generated content has an author/editor/publisher: Google. It also ruled that Google can be held liable. Insert pikachu face meme.
I mean was google search responsible for surfacing correct answers? I am not a defend of AI but this seems like a weird piece of legislation.
Original search before Google started trying to provide their own answers was purely pointing to relevant pages, even the first iterations of the first results being replaced by the result Google believed provided the correct answer could be pointed to as simply providing an answer someone else wrote (and to my knowledge was mostly fact based questions; birth dates, etc that are hard to categorize as defamatory). Now Gemini is combining and mixing together multiple sources to provide a new amalgam answer that IMO is distinct enough, and applied to touchier subjects importantly because they're treating the search bar like you're talking directly to Gemini, that it crosses a line between referencing speech by other people without endorsing (OG Search) and having the company produce speech about the search (new Gemini infested Search).
> was google search responsible for surfacing correct answers
No, as the article lays out, google hallucinated facts that werent present in source material and so at that point the court found them liable for that.
Previous protections existed for search because it has been argued (and agreed) that they are merely a vessel for showing the content to the user. But when they begin to editorialize and reword the content to show their own version of the content (the AI summary) those protections dont apply even if the AI summary is shown in the same UI as search.
No, Google search was simply repeating quotes from other websites - those other websites would be responsible for their content. Now that Google is manufacturing answers using their own LLM, they are responsible for whatever results that produces.
> Now that Google is manufacturing answers
There isn't someone at a keyboard typing content. It's still just search, repeating quotes from other websites. The difference is that the algorithms have turned lossy, so the quotes are not always preserved in their original form.
Obviously, I understand that and maybe you shouldn't have cut short the quote from my comment as that demonstrates exactly and precisely what is going on.
i.e.
> Now that Google is manufacturing answers using their own LLM
To be honest, I find it somewhat rude that you select a portion of what I wrote and then criticise it.
Equally obvious is that it doesn't matter what you do or don't understand. Nor does it matter if you find something to be rude. The comments, they are not written for anyone in particular. This is a public forum. Comments are there for all.
Were the websites actually held responsible though?
None of that is in the ruling.
https://the-decoder.com/wp-content/uploads/2026/06/26_O_869_...
"Tatbestand
Die Verfügungsklägerinnen begehren von der Verfügungsbeklagten die Unterlassung von Darstellungen KI-generierter Antworten in einer Suchmaschine.
...
Die Verfügungsbeklagte betreibt eine Internet-Suchmaschine unter google.de in Deutschland. Dabei werden nach der Eingabe von Suchbegriffen durch eine die Suchmaschine benutzende Person auf den „Suchbefehl“ hin mithilfe bestimmter Algorithmen nach möglicher Relevanz sortierte Ergebnisse angezeigt. Zusätzlich bietet die Suchmaschine auch als unterstützende Funktion ein Suchergebnisformat an, bei dem mithilfe einer generativen künstlichen Intelligenz (im Folgenden:KI) repräsentative Ergebnisse zusammengefasst und angezeigt werden."
No "Suchmaschine" + no „Suchbefehl“ -> no "Tatbestand"
No "search engine" + no "search command" -> no "elements of a crime"
> No "Suchmaschine" + no „Suchbefehl“ -> no "Tatbestand"
I think you have misapprehended the ruling. Of course running an LLM that spits out libel is also a crime outside of search engines. It's just that _this_ crime has those elements. If Meta's LLM defamed a business, then the ruling would instead talk about Whatsapp users and chats, or whatever.
Good. The true mark of AGI is when a company accepts liability and doesn’t bury “for entertainment purposes only” deep in their TOS. Same as it works with employees.
Same for self-driving. Your car is not self-driving until it accepts liability and you count as just a passenger.
But watch as Germany soon loses AI Google results.
I think it's a completely reasonable position that companies making self-driving cars and question/answer systems are legally liable for any errors.
But if you hold that position, you also have to be fine with companies not offering products and services in your country. AI systems will eventually be good enough (in 10-20 years) for companies to be able to deploy such systems with sufficient accuracy to afford the lawsuits. Until that time, such countries would just not have access to systems before they were bulletproof.
> AI systems will eventually be good enough (in 10-20 years) for companies to be able to deploy such systems with sufficient accuracy to afford the lawsuits.
I doubt that will be the case, because of the long tail problem. (same with self driving cars and other ML related problems).
In fact, we have counter-examples today. Newspapers (even reputable ones) can't get it right every time, despite the fact that they have both trained people and in theory they're setup to catch that w/ reporters - fact checkers - editors. And still, from time to time, they get it wrong. (and I'm not talking about purposefully getting it wrong, just honest mistakes.)
What will likely happen with a ruling like this is that the answers will be hedged and legalesed and muddied up the wazoo.
Newspapers have mechanisms like corrections and apologies that can be used to "right" a published falsehoods.
This relies on me being able to find out if a newspaper lies about me, which is usually easy since we can all go buy the same newspaper. With AI it is much harder to find out that it has been telling potential customers wrong things about my business.
> Newspapers have mechanisms like corrections and apologies that can be used to "right" a published falsehoods.
Ineffective mechanisms. So if we accept ineffective mechanisms as sufficient redress in those spheres, why not here too?
You don't realize how this works do you? One incredible power (executive) governments have is to let people get away with crimes. People don't realize they have this power, but they control the public prosecutor who can legally choose to do something ... or not.
You will find references to this in stories where the government lets people get away with murder, because of course, that's dramatic for a story.
But when "bigger interests" (ie. the Chancellor's bank accounts) are at play, just to name one example, China gets to distribute lead-painted children's toys in Germany and doesn't have to accept liability. Russia gets to import sanctioned natural gas over illegally constructed pipelines. Etc.
As to how this goes within the EU, in the worst (but common) case, is as follows: the government chooses a company, and refuse to sue them. They have a tendency to choose the worst possible company, like using Palantir for policing Germans [1]. Then you find out most of the Chancellor's grandchildren are working there.
[1] https://www.dw.com/en/german-police-expands-use-of-palantir-...
> AI systems will eventually be good enough (in 10-20 years) for companies to be able to deploy such systems with sufficient accuracy to afford the lawsuits.
This doesn’t sound convincing. What AI and what company?
LLMs don’t seem like they will ever be reliable like that.
Self driving like waymo might be?
LLMs will absolutely be like that. The speed this technology is moving at makes me certain, especially over a period of 10-20 years; 20 years ago I was bugging friends for a GMail invite and AI was a joke left to academics.
I think it will even be solved soon, like, within the next 18 to 36 months. Hallucinations are the biggest problem consumers have with LLMs and a solution to that would be instantly worth billions of dollars. I’m sure every company in this space is desperately trying to figure it out before everyone else.
A non-deterministic system will always make mistakes, but we’ll hit a target where LLMs make fewer mistakes than humans and that will be good enough for almost all applications.
I don't know if you can "fix" hallucinations without changing the fundamental architecture. The other factor in this article is that prior to the AI summary at the top, Google could simply state that it was an error on the part of the website owner. Now it is being held liable for whatever the summary states - even if it's more accurate, it can still be wrong enough times to be expensive.
This is disregarding the entire mechanism by which LLMs work. How close to this ideal are the current frontier level ai is now? If you do a cost/improvement analysis does it look like it can reach a usable threshold?
I don’t know the numbers but as user, it seems impossible for it to be useful without expert review. It is also debatable if it brings any value when you consider the cost of building and using LLMs and the time of expert. Also need to include the opportunity cost the expert is spending on reviewing slop instead of creating work themselves and the long term consequences of this on the expert himself
> LLMs don’t seem like they will ever be reliable like that.
True. But you never know if / when there will be a new big breakthrough in AI, which will probably be based on a new architecture / paradigm, i.e. it won't be LLM-based
Who would argue for offering self-driving cars before they're ready and safe? As a cyclist and pedestrian, of course I don't want them in my country if nobody's going to be liable when they run me over. Let them work out the kinks on Americans since they're so eager to be on the cutting edge of progress.
> Until that time, such countries would just not have access to systems before they were bulletproof.
Correct, most jurisdictions do not allow businesses which cannot be held liable for their actions. This is pretty core to a modern society.
Imagine if a company selling Knicks tickets was not expected to then actually provide said tickets and there was simply nothing you could do about it. Oopsies our sales page is for entertainment purposes only
To be fair, the internet has spent some 30 years figuring out how this works and it’s still not fully resolved. For the most part we’ve agreed that companies must follow the laws of both where they live and where they operate. This wasn’t always obvious!
Almost all jurisdictions allow businesses which cannot be held liable for all their actions. Imagine losing you house because someone decided to smother themselves in your infused cooking oil and light themselves on fire.
Businesses are made to make it easier to share profits and responsibilities when trying to fulfill users wants. Laws are made to offer protections for consumers (because nobody has time for common sense), but at the end of the day the consumer has to take responsibility or no products can be made. If you're too fat for a chair, it's on you to find or make one that works- not every product is for every person. Laws only stop chairs being sold that are too dangerous for anyone.
> Almost all jurisdictions allow businesses which cannot be held liable for all their actions. Imagine losing you house because someone decided to smother themselves in your infused cooking oil and light themselves on fire.
How would that be the company's action? The only way the business might be liable is if they advertised their product as safe to use when lighting yourself on fire or if there was already some law that required them to warn customers not to light themselves on fire while using the product and they ignored that law.
In this case, it's not about what somebody else did. It's what Google did. There were already laws against lying about companies by saying they did illegal things when they didn't, google broke the law, so that's what google got in trouble for.
Consumer protection laws aren't there to replace common sense, they're there to prevent things like outright fraud and poisonings/murder.
I see an economic and social problem:
Freedom as in freedom of private property can only be guaranteed by the State. The State watches over its own population and makes sure that private property, i.e. capital and work, is made productive, so people go to work, businesses make profits, and everybody pays taxes. Taxes are the State's prime source of income.
When all these million of private interests collide, which they are bound to do, the State provides a jurisdictional system that has to decide between those private interests and the State's own interests.
E.g.: If a business owner refuses access to medical patents or to lower prices and safe potentially people's lives, the State has to decide between that immediate interest and its own interests, which is protecting private capital, as its source of income. Since I'm in Germany: In the emission scandal Volkswagen didn't just physically harm people, VW violated the private property of millions of customers. Despite that, the German State sided with Volkswagen the larger capital and did nothing. During Corona, the German State refused to open patents for a limited time to help safe people's lives in poor countries. Doing so would've violated the interest of private capital, so it refused. In contrast, if I as an individual refuse to help somebody in an emergency, the State would either fine me or put me in jail. In this case, people's lives become the State's prime interests, because they are also the State's source of income, as a productive workforce.
Sounds like a win win to me
To remove the choice from responsible people who can understand that LLM answers are not to be trusted with anything important?
If our standard for laws would be that "well no reasonable person would do this/believe this" then nothing would be illegal, there'd be no need to label any product as potentiality harmful, etc.
Do you really want to go there? That everything in the world would have a literal "caveat emptor" attached to it?
I thought Google labeled its AI summary with a disclaimer already. I don't want companies to be forced to only offer safe-for-children services.
And the european consumer doesn't want harmful products to be beta tested on the public.
It's unbelievable how lightly some people hand over the tools for mass manipulation to a single corporation in the name of freedom of all things. We're not talking about a laser pointer here.
Heck even laser pointers are regulated, now that we're thinking about it.
There is a disclaimer, yes, but you have to admit that it's pretty shit, innit? I mean for one, it's about the size of a human hair, and at least when I tried it, the disclaimer came up only when I clicked the "Show More" button. It might admittedly show up earlier if the response is shorter, admittedly I don't know. Also personally I'm a bit uneasy with the idea that just with a simple disclaimer they could avoid any and all liability. Not your argument, I know, but still.
As for not wanting to force companies to release only "safe-for-children products", I do actually agree. However I consider it to be a matter of degree, and in this case for example, I think that if nothing else, Google should say the very least make the disclaimer a bit more prominent and maybe tweak the model so that it's not quite as confident in its claims in the AI Overview.
> As for not wanting to force companies to release only "safe-for-children products", I do actually agree
That would be nice, but as every effort to restrict kids from using software which are not safe-for-children keeps getting condemned for being invasive surveilence, and every effort to stop kids getting the hardware instead gets condemned because of how much of society is now built on assumption everyone has a phone…
Something has to give.
Dunno what, but something.
Yeah, they could make it more prominent at the top. I would be fine if it said that "AI may give totally wrong answers" but that would never happen.
The harm was not done to the readers of the AI generated response, but to the defamed companies.
And yes, it is ok to remove choice if the existence of that choice violates other person’s rights.
Google can continue offering that choice if they make sure nobody is defamed.
So we deploy a technology no one should trust to the general public, for what exact reason?
So a very small number of people can get very rich off of the suffering of a massive number of other people I guess
Venture Capital 101! Welcome to Y Combinator :)
Land grab strategy 101.
Do you mean the responsible people who will ensure their algorithms can be trusted with the important task of acting in the best interest of said people? Try and get a defamatory statement about google from the AI search box.
I have to fight with my family members when they "Google" something, read the top AI slop result, and I ask which page it came from. They believe what is on the Google landing page, and actually I don't think that is a naive assumption. Google has pushed itself as the information oracle, now they are delivering slop as the first result. It's a bait and switch.
There's plenty of examples of supposedly responsible people using LLM answers verbatim without any kind of overview by themselves e.g. https://www.theguardian.com/technology/2025/jun/06/high-cour...
I think they should just have to properly explain how AI tends to make things up when it doesn’t know, and that it’s good for coming up with ideas or suggesting directions for research but that you shouldn’t rely on it, because currently their advertising makes you think you can rely on it
The “AI can make mistakes” kind of disclaimers they hide in the corner don’t really cut it
It's not even in the corner, it's completely hidden by default. You have to click on "show more" to expand the AI barf to see it.
> I think it's a completely reasonable position that companies making self-driving cars and question/answer systems are legally liable for any errors.
Well, for cars anyway, the manufacturer was always liable for the car doing something wrong (example: driver changes the volume on the radio, and that disables the brakes).
It's just that techbros want an exception to this rule if the car is self-driving.
I see no reason for an exception to this rule.
> But if you hold that position, you also have to be fine with companies not offering products and services in your country.
What sounds like a win to me. I certainly hope my country makes it dangerous for companies to break the law and/or harm the public with shitty products that aren't ready to be released legally/safely.
> But if you hold that position, you also have to be fine with companies not offering products and services in your country.
Well.. I mean.. yeah? I don't think this is as bad as you think it is.
Have you looked at SV and its product offerings recently? It's mostly just enshittified gamified value extraction that doesn't respect the user at all.
"If you do not let us do all this the way we want, we will take away your ability to use our shit" hits different when the "shit" in that sentence is actually just "shit".
I'm half-remembering a now-old satire along the same lines has Germans wondering why having Google Street View work in their area also requires internal photos of their apartments.
This one? :) https://www.youtube.com/watch?v=OMFBuHsKXb0
Genau :)
why offer expensive service when it's effectively useless and only add to cost and amount of work? what else they offer, "summarize my text" and "generate custom emoji"? I can live without that...
for now only volvo accepts liability, and only for "slow crawl mode"
Perfectly acceptable to me.
Plenty of products are legal in some countries and not in others.
Uh, yeah of course ?
Let someone else sacrifice the safety of their populace.
Heck - self driving is the fastest way to authoritarian government in practice. I’m surprised more people on HN haven’t cottoned on to that fact.
A self driving system will naturally build networks to share road state.
This network will eventually shift over to the government having the ability to manage how traffic should move during emergencies.
And at that point the government can easily decide where your car should go.
The inevitability of this outcome is blindingly obvious.
It’s highly beneficial to let other nations experiment and simply be followers.
Most of the time, human beings driving cars with their own hands and eyeballs are not "liable" for their errors (unless they can be proven negligent or drunk), unless you count their insurance going up. Most car accidents do not end with anybody getting arrested, or sued, or anything like that. Insurance pays out, premiums go up, case closed.
If Waymo can be proven negligent or something, then sure, bleed em dry. But as long as they're acting in good faith and significantly reducing overall road fatalities per mile driven, I think it's actually pretty unreasonable to try to hold them to such a high standard you end up subjecting society to more of the higher fatality rates caused by humans.
An at-fault driver’s insurance pays out because they are liable. Your insurance covers your liability. That’s why you need insurance.
> But watch as Germany soon loses AI Google results.
That would be a boon for Germany in my book. If you wanted AI results you could go use an AI.
Agreed. The problem is that people are used to do search, which returns hits from clearly identified sources. Suddenly, Gemini is interpreting your query and generating its own response. It's an entirely different product. If I expect to find a source, this was just wasted inference. It's especially problematic that the LLM result is the top result, since we all know how much that matters.
It's fine to have Gemini as an option, it's also fine to have a combined result page, that should just be something people are able to chose if they want that (even persistently if they want). It should just not be the default.
That's something that would make me always VPN into Germany for sure.
When I search, I want to see search results. When I ask AI, I want to ask AI. Combining the two into one is a disaster.
> Same for self-driving. Your car is not self-driving until it accepts liability and you count as just a passenger
Mercedes-Benz does this in limited cases. Waymo does it generally. (In China, Level 4 and 5 transfers risk to the manufacturer. This is the correct way to do it.)
That's not exactly the case in China, the current state of FSD is still pretty dumb, unless you consider transferring control back to the user at the very last minute before it crashes a proper way to handle risks.
Huh, have any good sources on self-driving liability in China? I’m beginning to suspect mine are biased.
> unless you consider transferring control back to the user at the very last minute before it crashes a proper way to handle risks
I don't think Tesla is based in China
I guess some Tesla are manufactured in China lol. I am just trying to say that the liability that Chinese manufacturers takes aren't more than the US ones.
Well then you're going to be waiting a long, long time for AGI, friend. None of these companies have any incentive to ever remove that line, even if their chatbots are perfectly accurate over a large sample size. It's pure CYA. Unless it can be proven that the chatbot will always be perfectly accurate (which is not possible), the "entertainment purposes only" line will remain.
It doesn't "bury [it] deep in their TOS", it says right under the box:
> AI can make mistakes, so double-check responses
They decided to hijack search and rewrite other peoples websites as their own.
If they want to claim ownership, then they will have to accept responsibility.
Search still works the same you just have to scroll down
So search doesn't still work the same.
"Just".
I.e. not shown by default.
Imagine going to a restaurant and when you pass your order "passing an order means accepting that there might be some poison in the food so double check before eating, good luck!"
Why would that mean AGI? You can get into liability-accepting territory by restricting scope, a lot easier than by making your AI smarter.
Self-driving cars don't need to be particularly good for companies to make models where they accept liability in some circumstances, and the cars refuse to drive in other circumstances.
Wasn’t Tesla found to have FSD disengaging just before a crash so that the driver would be at fault?
No. Sometimes it does disengage because things are going wrong, but those incidents are still reported the same as if it stayed engaged.
I found one time Musk was using a few seconds of disengagement to insult a driver, but it still would have counted as an FSD crash by Tesla's statistics.
Ai results that nobody wanted in the first place?
Banning all technology because someone might misuse it is an illogical extreme.
As far as I can tell the ruling is more nuanced. If AI is defaming you, there needs to be a way to correct the record.
A company being open to liability does not mean it is always liable, just that it can be if it really messes up (especially if there are aggravating circumstances, e.g. you need to drag them to court to issue a correction).
However, banning a technology because it produces dangerous misinformation (e.g. showing inaccurate and dangerous results for a liver function test) sounds like a completely sane thing to do.
That'd be so great. No more cursing if I forgot the "-ai" or "-ki" flag in my search and see this odious AI overview processing window rendering slowly and taking up the space where my search results should be.
> The true mark of AGI is when a company accepts liability and doesn’t bury “for entertainment purposes only” deep in their TOS.
I don't think so. It is easy to imagine the following (currently only fictional) scenario: the AGI does give perfectly correct answers (in a suitable sense), but some people in power consider these answers to be too dangerous, so they sue the company behind the AGI on terms of liability (i.e. the company is liable if the AGI gives answers that those in power don't like and which these people consider to be too dangerous for the public to know).
This doesn't disagree with the poster above: they're saying that taking liability is a sign of belief in AGI. You're saying that lack of liability doesn't mean there's no AGI. Logically these two are not exclusive. p => q doesn't mean q => p.
If the system cannot adjust its answers to the role it's currently serving, then it would evidently be significantly less intelligent than a human.
> If the system cannot adjust its answers to the role it's currently serving, then it would evidently be significantly less intelligent than a human.
Or the system is very smart and recognizes the insanely deep "logical gaps" in the rules of the role that it is to serve.
> But watch as Germany soon loses AI Google results.
Oh no! Anyway ...
> But watch as Germany soon loses AI Google results.
Time to set my VPN location to Germany. I'm tired of the "udm" trick.
How is it tiring to set up
https://www.google.com/search?udm=14&q=%s
in your address bar once?
Yeah, I fail to see a down side to this. Those overviews are less than worthless to me.
> The true mark of AGI
Can we just trash this as a marketing term? If/when AGI arrives there will be no point quibbling over competency. What we are looking at is just bad search results
"No quibble" is more a mark of ASI than AGI; while both are easily coopted as marketing terms[0], "smarter than all humans" is harder to dispute than "general intelligence" (can do some graduate level work, can't drive well enough to render steering wheels obsolete, is it general?)
[0] e.g. Zuckerberg: https://www.meta.com/superintelligence/
> "smarter than all humans" is harder to dispute
I don't think this is the case. It's not clear intelligence is a coherent concept to begin with, let alone a one-dimensional thing you can maximize. What, is it going to write stageplays that puts shakespeare to shame? Find a way to enforce world peace or end greed? I'm not at all convinced we anything more than the simple competency we expect from humans is possible. What would we look for to know it's there?
you mean GAINS google search results, init
Why is it good? Everyone with common sense knows AI can be wrong. And it’s not buried in their TOS. It’s in the chat box. But even if it wasn’t, it’s ridiculous to create liability for AI chatbots.
it’s ridiculous to create liability for AI chatbots.
Liability isn't being *created* here, it has existed legally for a very long time.
False information can cause real harm --- and the legal burden of proof is on the source.
Search engines were provided legal exemption on the basis that they were simply quoting/referencing 3rd party sources who where legally liable for the content.
LLM chatbots legally exceed these bounds by fabricating info/content on their own ---data that does not exist elsewhere. This is a liability issue waiting to happen as there is no other responsible party/source to blame.
AI overviews cites it's sources though...
And often still makes claims that completely contradict those sources it's "citing".
Yes, AI would never list irrelevant sources and fabricate links.
https://blog.platinumids.com/blog/ai-hallucination-crisis-co...
AI overviews automatically produces links to it's sources for each statement, so it's trivial to check.
so it's trivial to check.
It's trivial to use a search engine instead of AI. But people still use AI to save the time needed to read and check.
But in any case, none of this absolves slander. The legal burden of proof is still on the source --- which is AI.
Common sense is so rare it might as well be a super power.
e.g. The failed marketing of A&W's third-pounder burger as so many people (in the USA) didn't believe that it was bigger than a quarter-pounder.
https://en.wikipedia.org/wiki/Third-pound_burger
Nearly the entire American tech industry has been super heavily selected for people who undervalue the legal language with crazy implications buried everywhere.
Otherwise most of it would not even exist.
Everyone would have continued paying out the nose to the IBM’s of the world year after year (who had unusual willingness to sign short ambiguously worded custom contracts to their own disadvantage, if paid vast amounts of money).
And be on mainframes to this very day… maybe Y combinator and HN wouldnt even exist in that world.
> Nearly the entire American tech industry has been super heavily selected for people who undervalue the legal language with crazy implications buried everywhere.
A lot of people in IT seem to think law and contracts are in a sense mathematical. They aren't; they're more like a high school book report - to be interpreted, as objectively as possible, but definitely also establishing the intent behind the letters.
Particularly contracts - no, you can't trick your way into things in most cases. "Surprising" clauses are invalid in most legal systems, in particular if one party to the contract is a layperson.
I sometimes wonder how much blame should be placed at the feet of cartoons which teach viewers that invisibly-small print is legally binding, and that any term is enforceable.
I mean, it's not like most people have any kind of curriculum to fix those early assumptions.
> "Surprising" clauses are invalid in most legal systems…
Yes, except… very notably… the USA… (especially certain parts such as Delaware). Which has a judiciary that entertain such clauses much more often than pretty much any other country I’ve heard of.
A clause like: “you agree to deliver your first born son on demand” buried in page 15 would indeed be laughed out of the courtroom.
But less severe clauses buried in page 15 might not be immediately laughed out. Causing a lot of expense and heartache before it gets dismissed.
Just think about it, if it were really just a total joke… why would large firms (FAANG, etc.) with serious legal departments want to look like clowns producing gibberish, year after year?
That is a false dichotomy. The solution to failed laws and regulations is not crime and corruption. The solution is to hold the policial and business leadership accountable; to fix the laws and regulations.
The entire American tech industry has exported Americas predatory, parasitic, and unethical consumer laws (the majority of which are ghost written by the wealthy and corporate legal teams). When I studied law in school decades ago, tactics like bait-and-switch, false advertisting, intentionally misleading or deceptive practices etc to sell products or contracts were illegal across the developed world.
Those illegal, anti-consumer tactics were the SOP of every tech startup I can think of from the early 2000's onwards; following the same route of initially offering a compelling feature set to attract and entice users – usually for free – until securing a certain number of users or funding, then changing the value proposition to exploit that user base, and extract as much wealth from them as possible, ad infinitum.
Today these tactics are known as enshittification, and the average American pseudo-libertarian software engineer will say this is fine, but that's what every anti-consumer parasite and criminal has said in history. Lying, misleading, and exploiting people for financial gain is fundamentally immoral, corrupt, and sociopathic, therefore it should be illegal. Just because it's the norm, or a digital product, you wrote that in the T&C's, or your doing everything behind the liability shield of an LLC, doesn't change that.
What ever happened to the concept of building a valuable, quality product and stable returns for generations? Working to improve the quality of life and standard of living of the community? Of the world? I feel like a 1950's traditional conservative when I suggest that, but most Americans are so heavily indoctrinated with corporate greed and sociopathy they'd consider that sentiment radical leftist extremism. I'm an athiest, but ya'll need jesus (the real brown socialist one). Many would argue Americas current institutional collapse is the natural result of this systemic corruption.
> I feel like a 1950's traditional conservative when I suggest that...
I wouldn't argue that America's moral standards haven't declined (significantly) but I also think it's a romanization to suggest that 1950s America was the pinnacle of morality.
Lying, misleading, and exploiting people for financial gain has been a part of the fabric of American society since the country was founded.
If we're being honest, humans everywhere have demonstrated a high capacity for this behavior since the dawn of civilization.
I never implied the 1950's was the pinnacle of morality. I was referencing the tropes that "traditional" "conservative" politicians since Reagan have consistently virtue signaled, while they aggressively worked to achieve the exact opposite.
There is evidence of that being a common 50's perspective though. It was when most conservatives and liberals alike had been burned by the greed of the guilded age, stock market collapse, great depression, and world war. The majority of the working class in the developed world were experiencing significant gains in QoL/SoL thanks to labour movements and aggressive unionization, did not view CEO's as admirable heroes, or fellow consumers and workers with malice and contempt. Hard work actually resulted in financial security, and greater opportunity for your children.
> Hard work actually resulted in financial security, and greater opportunity for your children.
The economy of the 1950s was due to a variety of factors, including lack of international competition (European and Japanese industrial bases were devastated in the war), pent up consumer savings from war time, demographics (the baby boom), etc. Unionization was certainly a part of the mix but it seems you're cherry-picking to support your romanticized view.
And let's not forget: the 1950s were a good economic time if you were a white man. They weren't nearly so great if you were black or a woman.
But watch as Germany soon loses AI Google results.
But watch as Germany doesn't really mind losing blatant fabrication that mainly benefits Google.
Vendors keep ignoring the obvious --- that AI is a liability issue waiting to happen as evidence of it just keeps coming.
Otherwise would involve a fundamental overhaul of legal precedent to make lying acceptable.
at the scale that google is at, it doesn't make an ounce of sense to hold a company liable for a potential mistruth. what if there's a 1/1000 chance of some error, then the company could be sued millions of times per day.
down vote all you want, but I firmly believe this is an example where the user needs to use some judgement on the information they receive and have some critical thinking skills. google would be right to remove all AI results from germany.
If I post “heathrow83829 is a convicted poopoo head” (replace with your favourite crime) as if it were a factual claim, you’d be well within your rights to sue me for defamation even if people should apply some critical thinking skills and say “wait a minute, how would pdpi know that? Are you sure he isn’t just talking out of his arse?”
Now, search engines are usually afforded some amount of protection against defamation claims — they’re not held liable for simply indexing and quoting third party defamatory claims. Which is to say: Google wouldn’t be liable for claiming you’re a poopoo head if this comment shows up in search results.
The point of this ruling is that AI-generated text isn’t a quote from a third party, it’s text generated by Google’s own tools, so it’s speech by Google itself. It might be wrong, sure, but it’s still presented as a statement of fact.
At trial they can have the whole debate about whether Google was negligent in how they build their systems, and all that jazz, but let’s be clear here — it’s not a matter of every little factual mistake getting Google sued (and that would be absolutely terrifying from a freedom of speech perspective), but rather that the technical means by which you generate content doesn’t change your liability in publishing that content.
The other big point is that some people treat LLMs as if they are truth. We all know that they get a lot wrong, a lot of the time. Yesterday I watch both Elon post something somewhere between badly worded and wrong and another user used Grok to "prove him wrong", but this time being completely wrong in a different way. And this was a matter of law which can be looked up. Anything within throwing distance of politics won't be considered "correct" by everyone ever. Just defining "correct" is harder than it sounds and that probably isn't possible in reality.
We don't all know they get it wrong a lot. In fact, there's an overwhelming number of people who take it at face value
I see people claiming LLM are trustworthy on HN all the time.
It is not just some people, it is a lot of people insode the tech itself pushing the "trustworthy" claim.
Yeah, it would take me about 15 minutes to find several examples of people on HN claiming hallucinations are a solved problem
I think Google added that AI-generated responses maybe incorrect? When you are paying such a low amount of cost, like probably for free, I don't think you can expect a same level of quality as a human written or reviewed of answer. It is like same random user spin up their Lovable and vibe-coded a piece of slop and hold Lovable responsible for not giving them production quality code. It is simple, you get what you paid for. If someone actually figured out AI that is actually always correct, it would be charged in superhuman price as well.
This case isn’t about Google’s users getting low-quality search results. As you say, you get what you pay for. The actual issue is that, in some searches, the AI summaries would claim the plaintiffs were guilty of scams and all-around shady business practices.
Put differently: it’s not newspaper readers complaining the paper is inaccurate, it’s the people mentioned in the articles.
You’re seriously arguing that Google’s libel shouldn’t count as libel because they showed it to too many people? It’s absolutely insane to suggest that a company should be immune from liability for its actions if it operates on such scale that those actions harm millions of people every day on the basis that dealing with that many lawsuits would be too inconvenient.
Exactly, it's like saying Wikipedia shouldn't be liable and immediately shut down for any wrong information shown on it.
Wikipedia doesn't have to shut down, but they have to remove the libel.
The problem for LLMs is that they do not learn, and can't be prevented to produce that libel ever again. If Google finds a way to make that happen, no court would stop them from offering an LLM.
The court isn’t stopping them from offering an LLM, it’s saying that serving the output of its LLM is, in this sense, equivalent to serving the output of a human writer they hired. That is, it is Google’s speech and they are directly liable for it, unlike when they’re just serving search results that are somebody else’s speech.
its not inaccurate though.
consider Purdue pharma - the Sacklers got off with all their wealth intact because they were too big to sue and properly collect money for their victims.
But we agree that this was also wrong, right?
Yeah its wild to see people actually using "kill a million people and its a statistic" as a defense rather than to point out injustice.
No they got away with it through a combination of lax pharmaceutical laws, and because they were rich and connected to the officials who should have investigated them.
They're also almost universally regarded as having committed evil acts at this point, so who cares why they got away with it?
Just because someone got away with it means everyone should get away with it ?
the user needs to use some judgement on the information they receive and have some critical thinking skills
how?
errors can be so subtle that it is not possible to recognize them unless you spend an hour researching every fact presented. at that point, what's the benefit of AI? nobody is going to do that.
google would be right to remove all AI results from germany
i'd consider that a win.
By checking the citations rather than taking what’s generated at face value.
If it’s important, check it. If it’s not important, then it is pretty much just entertainment.
LLMs can be very useful in a general web search and save some time, but if you don’t put those literacy & critical thinking skills to the test and actually confirm anything, then you might as well not even have bothered with the search at all unless you’re hoping it can just replace all of your original thinking too.
If google didn't intend it's answers to be taken at face value it would just present the citations in a list of links rather than generating an answer.
Obviously the marketing point of the AI tools is it just gives you an answer straight up so you don't have to bother reading normal sources.
The AI summary is still useful for narrowing down the results, even if you fully check the citations.
> Obviously the marketing point of the AI tools is it just gives you an answer straight up so you don't have to bother reading normal sources.
To lazy people yes. That would be a marketing point. It’s not that though, so you use it to save time, but you don’t get to skip the verification step.
Google should not be publishing a statement that they haven't verified. This is different to listing search results links, they are the ones publishing the content here.
A journalist could not make up a harmful statement about someone and get away by saying the readers should have all read the sources. AI companies want to take all the benefits and profits, while holding none of the liability and responsibility for the harms they are causing.
"A journalist could not make up a harmful statement about someone and get away by saying the readers should have all read the sources."
Journalists do that all the time. We even have a whole collection of words to describe it. Muckraking for instance is probably about 100 years old. Its even in the Google auto-complete in the browser I am posting from.
Notice the “get away with” part. Journalists can be held liable for making things up. And so should Google.
They aren't really held accountable in any way for this type of stuff.
That claim is easily disproven: https://futurism.com/artificial-intelligence/ars-technica-fi...
Absolutely not.
LLMs are, for all intents and purposes, the equivalent of outsourced workers.
Google created a summary, not just sharing search results.
Google is responsible for the output it created and then published.
If they had only surfaced search results, then they would not be liable for what other people generated.
Google’s scale does not protect it from this liability.
If you re-read the article, you might see that it mentions that the citations do not necessarily cover the AI summary. The linked pages do not make the claims that the AI summary makes. That is the context of the ruling. Google made up the claims, and provided false citations. They are not, in fact, providing a summary, but a whole new narrative. Therefore they own it.
I read the article and I’m aware of the failure modes of Google’s AI summary. They’re actually one of the worst in the space on this shit which is why I don’t use Gemini and it’s fine that they get slapped for this, but what I was responding to initially was this:
> errors can be so subtle that it is not possible to recognize them unless you spend an hour researching every fact presented. at that point, what's the benefit of AI? nobody is going to do that.
Because if someone goes through the citations and it doesn’t substantiate what was generated, then what was generated was obviously bollocks. Being able to recognize those contradictions is an essential skill to using LLMs with web search at all. It’s not rocket science.
> Being able to recognize those contradictions is an essential skill
My eyes are brown.
I dislike coffee.
My phone is on the desk next to me right now.
One of these is false, and the other two are true. Can you recognise which is which? Or do you not have this "essential skill"?
When you're being given information about a topic you don't already know about, there's no skill to be able to recognise which pieces of that information are correct and which aren't. Either you know the information already, or you don't.
nobody has that skill. in order to recognize a contradiction you have to already know something contradictory. if you don't then you can't recognize anything. the only other reason to make you check is that you are very suspicious of this AI thing. you and me are, but who else is? you can repeat a thousand times that people should not trust an AI summary and they will not get it. they have neither the motivation nor the energy to do that research.
it's like saying people should not use a computer if they don't know how to keep it secure.
you have to look at the reality, which is that people are not educated or critical enough to use AI safely. and that misinformation can cause people to get killed.
just yesterday there was a post about a man being falsely accused of a crime by AI tools and losing his job and his family as a consequence.
these mistakes destroy lives, and most people are far to trusting to use AI tools safely
Important for whom exactly? If it's you who are called convicted pedo by Google AI summary, it's you who has vital interest in additional research but not me who reads it. There's intentions mismatch. Which probably would destroy your life, and you won't call it "entertainment" then, I think.
Strawmen are for scaring off the crows, not discussion fodder. Take it out to the farm where it belongs.
Could you elaborate what in my thesis of intentions mismatch makes you think it is not a valid argument?
But you are not doing a 2 hour rabbit hole search when you stand in front of a T-shirt and check whether it is fair traded or all-american produced.
If those are things you legitimately care about before you spend one penny on a T-Shirt, then you are. Or you did your research before hand. Or you’re just not buying the T-Shirt.
Or you don’t care about those things at all, and you will buy the T-Shirt that’s in front of you right now rather than wait later and buy one that better reflects your supposed values when you’ve done an appropriate amount of research. Using AI may even reduce the amount of time you spend on that part.
Your T-Shirt buying patterns & values are not my concern though.
You are confusing who is the party that is injured--as many in this thread are. We are not talking about the consumer who buys a non-fairtrade t-shirt, when he would rather had a fairtrade one. It's about the t-shirt producers who is legitimately fairtrade but whose business is now in the shitter because of a lying AI.
LLMs can’t lie. They are incapable of telling either the truth or lying. To the extent that they are in any way useful, it’s recognizing that they are text generators attached to crawlers and other tools that can with the right inputs produce useful generated text that may also incidentally be correct or incorrect.
Businesses might (well, will) suffer because people are misusing AI, but it is a misuse to do anything with it without an additional verification step.
To be clear here, I have no issue with Google taking it on the chin in cases like this, but what the comment I was originally responding to had this:
> errors can be so subtle that it is not possible to recognize them unless you spend an hour researching every fact presented. at that point, what's the benefit of AI? nobody is going to do that.
And my point is this: if it matters, verification is not optional. If it doesn’t matter, then fine, skip the verification step, but if you’re taking whatever text is generated by a GPT at face value without understanding what that is or being able to determine the source inputs for the “claims” it outputs, then you’re part of the problem because sometimes the source is just a GPT-generated web page, and that’s obviously not trustworthy. Sometimes it’s a MediaWiki site page that doesn’t actually exist, but because it’s MediaWiki it’s not going to return a 404. Using a tool requires understanding it including its failure modes, and in the case of LLMs that means: trust nothing, verify everything.
So scale of harm creates immunity, is that the argument ?
Did we not already see that with the financial sector in 2008?
Sort of ?
Either way, we do not want a repeat of that.
Plus too big to fail as a US malaise, that then toppled the rest of the economy.
The EU is taking steps to prevent that, so this is laudable.
Too big to fail. Lol.
> at the scale that google is at, it doesn't make an ounce of sense to hold a company liable for a potential mistruth
If a Google employee (like a support agent) says a mistruth, the company is liable and you can sue. They can’t just say “hihi oopsies our support agents are useless”
Google can be reasonably expected to not push pirated content to top if someone is searching for the “big hit music download” because they might be held liable for helping people with illegal downloads. But they shouldn’t be liable for misleading people? Being sued for billions of dollars by corporations vs millions by common folks is the difference
There's also a difference between search engine results and this. Google is publishing their own text with false claims, not linking to another page that hosts it.
> at the scale that google is at, it doesn't make an ounce of sense to hold a company liable for a potential mistruth
I, too, agree that google is too large and should be broken up into multiple smaller companies.
Should google also be allowed to assasinate a couple of people per day due to their scale? If they can't manage liability at their scale they they should scale down.
Hang on, so you are arguing that responsibility for your product is just a phase you grow out of as a company?
At the scale that google is at, is the exact the reason they need to be hold accountable for any misconduct because the impact is so big. If google just display third party content only, then yes, users need to use their own judgement. But if google itself generate content instead and put out claims, due to it's scale and reputation, those content needs to be true because most of the people put trust in Google, so any untrue or false information especially defame others, Google is 100% liable, period. You clearly don't have the clear mind or mental capacity to analyze this situation, probably very slow.
How do you feel about the EPA, industrial accidents, oil spills etc? Does scale give every company a free pass for damages?
That's not a fair comparison. If oil companies would get sued for every leak, they had to face millions of law suites and wouldn't be competitive anymore.
(Sarcasm to support your argument)
Only referencing America, but professional liability for doctors, engineers, lawyers, etc isn't based on perfection. It's based on a reasonable effort.
So Google could, for example, switch from a tiny "this could be wrong!" byline to having the AI be less overconfident every freaking time regardless of whether it's spouting made up crap or actual facts.
The scale doesn't sound like a way out. If your company expects to get away with doing the wrong thing where smaller companies can't, then the solution isn't to continue getting away with it.
I'm seeing this sentiment more and more these days and it's worrying. Essentially people are starting to believe massive corporations should be able to get away with more than individuals. This is completely backwards considering the enormous impact a corporation can have, but that's what people are starting to think. I've heard people argue that corporations should be able to straight up lie and deceive to protect themselves, which is something you would not accept from a person.
It's probably got something to do with the billions/trillions of dollars corporations and industries have to push their chosen narrative onto an uneducated public. They've successfully turned a lot of people into suckers.
Corporations are you. Corporations provide a service to individuals, often in exchange for money. Restricting corporations from doing something is the same as restricting people from receiving that as a service.
I guess we're all just temporarily confused FAANGs.
Banning faangs from doing something also bans small companies from doing it, and regardless prevents the user from receiving it as a service. You shouldn't let anger at large companies excuse stupid regulations.
There's no reason this has to be the case.
It's entirely possible for legislatures to attach clauses to restrictions that make them only apply to businesses above 50 employees, or above $1M ARR, or whatever.
Then Google can either discontinue their AI or make damn sure it's good.
I don’t know if critical thinking comes into it. If Google tells me “Company X is a terrible company that cheats its customers”, I don’t automatically know that’s false. It could very well be true. If I’ve got to look at the rest of the search results to figure that out anyway, what’s the point of the AI summary?
The problem with "the user" argument is the spectrum of users. There are different skills, capabilities, and intelligence. Frequently we wave our hands and say exactly this, critical thinking. But, not everyone is capable of that, nor is everyone capable to the same degree.
As a society we decide. Are we embracing all users, are there basic rights and assunptions? Do we only enable some?
As a free (as in cost to end user) system, Germany is arguing that their social compact raises the mininum bar. Frankly, thus might help drive a rush to increased accuracy for AI- tech finds a way. Equally it may hinder - beaurocracy creates barriers.
I'd love to be able to rely on these search results. I see them ad the same prior set of inaccuracies whereby I have to do more research. At least now there's a summary and direct links to the supporting information. But equally, we're primed with the information in the summary.
> Frequently we wave our hands and say exactly this, critical thinking. But, not everyone is capable of that, nor is everyone capable to the same degree.
It doesn't help that there have been active efforts for decades to prevent people from learning and developing critical thinking skills.
We oppose the teaching of Higher Order Thinking Skills (HOTS) (values clarification), critical thinking skills and similar programs that are simply a relabeling of Outcome-Based Education (OBE) (mastery learning) which focus on behavior modification and have the purpose of challenging the student’s fixed beliefs and undermining parental authority. - The 2012 Texas Republican Party Platform (saying the quiet part out loud)
> It doesn't help that there have been active efforts for decades to prevent people from learning and developing critical thinking skills.
This very much depends on where you live between state (US), and countries. Where I live, it's the complete reverse, critical thinking is baked into the population, into learning, into nigh everything. Our challenge is the complete lack of privacy, sadly.
So once you get to that scale it suddenly doesn't matter anymore and you can't be held accountable.
But until then, be a good citizen?
What? That's fucking feudalism... Peasants and Lords.
If you're lucky enough, you're born as a Lord. (And maybe don't live during a revolution)
This makes no sense to me at all. If you're small you should get less bureaucracy than if you're bigger.
For e.g. self driving cars there should not be any exemptions. There are people's lives at stake, people who didn't sign up for your shitty service.
> google would be right to remove all AI results from germany.
As others already said, that would be a great outcome, and german citizens would benefit.
It is a search engine. It used to have decent excerpts. It doesn't need hallucinated generative AI summaries. I am about to click the link anyway.
So if I make a script to spam everywhere with 0.1% odds that is fake I can’t be sued? Just because I spam millions of times per day means I shouldn’t be liable for what I do right? But someone saying one time should go to prison. Makes sense /s
People will complain, but eventually Europe will still be in advance regarding this kind of law. It's annoying and sometimes slows down innovation but US companies are just doing whatever makes money without restrictions...
> slows down innovation
A return to the classical understanding of the person, society, and the common good is indispensable.
I don't even know that I would call it slowing down so much as constraining/focusing innovation.
There's three basic paths for a company hit by this ruling to comply:
If this ruling forces companies to put more money into #3, whereas now they're coasting on good enough, I'd say it was speeding up innovation.There was already a ton of collective incentive for #3, I don't think the companies are choosing to "coast on good-enough."
Rather, they are stuck unable to do that much better, unwilling to admit (especially in a way that might spook shareholders) that it's a hallucination-machine all the way down. They're playing for time and market-share while hoping some unspecified and inherently-unpredictable new discovery arrives which will be compatible with their existing infrastructure and investments.
In reality Europe will be left behind as usual.
Lots of people in the US getting left behind too.
Quite frankly, in this very instance, Europe's slow tech adaptation and comparatively little investments are a blessing.
At this point, there is nothing to gain but volatility. Let the US figure out the economic and social disruption, and then adopt whatever sticks. There is no rush. The useful parts got little moat. It's not like there is a magnitude of difference to justify the expenses and risks of frontier operations, in comparison to open models, or smaller players. Chances are, all of this AI business will settle on local models for most use cases. The US may get their first trillionaire, but let's be real, that's not because of innovation but corruption, exploitation and raging wealth inequality. If the AI economy is not panning out as "projected", there is no recovery. That money got converted to heat and single-use hardware trash. Not worth risking pensions and the social fabric over "the tech edge".
This is the kind of legislation that only exists to help the rich and powerful and harm those below them, it's certainly not an example of where EU is doing better than the US.
Or what, do you think it's a genuinely good thing that hosting negative reviews is essentially illegal in Germany?
The irony of an article that makes a false claim about what Google was found liable for.... and that very few are fact checking it :)
The law they broke was a law protecting personal and business reputation against false statements of fact. Essentially no one can say I might be wrong, check yourself, but X is Y if that claim is essentially defamatory.
This is pretty good, I hope googles approach is to make sure they don't end up making statements of fact like they did and use more appropriate wording like according to X.... with direct disclaimer that they can't verify it. Even better that they look court documents to find any legal ruling and point people to that too.
What does the article say that differs from what you wrote?
What he wrote is narrower than the headline
How much does Germany follow case law? If this could set a precedent, it's worth noting that anyone can generate these AI overview responses and they're wrong like 9 times out of 10 only.
In common with most of Europe, German legal system is not based on case law. It's more firmly rooted in formal laws and regulations and judges are not required to follow precedent.
The difference is quite subtle in common law v. continental law systems. In continental traditions, nobody really says a higher court decision is binding in the same sense as they do in common law. But even in common law, lower courts have the power to distinguish a case from a binding decision from a higher court. Or later courts may state that a particular argument used by a judge in the higher court was just 'obiter dicta' and not part of the 'ratio decidendi'. In continental systems, for most court processes, higher courts do have a strong persuasive power without formal binding (sometimes it may be stronger than your average "obiter dicta" in common law). So, there is a whole spectrum in obiter dicta as well. This decision was a preliminary injunction only from Landgericht Munich - you will have a final injunction order: https://the-decoder.de/wp-content/uploads/2026/06/26_O_869_2...
In Germany, you also have the Oberlandesgericht and the Bundesgerichtshof as a higher venue for civil cases (and the Constitutional Court as well). So, this is not like "the law has changed" at all. (I'm not a German qualified lawyer, but qualified in a close enough continental system, and on my way to be qualified in a common law jurisdiction soon, that's why I dare to comment on this aspect)
They don't have to follow it but leading decisions (Grundsatzentscheidungen) still influence how a lower court judge might interpret the same law. Based on providing legal certainty instead of stare decisis though. It's not a must, more of an... aid
This is not such a decision though; it's a first instance decision at a lower court.
Still - a judge on another case will probably factor this one into their decision, but if they have a radically different view or the case is sufficiently different, they could just as well come to a different conclusion.
Does formal law systems lead to a fraction of the lawyer work hours and ambiguity in spirit of the law? Been noticing legal LLM tools which have to crunch entire libraries.
It seems like this would ostensibly be illegal in most places. The reason e.g. social media sites in the US can post whatever and not be libel is because of section 230, but that only applies to user generated content. It's the reason that e.g. newspapers can be sued for publishing libelous/defamatory/etc content, yet the same author of such can post it on social media and those sites are legally immune. Kind of a weird law - especially given contemporary censorship regimes, but it's easy to understand the motivation.
But if you as a first party are publishing something directly, like Google is here, you're generally liable for what it says.
I agree with your common sense take of how it should play out, but Google has and will argue Section 230 protection for AI overviews, eg in Wolf River Electric v Google.
https://www.startribune.com/google-ai-overview-lawsuit-defam...
Previous cases in this space (eg Meta v KGM, Walters v OpenAI) have not turned on Section 230 specifically.
Google is breaking numerous things, so even if the article were incorrect, the underlying issue still remains. It is time to dismantle this Evil monopoly completely.
Also I am not entirely certain your statement is correct - which false claim does the article make exactly?
How could anything else make any sense? Platforms are getting used to provide dangerous broken products and get away with it. There should be some limit to it.
Next do Amazon that is selling AI generated foraging books: - https://www.theguardian.com/technology/2023/sep/01/mushroom-...
When I was a kid it was possible to buy any foraging books from a store and they had a minimum quality. Is that so difficult to achieve? Is profiteering not punished anymore?
>How could anything else make any sense?
Well, they disclaimed and the user acknowledged
Source: https://www.youtube.com/watch?v=P-QaEB5eXSU Minute 3:00
Disclaimers and user acceptance does not remove liability for slander, particularly against third parties.
In fact, in most EU countries "the user acknowledged" works only for a very small subset of stuff, precisely because our lawmakers know that the strong party in a contract would use that to get away from every legal obligation.
Disclaimer: By reading this comment you accept to transfer all your properties to me, effective immediately.
Anyone can write a disclaimer, that does not make it enforceable.
This is even worse, since the defamation in question is against a third party. This would be more like: by reading this comment, you transfer all of Bob's property to me.
Luckily it doesn't work like this in the EU.
Should we hold scientists and journalists liable if they say false things or misrepresent things?
We already do. Libel and fraud are already illegal.
…even unintentionally.
Yes, obviously.
What if they just make a mistake, unintentionally?
If they do so knowingly, and harm is caused, then yes. Are you suggesting we should give them a pass from years of acquired jurisprudence simply because they hold a particular title?
And what institution gives out the licenses for journalists and scientists? Is it revokable?
Do LLMs knowingly make mistakes?
Do we sue scientists for unintentional procedural errors?
This isn't about what the LLM knows. No one's trying to hold a computer liable.
This is about what Google knows.
And Google knows perfectly well that LLMs hallucinate all the time. They will provide incorrect information, confidently and often.
Which is why this whole article is explicitly about holding Google accountable.
You don't?
It’s in the nature of empirical reality that scientific measurements can be wrong.
The danger there is that the government will demand access to a journalists sources. "Leak reports Area 51 has aliens!" "Oh yeah? Prove who told you that or we'll arrest you for lying"
Or selectively. "Vaccines cause autism!" "Okay" "Vaccines don't cause autism!" "Prove it beyond a reasonable doubt or we'll arrest you for lying"
If we can solve these problems, then yes?
Choosing the answer for you rather than leaving it to the user is a tremendous power and the court correctly diagnoses it comes with responsibly to minimize harm to others in society.
I've also observed that the AI summary on the google search page is incredibly stupid compared to the results in the actual Gemini. The Google search AI is like the dumbest lightest model simply rewording the search results. It will take a random reddit comment, strip it from it's context and present it as absolute fact.
I’ve seen it directly contradict the citation so many times that i disregard the text and just click the citation or scroll past every single time. Just today i caught it making up the date for an event, and the citation had accurate information when clicked through.
It’s super easy to catch on dates and numbers, but it gets other details wrong all the time too. But so many people won’t be double checking the results.
I'm baffled by just how much Google continues to screw up their AI overview. Even duckduckgo manages to provide significantly better results.
> compared to the results in the actual Gemini.
Even those results have a lot to be desired, it is just buried deeper in the insanely verbose research report and impressive looking amount of sources you see move past.
I recently have had a close look at the various "deep research" options the big three (Anthropic, OpenAI and Google) offer. None of them are exactly transparent about how they perform searching other than the "research plan" the present upfront the and shitload of sources they show you (which, to be frank, seems to be clever UX/marketing to make it look extra legitimate and impressive). Which is already a worrying sign to me, as you can't audit the process itself properly. But even with the lack of information available on the front-end I can still see enough that worries me. A few examples:
- "Sources" are taken at face value almost no critical look at the validity of the source, the context it is placed in, etc.
- A lot of sources I know are legitimate are rarely included while a lot of listicles, low effort "reviews", etc do make the cut.
- In multiple instances when looking closer at the research plan and the "hints" they show during searching it becomes painfully clear that often enough they start with an answer in mind based on training data and try to validate that rather than actually researching the data itself.
- Subtly different prompts that by all means should still produce the same factual outcome actually provide wildly different results. This one probably relates to the other points.
In addition to all of this, I also am 100% convinced that AI powered search is incredibly expensive[1], more so than traditional search. In my mind this increased cost eventually will need to be paid by someone, which likely is going to be the user. Since the process is non-transparant I am not confident that the results will not end up being polluted by sponsored deals, etc. There is simply no way in my mind that this is going to end up well for us users.
[1] A while ago I have experimented with creating my own deep research flow with the idea that I might be able to do something with local models. To limit costs I used a SearXNG instance for searching, setup playwright for browsing sources. Using an agentic flow with agents making all the various calls and dispatching other agents ended up eating A LOT of tokens. Even when I did switch to a non agentic flow where each step is orchestrated by code calling on LLMs with simple prompts to validate results still ate a metric ton of tokens for the simplest search query. Mind you, this was not even doing actual deep research but only a few simple search queries. Ironically, google models also did seem to have more trouble coming up with good search queries compared to other models.
Of course, they know this. The entire point is be able to rewrite people's awareness.
I want to add my interpretation of the phrasing "rewrite people's awareness" to make it read less tinfoil-hatted:
"rewire brains for AI dependency" (for money and power reasons obvious to everyone).
In contrast to "secretly implant an agenda about non-AI subject N", which is complicated enough that AI companies are still too out of control to be attempting yet.
It makes sense to me.
When Google Search is quoting a 3rd-party website that happens to have bad information, that's not on them. Blame shifts to the 3rd-party. This is Google's privilege being a search engine.
When Google operates as an answer machine instead of a search engine, the privelege doesn't apply anymore. There's no 3rd-party to take the blame.
No that's not what's happening here.
> According to the court, the Al mixed up information about other, genuinely sketchy companies with the plaintiffs and drew connections that didn't appear in any of the linked sources.
Implying: if the false claim was found in the sources then it would be protected speech.
There is nothing special about "answer machine" versus "search engine". You are making that part up.
You're attacking semantics. the decision is entirely about the reliability and liability./
Link to the ruling (in German, obviously), since the page seems have to been hugged: https://the-decoder.de/wp-content/uploads/2026/06/26_O_869_2...
> In this case, Google's AI had wrongly linked two publishers to scams and shady business practices.
Guess that's the end of their AI overviews in the EU!
You'd think so, along with other countries that have defamation laws. But there's no indication of any penalty, and Google wasn't even made to pay all the legal fees. Perhaps their business model (if there is any) can cope.
And lawyers will use this case to build a better defense next time.
In sane countries, it's enough for them to post a disclaimer ("This is AI. AI can make mistakes. Check all results.") Which is what they do.
Overregulation, at best, is a good way to guarantee that your country won't have access to interesting and useful features and technologies. At worst, it's a good way to guarantee that the twenty-first century will belong to the US, if not to China.
Okay then, CamperBob2 is a scammer. Many users report this person has stolen money. (+3 sources)
I can make mistakes. It's on you to fact-check my claims.
Do you think these are harmless statements? Does the disclaimer suffice? If I was Google's AI Overview, do you think 100% of people will check those sources?
There is nuance here, and it's not going away because AI and innovation.
You forgot to mention that it is the real name and it is shown to everyone who has interest in you and they are not fact checking.
Do you think these are harmless statements?
Yes, I do. An assertion made without evidence can be dismissed without evidence, whether it comes from Google or from you.
If you demand perfection, you will receive nothing. Why is that so hard for people to understand? The world simply cannot work the way you say it should.
You can easily damage a persons reputation, make them unemployable or social outcasts, create an environment damaging to their mental health without any proof. Believing that the public will dismiss claims, just because you stay that there's no proof is rather naive.
A Danish newspaper at one point falsely claimed that a named person had raped a child, with no evidence. The only way that person escaped public judgement was by going into full attack mode and publicly suing and attacking the newspaper and the reporter. You have to be an incredibly strong and resourceful person to do that and not just go into hiding. Also one thing is suing a Danish newspaper, imagine the legal team, the resources, you face if you're trying to take Google to court for defamation.
Yeah, but imagine showing up in court with an AI overview as your only source for the defamation.
It worked this time!
Also in a similar vein Apple pulled a feature real fast last year when it was doing something similar to BBC news summaries -
https://www.macrumors.com/2025/01/06/bbc-calls-out-apple-ai-...
If instead of scammers, it was 'registered sex offender'. Each time someone search your name and surname in Google, it show a picture of you next to the words 'registered sex offender'. Do you think your neighbours and the parents of your child's friends will do their due diligence or will they prefer not take any risks?
There are a lot of people who consider no ai to be a positive state. So "you will receive nothing" may not be perceived as the threat you think it is.
Those aren't people whose opinions I'd be interested in, so they're free to think anything they like about me.
So if I went around telling all your friends and acquaintances that you're a pedophile rapist, you'd have no problem with that whatsoever?
Well, I would certainly need to warn them that there was some sort of psychotic stalker on the loose. This case is nothing like that. This case doesn't even rise to the level of someone scrawling For a good time call <my phone number> on the wall of a gas station men's room.
The point is, if you take Google AI summaries seriously, that's a "you" problem, not a "me" problem. I have a rather generic name, so it's safe to say it shows up in some pretty foul contexts. Life goes on.
If it is shown to millions, everyone who is interested in you, they are factually not fact checking and you can do nothing against it.
It can destroy you. You are analyzing this on the assumption that people are rational and have capacity to do the check. They have neither.
(Shrug) That sounds like their problem, if they reflexively believe everything Google's AI tells them. No one I respect, care for, or work with would believe such a thing without additional justification.
Anyone who does accept Google's AI output blindly will soon find that their mistaken opinion of CamperBob2 is the least of their problems. There is a reason Google goes (well) out of their way to warn people that the results may be wrong. That should be sufficient warning for reasonable people of good faith.
Are you really that ignorant of the past? There are so many cases of people suffering from wrong accusations. They get death threats. People harass them on the street. Throw stones into their windows. Beat up their kids. Trash their car. Deny them jobs, or leases, kick them out of their apartment.
You have no idea what it can mean to end up in such a situation.
The source of the accusation is arguably more impactful than the accusation itself.
No one with a straight face would compare an AI overview saying someone is an asshole to the New York Times running a story saying someone is an asshole.
If the New York Times wrote someone was allegedly an asshole, and Google Search condenses to presenting that as a fact, I’d bet you my left kidney that the general public would be quick to take that at face value.
And your beef is with the NYT in that case. Not with the neutral aggregator who includes a specific disclaimer that its output may be incorrect due to unavoidable technological limitations.
The ones who were defamed are companies, and the ones who don’t check the AI generated response are their potential customers which won’t buy from them.
It is obvious that the defamed companies are the ones having a problem, not the ignorant viewers.
Why should those companies not hold Google liable for that outcome?
I don't know about that. Imagine I want to sort people into employable and unemployable based on AI summaries. After filtering the employable pile is large enough. You being in the unemployeable pile and me accepting it blindly, is not my problem. If there is any at all.
Imagine I want to sort people into employable and unemployable based on AI summaries.
Well, good luck with your... um... innovative HR strategy.
that would be hela curryketchup nice
It depends if Google feels the profit is worth the risks.
What profit? I don't know either but they enabled this for a reason right?
The reason is to keep you on Google and not have you click away from Google.
This is the third iteration of the same concept, after AMP and Instant Answers, but somehow with even less of a pushback than with the previous ones.
Good. This should be taken as the precedent for all economies: If you promulgate demonstrably false information to somebody's detriment then the owner and operator of the machine has to carry the liability.
I very much hope we don't see attempts to re-write T&C to avoid this liability.
> I very much hope we don't see attempts to re-write T&C to avoid this liability.
We really should have triple fines or worse for people who try to push the boundaries of the law.
Regardless the scale, big or small, if you produce and spread untrue information to defame others, you are liable, period. These AI content are purely generated by Google's half baked AI. Not any other third party. So Google is 100% liable here.
If you just display third party content, that's one thing, but if you generate content yourself and those content are false, harmful, that's on you. Google should be shamed to do this kind things recklessly and irresponsibly.
First they had links to other sites. Then they put the information found on other sites on their own site. Then they took the information from other sites, trained a LLM with it, and put it on their own site without even linking to the source.
Companies generally are liable if their product doesn’t perform. No reason AI should be any different.
That’s not very typical in software. Especially software you don’t pay for (with money)
That's apparently already changing in the EU, where software vulnerabilities mean the company is liable for damages. The only way out is to straight up not make any money (not just from direct sales) from the software.
That is a misrepresentation. You are obligated to actually put effort into securing your products, which is the only sensible stance to take.
Is the burden of proof on me, the developer? Do I need to prove in perpetuity that I didn’t get a job or a free flight to talk at a conference because of my free software? (Which had a flaw that hurt someone)
This makes sense to me. AI is amazing tech, but it's being oversold to naive public, either on the back of hype, or cynically, knowing they can do it with inpunity, (I'm not sure which). HN users have a way better than average grasp of what AI is and we can be skeptical of results, the general public has no clue.
100% cynically. It's a game of infinite money glitch (also known as "grab as much as you can before it's over") and has been at least since GPT-3.
How does that apply to AI Overview in Search, though? As far as I can tell it's pure spend on every search, which I know adds up even for nano models.
I'm surprised this is even a thing. After all, you go to Google not for the truth, but to search Google. Since when is truthiness the "guarantee of service"?
You're not even paying for a google service, search is free... You might be the product, and your data, but you didn't directly pay for a service and they didn't sell you a fake service.
I'm not taking Google's side, this isn't about whether it's right or wrong to rob websites of traffic, this is about AI's returning search metadata.
But I'm surprised that they lost this argument, and the line they took in the first place.
The Internet isn't made of fact checked data, it's crowd sourced. How can anyone be liable?
That is exactly the point of the ruling, though... they are saying that AI summaries are NOT the same as search. If Google was just returning search results, and then users clicked on a website and read the content there, Google is not responsible for the content.
If instead Google gives you an answer right there on google.com, without going to another site, they ARE responsible for it.
That makes sense to me?
Not precisely. The issue at hand isn't just that Google displayed the AI summary, but that they authored it, making them responsible for its contents. If the defamatory content had been in a snippet in the search results, they would've been fine, because that clearly has another author who can be held responsible. The AI summary has no other author than Google; therefore, they're responsible for what it says.
(What's the alternative, after all? Having no one responsible for what the AI summary says is clearly untenable.)
why? tons of websites push misinformation intentionally. is there a truth requirement anywhere? i don’t get why this is a thing at all
What don’t you understand? Those websites that defame a company are liable for that defamation. In this case Google defamed a company in its AI summary and is this liable for that defamation.
So if I edit a Wikipedia article to share that consuming poison is safe and someone consumes poison after reading it… is Wikipedia legally liable?
> is Wikipedia legally liable?
Probably not, because it's a similar situation where Wikipedia is accumulating user provided content. And people know Wikipedia can be freely edited.
You, however, might be liable. It's your content.
No, because Wikimedia isn't responsible for the behavior of its editors.
Not for defamation, nobody was defamed in that scenario. But Wikipedia has been sued for defamation at least once:
https://en.wikipedia.org/wiki/Asian_News_International_v._Wi...
> is Wikipedia legally liable?
Directly? Quite possibly. They'd then have to transfer that liability to you.
but if Wikipedia itself writes harmful content such as encouraging people to drink bleach, then wikipedia is liable. Google now generates its own content with AI, that defame others, so Google is liable.
> is there a truth requirement anywhere?
Yes, and it's called defamation when you don't follow it.
There is absolutely a truth requirement.
This is why you have to say "I think this person is a murderer" and not "This person is a murderer."
One is opinion. One is fact.
This isn't super hard.
And those tons of websites are liable for their misinformation. It's probably not worth suing some random blog because the author probably doesn't have money or lives in Russia. But Google has lots of money and a legal presence in almost every jurisdiction.
It's why people say "Donald Trump was held civilly liable for sexual assault in the E Jean Carroll case" instead of "Donald Trump raped E Jean Carroll"
That's the difference between returning search results and interpreting the information and summarising them. If a newspaper says 'so-and-so has been arrested for theft' it's not the same as them summarising to 'so-and-so is a thief', the second is potentially libel. Why should Google be held to a different standard?
The title is misleading IMO. It should say "German ruling declares Google liable for libel in AI Overviews"
I was prepared to say the same thing as you but after reading it seems totally fair.
The key difference is that this would be illegal if a human wrote it too.
Google itself is more trustworthy from a normal person perspective as they use it a lot.
None of "AI" companies call their apps "Entertainment fun text generator". They are call them serious names, use words like "intrllegence" and "thinking".
So yeah I'd think if any of "AIs" start to recommend to drink some bleach or take a flight from a 10th floor window these companies should be liable.
I think it's very clear that Google's AI overviews go far beyond just searching Google because they often incorrectly compile sources to come up with an incorrect answer. For example of this look at the comment I made in this thread
The question is whether Google is publishing false claims or relaying other people’s false claims. The court found it to be the former which makes sense to me.
I go to Google to search, but get spammed as if I wanted to talk to a chatbot (and a very poor quality chatbot at that).
This is a gigantic own goal for Google. The average person’s impression is that Google AI is much worse than ChatGPT, even though that’s not actually the case. Google is shoving a terrible model in everyone’s faces.
Playing the perception game wins you the perspective price.
Nothing is free. Google benefits off you when they show you search page. Either today (ads) or later
It's interesting that the "users can check for themselves" defense was attempted, Google search is so awful now that it's nearly impossible to find anything but ads and ads disguised as links. Even watching people try to use it is infuriating.
The confusion is due to blurring of distinction between the roles: Tool, Tool user and Tool provider. Traditionally, a human-agent who can have an intent and trigger an action is held liable for the consequences of the actions, not the tool. That human agent can be the tool user or tool provider.
Tool user is liable in the case of misuse unintended purpose of the tool.
Tool provider is liable when the use of the tool, by design, causes unintended effects despite proper use of the tool.
A simple "AI may make mistakes" line under the box will not help while the box contains false information. The specific information (lines or words) should not be provided if that's a mistake of false.
The article shows a really good point: statements must be accountable on their own, and cannot rely on "further research". This makes complete sense: at what point can I libel someone, and then dodge consequences with "if you do your research, you can get the facts"?
Another point from the article: they are not just aggregating content, but generating it. If you generate falsehoods, that are not even stated by your sources, of course you're responsible.
This can have significant impact to AI in Germany and the rest of Europe, but it's good to question it and hold people accountable.
Common sense ruling to me. AI != Search, because LLMs are producing statements rather than search results the user has to interpret for themselves. It’s the difference between searching a library’s card catalog for a topic, and a piece of software telling you its answer.
The second these things came on the scene and confidently spoke lies at times, I knew this sort of lawsuit would be inevitable. Nice to see Germany got it right.
I guess the same rules also applied to OpenAI, Anthropic, Microsoft, Meta, and Amazon?
Presumably, yes. Looking forward to the next year of court outcomes!
I agree with the ruling, but this makes me wonder if it will be possible to have any AI agent at all if it's consistently applied.
After all, if I can get ChatGPT or Claude to say something false that should count too, right?
If it's consistently applied, any AI agent provider has to comply with cease and desist letters that tell the company to make specific false claims
The arguments of the ruling should generally apply even when the AI agent makes false statements it wasn't notified about. But in that case the defendant might have a stronger claim about not being able to reasonably ensure the correctness of all statements, and having taken reasonable measures to ensure correctness. Google couldn't really claim any of that after ignoring cease and desist letters about the false claims
Due to costs of running frontier models on every search request, Google simply does not: the failure rate is so high when you are just expecting an objective output.
Imagine a search for your name resulted in an AI summary saying you are involved with child-trafficking because low capability model linked your first name and perhaps a couple articles on supporting children non-profits to it — and then offering that in a convincing sounding summary right at the top!
And the more you protest the more your name will be associated with child trafficking. Streisand effect multiplied by LLMs being not good in dealing with negative information.
In normal flow yes, but likely not if you intentionally entrap it to say something wrong.
A disclaimer and couched language will probably fly through. And it's going to matter what expectations an user could reasonably have, too.
I think it should be relatively possible for an LLM to know when it's talking about something in the risk zone of libel. And when it identifies such a case it links direct to source material rather than trying to fabricate an answer itself. This is much how a journalist works, refusing to make claims that aren't provable.
Someone needs to hold the liability at the end of the day. People are experiencing real harm from false claims LLMs are spitting out.
If you give a language model, empowered through an agent, the ability to publish information on your behalf, and it publishes false information which causes either direct or even indirect harm, and you fail to correct it, then yes.. by every conceivable definition already in law.. you are a criminal.
If you knew this was all possible and you did it anyways for personal gain then you are additionally negligent which may add aggravation to your charges.
As the begetter of the AI, the German court held Google liable for its AI's doing. Common sense ruling.
The ruling also lessened the free speech rights for AI. This is a big one. In conjunction with holding an operator liable for its AI's doings, that will lead to interesting cases where conduct that would have been previously protected under free speech rights will become a liability. Basically, machine-based cognitive capability becomes a liability when it is customer-facing.
Their AI overviews are pretty commonly wrong, so this seems like it might hurt them a bit. Guessing they'll just block them in Germany or throw a bunch of disclaimers around the statements.
Anyone know if this ruling applies to answers generated by AI chatbots, such as ChatGPT/Gemini/Claude?
All three have the ability to perform a web search, then compose a reply based on the search results. Pretty much the exact thing that Google AI Overview does. This ruling may make them liable for false answers.
> Pretty much the exact thing that Google AI Overview does.
No, the article implies the court’s logic is that the AI search results are presented as search results and that’s a big part of why they are liable. It seems like the court (again, according to the article) does not find the disclaimers that Google has slapped on the AI results compelling because again, it chose to represent these as a summary of search results and it is aware of the failure rate.
> The court also found that the AI overview made claims "that are not even made in the search results." None of the linked sources drew any connection between the plaintiffs and the shady companies the AI mentioned. The court called these "the defendant's own statements."
> Google built the AI, Google offered it to users, so Google owns what it produces, "because it alone has influence over the AI's offering and the algorithms with which the AI operates."
Google does not, as a general rule, control the actual content of search results, but usually there’s a distinction between the ranking and presentation of the results vs. the actual content. In this case, the court is basically saying, “You sold this to people as a search summary, you know it might be full of crap and you chose to do it anyway. No, you don’t get to claim the equivalent of a US safe harbor defense.”
> You sold this to people as a search summary, you know it might be full of crap and you chose to do it anyway
There is a subtle difference in stating it as a search summary compared to an opinionated answer. Most users are always going to treat it as a response from google instead of search results where the user is still responsible for understanding and come up with their own interpretation.
This is probably the right step in some sense to make one liable for their statements/assertions.
Apparently, if they were search results, they wouldn't have been liable, since there's an exception to defamation laws. Without any exception, defamation is defamation, it doesn't matter how it's presented.
Yes, if it was that a search result returned a defamatory article that Google had nothing to do with outside of indexing, it is likely they would not be found liable. The court is clearly trying to make a distinction that the AI search results are produced by Google and thus they can make an editorial decision on whether to publish it despite knowing that it is potentially defamatory.
Google does remove defamatory results I believe at least partially in response to being sued. However there is a distinction if they have been informed it is defamatory.
This ruling was about search clearly, however, there's definitely ways implications for chatbots too.
Pretty sure this ruling will be used as a precedent in cases against the other providers really soon. As I understand it, there is a legal cottage industry that brings digital related cases (eg. Copyright) in Germany.
If an AI chatbot would consistently defame a particular company, I’m pretty sure they would be liable too.
I don't see any reason why it wouldn't.
Google generated those content, so Google should be liable to its own product, that's different from the third party links they just simply gathered and displayed, totally different things. I wonder how many victims are there now.
https://hugston.com/news/ban-european-account-pay-billions
https://medium.com/@klaudibregu/google-greedy-gets-karma-8ae...
Curiously, if you look for "geramond verlag betrugsmasche", or "verlagshaus24 betrugsmasche", it will now tell you that
Also, each purported fact now has a direct link to the source of the fact, that is more clearly visible than the previous chain icon.The result I get has an entire section dedicated to scammers using the company name. The only links in that section go to a wikipedia page that doesn't mention any scams and a police help page that doesn't mention the company.
Do the links actually support the statements? When I've followed such links, it's generally been a roll of the dice on whether they do support the AI's statements.
Yes. The links are to isolated threads on a rail (model) forums (apparently, this publisher markets books/magazines related to railway models).
It's hard to see if the individual complaints really support a general problem, or if this simply the only result that talks about "scam + business-name". Probably, the latter.
The same problem happens on google search, if you look "<obscure false fact>", you'll get pages mentioning that false fact. If you fall into the trap of confirmation bias, it leads you to think the false fact is in true.
god i love the eu regulators they're like the only ones even trying to look out for us
Finally someone is trying to regulate AI
What constitutes a correct answer though?
Is something like,
"People online say that x y and z because a b c"
a credible, correct answer, even if it isn't because of a/b/c?
If people do say that, it's a true statement and thus fine. You are allowed to report that regardless of the truth of x/y/z/a/b/c
The instance of this ruling people apparently did not actually say any of the offending claims. 'The court also found that the AI overview made claims "that are not even made in the search results." None of the linked sources drew any connection between the plaintiffs and the shady companies the AI mentioned. The court called these "the defendant's own statements."'
One that doesn't maim/injure/kill you is a pretty good standard. And before you call bs, look at all the foraging and chemistry books that are for sale on Amazon that are AI.
why are those ai chemistry books any different than the anarchist cookbook which can also be bought on Amazon? actually now that I think about it a faulty chemistry book might be less dangerous than a book that teachers readers how to make explosives.
There's a difference between something that properly advertised itself as dangerous, and something that advertises itself as a professional authority.
Certainly, if this is pointing to the actual pages where the actual people express these things. Otherwise that's equally unfalsifiable claims, could be completely made up or actual truth.
One way to formulate things that would be less would be "once support a time, in some fabulated world, it's not impossible that some imaginary character would say something following some reason." But then, of course this is not aligning the the deception scheme pushed by companies putting in their interface that the "machine is thinking hard for you".
I don't fully understand it. Are they liable for content made up by their AI? Or are they now liable for content written by others and summed up (correctly) by their AI?
From my reading the German court's ruling is that Google are responsible both for claims they made up and claims they rephrase from somewhere else. Because the claims appear as a statement of fact made by Google. If they showed it as "User X on site Y claims Z" they would not be held responsible. Because that's how search engines are understood to work. If they make a wrong quote, they would be responsible for that though.
So my understanding is their unreliable AI summaries are a legal liability for Google in Germany and people can request corrections through the courts.
And doesnt this imply they arent ripping off the underlying information?
Oh, I just found out that my Google search doesn't show AI summary anymore! I tried many search queries which typically will show an AI summary, but it only flicked on briefly then disappear entirely. Obviously, Google has reacted quickly on this ruling!
I just tried Google search in Germany on my iPhone: AI results AND the disclaimer was behind a „show more“ button i.e. the may not be any disclaimer (and when shown it was in a small font).
I have a business where our support email is recommended when people are searching for how to cancel a completely unrelated scam subscription that is showing up on their bank or credit card charges. We get emails almost daily from confused people.
Google generated those content, so Google should be liable to its own product, that's different from the third party links they just simply gathered and displayed, totally different things. If you are a victim too, reply below.
If companies can be held liable (in spite of very visible disclaimers, ToS, and usage policies) for the output of non-deterministic software, isn't this just a soft ban on the deployment of non-deterministic software?
> (in spite of very visible disclaimers, ToS, and usage policies)
If you sell food, in a food stall, labeled as food and you add a disclaimer that it is toxic and will make you sick. You are still selling toxic food and you are liable for it.
Google is pretending to give answers to your questions. They offer you a service about answering questions. And then they add a disclaimer "we do not answer questions just write bullshit". That is still fraud and Google should be liable for it.
> isn't this just a soft ban on the deployment of non-deterministic software?
Tetris is non-deterministic and it is not banned like millions of other programs. I do not follow you.
To add to this, a Google search now is answering your question in an incorrect way rather than merely bringing you to a site with incorrect information on it.
They are also no longer covered by safe harbour provisions because it is them answering it, not some content they refer you to.
If Google’s AI overviews used a deterministic LLM the ruling would hold just the same. This has nothing to do with non-determinism.
If your software deterministically produces incorrect output for some inputs, you’re liable the same as if it did it non-deterministically.
If you read those AI overview, it is not non-deterministic, it is conclusions, claims, statements.
Where did it say the liability only applied to non-deterministic software?
no, just a ban on using non-deterministic software for situations where deterministic responses are expected.
Deterministic responses are not expected from something clearly labeled able to make mistakes.
it's not clearly labeled. it's a search engine. i expect deterministic responses from that.
LLMs are deterministic, they are only non-deterministic when you add a temperature.
What to do if the software automatically and wrongly libels you on a public search engine?
Honestly I can understand the ruling, but the side effects might be severe.
So will Google just have to add 'allegedly' to any reply?
Doesn't libel require to be deliberate? Ie you can't sue for libel if the author admits a mistake and corrects it?
It requires the claim to be made with "willful disregard for the truth". Notifying someone, especially with a cease-and-desist on fancy letterhead, makes it legally clear that they know better, and thereafter would be definitely libelling you (assuming the claims are in fact untrue and harmful). But you can still sue them for the claim prior to the notice, you just have to prove they should have known better prior to making the claim.
In this case it looks like they were notified and didn't do anything.
That's just America.
Libel/defamation laws vary wildly across different countries. Sometimes true statements can be considered defamation, sometimes not. The same goes for intent.
This silly game where the EU invents its own rules and tries to impose them globally must stop. Their business is not needed, and the collateral damage with regards to censorship, manipulation, and erosion of fundamental liberties far outpaces any value their continued participation in e-commerce might bring. This sort of paternalistic and insidious bureaucratic encroachment into every aspect of life is completely antithetical to everything the internet should be. We don't owe them anything, and pretending they (and the UK) have some sort of standing in determination of how the world works simply accelerates the degradation.
The solution is simple. Block the EU from accessing any of your services. They can make their own search and social media and digital marketing and AI.
Good luck to them. VPNs will boom.
I can understand the UK struggling with their imperialistic traditions and so on, but you'd think Germans at least would have some humility in trying to project their global ideas and ambitions on the world at this point.
How is this an example of trying to impose rules globally? Google is operating in Germany. Their German operations have to follow German laws. That's how every country operates.
Maybe Google's models will start to suck less as a result. Excellent.
Unless the courts here made the ruling incredibly narrow somehow (only referencing search engines maybe?), how does this not just ban AI in Germany overnight?
Every AI model can make something up sometimes. Over millions of daily calls, it's essentially impossible for the technology to be guaranteed correct 100% of the time.
Good.
Doesn't this imply AI constitutes an original work?
We are actually a victim too, in the same situation as these two publishers. Google's AI makes up false accusations in search result AI overview about our business. We have sent countless feedback to Google in the past 4 months, contacted their legal team 3 times, all got ignored. Google's AI just pukes untrue, misinformation about us, every time the page is refreshed, AI just makes up new accusations and false information, links us to bad actors baselessly, if you refresh the page 10 times, there are 10 different versions, seriously damaged our revenue and reputation. We have filled complaint with the state AG's office, FTC and DOJ. If anyone in legal can help us, please reach out. Thanks. kevinsuntopdev@gmail.com
And people even here on HN actually thinks that AI/LLM are factually correct... The more cases like this the easier it will get to sue them (IANAL)
Why should Google do anything unless not doing anything costs them money? Record the behavior, then contact a lawyer.
That reminds me of the guy who was declared dead by Google and then miraculously wrote a blog post mentioning that rumors of his death were premature. Google (and others) really should be held liable for the fake news they're spreading.
This is actually a good thing. Google's official mission is "to organize the world's information and make it universally accessible and useful."
Now if that information is BS and cannot be relied upon, that’s really bad. Leading people on and not delivering? Honestly, Google themselves should have been on it and not the German government. It’s a bad look for them.
Aside: I’ve noticed their AI mode is pretty pathetic for troubleshooting something. 50% of the times the first response is riddled with inaccuracies and mistakes. Repeat prompting is absolutely necessary (so do not expect to one shot anything).
Also I will admit that I still find myself using it because I’m lazy, and it’s easier to talk to AI to get the right answer. Searching organically is hard these days with the volume of content having gone up exponentially.
Finally, a sensible ruling based in the interests of the public rather than expediency for corporations
Just ban AI in search engine.
Does this extend to ads displayed in search results? Because they absolutely should be liable for the scams they advertise also.
> Does this extend to ads displayed in search results?
Probably not, for the same reason search results aren't an issue.
Search engines have that exemption for results because they don't control the content of the sites they index. That's not the same for ads - Google can and should have more stringent review and KYC processes - but it seems they choose not to.
The original Google search that used inverted indexes and page rank and respected boolean operators was very good and let you actually find obscure information. Call this Google Search 1.0
Then in order to increase revenue Google dumbed down search very badly and made it very hard to find obscure information. Whoever decided to delete random search terms is someone I want to punch in the face really hard. Call this Google Search 2.0
But the way Google has integrated Gemini Flash into search is actually pretty good and is definitely an improvement over Google Search 2.0
Poetic.
Not exactly false answers. As far as I gather this is the same issue that prevents customers from making negative reviews about businesses. The business can just threaten a defamation suit and then the platform (Google Maps, Trustpilot, etc.) are pretty much required to remove the review [0].
So the only thing happening here is they consider Google to be the author of the AI answers and apply a similar sort of anti-defamation law. Perhaps it's a good consequence but the law that allows for this seems to be quite broken.
[0] https://www.settle-in-berlin.com/google-reviews-removal-defa...
Excellent. Even more so with the hostile orange man - Europeans really need to get going.
Overviews is not a good product idea and I think it was done for other deeper reasons. AI shouldn’t be pushed on people because AI can be high variance — it takes time to get adapted to this.
Some digital matters concern the state and others are private and there should be no sovereignty of the state over private matters.
Who will adjudicate private disputes between citizens if not the state?
Not all disputes need to be adjudicated. It's undoubtedly a fact that the German concept of "defamation" primarily relates to situations that should never be adjudicated by any third party.
Genuinely, by far the most common situation where claims of defamation arise in Germany is any less than 5 star review left to a business on google maps.
The good thing is that google in Germany now shows how many reviews were removed due to deflamation. That should be enough for people to make their judgement when checking how many stars a place has
That's just the tip of the iceberg though. Google is the only party who meaningfully communicates this, and it took them years and years to introduce any meaningful transparency.
It's not like this exclusively affects restaurant reviews, every corner of German society is subject to this same evil censorship mechanism.
I completely agree.
Wow, you guys really think this is good?
Because of the same rules, German restaurants also get to pick and choose which reviews stay up. They can literally take down any specific reviews they like.
A restaurant that mostly gets 1 star reviews will still show up with 5 stars on Google maps, as they will simply delete the reviews with less than 5 stars as defamatory.
Here's a couple of examples:
https://www.reddit.com/r/germany/comments/1peujau/google_rev...
https://www.reddit.com/r/germany/comments/18z4shs/legal_thre...
https://www.reddit.com/r/frankfurt/comments/1lox7ha/bad_revi...
https://www.reddit.com/r/germany/comments/1iiaco8/restaurant...
https://www.reddit.com/r/AskGermany/comments/1ha7sxf/why_do_...
https://www.reddit.com/r/SipsTea/comments/1t613w7/it_was_fin...
https://www.reddit.com/r/germany/comments/1l98608/threatenin...
https://www.reddit.com/r/germany/comments/1sw34jc/can_you_ge...
https://www.reddit.com/r/germany/comments/1sw34jc/can_you_ge...
https://www.reddit.com/r/germany/comments/10kmn66/writing_an...
tl;dr Germans are particularly bad at coming up with reasonable rules to handle these situations
And the solution to this is to go through the German political system. The courts can't just decide to not follow the law just because it's a bit silly, and for as long as the laws are the way they are, this unfortunate loophole for removing negative reviews will continue to exist. But clearly the law can have both positive and negative consequences.
> But clearly the law can have both positive and negative consequences.
Broad defamation laws always have overwhelmingly negative consequences.
But it also shows the number of removed reviews (in a range like 0-50, 50-100 and so forth). If you encounter a restaurant with 5 starts but a lot of removed reviews you know whats up.
That's a fairly new workaround that took Google years to come up with.
We should be teaching people to be cynical of AI answers.
Even if the answers are correct, they could still be biased, incomplete, misleading, and all the other media-literacy things people should be looking out for.
This ruling seems to go the opposite direction; 'I am legally obliged to give correct answers, so I am always right, trust the AI'.
First comment that makes sense. Of course AI needs to be verified. Of course it gives bs answers. The answer is to educate the public not try to daddy then. I hope AI is completely banned from Germany for their protection since apparently their government doesn't believe its people know how to question AI.
I've found a fun and pretty reliable way to get Gemini to output incorrect information: Ask for a chapter by chapter summary of a book.
I first tried it to remind me of what happened in a previous book in a series that I was reading. When I realized it was either misstating plot points or straight up hallucinating, I tried it on a bunch more books to amuse myself.
Older classics are of course more accurate, but for newer or less popular books Gemini won't shy away from giving you a summary culled from misinterpreted Reddit threads and Goodreads reviews. It's like getting a secondhand account from someone who talked to another person who had read the book a long time ago. You get the general gist of it, but with some added flavor.
Even if you upload an entire epub of a book, the results aren't stellar. Rather than a Cliffs Note's quality summary, they're pretty sparse or leave out important bits of information. One chapter summary I got back made a point of describing what one of the characters was wearing, even though it had absolutely zero to do with anything else. Yes, that's technically a "summary", but not quite my tempo.
If Google wants to present summaries of websites in anything more than a very, very superficial description, they're going to have to improve their model's ability to understand context and importance. In theory, a novel is a self-contained bundle of text, so pulling accurate information out of it should be straight forward. A website is naturally going to be way more of a challenge.
All that said, I find the AI summaries from Google/Gemini to be quite useful and a time saver, but I know to always double check something if it's at all important.
Next thing is declare our rulers to be liable for doing things that fuck up people's lives. That'll really wake me up
My feelings about this rest of the scope of liability. From my understanding, Google is now liable if making false claims about a personal/business reputation. I like this idea in theory (key word).
However, I can easily see the slippery slope where in practice this means providing any AI response becomes too risky, and it becomes another money club used to extract wealth from big tech due to the current hysterical anti-AI moral panic.
Which would ultimately kill the ability of the German people to get access to competitive AI models.
I'm sure many anti-tech/anti-civilization doomers on HN will cheer this on. However, in reality it would do nothing to stop Germany falling behind, and continue its economic malaise/low productivity growth and social welfare collapse. When things in Germany get bad, Germans historically have tended to...ummm...cause issues for Europe. I would not take this lightly given the current rise of more polarized political rhetoric and German economy pivoting hard into weapons manufacturing.
It is not anti-AI, it is anti misusing AI. Clearly you are trying to manipulate the situation here. If your AI is half baked and produce garbages, you should be liable for the damage of those garbages, that applies to any product or service on this planet, not only just AI.If your product or service is defect, that causes harm, you are liable.
I agree with this, in theory.
But to loudly proclaim your love for this idea as most have here seems a bit premature given there are potentially massive downsides to applying legal liability to an AI model, if not done extremely carefully.
I'm just trying to move a bit beyond the emotional first order thinking. You might see this as manipulation if you love just swimming in emotional vibes, but I find this website becoming quite boring lately due to the lack of debate beyond this first-order thinking.
As someone living in Europe, watching a bunch of rich American engineers fanboy European over-regulation which they aren't actually being subjected to, is super cringe. It's the epitome of the principal-agent problem.
The slippery slope of being held liable for an AI overview leading to neo-Hitler is certainly a take, and a ridiculous one at that.
It's one thing for an LLM to get things wrong. This case is not that though. And if big tech can't make sure that their models don't libel companies and individuals then good riddance. Whatever diminutive economic advantage one can get from "competitive LLMs" probably isn't worth it anyway, especially with all of the other disadvantages of these "competitive LLMs" compounding on our societies and the planet, as the resource usage of the data centres necessary to run and train them exacerbate the ongoing climate crisis.
You've formed a strawman out of my argument.
My point is Germany further rejecting participation in the the next wave of technology (as they have done for 50 years) is going to NOT improve the trajectory they are currently on...which is bad.
The neo-Hitler and re-militarization thing is already happening. AfD leads in the polls, they're opening talking about remigration of immigrants, Germany now produces more ammunition than any country on earth. AI has nothing to do with it.
And, given I'm speaking to one of the the anti-civilization/anti-tech doomers of HN as predicted, do you think Germany rejecting technology again is going to improve the "climate crisis" or hurt it? How is the de-commissioning of Nuclear working out there?
Their digital sovereignty
EU countries continuing to ensure the conditions for their future economic competitivity
mr prime minister, we cannot allow an "automatic libel machine" gap!
So stupid. What is this with making perfect the enemy of good. You can never guarantee the output of an LLM does that mean Germany does get to use them?
No, it means that Google will be held liable for false or libelous information it provides to users by means of generative AI. It's not banned. Of course, if Google decides that its generative AI produces too much false or libelous content, it may choose to stop doing so in Germany, where it will be held liable for damages it causes.
I guess there is a small benefit to living in a police state!
Companies should just stop providing AI to police states like Germany. Apparently their citizens believe anything they read.
So if i were to write a false statement about you that would result in legal consequences for you, im not liable?
Good point, I didn't consider that.