Without even looking at the AI part, I have a single question: Did anybody investigate? That's it.
Whether it's AI that flagged her, or a witness who saw her, or her IP address appeared on the logs. Did anybody bothered to ask her "where were you the morning of july 10th between 3 and 4pm. But that's not what happened, they saw the data and said "we got her".
But this is the worst part of the story:
> And after her ordeal, she never plans to return to the state: “I’m just glad it’s over,” she told WDAY. “I’ll never go back to North Dakota.”
That's the lesson? Never go back to North Dakota. No, challenge the entire system. A few years back it was a kid accused of shoplifting [0]. Then a man dragged while his family was crying [1]. Unless we fight back, we are all guilty until cleared.
The thing about the legal system is there's no incentive to investigate to find the truth.
The incentive is to prosecte and prove the charges.
Speaking from the experience of being falsely accused after calling 911 to stop a drunk woman from driving.
The narrative they "investigated" was so obviously false, bodycam evidence directly contradicted multiple key facts. Officials are interested only seeking to prove the case. Thankfully the jury came to the right verdict.
Yes, of course someone should have investigated, but the larger point here is that people don’t because they are being sold a false narrative that AI is infallible and can do anything.
We could sit here all day arguing “you should always validate the results”, but even on HN there are people loudly advocating that you don’t need to.
"The trauma, loss of liberty, and reputational damage cannot be easily fixed,” Lipps' lawyers told CNN in an email.
That sounds a LOT like a statement you make for before suing for damages, not to mention they literally say "Her lawyers are exploring civil rights claims but have yet to file a lawsuit, they said."
This lady probably just wants to go back to normal life and get some money for the hell they put her in. She has never been on a airplane before, I doubt she is going to take on the entire system like you suggest. Easier said than done to "challenge the entire system", what does that even mean exactly?
It was worse than that, the reporting from an earlier story[0]
...Unable to pay her bills from jail, she lost her home, her car and even her dog.
There is not a jury in the country that will side against the woman. I am not even sure who will make the best pop culture mashup - John Wick or a country song writer?
(Also, what happened to journalism - no Oxford comma?)
It absolutely was. There's no question of this. Now we need to ask how was the system marketed, what did the police pay for it, how were they trained to use it?
> anybody bothered to ask her "where were you the morning of july 10th between 3 and 4pm.
Legally that amounts "hearsay" and cannot have any value. Those statements probably won't even be admissible in court without other supporting facts entered in first.
> we are all guilty until cleared.
This is not at a phenomenon that started with AI. If you scratch the surface, even slightly, you'll find that this is a common strategy used against defendants who are perceived as not being financially or logistically capable of defending themselves.
We have a private prison industry. The line between these two outcomes is very short.
The vendor they used, Clearview AI, does not allow you to request data deletion unless you live in one of the half-dozen states that legally mandate it.
Sadly this is really the only tool we have right now. Just have to keep spamming them with delete requests because once they delete it’ll end up back in their database eventually.
To get your data deleted in the states that require it you have to submit a photo of yourself which I really don't want to do for a sketchy company with ties to evil billionaire Peter Thiel.
First, the detective used the FaceSketchID system, which has been around since around 2014. It is not new or uniquely tied to modern AI.
Second, the system only suggests possible matches. It is still up to the detective to investigate further and decide whether to pursue charges. And then it is up to court to issue the warrant.
The real question is why she was held in jail for four months. That is the part that I do not understand. My understanding is that there is 30-day limit (the requesting state must pick up the defendant within 30 day).
Regarding the individual involved, Angela Lipps, she has reportedly been arrested before, so it is possible she was on parole. So maybe they were holding her because of that?
In the US there are no consequences for people in power failing to follow procedures, laws or regulations - except for being told to stop doing whatever illegal thing they're doing, and possibly getting sued way down the line, which gets paid by taxpayers.
From reading more into the case, it seems the issue may be related to how her lawyer handled the case.
They probably did “identity challenge” arguing that she is not the right person. But from Tennessee’s perspective, she was considered the correct person to be arrested, so there was no “mistaken identity” in their system. In other words, North Dakota Wanted person x and here is person x.
Once a judge in North Dakota reviewed the full evidence (and found that person they issued warrant for arrest is not one they want), the case was dismissed.
The judge likely issued the warrant based on the detective’s sworn testimony. In most cases, a judge does not have the ability or detailed knowledge to independently verify whether the detective completed all necessary checks.
This situation likely resulted from either sloppy investigative work or an honest mistake: the detective believed her booking photo matched the individual captured on camera.
This isn’t how it works, you can invoke your right to a speedy trial at any point you want. You can spend 2 months waiting and then invoke it if you want.
The timer starts from when you invoke it, though.
The 2 issues, which she may be caught in, are that it’s “speedy” from the perspective of a court, and that it really means “free from undue delays”.
There is no general definition of a speedy trial, but I think the shortest period any state defines is a month (with some states considering several months to still be “speedy”).
A trial can still be speedy even past that window if the prosecution can make a case that they genuinely need more time (like waiting for lab tests to come back).
It’s basically only ever not speedy if the prosecution is just not doing anything.
You get charged with something and if you want to have the trial right now, before you have any idea what's going on, then you can insist, which basically nobody does because it's pretty crazy to go in blind
Actually most criminal defense attorneys recommend not waiving your speedy trial rights. Yes, the defense goes in blind. But so does the prosecution, and they're the ones that have to make a case.
The usual result for defendants that don't waive their speedy trial rights is an acquittal if the case goes to trial (between 50-60%), which doesn't sound like a lot but prosecutors are expected to win >90% of their trials. Additionally, in many counties they don't have sufficient courtrooms to handle all the criminal trials within the speedy trial timeframe, so if the trial date comes and a courtroom is not available the case is dismissed with prejudice. Nonviolent misdeameanors are the lowest priority for a courtroom (and by that I mean even family law cases have priority over nonviolent misdos in most counties), so those cases are frequently dismissed a day or two before the trial date. Consequently, most prosecutors will offer better and better plea bargains as the trial date approaches.
This is even more true for murders, which is why murder suspects don't usually get charged for a year or two after the crime.
> The real question is why she was held in jail for four months. That is the part that I do not understand. My understanding is that there is 30-day limit (the requesting state must pick up the defendant within 30 day). Regarding the individual involved, Angela Lipps, she has reportedly been arrested before, so it is possible she was on parole. So maybe they were holding her because of that?
As the article gestures towards, challenging the extradition can greatly extend the timeline, from 30 days after the arrest to 90 days after a formal identity hearing. Which isn't fair and isn't intuitive, but is unfortunately a long-standing part of the system. (Even worse, this kind of mistaken identity can't be challenged in an extradition hearing; the question isn't whether she's the person who committed the crime but whether she's the person identified in the warrant.)
Her picture was used as part of a fake id card, in the commission of a crime. The fuzzy camera footage looked like her (from stills I've seen) and her picture was on the fake ID. Those 2 circumstantial items were, apparently, enough to have a warrant issued.
They picked her up in TN and held her for 4 months, even after:
The ND police knew the ID was fake and the person using it was not her.
The ND police knew she had been in TN before, during, and after the crime.
She is still technically a suspect, even after all of this has come out.
> It is still up to the detective to investigate further and decide whether to pursue charges. And then it is up to court to issue the warrant.
This is how it should work, but I still think it is important to discuss these failures in the context of AI risks.
One of the largest real-world dangers of AI (as we define that now) is that it is often confidently wrong and this is a terrible situation when it comes to human factors.
A lot of people are wired in such a way that perceived confidence hacks right through their amygdala and they immediately default to trust, no matter how unwarranted.
I've been hearing "it's not just... it's a" touted as an AI sign recently, personally I think it's an AI sign because it's a human thinking shortcut sign, and AI copies it, but it would be funny if AI wrote the article and then hallucinated this specific money quote.
I doubt this happened here, but FWIW, AI does have a habit of "cleaning up" (read: hallucinating) interview transcript quotes if you ask it to go through a transcript and pull quotes. You have to prompt AI very specifically to get it to not "clean up" the quotes when you ask it to do that task.
... which is why the institutions that assign responsibility and consequences need to make it really clear that excuse won't fly. With illustrative examples.
There is enormous variability in how hard a tool is to use correctly, how likely it is to go wrong, and how severe the consequences are. AI has a wide range on all those variables because its use cases vary so widely compared to a hammer.
The use case here is police facial recognition. Not hitting nails. The parent wasn't saying "AI is a liability" with no context.
When somebody uses a tool to hurt somebody, they need to be held accountable. If I smack you with a hammer, that needs to be prosecuted. Using AI is no different.
The problem here is incidental to the tool; it was done by the cops and therefore nobody will be held accountable.
Systems are also a tool. Whoever institutes and helps build the system that systematically results in harm is also responsible.
That would be the vendors, the system planners, and the institutions that greenlit this. It would also include the larger financial tech circle that is trying to drive large scale AI adoption. Like Peter Thiel, who sees technology as an "alternative to politics". I.e. a way to circumvent democracy [1]
This tool, however, is specifically built for mass surveillance. It serves no other purpose. The tool is broken, and everybody knows it. The tool makers are at least as guilty as those who use it.
Yes. But doing the investigation negates much of the incentive for using AI.
Look for similar to play out elsewhere --- using unreliable tools for decision making is not a good, responsible business plan. And lawyers are just waiting to press the point.
In this case it sounds as though AI could have been used to generate preliminary leads. When someone calls a tip line with information, police don’t just take their word for it, they investigate it. They know that tips they receive may be incorrect. They should have done the exact same here, but they didn’t.
I’m very opposed to AI in general, but this one is clearly human failure.
The noteworthy AI angle is the undeserved credence police gave to AI information. But that is ultimately their failure; they should be investigating all information they receive.
The failure starts with tool vendors who market these statistical/probabilistic pattern searchers as "intelligent". The Fargo police failed to fully evaluate these marketing claims before applying them to their work.
So in the same way that the failure rolled down hill, liability needs to roll back up.
The article says that the Fargo police claimed to have done "additional investigative steps independent of AI". (Perhaps they're lying, or did a poor job because they thought the extra steps were a formality.)
We will find out. But relying on AI is likely to cost the city of Fargo in one way or another. They say they have already stopped using AI and returned to good old fashioned human investigation.
Look, I'm generally considered AI's most vociferous detractor.
But...
> there is no way to tell if you are using it "correctly".
This simply isn't true, at least in cases like this.
I know common sense isn't really all that common, but why would you give more credence to an untested tool than an untested crack-addled human informant?
The entire point of the informant, or the AI in this instance, is to generate leads. Which subsequently need to be checked.
What kind of outcome results from misuse? Clearly a hammer's misuse has very little in common with a global, hivemind network used in high-stake campaigns.
Now, if I misused a hammer and it hurt everyone's thumb in my country, then maybe what you said would have some merit.
Otherwise, I'd say it's an extremely lazy argument
AFAIK the actual cause for our high incarceration rate is that we have longer sentences. The conviction rate, for example, as compared to the UK is similar.
This appears to be The Sort in action again. The 50% of Americans below IQ 100 also need jobs and so on. Perhaps with AI pushing out people from high-intelligence jobs, we will get a large number of intelligent people in jobs like police or retail pharmacists or so on. Currently, these guys can barely read text and follow instructions. In fact, most of them are likely functionally illiterate and are coaxed through their programs by a system that is punished if it does not pass people.
The average policeman will find his brain sorely taxed by the average incident report form. Describing the phrase "false positive" to them is like trying to explain calculus to a mouse.
Has it not been fairly common to require police officers to have a bachelor’s degree? Or an associate’s? I think recently that has been relaxed but I’ve lived in places where it was absolutely a requirement.
Police departments are known to avoid hiring people that get high marks in school, under the principle that such individuals will become bored with the job and quit. They literally look for average people with average intelligence: C students.
Now factor in the slow decline of our educational institutions, where grade inflation has systematically diminished the credibility of a degree. I would wager that many C students today would have failed out completely 30 years ago.
In that light, it is not surprising that people are seeing ICE agents behave like brown shirts. No one in power wants those people asking any kind of hard questions about what they are being ordered to do.
Without even looking at the AI part, I have a single question: Did anybody investigate? That's it.
Whether it's AI that flagged her, or a witness who saw her, or her IP address appeared on the logs. Did anybody bothered to ask her "where were you the morning of july 10th between 3 and 4pm. But that's not what happened, they saw the data and said "we got her".
But this is the worst part of the story:
> And after her ordeal, she never plans to return to the state: “I’m just glad it’s over,” she told WDAY. “I’ll never go back to North Dakota.”
That's the lesson? Never go back to North Dakota. No, challenge the entire system. A few years back it was a kid accused of shoplifting [0]. Then a man dragged while his family was crying [1]. Unless we fight back, we are all guilty until cleared.
[0]: https://www.theregister.com/2021/05/29/apple_sis_lawsuit/
[1]: https://news.ycombinator.com/item?id=23628394
The thing about the legal system is there's no incentive to investigate to find the truth.
The incentive is to prosecte and prove the charges.
Speaking from the experience of being falsely accused after calling 911 to stop a drunk woman from driving.
The narrative they "investigated" was so obviously false, bodycam evidence directly contradicted multiple key facts. Officials are interested only seeking to prove the case. Thankfully the jury came to the right verdict.
Yes, of course someone should have investigated, but the larger point here is that people don’t because they are being sold a false narrative that AI is infallible and can do anything.
We could sit here all day arguing “you should always validate the results”, but even on HN there are people loudly advocating that you don’t need to.
We can barely convince powers thar be that eye-witness testimony is unreliable, after all.
I think you missed many important points.
"The trauma, loss of liberty, and reputational damage cannot be easily fixed,” Lipps' lawyers told CNN in an email.
That sounds a LOT like a statement you make for before suing for damages, not to mention they literally say "Her lawyers are exploring civil rights claims but have yet to file a lawsuit, they said."
This lady probably just wants to go back to normal life and get some money for the hell they put her in. She has never been on a airplane before, I doubt she is going to take on the entire system like you suggest. Easier said than done to "challenge the entire system", what does that even mean exactly?
It was worse than that, the reporting from an earlier story[0]
There is not a jury in the country that will side against the woman. I am not even sure who will make the best pop culture mashup - John Wick or a country song writer?(Also, what happened to journalism - no Oxford comma?)
[0] https://news.ycombinator.com/item?id=47356968
As an aside AP Style is not use an Oxford comma, and that's been the rule for 50+ years https://www.prnewsonline.com/explainer-how-to-use-oxford-com...
This is upsetting.
Yes, finding out how badly wrong you were is never fun. Of course the lack of ubiquitous Oxford comma use is itself and separately displeasing.
AP Style is simply wrong on this, then.
You have more faith in the country than I do.
anyone in the chain of responsibility should be punished so severely that they will be still crying about it in 2030
Indeed let out on Christmas Eve with no money 1000 miles from your homeland.
Where your home was lost to foreclosure because one JUDGE did not look at the paperwork.
There should be a way to personally sue somebody when they don't do their job. Protecting the innocent. The JUDGE failed badly here.
Flimsy evidence would mean no warrant. Do your basic investigation please... Rubberstamping JUDGE caused this.
Why are they not named? Like they are a spectator. Infact they are the cause.
> Whether it's AI that flagged her
It absolutely was. There's no question of this. Now we need to ask how was the system marketed, what did the police pay for it, how were they trained to use it?
> anybody bothered to ask her "where were you the morning of july 10th between 3 and 4pm.
Legally that amounts "hearsay" and cannot have any value. Those statements probably won't even be admissible in court without other supporting facts entered in first.
> we are all guilty until cleared.
This is not at a phenomenon that started with AI. If you scratch the surface, even slightly, you'll find that this is a common strategy used against defendants who are perceived as not being financially or logistically capable of defending themselves.
We have a private prison industry. The line between these two outcomes is very short.
IANAL but AFAIK custodial interrogation triggers Miranda, lawyers, and those awful awful civil liberties we’re trying to get rid of.
Better just to apply Musk or Altman software to the problem and avoid it entirely.
The vendor they used, Clearview AI, does not allow you to request data deletion unless you live in one of the half-dozen states that legally mandate it.
https://www.clearview.ai/privacy-and-requests
I have suddenly becomes very interested in New York's S1422 Biometric Privacy Act.
Sadly this is really the only tool we have right now. Just have to keep spamming them with delete requests because once they delete it’ll end up back in their database eventually.
related https://noyb.eu/en/criminal-complaint-against-facial-recogni...
To get your data deleted in the states that require it you have to submit a photo of yourself which I really don't want to do for a sketchy company with ties to evil billionaire Peter Thiel.
This is a weak or misleading story about AI.
First, the detective used the FaceSketchID system, which has been around since around 2014. It is not new or uniquely tied to modern AI.
Second, the system only suggests possible matches. It is still up to the detective to investigate further and decide whether to pursue charges. And then it is up to court to issue the warrant.
The real question is why she was held in jail for four months. That is the part that I do not understand. My understanding is that there is 30-day limit (the requesting state must pick up the defendant within 30 day). Regarding the individual involved, Angela Lipps, she has reportedly been arrested before, so it is possible she was on parole. So maybe they were holding her because of that?
Can someone clarify how that process works?
In the US there are no consequences for people in power failing to follow procedures, laws or regulations - except for being told to stop doing whatever illegal thing they're doing, and possibly getting sued way down the line, which gets paid by taxpayers.
From reading more into the case, it seems the issue may be related to how her lawyer handled the case.
They probably did “identity challenge” arguing that she is not the right person. But from Tennessee’s perspective, she was considered the correct person to be arrested, so there was no “mistaken identity” in their system. In other words, North Dakota Wanted person x and here is person x.
Once a judge in North Dakota reviewed the full evidence (and found that person they issued warrant for arrest is not one they want), the case was dismissed.
Yes but a judge issued the warrant in the first place.
Cops did not do a proper investigation and the judge green-lighted it.
It is all on the JUDGE or possibly a magistrate who approved a faulty warrant.
The judge failed the poor woman. FIRE him.
Then sue Clearview for big bucks.
The judge likely issued the warrant based on the detective’s sworn testimony. In most cases, a judge does not have the ability or detailed knowledge to independently verify whether the detective completed all necessary checks.
This situation likely resulted from either sloppy investigative work or an honest mistake: the detective believed her booking photo matched the individual captured on camera.
Her booking photo from a prior arrest can be found here: https://mugshots.com/US-States/Tennessee/Carter-County-TN/An...
Do we have recording of the suspect they used for the match?
There's a screenshot from a security camera with the _actual_ suspect in the article that was posted to HN last week.
https://www.grandforksherald.com/news/north-dakota/ai-error-...
what's with the weird obsession all over the thread that it is the JUDGE who is the only person at fault here?
It’s the same poster, I assumed they were ai at first but the account is from 2017.
Some people are just weird
[deleted]
This isn’t how it works, you can invoke your right to a speedy trial at any point you want. You can spend 2 months waiting and then invoke it if you want.
The timer starts from when you invoke it, though.
The 2 issues, which she may be caught in, are that it’s “speedy” from the perspective of a court, and that it really means “free from undue delays”.
There is no general definition of a speedy trial, but I think the shortest period any state defines is a month (with some states considering several months to still be “speedy”).
A trial can still be speedy even past that window if the prosecution can make a case that they genuinely need more time (like waiting for lab tests to come back).
It’s basically only ever not speedy if the prosecution is just not doing anything.
You get charged with something and if you want to have the trial right now, before you have any idea what's going on, then you can insist, which basically nobody does because it's pretty crazy to go in blind
Actually most criminal defense attorneys recommend not waiving your speedy trial rights. Yes, the defense goes in blind. But so does the prosecution, and they're the ones that have to make a case.
The usual result for defendants that don't waive their speedy trial rights is an acquittal if the case goes to trial (between 50-60%), which doesn't sound like a lot but prosecutors are expected to win >90% of their trials. Additionally, in many counties they don't have sufficient courtrooms to handle all the criminal trials within the speedy trial timeframe, so if the trial date comes and a courtroom is not available the case is dismissed with prejudice. Nonviolent misdeameanors are the lowest priority for a courtroom (and by that I mean even family law cases have priority over nonviolent misdos in most counties), so those cases are frequently dismissed a day or two before the trial date. Consequently, most prosecutors will offer better and better plea bargains as the trial date approaches.
This is even more true for murders, which is why murder suspects don't usually get charged for a year or two after the crime.
Apparently I set up a unit test for Cunningham's Law today.
> The real question is why she was held in jail for four months. That is the part that I do not understand. My understanding is that there is 30-day limit (the requesting state must pick up the defendant within 30 day). Regarding the individual involved, Angela Lipps, she has reportedly been arrested before, so it is possible she was on parole. So maybe they were holding her because of that?
As the article gestures towards, challenging the extradition can greatly extend the timeline, from 30 days after the arrest to 90 days after a formal identity hearing. Which isn't fair and isn't intuitive, but is unfortunately a long-standing part of the system. (Even worse, this kind of mistaken identity can't be challenged in an extradition hearing; the question isn't whether she's the person who committed the crime but whether she's the person identified in the warrant.)
That is my assumption. I assume who ever was representing her made a mistake and challenged the warrant and that caused delay in the extradition.
I wish I could find the link, but I believe she was in jail on parole violation, unrelated to anything that the "AI" flagged her on.
Her picture was used as part of a fake id card, in the commission of a crime. The fuzzy camera footage looked like her (from stills I've seen) and her picture was on the fake ID. Those 2 circumstantial items were, apparently, enough to have a warrant issued.
They picked her up in TN and held her for 4 months, even after:
The ND police knew the ID was fake and the person using it was not her. The ND police knew she had been in TN before, during, and after the crime.
She is still technically a suspect, even after all of this has come out.
Ok. The mistake was made by North Dakota police (and they blame AI - the AI just gave them a possible match. Whatever.).
What I still do not understand is why she spent nearly six months in a Tennessee jail. That part remains unclear and needs further explanation.
She was not
Source: I live in Fargo and have been following this story closely. Everyone here is pissed
Thanks for clarifying.
I wonder who is slandering her more... WOW
Maybe the citys insurance carrier hired a FIRM...
They will be taking a hit.
That is the first I have heard of that. A small unexplained blurb in this article. Already in jail on parole violation..
Maybe she objected to the extradition order without good counsel.
"I aint never been to N.Dakota". She found out the hard way how the law works..
What about the banks being hit. Surely they have good cameras. This was bad mojo. I would think a Wells Fargo/BoA has a unit for this stuff.
Finincial crimes handled like this. The banks will be sued too I suspect.. Deep pockets settle out.
> It is still up to the detective to investigate further and decide whether to pursue charges. And then it is up to court to issue the warrant.
This is how it should work, but I still think it is important to discuss these failures in the context of AI risks.
One of the largest real-world dangers of AI (as we define that now) is that it is often confidently wrong and this is a terrible situation when it comes to human factors.
A lot of people are wired in such a way that perceived confidence hacks right through their amygdala and they immediately default to trust, no matter how unwarranted.
Earlier discussion (405 comments):
https://news.ycombinator.com/item?id=47356968
Money quote from someone quoted in the article:
"[I]t’s not just a technology problem, it’s a technology and people problem."
I can't. I just can't.
I've been hearing "it's not just... it's a" touted as an AI sign recently, personally I think it's an AI sign because it's a human thinking shortcut sign, and AI copies it, but it would be funny if AI wrote the article and then hallucinated this specific money quote.
I doubt this happened here, but FWIW, AI does have a habit of "cleaning up" (read: hallucinating) interview transcript quotes if you ask it to go through a transcript and pull quotes. You have to prompt AI very specifically to get it to not "clean up" the quotes when you ask it to do that task.
And why not?
If you look at examples of people quoting on the internet, lots are out of context, paraphrased, or made up.
AI is just mimicking what it has seen.
Wow thought the bar for probable cause for an arrest warrant would be much higher. Especially to drag soneone from another state.
A lot of dumb shit happens in this arena, where if you had just one smart cop, it could have been prevented. Here’s one from 2023:
https://youtu.be/lPUBXN2Fd_E
AI is a liability issue waiting to happen. And this is just another example.
It's the opposite, it's absolution from liability. "The AI did it" is the ultimate excuse to avoid accepting responsibility and consequences.
Courts are already refusing to accept this excuse.
https://pub.towardsai.net/the-air-gapped-chronicles-the-cour...
Good to know it's not a fait accompli yet, but I wont be surprised to see corporations pushing for this hard.
... which is why the institutions that assign responsibility and consequences need to make it really clear that excuse won't fly. With illustrative examples.
It’s a tool. Used incorrectly will lead to errors. Just like a hammer, used incorrectly could hit the users finger.
There is enormous variability in how hard a tool is to use correctly, how likely it is to go wrong, and how severe the consequences are. AI has a wide range on all those variables because its use cases vary so widely compared to a hammer.
The use case here is police facial recognition. Not hitting nails. The parent wasn't saying "AI is a liability" with no context.
When somebody uses a tool to hurt somebody, they need to be held accountable. If I smack you with a hammer, that needs to be prosecuted. Using AI is no different.
The problem here is incidental to the tool; it was done by the cops and therefore nobody will be held accountable.
Systems are also a tool. Whoever institutes and helps build the system that systematically results in harm is also responsible.
That would be the vendors, the system planners, and the institutions that greenlit this. It would also include the larger financial tech circle that is trying to drive large scale AI adoption. Like Peter Thiel, who sees technology as an "alternative to politics". I.e. a way to circumvent democracy [1]
[1] https://stavroulapabst.substack.com/p/techxgeopolitics-18-te...
This tool, however, is specifically built for mass surveillance. It serves no other purpose. The tool is broken, and everybody knows it. The tool makers are at least as guilty as those who use it.
The tool, like Google search, is likely biased towards returning results regardless of confidence.
Used incorrectly will lead to errors.
Only one small little problem --- there is no way to tell if you are using it "correctly".
The only way to be sure is to not use it.
Using it basically boils down to, "Do you feel lucky?".
The Fargo police didn't get lucky in this case. And now the liability kicks in.
Some basic investigatory police work (the kind they did before AI) would have revealed the mistake before an innocent woman’s life was destroyed.
Yes. But doing the investigation negates much of the incentive for using AI.
Look for similar to play out elsewhere --- using unreliable tools for decision making is not a good, responsible business plan. And lawyers are just waiting to press the point.
In this case it sounds as though AI could have been used to generate preliminary leads. When someone calls a tip line with information, police don’t just take their word for it, they investigate it. They know that tips they receive may be incorrect. They should have done the exact same here, but they didn’t.
I’m very opposed to AI in general, but this one is clearly human failure.
The noteworthy AI angle is the undeserved credence police gave to AI information. But that is ultimately their failure; they should be investigating all information they receive.
...but this one is clearly human failure.
Absolutely.
The failure starts with tool vendors who market these statistical/probabilistic pattern searchers as "intelligent". The Fargo police failed to fully evaluate these marketing claims before applying them to their work.
So in the same way that the failure rolled down hill, liability needs to roll back up.
AI can provide leads. Someone still needs to verify them and decide.
Generating and verifying bad leads costs money. Not verifying bad leads can cost much more.
At some point, you have to decide if wasting good money on bad intel makes sense.
The article says that the Fargo police claimed to have done "additional investigative steps independent of AI". (Perhaps they're lying, or did a poor job because they thought the extra steps were a formality.)
Now the "qualified" immunity kicks in.
We will find out. But relying on AI is likely to cost the city of Fargo in one way or another. They say they have already stopped using AI and returned to good old fashioned human investigation.
https://www.lawlegalhub.com/how-much-is-a-wrongful-arrest-la...
Look, I'm generally considered AI's most vociferous detractor.
But...
> there is no way to tell if you are using it "correctly".
This simply isn't true, at least in cases like this.
I know common sense isn't really all that common, but why would you give more credence to an untested tool than an untested crack-addled human informant?
The entire point of the informant, or the AI in this instance, is to generate leads. Which subsequently need to be checked.
There is no "correct" way to use AI in order to avoid bad results. The only prudent approach is to assume all results are bad until proven otherwise.
But this approach negates much of the incentive to pay for AI.
What kind of outcome results from misuse? Clearly a hammer's misuse has very little in common with a global, hivemind network used in high-stake campaigns.
Now, if I misused a hammer and it hurt everyone's thumb in my country, then maybe what you said would have some merit.
Otherwise, I'd say it's an extremely lazy argument
Unlike hammers people preface things with "claude says", etc. I never see that kind of distancing with tools that aren't AI.
Dynamite is a tool. But we don't hand it out to anyone who wants to play with it.
We used to until quite recently. Anybody could buy dynamite at the hardware store. We had to end this because of criminals using it to hurt people.
I admit I was surprised to see you could buy dynamite in a hardware store until 1970.
Look for AI to follow a similar trajectory over time.
Impossible at this point. You cannot download dynamite.
Yes, regulation is inevitable.
Regulation is impossible. The AI barons literally control the federal government, so not even state regulations get tried.
Except this time the criminals are police.
They are far more often than anyone wants to admit. That's how we got 25% of the world's prison population.
AFAIK the actual cause for our high incarceration rate is that we have longer sentences. The conviction rate, for example, as compared to the UK is similar.
AI feels closer to a firearm than a hammer when accessing law enforcement's ability to quickly do massive, unrecoverable harm.
This appears to be The Sort in action again. The 50% of Americans below IQ 100 also need jobs and so on. Perhaps with AI pushing out people from high-intelligence jobs, we will get a large number of intelligent people in jobs like police or retail pharmacists or so on. Currently, these guys can barely read text and follow instructions. In fact, most of them are likely functionally illiterate and are coaxed through their programs by a system that is punished if it does not pass people.
The average policeman will find his brain sorely taxed by the average incident report form. Describing the phrase "false positive" to them is like trying to explain calculus to a mouse.
Has it not been fairly common to require police officers to have a bachelor’s degree? Or an associate’s? I think recently that has been relaxed but I’ve lived in places where it was absolutely a requirement.
I don’t think they’re as stupid as you suggest.
Police departments are known to avoid hiring people that get high marks in school, under the principle that such individuals will become bored with the job and quit. They literally look for average people with average intelligence: C students.
Now factor in the slow decline of our educational institutions, where grade inflation has systematically diminished the credibility of a degree. I would wager that many C students today would have failed out completely 30 years ago.
In that light, it is not surprising that people are seeing ICE agents behave like brown shirts. No one in power wants those people asking any kind of hard questions about what they are being ordered to do.
Police departments won the right to discriminate _against_ intelligence in 1997 on the Jordan vs The City of New London case[1].
They literally aim to be dumber than average.
[1] https://en.wikipedia.org/wiki/Wonderlic_test
Having a degree is a very low bar for intelligence.
> I don’t think they’re as stupid as you suggest.
I'll just leave this here:
https://abcnews.com/US/court-oks-barring-high-iqs-cops/story...
It says the national average is just slightly above average IQ. Sounds fine.
> Sounds fine.
Really? Maybe your perception of the "average" person is colored by where you live and who you interact with.
In any case, the dumber they are, the more lethal they are.
https://www.sciencedirect.com/science/article/abs/pii/S01602...
This has been posted at least twice before on HN.