> The direction that AI takes is not written into the technology itself. It depends on our choices, on the institutions that host and underpin that choosing, and on our ability to manage innovation responsibly to serve the common good.
This is such a key point -- the problem is that the $T that are being poured into AI are doing so for one purpose alone: ROI
The choice has already been made, and it's the wrong one.
This is the same position of Yuval Noah Harari in his recent Economist interview: The ones in power (Musk etc.) try to frame the bad AI future as inevitable, instead of shouldering responsibility. https://www.youtube.com/watch?v=FNZhxTtOL-I
It's the correct choice according to the fundamental principles of capitalism, as set down by Milton Friedman.[1] and Lewis Powell.[2] Friedman set down the theory:
In an ideal free market resting on private property, no individual can coerce any other, all cooperation is voluntary, all parties to such cooperation benefit or they need not participate. There are no “social” values, no “social” responsibilities in any sense other than the shared values and responsibilities of individuals. Society is a collection of individuals and of the various groups they voluntarily form. ...
(But) the doctrine of “social responsibility” taken seriously would extend the scope of the political mechanism to every human activity. It does not differ in philosophy from the most explicitly collectivist doctrine. It differs only by professing to believe that collectivist ends can be attained without collectivist means. That is why, in my book “Capitalism and Freedom,” I have called it a “fundamentally subversive doctrine” in a free society, and have said that in such a society, “there is one and only one social responsibility of business—to use its resources and engage in activities designed to increase its profits so long as it stays within the rules of the game, which is to say, engages in open and free competition without deception or fraud.”
Then Powell (the Powell who became a Supreme Court Justice) and the U.S. Chamber of Commerce set out to change the rules of the game.
The 1971 "Powell memo" was written when the values of the 1960s had permeated society.
The objective of the memo was to turn that around. It's a battle plan to use corporate power to change government. When it was written, few companies had lobbyists. Now, corporate lobbying is huge, and effective.
The extent to which this well-funded project worked doesn't have to be described today.
What's worth realizing is that it wasn't always that way.
The U.S. Government of the Lyndon Johnson era ("Great Society", "War on Poverty") was organized to deal with something like the impact of AI on employment. The current government is not. It is permeated with people who think it the government shouldn't do anything about it.
If eg Putin had been more greedy instead of power hungry, we wouldn't have had his war against Ukraine. Same for many other wars and conflicts, including the US war on Iran, or even WW2 if you want to go back in time.
Greed, for lack of a better word, is good. Greed is right. Greed works. Greed clarifies, cuts through, and captures the essence of the evolutionary spirit.
You're quoting Wall Street (1987), but it's not clear whether you're in support of the sentiment or not. Plenty of people around here weren't born when that film came out.
Greed is good for the Greedy, but disasterous for anyone you'd ever want live with. Greed simplifies and creates clarity by destroying and debasing acts of humanity that are actually worthy to something that can exist as a row in a ledger. A society cannot thrive on greed alone. A society based only on greed abandons every moral check against corruption and manipulation imaginable. There is nothing more frightening to a crook than a world full of decent people that can't empathize with greed, and will actually talk amongst themselves about it, and react. Without greed, people can't be easily greased or goaded. They can't be controlled or cajoled or moved with mere trickery. You are forced to appeal in the one thing the crook doesn't have in abundance. Virtue, and rightness.
Yeah. And the quality of AI will only get worse from here unless we can commoditize open weight models.
If Google and Anthropic and such are able to own access, they will enshittify it horribly. Because they are powerless to do anything but, in the name of profit.
I think it's just as likely that these companies will choose different strategies and go after different markets. Though there are some similarities, Apple didn't do the same thing as Microsoft. And although they do copy features, Android isn't the same as iPhone.
In particular, Google depends a lot more on advertising than Anthropic does, so their incentives are different.
I'd say that the outcome mostly depends on whether something resembling AGI is achieved since that would more or less inevitably end up leading to something resembling ASI ('super-intelligence') which in turn makes it questionable whether the resulting entity can be counted upon to 'serve the common good' - whatever that may be. The quest for ROI has at most a temporary influence on the direction if this path is followed unless it actually keeps AGI (and thus ASI) from being achieved.
That's the thing about modern chatbots--they are nothing new technologically. They're just taking the same old neural networks and brute-forcing everything with ridiculous computing power and energy consumption. Autonomous agents just feed everything back into a loop that's even more wasteful.
There is interesting debate about whether, and how much, LLMs were used to write parts of this encyclical. Generally encyclicals are written by a group of Cardinals and/or Bishops, with significant input from the Pope, so it seems possible that while the Pope's sections and speeches have been human-written, some of the other writers used LLMs in some capacity.
The argument boil down to: Pangram seems to flag some sections, some sections have tells or AI smell, and despite being written in a similar register, previous writings of both the Pope and previous encyclicals are not flagged.
I'm not sure that it's a disaster if parts have been AI assisted, but it's an interesting question to ponder, and would be interested in the HN view.
My view is that it doesn't really matter, and focusing on it is likely an attempt to distract from the substance of the encyclical, which could be seen as critical of AI insofar as it calls for stronger regulation and more conscientious use of AI. As an aside, I think the arguments put forward in the blog post you link are pretty weak, but again, getting into that would just distract from the actual points being made in the encyclical.
I have caught myself writing in a manner similar to LLM output. I put it down to reading so much of LLM content at work and online that it bleeds into my own style.. Nothing egregious like the "it's not this, it's that" pattern, but more subtle like the cadence and the balance of active vs passive voice.
What I'm saying is that it's possible that some of these tells are becoming part of standard writing style now.
The encyclical is not anti-AI, so there's not really any contradiction or hypocrisy if AI was used to write it. It's more of a criticism of the "technocratic paradigm" and the potential for abuse of power and lack of accountability.
I think it's a matter of incentives. American billionaires are mostly highly financially exposed to AI, so are not really incentivised to say anything interesting about it except "it's great, you should buy it".
It is fascinating (this word doesn't quite capture sentiment, but the best I can think of right now) how any mention of the Catholic Church, on whatever specific topic — in this case, an encyclical about AI — triggers the immediate desire in some to go full ad hominem.
"billionaire", "in some Vatican sense". Would you elaborate? This just sounds like such a low and lazy attack
On one hand I appreciate the considerate scholarship on display, and since there are so many Catholics maybe some will even listen to their purported leader. On the other hand I find any talk of greed, selfishness, manipulation and degradation of humanity... really rich coming from a pope. Maybe I'm supposed to forgive, but I'm not going to forget that they spent decades abusing, hiding abuse, trying to force abortion to the margins along with birth control even when HIV became an obvious pandemic. I'm tired of the leader of an organization that owns a whole city preaching poverty to the masses, even if this particular leader has better PR than most.
Enough people have made these same points without the taint of that organization that I don't see this offering as necessary for the discussion, it mostly seems like a way to insert religion into something that simply doesn't require it. Faith can be a beautiful thing, personal and meaningful, but a giant organization with power and money is always going to be just that and should be treated with according suspicion.
The most common form of male birth control also happens to be the primary recommended means to avoid HIV and other STI's: the condom. Sorry if I was unclear and made it seem like it was improperly related.
The only 100% effective form of birth control, which is also the most effective at avoiding HIV and other STIs, is the one that the Catholic Church recommends. Why do you want them to promote less effective alternatives instead?
It's not 100% effective, in the real world you don't start with a hypothetical value and then insist it's real despite thousands of years of evidence to the contrary: You Modify Your Hypothesis.
There are many practices that have been widespread for thousands of years, which we would nevertheless all consider wrong. Why is this issue different?
I'd argue that it isn't something to promote harmful policies over, recognizing reality as it exists is essential. You can want to change the world, but until you have you need to work with what you have and not doom so many people to horrible lives and deaths as a result of your ultimate goal.
Edit: You ultimately can't claim a fictional effectiveness rate for something that's never demonstrated that level of success. Theory and Practice DIFFER.
The abuse, sure, but the birth control stance is really the only one compatible with the Catholic idea of sex only being appropriate in the context of a monogamous, lifelong married couple. I'm not really sure what you're getting at with that one, since there wouldn't be an HIV or STI epidemic in that world.
In this world with real people in it a lot of needless suffering and death resulted from that allegedly principled stance. I'm not sure what to take away from the idea that in a different, imaginary world this would have been harmless, except to conclude that you blame people for not meeting a particular ideal of "love" and sex.
Again, one promulgated by people who see fit to protect pedophiles and allow them to continue their abuse.
My feeling is that freedom of religion is important, but I also believe that there have to be limits and public health crises are one of those limiting factors. I would point today to people who use a religious exemption to avoid vaccination as an example of people who's (perhaps) sincere expression of religious belief is incompatible with their duty to the society they're a part of.
Is the Catholic Church against condoms being used for gay sex? It's my understanding that they're only against them because they're against purely recreational sex. If the sex can't result in a child anyway then I dont think they're going to treat it as a double sin.
I'm unsure how this relates to my comment, unless you're under the impression that HIV is exclusively spread through "gay sex" as you put. I certainly don't want to make the assumption about you though, given how archaic and bigoted that view is. If you could clarify what you mean I'd appreciate it.
Not exclusively but predominantly. Which makes me doubtful that the Catholic Church moved the numbers much on the HIV pandemic. Assuming of course that they weren't against condom use between men, which I'm not sure if they had or have a stance on.
89. Living out justice in the Church means purifying ecclesial relationships and structures from distortions that give rise to inequality, lack of transparency and abuse of power. In this regard, listening to the victims of spiritual, economic, institutional, sexual and power-based abuse, as well as abuses of conscience, is an integral part of a journey toward justice, which includes acknowledging the harm done, just reparation and taking steps to prevent it from happening again. Every power is at the service of communion and mission. All authority is at the service of the People of God. This ministry of service is expressed not only through our faith celebrated and lived in the Sacraments, and in the adoption of a synodal style, but also in the concrete sharing of goods. Following the example of the early Church, ecclesial resources need to be shared so that no one among us may be in need (cf. Acts 434, and so that their administration may support the mission of proclaiming the Gospel to the poorest. Regular assessments of the exercise of ministerial responsibilities should be encouraged, not as judgments on individuals, but as tools for learning and correction oriented toward mission. 116 Only to the extent that we are open to the action of the Holy Spirit will these principles of Social Doctrine become incarnate in ecclesial life. In this way, the Church will be able to bear credible witness to society that seeking the common good together, with shared responsibility and fraternity, is not a utopia, but a real possibility.
I did read it, but words are cheap, even very flowery words that make all sorts of promises. There's no particular reason to trust that organization's leadership, so any promises should be believed only once results materialize and last for some period of time. I for example would hesitate to credit the church with stamping out abuse just because no major stories have broken this year (that I'm aware of).
Likewise when it comes to changing their practical stances and teaching on matters that have very real impacts on health and wellbeing of so many, I'll believe it when I see the results for some time.
Catholic nuns and religious were some of the first and only to help gay men during the AIDS crisis, so your moral pearl clutching is somewhat hyperbolical. Yes the church has particular moral beliefs you disagree. Join the club.
I'm disinterested in their beliefs, only their actions. Their actions overall speak much more loudly of a callous disregard for life (see: HIV in Africa) and the wellbeing of their most helpless charges (children). I'm not damning Catholics here, I'm talking about the central political and financial entity at the heart of it only.
> The direction that AI takes is not written into the technology itself. It depends on our choices, on the institutions that host and underpin that choosing, and on our ability to manage innovation responsibly to serve the common good.
This is such a key point -- the problem is that the $T that are being poured into AI are doing so for one purpose alone: ROI
The choice has already been made, and it's the wrong one.
This is the same position of Yuval Noah Harari in his recent Economist interview: The ones in power (Musk etc.) try to frame the bad AI future as inevitable, instead of shouldering responsibility. https://www.youtube.com/watch?v=FNZhxTtOL-I
The whole interview is well worth your time.
It's the correct choice according to the fundamental principles of capitalism, as set down by Milton Friedman.[1] and Lewis Powell.[2] Friedman set down the theory:
In an ideal free market resting on private property, no individual can coerce any other, all cooperation is voluntary, all parties to such cooperation benefit or they need not participate. There are no “social” values, no “social” responsibilities in any sense other than the shared values and responsibilities of individuals. Society is a collection of individuals and of the various groups they voluntarily form. ...
(But) the doctrine of “social responsibility” taken seriously would extend the scope of the political mechanism to every human activity. It does not differ in philosophy from the most explicitly collectivist doctrine. It differs only by professing to believe that collectivist ends can be attained without collectivist means. That is why, in my book “Capitalism and Freedom,” I have called it a “fundamentally subversive doctrine” in a free society, and have said that in such a society, “there is one and only one social responsibility of business—to use its resources and engage in activities designed to increase its profits so long as it stays within the rules of the game, which is to say, engages in open and free competition without deception or fraud.”
Then Powell (the Powell who became a Supreme Court Justice) and the U.S. Chamber of Commerce set out to change the rules of the game. The 1971 "Powell memo" was written when the values of the 1960s had permeated society. The objective of the memo was to turn that around. It's a battle plan to use corporate power to change government. When it was written, few companies had lobbyists. Now, corporate lobbying is huge, and effective.
The extent to which this well-funded project worked doesn't have to be described today. What's worth realizing is that it wasn't always that way.
The U.S. Government of the Lyndon Johnson era ("Great Society", "War on Poverty") was organized to deal with something like the impact of AI on employment. The current government is not. It is permeated with people who think it the government shouldn't do anything about it.
[1] https://www.nytimes.com/1970/09/13/archives/a-friedman-doctr...
[2] https://reclaimdemocracy.org/powell_memo_lewis/
It ain't capitalism if we have the FOMC chop off the invisible hand with 20 years of LIRP/ZIRP
> It's the correct choice according to the fundamental principles of capitalism
sure, it aligns with the principles of capitalism
that doesn't make it the right choice
I agree with your summary. Greed has won out over everything else.
Ha, if only! I'd like to live in that utopia!
If eg Putin had been more greedy instead of power hungry, we wouldn't have had his war against Ukraine. Same for many other wars and conflicts, including the US war on Iran, or even WW2 if you want to go back in time.
Greed, for lack of a better word, is good. Greed is right. Greed works. Greed clarifies, cuts through, and captures the essence of the evolutionary spirit.
You're quoting Wall Street (1987), but it's not clear whether you're in support of the sentiment or not. Plenty of people around here weren't born when that film came out.
Greed is good for the Greedy, but disasterous for anyone you'd ever want live with. Greed simplifies and creates clarity by destroying and debasing acts of humanity that are actually worthy to something that can exist as a row in a ledger. A society cannot thrive on greed alone. A society based only on greed abandons every moral check against corruption and manipulation imaginable. There is nothing more frightening to a crook than a world full of decent people that can't empathize with greed, and will actually talk amongst themselves about it, and react. Without greed, people can't be easily greased or goaded. They can't be controlled or cajoled or moved with mere trickery. You are forced to appeal in the one thing the crook doesn't have in abundance. Virtue, and rightness.
We must be greedy for justice, greedy for kindness, greedy for people taking responsibility for their actions...
There is no such thing as an evolutionary spirit.
Greed is "good" for a few, clearly.
Its certainly the wrong choice if by all indications the R is apparently missing in your ROI.
Yeah. And the quality of AI will only get worse from here unless we can commoditize open weight models.
If Google and Anthropic and such are able to own access, they will enshittify it horribly. Because they are powerless to do anything but, in the name of profit.
I think it's just as likely that these companies will choose different strategies and go after different markets. Though there are some similarities, Apple didn't do the same thing as Microsoft. And although they do copy features, Android isn't the same as iPhone.
In particular, Google depends a lot more on advertising than Anthropic does, so their incentives are different.
I'd say that the outcome mostly depends on whether something resembling AGI is achieved since that would more or less inevitably end up leading to something resembling ASI ('super-intelligence') which in turn makes it questionable whether the resulting entity can be counted upon to 'serve the common good' - whatever that may be. The quest for ROI has at most a temporary influence on the direction if this path is followed unless it actually keeps AGI (and thus ASI) from being achieved.
I am afraid that anything created by humans will be, inherently, influenced in its ultimate development by human nature.
Therefore, whatever AGI looks like will be selfish, violent, and short sighted...
But our current crop of chatbots are not it. They're just parrots that happen to be pretty bad for the climate.
That's the thing about modern chatbots--they are nothing new technologically. They're just taking the same old neural networks and brute-forcing everything with ridiculous computing power and energy consumption. Autonomous agents just feed everything back into a loop that's even more wasteful.
There is interesting debate about whether, and how much, LLMs were used to write parts of this encyclical. Generally encyclicals are written by a group of Cardinals and/or Bishops, with significant input from the Pope, so it seems possible that while the Pope's sections and speeches have been human-written, some of the other writers used LLMs in some capacity.
The argument boil down to: Pangram seems to flag some sections, some sections have tells or AI smell, and despite being written in a similar register, previous writings of both the Pope and previous encyclicals are not flagged.
I'm not sure that it's a disaster if parts have been AI assisted, but it's an interesting question to ponder, and would be interested in the HN view.
Best post I've found arguing that there was LLM use: https://substack.com/inbox/post/199316613
Australian startup apparently verifying (without much detail) that it's not: https://www.proudlyhuman.org/news-room/media-releases/pope-l...
My view is that it doesn't really matter, and focusing on it is likely an attempt to distract from the substance of the encyclical, which could be seen as critical of AI insofar as it calls for stronger regulation and more conscientious use of AI. As an aside, I think the arguments put forward in the blog post you link are pretty weak, but again, getting into that would just distract from the actual points being made in the encyclical.
I have caught myself writing in a manner similar to LLM output. I put it down to reading so much of LLM content at work and online that it bleeds into my own style.. Nothing egregious like the "it's not this, it's that" pattern, but more subtle like the cadence and the balance of active vs passive voice.
What I'm saying is that it's possible that some of these tells are becoming part of standard writing style now.
The encyclical is not anti-AI, so there's not really any contradiction or hypocrisy if AI was used to write it. It's more of a criticism of the "technocratic paradigm" and the potential for abuse of power and lack of accountability.
who gives a shit tbh, argue with the substance, not the style.
It blows my mind that the Pope can say more relevant words about AI than American billionaires.
I think it's a matter of incentives. American billionaires are mostly highly financially exposed to AI, so are not really incentivised to say anything interesting about it except "it's great, you should buy it".
Saying relevant words is part of the Pope's job description. Not so for billionaires
the Pope is an American billionaire at this point, in some Vatican sense.
It is fascinating (this word doesn't quite capture sentiment, but the best I can think of right now) how any mention of the Catholic Church, on whatever specific topic — in this case, an encyclical about AI — triggers the immediate desire in some to go full ad hominem.
"billionaire", "in some Vatican sense". Would you elaborate? This just sounds like such a low and lazy attack
did you read the comment i responded to?
not some ad hominem attack at all: the parent comment remarked the pope knew more about AI than American billionaires.
The pope is American, is in charge of the Holy See, which has assets in the billions. Thus the irony of the comment to which i responded.
[dead]
[dead]
On one hand I appreciate the considerate scholarship on display, and since there are so many Catholics maybe some will even listen to their purported leader. On the other hand I find any talk of greed, selfishness, manipulation and degradation of humanity... really rich coming from a pope. Maybe I'm supposed to forgive, but I'm not going to forget that they spent decades abusing, hiding abuse, trying to force abortion to the margins along with birth control even when HIV became an obvious pandemic. I'm tired of the leader of an organization that owns a whole city preaching poverty to the masses, even if this particular leader has better PR than most.
Enough people have made these same points without the taint of that organization that I don't see this offering as necessary for the discussion, it mostly seems like a way to insert religion into something that simply doesn't require it. Faith can be a beautiful thing, personal and meaningful, but a giant organization with power and money is always going to be just that and should be treated with according suspicion.
> abortion to the margins along with birth control even when HIV became an obvious pandemic.
How are abortion and birth control at all related to HIV? I don't understand the connection.
Condoms prevents the spread of HIV.
The most common form of male birth control also happens to be the primary recommended means to avoid HIV and other STI's: the condom. Sorry if I was unclear and made it seem like it was improperly related.
My bad, I was thinking hormonal.
That's completely understandable and probably down to my hasty phrasing!
The only 100% effective form of birth control, which is also the most effective at avoiding HIV and other STIs, is the one that the Catholic Church recommends. Why do you want them to promote less effective alternatives instead?
It's not 100% effective, in the real world you don't start with a hypothetical value and then insist it's real despite thousands of years of evidence to the contrary: You Modify Your Hypothesis.
There are many practices that have been widespread for thousands of years, which we would nevertheless all consider wrong. Why is this issue different?
I'd argue that it isn't something to promote harmful policies over, recognizing reality as it exists is essential. You can want to change the world, but until you have you need to work with what you have and not doom so many people to horrible lives and deaths as a result of your ultimate goal.
Edit: You ultimately can't claim a fictional effectiveness rate for something that's never demonstrated that level of success. Theory and Practice DIFFER.
The abuse, sure, but the birth control stance is really the only one compatible with the Catholic idea of sex only being appropriate in the context of a monogamous, lifelong married couple. I'm not really sure what you're getting at with that one, since there wouldn't be an HIV or STI epidemic in that world.
In this world with real people in it a lot of needless suffering and death resulted from that allegedly principled stance. I'm not sure what to take away from the idea that in a different, imaginary world this would have been harmless, except to conclude that you blame people for not meeting a particular ideal of "love" and sex.
Again, one promulgated by people who see fit to protect pedophiles and allow them to continue their abuse.
>trying to force abortion to the margins along with birth control
Do you have some suggestions about the doctrines that Catholics should assent to?
My feeling is that freedom of religion is important, but I also believe that there have to be limits and public health crises are one of those limiting factors. I would point today to people who use a religious exemption to avoid vaccination as an example of people who's (perhaps) sincere expression of religious belief is incompatible with their duty to the society they're a part of.
You're accountable to more than God.
Is the Catholic Church against condoms being used for gay sex? It's my understanding that they're only against them because they're against purely recreational sex. If the sex can't result in a child anyway then I dont think they're going to treat it as a double sin.
Restricting the availability of the condoms restricts them for everyone.
I'm unsure how this relates to my comment, unless you're under the impression that HIV is exclusively spread through "gay sex" as you put. I certainly don't want to make the assumption about you though, given how archaic and bigoted that view is. If you could clarify what you mean I'd appreciate it.
Not exclusively but predominantly. Which makes me doubtful that the Catholic Church moved the numbers much on the HIV pandemic. Assuming of course that they weren't against condom use between men, which I'm not sure if they had or have a stance on.
Did you read it?
89. Living out justice in the Church means purifying ecclesial relationships and structures from distortions that give rise to inequality, lack of transparency and abuse of power. In this regard, listening to the victims of spiritual, economic, institutional, sexual and power-based abuse, as well as abuses of conscience, is an integral part of a journey toward justice, which includes acknowledging the harm done, just reparation and taking steps to prevent it from happening again. Every power is at the service of communion and mission. All authority is at the service of the People of God. This ministry of service is expressed not only through our faith celebrated and lived in the Sacraments, and in the adoption of a synodal style, but also in the concrete sharing of goods. Following the example of the early Church, ecclesial resources need to be shared so that no one among us may be in need (cf. Acts 434, and so that their administration may support the mission of proclaiming the Gospel to the poorest. Regular assessments of the exercise of ministerial responsibilities should be encouraged, not as judgments on individuals, but as tools for learning and correction oriented toward mission. 116 Only to the extent that we are open to the action of the Holy Spirit will these principles of Social Doctrine become incarnate in ecclesial life. In this way, the Church will be able to bear credible witness to society that seeking the common good together, with shared responsibility and fraternity, is not a utopia, but a real possibility.
https://www.vatican.va/content/leo-xiv/en/events/event.dir.h...
I did read it, but words are cheap, even very flowery words that make all sorts of promises. There's no particular reason to trust that organization's leadership, so any promises should be believed only once results materialize and last for some period of time. I for example would hesitate to credit the church with stamping out abuse just because no major stories have broken this year (that I'm aware of).
Likewise when it comes to changing their practical stances and teaching on matters that have very real impacts on health and wellbeing of so many, I'll believe it when I see the results for some time.
Catholic nuns and religious were some of the first and only to help gay men during the AIDS crisis, so your moral pearl clutching is somewhat hyperbolical. Yes the church has particular moral beliefs you disagree. Join the club.
I'm disinterested in their beliefs, only their actions. Their actions overall speak much more loudly of a callous disregard for life (see: HIV in Africa) and the wellbeing of their most helpless charges (children). I'm not damning Catholics here, I'm talking about the central political and financial entity at the heart of it only.
Always interesting to see older philosophical texts discussed here. Often find surprising parallels to modern tech ethics debates.
I'm not sure if I'm misunderstanding you, but Magnifica Humanitas was published earlier this year.